Hallvard Furuseth [Sun, 5 May 2013 12:26:05 +0000 (14:26 +0200)]
Fix mdb_env_close(unopened MDB_env).
Do not try to scan me_dbxs in a closed/never-opened MDB_env.
Broken by
7d643d3acb94a6d7306330b550fce0e5755444a9
and
151c416b460b02cd96502a47189cda19ab651ada.
Hallvard Furuseth [Sat, 4 May 2013 10:53:43 +0000 (12:53 +0200)]
Simplify/cleanup mdb_cursor_init, C_ALLOCD.
No real change.
mdb_cursor_init() checks if it needs mx, so pass it unconditionally.
Set C_ALLOCD for shadow cursors, for clarity. (It was always set as
it should anyway from the origin cursor, which would have C_ALLOCD.)
Hallvard Furuseth [Sat, 4 May 2013 10:41:28 +0000 (12:41 +0200)]
MDB warning cleanup.
Unused function when MDB_DEBUG. Unused 'excl' param.
Hallvard Furuseth [Sat, 4 May 2013 10:34:49 +0000 (12:34 +0200)]
mdb_dcmp(): Assume the database has MDB_DUPSORT.
There was little point in returning EINVAL when not: Comparing (A,B)
and (B,A) would claim (A > B && B > A), which could confuse callers.
Hallvard Furuseth [Sat, 4 May 2013 10:34:20 +0000 (12:34 +0200)]
Update MDB doc: Cursors, DB handles, data lifetime
Hallvard Furuseth [Sat, 4 May 2013 10:34:02 +0000 (12:34 +0200)]
Allow mdb_cursor_close() after readonly txn ends.
Catch mdb_cursor_renew(write txn's cursor). Add flag C_UNTRACK, so
mdb_cursor_close need not peek inside a possibly-freed readonly txn.
Hallvard Furuseth [Sat, 4 May 2013 10:33:06 +0000 (12:33 +0200)]
Close cursors when commit(writer). Factor out code.
Close remaining cursors when committing a write txn. The doc says this
happens, and it avoids cursor tracking when updating mainDB + freeDB.
Rename mdb_cursor_merge() -> mdb_cursors_close() for code reuse,
and add a merge option. Simplify its loop a bit.
Factor out cleanup of DBIs.
Hallvard Furuseth [Sat, 4 May 2013 10:31:31 +0000 (12:31 +0200)]
Plug txn leak and MDB_NOTLS slot leaks on error.
On mdb_env_copy() error: Abort the txn.
On mdb_txn_renew0() error: Release new MDB_NOTLS reader slot.
Hallvard Furuseth [Sat, 4 May 2013 08:03:19 +0000 (10:03 +0200)]
Cleaner "Support mdb_txn_abort(a reset txn)".
Redo
8a562f560edcc6417672df453c4e626dfb4c4024 so !mt_dbxs indicates
"txn was reset", so mt_numdbs gets one magic value instead of two.
Hallvard Furuseth [Sat, 4 May 2013 08:03:19 +0000 (10:03 +0200)]
Update fixes for dbi_open/close, ITS#7515.
Reset me_dbflags[dbi] when closing DBI, to get rid of MDB_VALID flag.
mdb_env_close(): Re-fix DB-name memleak. DBIs > me_numdbs may exist.
Howard Chu [Thu, 2 May 2013 05:43:35 +0000 (22:43 -0700)]
Allow reading freelist while working on it
The circular dependency issues appear to have been resolved.
Still, need to watch closely, maybe revert this change if
problems arise.
Howard Chu [Wed, 1 May 2013 04:09:09 +0000 (21:09 -0700)]
Avoid assert
Due to underfilled branch page. We're in the process of merging/moving
nodes to it because we already know it's underfilled. Took this approach
rather than just removing the assert in mdb_page_search_root, because
that assert may yet catch other situations we don't know about.
(Although, it has been there since the original commit of mdb.c and
has never triggered any other times...)
Hallvard Furuseth [Mon, 22 Apr 2013 18:46:56 +0000 (20:46 +0200)]
Plug mdb_cursor_renew() memleak
Howard Chu [Sun, 21 Apr 2013 02:30:06 +0000 (19:30 -0700)]
Fix MDB_LAST, reset cursor index
Howard Chu [Sat, 20 Apr 2013 16:47:33 +0000 (09:47 -0700)]
Hallvard Furuseth [Thu, 18 Apr 2013 06:00:03 +0000 (08:00 +0200)]
Tweak comments/readability.
Show MDB_PERSISTENT/MDB_VALID/DB_VALID relationship.
mdb_txn_renew0(): Remove obsolete "cannot fail" comment.
Howard Chu [Thu, 18 Apr 2013 02:17:03 +0000 (04:17 +0200)]
Add MDB_NOTLS envflag.
Hallvard Furuseth [Thu, 18 Apr 2013 02:16:07 +0000 (04:16 +0200)]
Catch MDB txn reuse/sync errors.
Hallvard Furuseth [Thu, 18 Apr 2013 02:15:45 +0000 (04:15 +0200)]
Support mdb_txn_abort(a reset txn).
Hallvard Furuseth [Thu, 18 Apr 2013 02:15:13 +0000 (04:15 +0200)]
Cleanup for TLS key and read-only filesystem.
Move key init into mdb_env_setup_locks().
Don't create unused TLS key when read-only filesystem.
Drop internal flag MDB_ROFS, we can instead test either
!me_txns, !mt_u.reader or me_lfd==INVALID_HANDLE_VALUE.
Hallvard Furuseth [Thu, 18 Apr 2013 02:13:43 +0000 (04:13 +0200)]
Clean up MDB_env setup.
Malloc before I/O. Avoids possible malloc error after I/O.
Don't allocate dirty & free lists when MDB_RDONLY.
Factor out code.
Howard Chu [Wed, 17 Apr 2013 21:33:41 +0000 (14:33 -0700)]
Fix mdb_rebalance
Don't do anything with (fake root) subpages
Howard Chu [Tue, 16 Apr 2013 19:05:03 +0000 (12:05 -0700)]
ITS#7574 blind fix
Fix
227329c8e1fb818548f1936f5dc1349ffd733523, don't persist
the MDB_VALID bit in db.md_flags.
Howard Chu [Mon, 15 Apr 2013 16:30:39 +0000 (09:30 -0700)]
fix prev commit
Howard Chu [Mon, 15 Apr 2013 06:59:54 +0000 (23:59 -0700)]
Fix mdb_rebalance
Need to check NUMKEYS as well as fill threshold, when deciding
whether to change anything. Don't let the page drop below the
minimum number of keys.
Howard Chu [Sun, 14 Apr 2013 23:53:30 +0000 (16:53 -0700)]
Revert "ITS#7570 fix MDB_GET_MULTIPLE and MDB_NEXT_MULTIPLE"
This reverts commit
372b3ddb85bef953fb548fbf2b9ef0cba4e3c8ed.
Original code works as designed.
Claude Brisson [Wed, 10 Apr 2013 12:40:49 +0000 (14:40 +0200)]
ITS#7570 fix MDB_GET_MULTIPLE and MDB_NEXT_MULTIPLE
Howard Chu [Mon, 8 Apr 2013 02:21:46 +0000 (19:21 -0700)]
Fix
227329c8e1fb818548f1936f5dc1349ffd73352
mdb_dbi_close() must allow any dbi < maxdbs, since opens in an
uncommitted txn don't increment env->me_numdbs.
Howard Chu [Fri, 5 Apr 2013 10:44:56 +0000 (03:44 -0700)]
Revert "Fix
6beaad52129da5353fd40c0ec48e6a78c4f71a2e"
This reverts commit
8eef7a4275eda8f2fa2e0d1e67c1d5cbcd91607e.
Previous commit was correct, duplicate keys should not appear here.
Howard Chu [Tue, 2 Apr 2013 18:25:44 +0000 (11:25 -0700)]
Fixes for dbi_open/close, ITS#7515
Howard Chu [Thu, 4 Apr 2013 23:16:25 +0000 (16:16 -0700)]
ITS#7561 Fix mdb_drop bugs
Check for overflow pages, reset cursor position properly.
Howard Chu [Thu, 4 Apr 2013 14:25:21 +0000 (07:25 -0700)]
Fix
6beaad52129da5353fd40c0ec48e6a78c4f71a2e
In APPEND don't immediately reject matching key, since this
is valid for APPENDDUP.
Howard Chu [Thu, 4 Apr 2013 01:26:41 +0000 (18:26 -0700)]
Fix
d3990eb2f8b2944782604a2ca7bd52e08c58fc3e
Append mode should *reject* keys that are too small.
Also allow APPENDDUP in mdb_put().
Howard Chu [Tue, 2 Apr 2013 03:34:40 +0000 (20:34 -0700)]
Fix rebalance when collapsing root page
When fixing other cursors, must also fix their depth.
Howard Chu [Tue, 2 Apr 2013 02:49:40 +0000 (19:49 -0700)]
More fixes for MDB_LAST
Make sure C_INITIALIZED gets set on successful call
Howard Chu [Tue, 2 Apr 2013 02:06:05 +0000 (19:06 -0700)]
ITS#7556 fix cursor_sibling for PREV
Hallvard Furuseth [Sun, 31 Mar 2013 21:50:02 +0000 (23:50 +0200)]
Plug MDB memory leak of DB names.
Hallvard Furuseth [Sun, 31 Mar 2013 21:48:10 +0000 (23:48 +0200)]
Silence signedness warning
Hallvard Furuseth [Sun, 31 Mar 2013 21:47:45 +0000 (23:47 +0200)]
Fix MDB_DEBUG compile when no varargs macros.
Variables mdb_debug, mdb_debug_start were used undefined.
Howard Chu [Sun, 31 Mar 2013 19:56:08 +0000 (20:56 +0100)]
Tweak mode_t def per Hallvard's suggestion
Howard Chu [Sun, 31 Mar 2013 16:10:43 +0000 (08:10 -0800)]
MSVC doesn't define mode_t
Howard Chu [Fri, 29 Mar 2013 08:08:06 +0000 (01:08 -0700)]
Fix CURSOR_LAST/CURSOR_PREV
Howard Chu [Tue, 26 Mar 2013 18:24:59 +0000 (11:24 -0700)]
ITS#7553 better fix
Howard Chu [Mon, 25 Mar 2013 19:15:53 +0000 (12:15 -0700)]
ITS#7553 deinit cursor after deleting last node
Howard Chu [Mon, 25 Mar 2013 18:53:32 +0000 (11:53 -0700)]
Silence stupid warnings
Claude Brisson [Fri, 22 Mar 2013 15:00:21 +0000 (16:00 +0100)]
bugfix for mdb_cursor_put with MDB_MULTIPLE
If the variable dkey.mv_size is non-zero, then it means dkey
contains some original data which has to be put back in the
child db, typically when the child db has just been created.
But when using MDB_MULTIPLE, if this variable has not been
reset to zero, we may come back to this section and wrongly
think that there is some original data to be taken care of.
Howard Chu [Wed, 13 Mar 2013 00:29:34 +0000 (17:29 -0700)]
ITS#7538 fallout from ITS#7536 fix.
Allow leaf pages to have only 1 key.
Howard Chu [Wed, 13 Mar 2013 00:25:49 +0000 (17:25 -0700)]
Precalculate max size of a node
Howard Chu [Mon, 11 Mar 2013 17:28:25 +0000 (10:28 -0700)]
Howard Chu [Mon, 11 Mar 2013 13:04:54 +0000 (06:04 -0700)]
Check for MADV_RANDOM
Android supports madvise but not posix_madvise
Howard Chu [Mon, 11 Mar 2013 12:59:07 +0000 (05:59 -0700)]
Howard Chu [Mon, 11 Mar 2013 12:49:14 +0000 (05:49 -0700)]
Turn off readahead on main mmap
It's harmful when the DB is larger than RAM.
Howard Chu [Wed, 6 Mar 2013 20:30:37 +0000 (12:30 -0800)]
ITS#7536 fix mdb_rebalance
A page must always have at least 2 keys (unless it's a root page)
Howard Chu [Mon, 25 Feb 2013 13:06:51 +0000 (05:06 -0800)]
Use CC/LDFLAGS to link the shared lib
Howard Chu [Mon, 25 Feb 2013 10:02:15 +0000 (02:02 -0800)]
Install manpages too
Howard Chu [Mon, 25 Feb 2013 07:40:34 +0000 (23:40 -0800)]
Delete obsolete @todo
We do it now... (coalescing freelist entries in page_alloc)
Howard Chu [Mon, 25 Feb 2013 07:39:17 +0000 (23:39 -0800)]
Add sample apps for comparison/education purposes
Howard Chu [Mon, 25 Feb 2013 06:50:51 +0000 (06:50 +0000)]
Add $(DESTDIR) to make install target
Howard Chu [Wed, 20 Feb 2013 13:08:52 +0000 (05:08 -0800)]
Update error code instances
Hallvard Furuseth [Wed, 20 Feb 2013 11:19:45 +0000 (12:19 +0100)]
Tweak MDB_INCOMPATIBLE description
Hallvard Furuseth [Wed, 20 Feb 2013 08:08:41 +0000 (09:08 +0100)]
Tweak prev commit: Restore if-test as an assert
Hallvard Furuseth [Tue, 19 Feb 2013 21:03:41 +0000 (22:03 +0100)]
ITS#7515 Fix mdb_txn_commit(nested txn).
Don't modify the parent txn until the current txn cannot fail.
Don't assume new dirty child pgnos > dirty parent pgnos.
Page alloc/touch: Fail if child+parent dirty pages would exceed
dirty_list's maxsize. Avoids an error situation in commit.
Hallvard Furuseth [Tue, 19 Feb 2013 21:03:04 +0000 (22:03 +0100)]
ITS#7512 Plug mdb_txn_abort(nested txn) page leaks.
Also catch mdb_cursor_shadow() errors.
Hallvard Furuseth [Tue, 19 Feb 2013 21:02:37 +0000 (22:02 +0100)]
Check DB flags when refreshing a stale MDB DBI.
It's hairy to figure out when a DBI is valid. Catch destructive
user errors, and flags which another process changed under us.
Hallvard Furuseth [Tue, 19 Feb 2013 21:02:15 +0000 (22:02 +0100)]
mdb_cursor_prev,mdb_cursor_next: Fix return value.
Return mdb_node_read()'s return value if it fails, not 1.
(Can happen if mdb_page_get() fails and NDEBUG is #defined.)
Hallvard Furuseth [Tue, 19 Feb 2013 21:01:29 +0000 (22:01 +0100)]
ITS#7515 mdb_dbi_open(): Also open in parent txns.
This makes aborting nested and non-nested txns more
similar: The new DBI is available to the surrounding
context (parent txn and MDB_env respectively).
Hallvard Furuseth [Tue, 19 Feb 2013 20:17:33 +0000 (21:17 +0100)]
mdb_page_alloc(): Handle freeDB txnid range holes.
A txn writes no freeDB entry if previous txn dropped mainDB and a read
txn prevents freelist entry reuse. This surprised mdb_page_alloc (and
mdb_txn_commit too before
65c053a6e7f6973c1d09710aa1bd57b218206fcb).
Hallvard Furuseth [Tue, 19 Feb 2013 20:15:26 +0000 (21:15 +0100)]
ITS#7517 Document that dirty DBs may not be closed
Hallvard Furuseth [Tue, 19 Feb 2013 20:14:23 +0000 (21:14 +0100)]
ITS#7485 Document key/data size limits in lmdb.h.
mdb.c already describes them. The user doc should too.
Hallvard Furuseth [Sun, 17 Feb 2013 07:42:14 +0000 (08:42 +0100)]
Revert "ITS#7515 update parent's mt_next_pgno on child commit"
This reverts commit
fd4861bf00fb0b86a9f3b80d16cbe363a8eac227.
It duplicated earlier code.
Howard Chu [Sun, 17 Feb 2013 00:49:53 +0000 (00:49 +0000)]
ITS#7515 update parent's mt_next_pgno on child commit
Howard Chu [Sun, 17 Feb 2013 00:48:43 +0000 (00:48 +0000)]
Don't limit retries when coalescing freelist
Try to use whatever's available.
Hallvard Furuseth [Sat, 16 Feb 2013 18:11:20 +0000 (19:11 +0100)]
ITS#7515 Fix MDB parent/child txn interaction.
mdb_txn_commit(child): Copy more state. Copy all of mt_dbs:
Include mainDB, and even freeDB since mdb_drop() can update
it. Don't skip DBs with unchanged root, this could break
when the new was newly opened and the old unused junk.
mdb_page_get(): Search parents' dirty lists.
Hallvard Furuseth [Sat, 16 Feb 2013 18:08:54 +0000 (19:08 +0100)]
ITS#7377 Catch MDB failure updating root pointers.
"cannot fail" was wrong, it fails at least when exceeding mapsize.
Hallvard Furuseth [Sat, 16 Feb 2013 18:08:37 +0000 (19:08 +0100)]
ITS#7517 Don't save dropped dirty MDB databases.
mdb_txn_commit's attempt to save the DB could corrupt another DB if
another thread had called mdb_dbi_open and reused the closed DBI.
Hallvard Furuseth [Sat, 16 Feb 2013 18:07:16 +0000 (19:07 +0100)]
mdb_stat -ff[f]: show contiguous page spans.
Hallvard Furuseth [Sat, 16 Feb 2013 18:06:28 +0000 (19:06 +0100)]
Add error code MDB_MAP_RESIZED.
Howard Chu [Thu, 14 Feb 2013 19:20:45 +0000 (19:20 +0000)]
Don't memmove freelist entry when chunking it
Just advance the pointer, now that it's no longer a complex struct
Hallvard Furuseth [Tue, 12 Feb 2013 11:27:59 +0000 (12:27 +0100)]
ITS#7455 Save freelist in single-page chunks
Hallvard Furuseth [Wed, 23 Jan 2013 14:47:35 +0000 (15:47 +0100)]
Freelist cleanup/streamlining
Drop unneeded definitions, redundant code.
Hallvard Furuseth [Thu, 7 Feb 2013 07:17:30 +0000 (08:17 +0100)]
ITS#7512 Fix MDB page leak when malloc error.
mdb_page_alloc(): Delay moving me_pgfirst,me_pglast
until malloc(MDB_oldpages to hold the IDs) succeeds.
Howard Chu [Mon, 4 Feb 2013 18:53:17 +0000 (10:53 -0800)]
Revert "ITS#7515 fix mdb_dbi_open/close"
This reverts commit
48dc782ea612f85e8356a50bfbafe22e5be121cf.
Howard Chu [Fri, 1 Feb 2013 14:19:23 +0000 (06:19 -0800)]
NULL key is allowed for MDB_CURRENT
Howard Chu [Fri, 1 Feb 2013 04:05:48 +0000 (20:05 -0800)]
ITS#7515 fix mdb_dbi_open/close
If a DBI handle is opened by a txn that aborts, the DBI handle
should no longer be valid.
Howard Chu [Tue, 29 Jan 2013 02:45:36 +0000 (18:45 -0800)]
ITS#7511 make sure cursor is marked valid after successful put
Howard Chu [Tue, 29 Jan 2013 01:08:24 +0000 (01:08 +0000)]
ITS#7505 fix prev commit, partial revert
Howard Chu [Sun, 27 Jan 2013 17:55:21 +0000 (17:55 +0000)]
ITS#7505 Fix mdb_update_key when key is too big
Howard Chu [Sat, 26 Jan 2013 18:55:35 +0000 (18:55 +0000)]
Fix MDB_CURRENT updating for dupsort items
Hallvard Furuseth [Wed, 16 Jan 2013 17:42:57 +0000 (18:42 +0100)]
Update MDB comments: Caveats, flags, etc.
Howard Chu [Tue, 15 Jan 2013 12:24:07 +0000 (04:24 -0800)]
ITS#7485 data sizes limited to 32 bits
That's all we have space for in a node record.
Howard Chu [Mon, 14 Jan 2013 21:46:25 +0000 (13:46 -0800)]
ITS#7485 more size checks
Howard Chu [Mon, 14 Jan 2013 21:12:15 +0000 (13:12 -0800)]
ITS#7485 check maxkeysize in mdb_cursor_put
Also MDB_MAXKEYSIZE is redefinable at compile time.
Howard Chu [Mon, 14 Jan 2013 20:57:54 +0000 (12:57 -0800)]
Howard Chu [Fri, 11 Jan 2013 19:45:25 +0000 (11:45 -0800)]
ITS#7491 check for filled dirty page list
Very large single transactions will fail. It's not just a problem when
nested transactions are used. We could make this dynamically sized,
but I'm not sure what the point is.
Howard Chu [Thu, 10 Jan 2013 10:58:55 +0000 (02:58 -0800)]
Happy New Year
Hallvard Furuseth [Wed, 9 Jan 2013 19:49:43 +0000 (20:49 +0100)]
Delete liblmdb/mfree. It has moved into mdb_stat.
Howard Chu [Wed, 9 Jan 2013 16:30:13 +0000 (08:30 -0800)]
ITS#7455 don't give up so soon
If we're operating on the freelist, see if our current pghead
can satisfy the request before giving up and using new pages.
Howard Chu [Wed, 19 Dec 2012 21:20:24 +0000 (13:20 -0800)]
Fix typo
Howard Chu [Wed, 19 Dec 2012 21:09:56 +0000 (13:09 -0800)]
Add initial mdb utility manpages
Howard Chu [Wed, 12 Dec 2012 06:21:52 +0000 (22:21 -0800)]
init_meta should write mapsize
Was leaving it 0 initially.