Howard Chu [Mon, 25 Mar 2013 18:53:32 +0000 (11:53 -0700)]
Silence stupid warnings
Claude Brisson [Fri, 22 Mar 2013 15:00:21 +0000 (16:00 +0100)]
bugfix for mdb_cursor_put with MDB_MULTIPLE
If the variable dkey.mv_size is non-zero, then it means dkey
contains some original data which has to be put back in the
child db, typically when the child db has just been created.
But when using MDB_MULTIPLE, if this variable has not been
reset to zero, we may come back to this section and wrongly
think that there is some original data to be taken care of.
Howard Chu [Wed, 13 Mar 2013 00:29:34 +0000 (17:29 -0700)]
ITS#7538 fallout from ITS#7536 fix.
Allow leaf pages to have only 1 key.
Howard Chu [Wed, 13 Mar 2013 00:25:49 +0000 (17:25 -0700)]
Precalculate max size of a node
Howard Chu [Mon, 11 Mar 2013 17:28:25 +0000 (10:28 -0700)]
Howard Chu [Mon, 11 Mar 2013 13:04:54 +0000 (06:04 -0700)]
Check for MADV_RANDOM
Android supports madvise but not posix_madvise
Howard Chu [Mon, 11 Mar 2013 12:59:07 +0000 (05:59 -0700)]
Howard Chu [Mon, 11 Mar 2013 12:49:14 +0000 (05:49 -0700)]
Turn off readahead on main mmap
It's harmful when the DB is larger than RAM.
Howard Chu [Wed, 6 Mar 2013 20:30:37 +0000 (12:30 -0800)]
ITS#7536 fix mdb_rebalance
A page must always have at least 2 keys (unless it's a root page)
Howard Chu [Mon, 25 Feb 2013 13:06:51 +0000 (05:06 -0800)]
Use CC/LDFLAGS to link the shared lib
Howard Chu [Mon, 25 Feb 2013 10:02:15 +0000 (02:02 -0800)]
Install manpages too
Howard Chu [Mon, 25 Feb 2013 07:40:34 +0000 (23:40 -0800)]
Delete obsolete @todo
We do it now... (coalescing freelist entries in page_alloc)
Howard Chu [Mon, 25 Feb 2013 07:39:17 +0000 (23:39 -0800)]
Add sample apps for comparison/education purposes
Howard Chu [Mon, 25 Feb 2013 06:50:51 +0000 (06:50 +0000)]
Add $(DESTDIR) to make install target
Howard Chu [Wed, 20 Feb 2013 13:08:52 +0000 (05:08 -0800)]
Update error code instances
Hallvard Furuseth [Wed, 20 Feb 2013 11:19:45 +0000 (12:19 +0100)]
Tweak MDB_INCOMPATIBLE description
Hallvard Furuseth [Wed, 20 Feb 2013 08:08:41 +0000 (09:08 +0100)]
Tweak prev commit: Restore if-test as an assert
Hallvard Furuseth [Tue, 19 Feb 2013 21:03:41 +0000 (22:03 +0100)]
ITS#7515 Fix mdb_txn_commit(nested txn).
Don't modify the parent txn until the current txn cannot fail.
Don't assume new dirty child pgnos > dirty parent pgnos.
Page alloc/touch: Fail if child+parent dirty pages would exceed
dirty_list's maxsize. Avoids an error situation in commit.
Hallvard Furuseth [Tue, 19 Feb 2013 21:03:04 +0000 (22:03 +0100)]
ITS#7512 Plug mdb_txn_abort(nested txn) page leaks.
Also catch mdb_cursor_shadow() errors.
Hallvard Furuseth [Tue, 19 Feb 2013 21:02:37 +0000 (22:02 +0100)]
Check DB flags when refreshing a stale MDB DBI.
It's hairy to figure out when a DBI is valid. Catch destructive
user errors, and flags which another process changed under us.
Hallvard Furuseth [Tue, 19 Feb 2013 21:02:15 +0000 (22:02 +0100)]
mdb_cursor_prev,mdb_cursor_next: Fix return value.
Return mdb_node_read()'s return value if it fails, not 1.
(Can happen if mdb_page_get() fails and NDEBUG is #defined.)
Hallvard Furuseth [Tue, 19 Feb 2013 21:01:29 +0000 (22:01 +0100)]
ITS#7515 mdb_dbi_open(): Also open in parent txns.
This makes aborting nested and non-nested txns more
similar: The new DBI is available to the surrounding
context (parent txn and MDB_env respectively).
Hallvard Furuseth [Tue, 19 Feb 2013 20:17:33 +0000 (21:17 +0100)]
mdb_page_alloc(): Handle freeDB txnid range holes.
A txn writes no freeDB entry if previous txn dropped mainDB and a read
txn prevents freelist entry reuse. This surprised mdb_page_alloc (and
mdb_txn_commit too before
65c053a6e7f6973c1d09710aa1bd57b218206fcb).
Hallvard Furuseth [Tue, 19 Feb 2013 20:15:26 +0000 (21:15 +0100)]
ITS#7517 Document that dirty DBs may not be closed
Hallvard Furuseth [Tue, 19 Feb 2013 20:14:23 +0000 (21:14 +0100)]
ITS#7485 Document key/data size limits in lmdb.h.
mdb.c already describes them. The user doc should too.
Hallvard Furuseth [Sun, 17 Feb 2013 07:42:14 +0000 (08:42 +0100)]
Revert "ITS#7515 update parent's mt_next_pgno on child commit"
This reverts commit
fd4861bf00fb0b86a9f3b80d16cbe363a8eac227.
It duplicated earlier code.
Howard Chu [Sun, 17 Feb 2013 00:49:53 +0000 (00:49 +0000)]
ITS#7515 update parent's mt_next_pgno on child commit
Howard Chu [Sun, 17 Feb 2013 00:48:43 +0000 (00:48 +0000)]
Don't limit retries when coalescing freelist
Try to use whatever's available.
Hallvard Furuseth [Sat, 16 Feb 2013 18:11:20 +0000 (19:11 +0100)]
ITS#7515 Fix MDB parent/child txn interaction.
mdb_txn_commit(child): Copy more state. Copy all of mt_dbs:
Include mainDB, and even freeDB since mdb_drop() can update
it. Don't skip DBs with unchanged root, this could break
when the new was newly opened and the old unused junk.
mdb_page_get(): Search parents' dirty lists.
Hallvard Furuseth [Sat, 16 Feb 2013 18:08:54 +0000 (19:08 +0100)]
ITS#7377 Catch MDB failure updating root pointers.
"cannot fail" was wrong, it fails at least when exceeding mapsize.
Hallvard Furuseth [Sat, 16 Feb 2013 18:08:37 +0000 (19:08 +0100)]
ITS#7517 Don't save dropped dirty MDB databases.
mdb_txn_commit's attempt to save the DB could corrupt another DB if
another thread had called mdb_dbi_open and reused the closed DBI.
Hallvard Furuseth [Sat, 16 Feb 2013 18:07:16 +0000 (19:07 +0100)]
mdb_stat -ff[f]: show contiguous page spans.
Hallvard Furuseth [Sat, 16 Feb 2013 18:06:28 +0000 (19:06 +0100)]
Add error code MDB_MAP_RESIZED.
Howard Chu [Thu, 14 Feb 2013 19:20:45 +0000 (19:20 +0000)]
Don't memmove freelist entry when chunking it
Just advance the pointer, now that it's no longer a complex struct
Hallvard Furuseth [Tue, 12 Feb 2013 11:27:59 +0000 (12:27 +0100)]
ITS#7455 Save freelist in single-page chunks
Hallvard Furuseth [Wed, 23 Jan 2013 14:47:35 +0000 (15:47 +0100)]
Freelist cleanup/streamlining
Drop unneeded definitions, redundant code.
Hallvard Furuseth [Thu, 7 Feb 2013 07:17:30 +0000 (08:17 +0100)]
ITS#7512 Fix MDB page leak when malloc error.
mdb_page_alloc(): Delay moving me_pgfirst,me_pglast
until malloc(MDB_oldpages to hold the IDs) succeeds.
Howard Chu [Mon, 4 Feb 2013 18:53:17 +0000 (10:53 -0800)]
Revert "ITS#7515 fix mdb_dbi_open/close"
This reverts commit
48dc782ea612f85e8356a50bfbafe22e5be121cf.
Howard Chu [Fri, 1 Feb 2013 14:19:23 +0000 (06:19 -0800)]
NULL key is allowed for MDB_CURRENT
Howard Chu [Fri, 1 Feb 2013 04:05:48 +0000 (20:05 -0800)]
ITS#7515 fix mdb_dbi_open/close
If a DBI handle is opened by a txn that aborts, the DBI handle
should no longer be valid.
Howard Chu [Tue, 29 Jan 2013 02:45:36 +0000 (18:45 -0800)]
ITS#7511 make sure cursor is marked valid after successful put
Howard Chu [Tue, 29 Jan 2013 01:08:24 +0000 (01:08 +0000)]
ITS#7505 fix prev commit, partial revert
Howard Chu [Sun, 27 Jan 2013 17:55:21 +0000 (17:55 +0000)]
ITS#7505 Fix mdb_update_key when key is too big
Howard Chu [Sat, 26 Jan 2013 18:55:35 +0000 (18:55 +0000)]
Fix MDB_CURRENT updating for dupsort items
Hallvard Furuseth [Wed, 16 Jan 2013 17:42:57 +0000 (18:42 +0100)]
Update MDB comments: Caveats, flags, etc.
Howard Chu [Tue, 15 Jan 2013 12:24:07 +0000 (04:24 -0800)]
ITS#7485 data sizes limited to 32 bits
That's all we have space for in a node record.
Howard Chu [Mon, 14 Jan 2013 21:46:25 +0000 (13:46 -0800)]
ITS#7485 more size checks
Howard Chu [Mon, 14 Jan 2013 21:12:15 +0000 (13:12 -0800)]
ITS#7485 check maxkeysize in mdb_cursor_put
Also MDB_MAXKEYSIZE is redefinable at compile time.
Howard Chu [Mon, 14 Jan 2013 20:57:54 +0000 (12:57 -0800)]
Howard Chu [Fri, 11 Jan 2013 19:45:25 +0000 (11:45 -0800)]
ITS#7491 check for filled dirty page list
Very large single transactions will fail. It's not just a problem when
nested transactions are used. We could make this dynamically sized,
but I'm not sure what the point is.
Howard Chu [Thu, 10 Jan 2013 10:58:55 +0000 (02:58 -0800)]
Happy New Year
Hallvard Furuseth [Wed, 9 Jan 2013 19:49:43 +0000 (20:49 +0100)]
Delete liblmdb/mfree. It has moved into mdb_stat.
Howard Chu [Wed, 9 Jan 2013 16:30:13 +0000 (08:30 -0800)]
ITS#7455 don't give up so soon
If we're operating on the freelist, see if our current pghead
can satisfy the request before giving up and using new pages.
Howard Chu [Wed, 19 Dec 2012 21:20:24 +0000 (13:20 -0800)]
Fix typo
Howard Chu [Wed, 19 Dec 2012 21:09:56 +0000 (13:09 -0800)]
Add initial mdb utility manpages
Howard Chu [Wed, 12 Dec 2012 06:21:52 +0000 (22:21 -0800)]
init_meta should write mapsize
Was leaving it 0 initially.
Howard Chu [Wed, 12 Dec 2012 00:01:25 +0000 (16:01 -0800)]
Update version date
Howard Chu [Tue, 11 Dec 2012 20:55:47 +0000 (12:55 -0800)]
Make sure init_meta is at beginning of file
Howard Chu [Tue, 11 Dec 2012 20:46:17 +0000 (12:46 -0800)]
More for prev commit
Use the latest meta page
Howard Chu [Tue, 11 Dec 2012 20:03:19 +0000 (12:03 -0800)]
Partially revert
65d40eb5d2c7c28df05e2c1d9b21d90e2a82e0b5
Allow both increasing and decreasing the environment size.
But don't allow decreasing below the currently occupied space.
Howard Chu [Tue, 11 Dec 2012 01:21:24 +0000 (17:21 -0800)]
FIx prev commit
Howard Chu [Mon, 10 Dec 2012 20:21:36 +0000 (12:21 -0800)]
Add error checks to txn_commit
These ops could fail if the map is full.
Howard Chu [Mon, 10 Dec 2012 20:16:50 +0000 (12:16 -0800)]
ITS#7455 simplify
Don't try to reclaim overflow pages while operating on
the freelist (for now). The circular dependencies are much like
the single-page case, but worse. Maybe look into this in the
future, but it's not absolutely necessary now.
Howard Chu [Fri, 7 Dec 2012 21:19:36 +0000 (13:19 -0800)]
ITS#7467 add verbose freelist info
Howard Chu [Fri, 7 Dec 2012 20:06:30 +0000 (12:06 -0800)]
ITS#7465 fail if both -a and -s specified
Howard Chu [Fri, 7 Dec 2012 01:39:49 +0000 (17:39 -0800)]
Yet more retries for overflow page scan
Howard Chu [Thu, 6 Dec 2012 16:25:43 +0000 (08:25 -0800)]
Freelist fixes
Keep list sorted if it grows during a write. Don't free pghead until
we're sure our writes are all finished.
Howard Chu [Tue, 4 Dec 2012 14:30:22 +0000 (06:30 -0800)]
Fix mdb_env_copy
Just write in 2^31 sized chunks
Howard Chu [Tue, 4 Dec 2012 00:53:24 +0000 (16:53 -0800)]
Rename mdb_open/close mdb_dbi_open/close
Avoid possible symbol clashes with MDB-Tools libmdb
Howard Chu [Mon, 3 Dec 2012 21:57:46 +0000 (13:57 -0800)]
Fix
5c1ee7f7ba9580d5ff29e74cbb02eee335c33d94
mdb_cursor_sibling() no longer pops cursor before returning.
Howard Chu [Mon, 3 Dec 2012 21:57:28 +0000 (13:57 -0800)]
Fix mdb_midl_shrink
Howard Chu [Fri, 30 Nov 2012 20:54:19 +0000 (12:54 -0800)]
More retries for overflow page scan
Howard Chu [Fri, 30 Nov 2012 20:45:34 +0000 (12:45 -0800)]
Rename libmdb to liblmdb
Howard Chu [Fri, 30 Nov 2012 20:39:53 +0000 (12:39 -0800)]
Add install target
Howard Chu [Fri, 30 Nov 2012 20:30:51 +0000 (12:30 -0800)]
Rename libmdb to liblmdb, lmdb.h
Avoid naming conflicts with other mdb* packages. Bump version to 0.9.5
Howard Chu [Thu, 29 Nov 2012 01:13:26 +0000 (17:13 -0800)]
ITS#7455 use freelist for overflow pages
Howard Chu [Wed, 28 Nov 2012 23:34:44 +0000 (15:34 -0800)]
Drop unneeded #include <time.h>
Howard Chu [Wed, 28 Nov 2012 23:34:19 +0000 (15:34 -0800)]
Doc layout tweaks
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
mdb_env_sync(,force=1): Override MDB_MAPASYNC.
With MDB_MAPASYNC, the API provided no way to ensure full sync.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
mdb_cursor_dbi(): Assume valid args.
It returned 0, which is the freelist DBI and is valid for a few uses.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
API change: mdb_drop() only accepts del=0 or 1.
This allows for other flag values in the future.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
API change: mdb_env_set_maxdbs(, *named* DBs).
Let the user specify the number of databases he will actually
use. Adding 2 for FREE_DBI and MAIN_DBI is an internal matter.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
ITS#7453 Fix reset of MDB_NOSYNC/MDB_NOMETASYNC.
mdb_env_open() with these flags did not open a synchronizing
file descriptor. Thus disabling them later did not work.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
Tweak MDB_FIXEDMAP handling.
Drop mmap()'s MAP_FIXED flag, so it returns another address instead
of unmapping existing overlapping pages. Verify the returned address.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
Renumber MDB_NOSUBDIR. Check mdb_env_open() flags.
MDB_NOSUBDIR was == MDB_REVERSEKEY. That affected the freelist:
Env flags are stored in mm_flags alias mm_dbs[FREE_DBI].md_flags.
It stays a persistent flag, in case mdb_env_open someday wants
to pick/verify which lockfile to use with the datafile.
Catch bad flags so they will no longer make it into the data file.
Howard Chu [Mon, 5 Nov 2012 13:06:06 +0000 (05:06 -0800)]
ITS#7432 fix typo in mdb_midl_sort
Wasn't pushing the optimal half of the array onto the stack,
thus used more stack than expected -> overrun.
Howard Chu [Thu, 25 Oct 2012 13:55:46 +0000 (06:55 -0700)]
Better fix for MDB_DEBUG tweak -- from hbf
Howard Chu [Thu, 25 Oct 2012 13:54:09 +0000 (06:54 -0700)]
Revert "MDB_DEBUG tweak for old compilers"
This reverts commit
ef2092eac9f35b77eae250801c96dfa515c6d4b2.
Howard Chu [Thu, 25 Oct 2012 12:28:55 +0000 (05:28 -0700)]
MDB_DEBUG tweak for old compilers
Howard Chu [Tue, 23 Oct 2012 11:48:04 +0000 (04:48 -0700)]
Fix flag checks in mdb_open
Howard Chu [Tue, 23 Oct 2012 11:32:55 +0000 (04:32 -0700)]
fix Windows mdb_env_copy(), WriteFile overflow
Howard Chu [Tue, 23 Oct 2012 00:03:15 +0000 (17:03 -0700)]
Make sure mdb_open flags are committed on main DB
If no data was modified in the txn, mdb_open flag changes were dropped.
Howard Chu [Mon, 22 Oct 2012 23:13:06 +0000 (16:13 -0700)]
Minor cleanup
Howard Chu [Mon, 22 Oct 2012 22:58:46 +0000 (15:58 -0700)]
Fix MDB_PREV
Could return 1 garbage record before actual data, if starting from EOF
Howard Chu [Tue, 16 Oct 2012 23:52:13 +0000 (16:52 -0700)]
Tweak output, add free page info
Howard Chu [Tue, 16 Oct 2012 23:22:21 +0000 (16:22 -0700)]
Add mdb_env_info() to retrieve interesting env params
Howard Chu [Tue, 16 Oct 2012 22:28:20 +0000 (15:28 -0700)]
Merge mdb_stata into mdb_stat
Howard Chu [Tue, 16 Oct 2012 21:53:25 +0000 (14:53 -0700)]
Make sure increases of mapsize are persisted
Hallvard Furuseth [Mon, 15 Oct 2012 15:04:48 +0000 (17:04 +0200)]
ITS#7377 Errorcheck Windows calls and thread keys.
Hallvard Furuseth [Mon, 15 Oct 2012 15:04:43 +0000 (17:04 +0200)]
Explicitly unlock the MDB lockfile on Windows.