Howard Chu [Wed, 12 Dec 2012 06:21:52 +0000 (22:21 -0800)]
init_meta should write mapsize
Was leaving it 0 initially.
Howard Chu [Wed, 12 Dec 2012 00:01:25 +0000 (16:01 -0800)]
Update version date
Howard Chu [Tue, 11 Dec 2012 20:55:47 +0000 (12:55 -0800)]
Make sure init_meta is at beginning of file
Howard Chu [Tue, 11 Dec 2012 20:46:17 +0000 (12:46 -0800)]
More for prev commit
Use the latest meta page
Howard Chu [Tue, 11 Dec 2012 20:03:19 +0000 (12:03 -0800)]
Partially revert
65d40eb5d2c7c28df05e2c1d9b21d90e2a82e0b5
Allow both increasing and decreasing the environment size.
But don't allow decreasing below the currently occupied space.
Howard Chu [Tue, 11 Dec 2012 01:21:24 +0000 (17:21 -0800)]
FIx prev commit
Howard Chu [Mon, 10 Dec 2012 20:21:36 +0000 (12:21 -0800)]
Add error checks to txn_commit
These ops could fail if the map is full.
Howard Chu [Mon, 10 Dec 2012 20:16:50 +0000 (12:16 -0800)]
ITS#7455 simplify
Don't try to reclaim overflow pages while operating on
the freelist (for now). The circular dependencies are much like
the single-page case, but worse. Maybe look into this in the
future, but it's not absolutely necessary now.
Howard Chu [Fri, 7 Dec 2012 21:19:36 +0000 (13:19 -0800)]
ITS#7467 add verbose freelist info
Howard Chu [Fri, 7 Dec 2012 20:06:30 +0000 (12:06 -0800)]
ITS#7465 fail if both -a and -s specified
Howard Chu [Fri, 7 Dec 2012 01:39:49 +0000 (17:39 -0800)]
Yet more retries for overflow page scan
Howard Chu [Thu, 6 Dec 2012 16:25:43 +0000 (08:25 -0800)]
Freelist fixes
Keep list sorted if it grows during a write. Don't free pghead until
we're sure our writes are all finished.
Howard Chu [Tue, 4 Dec 2012 14:30:22 +0000 (06:30 -0800)]
Fix mdb_env_copy
Just write in 2^31 sized chunks
Howard Chu [Tue, 4 Dec 2012 00:53:24 +0000 (16:53 -0800)]
Rename mdb_open/close mdb_dbi_open/close
Avoid possible symbol clashes with MDB-Tools libmdb
Howard Chu [Mon, 3 Dec 2012 21:57:46 +0000 (13:57 -0800)]
Fix
5c1ee7f7ba9580d5ff29e74cbb02eee335c33d94
mdb_cursor_sibling() no longer pops cursor before returning.
Howard Chu [Mon, 3 Dec 2012 21:57:28 +0000 (13:57 -0800)]
Fix mdb_midl_shrink
Howard Chu [Fri, 30 Nov 2012 20:54:19 +0000 (12:54 -0800)]
More retries for overflow page scan
Howard Chu [Fri, 30 Nov 2012 20:45:34 +0000 (12:45 -0800)]
Rename libmdb to liblmdb
Howard Chu [Fri, 30 Nov 2012 20:39:53 +0000 (12:39 -0800)]
Add install target
Howard Chu [Fri, 30 Nov 2012 20:30:51 +0000 (12:30 -0800)]
Rename libmdb to liblmdb, lmdb.h
Avoid naming conflicts with other mdb* packages. Bump version to 0.9.5
Howard Chu [Thu, 29 Nov 2012 01:13:26 +0000 (17:13 -0800)]
ITS#7455 use freelist for overflow pages
Howard Chu [Wed, 28 Nov 2012 23:34:44 +0000 (15:34 -0800)]
Drop unneeded #include <time.h>
Howard Chu [Wed, 28 Nov 2012 23:34:19 +0000 (15:34 -0800)]
Doc layout tweaks
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
mdb_env_sync(,force=1): Override MDB_MAPASYNC.
With MDB_MAPASYNC, the API provided no way to ensure full sync.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
mdb_cursor_dbi(): Assume valid args.
It returned 0, which is the freelist DBI and is valid for a few uses.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
API change: mdb_drop() only accepts del=0 or 1.
This allows for other flag values in the future.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
API change: mdb_env_set_maxdbs(, *named* DBs).
Let the user specify the number of databases he will actually
use. Adding 2 for FREE_DBI and MAIN_DBI is an internal matter.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
ITS#7453 Fix reset of MDB_NOSYNC/MDB_NOMETASYNC.
mdb_env_open() with these flags did not open a synchronizing
file descriptor. Thus disabling them later did not work.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
Tweak MDB_FIXEDMAP handling.
Drop mmap()'s MAP_FIXED flag, so it returns another address instead
of unmapping existing overlapping pages. Verify the returned address.
Hallvard Furuseth [Tue, 27 Nov 2012 22:56:47 +0000 (23:56 +0100)]
Renumber MDB_NOSUBDIR. Check mdb_env_open() flags.
MDB_NOSUBDIR was == MDB_REVERSEKEY. That affected the freelist:
Env flags are stored in mm_flags alias mm_dbs[FREE_DBI].md_flags.
It stays a persistent flag, in case mdb_env_open someday wants
to pick/verify which lockfile to use with the datafile.
Catch bad flags so they will no longer make it into the data file.
Howard Chu [Mon, 5 Nov 2012 13:06:06 +0000 (05:06 -0800)]
ITS#7432 fix typo in mdb_midl_sort
Wasn't pushing the optimal half of the array onto the stack,
thus used more stack than expected -> overrun.
Howard Chu [Thu, 25 Oct 2012 13:55:46 +0000 (06:55 -0700)]
Better fix for MDB_DEBUG tweak -- from hbf
Howard Chu [Thu, 25 Oct 2012 13:54:09 +0000 (06:54 -0700)]
Revert "MDB_DEBUG tweak for old compilers"
This reverts commit
ef2092eac9f35b77eae250801c96dfa515c6d4b2.
Howard Chu [Thu, 25 Oct 2012 12:28:55 +0000 (05:28 -0700)]
MDB_DEBUG tweak for old compilers
Howard Chu [Tue, 23 Oct 2012 11:48:04 +0000 (04:48 -0700)]
Fix flag checks in mdb_open
Howard Chu [Tue, 23 Oct 2012 11:32:55 +0000 (04:32 -0700)]
fix Windows mdb_env_copy(), WriteFile overflow
Howard Chu [Tue, 23 Oct 2012 00:03:15 +0000 (17:03 -0700)]
Make sure mdb_open flags are committed on main DB
If no data was modified in the txn, mdb_open flag changes were dropped.
Howard Chu [Mon, 22 Oct 2012 23:13:06 +0000 (16:13 -0700)]
Minor cleanup
Howard Chu [Mon, 22 Oct 2012 22:58:46 +0000 (15:58 -0700)]
Fix MDB_PREV
Could return 1 garbage record before actual data, if starting from EOF
Howard Chu [Tue, 16 Oct 2012 23:52:13 +0000 (16:52 -0700)]
Tweak output, add free page info
Howard Chu [Tue, 16 Oct 2012 23:22:21 +0000 (16:22 -0700)]
Add mdb_env_info() to retrieve interesting env params
Howard Chu [Tue, 16 Oct 2012 22:28:20 +0000 (15:28 -0700)]
Merge mdb_stata into mdb_stat
Howard Chu [Tue, 16 Oct 2012 21:53:25 +0000 (14:53 -0700)]
Make sure increases of mapsize are persisted
Hallvard Furuseth [Mon, 15 Oct 2012 15:04:48 +0000 (17:04 +0200)]
ITS#7377 Errorcheck Windows calls and thread keys.
Hallvard Furuseth [Mon, 15 Oct 2012 15:04:43 +0000 (17:04 +0200)]
Explicitly unlock the MDB lockfile on Windows.
Hallvard Furuseth [Mon, 15 Oct 2012 15:04:11 +0000 (17:04 +0200)]
mdb_env_open(): Keep mdb_env_set_flags() flags.
Hallvard Furuseth [Mon, 15 Oct 2012 15:03:46 +0000 (17:03 +0200)]
More for mdb_env_copy().
Fix #define _GNU_SOURCE = 1 to match any command line -D_GNU_SOURCE.
.gitignore += mdb_copy.
Hallvard Furuseth [Mon, 15 Oct 2012 15:03:33 +0000 (17:03 +0200)]
Fix last commit.
Add flag MDB_ENV_ACTIVE instead of using the state of me_lfd.
Require MDB_RDONLY for MDB_ROFS. Swap reader/writer mutex lock.
Howard Chu [Sat, 13 Oct 2012 11:05:00 +0000 (04:05 -0700)]
Support read access on read-only media
Howard Chu [Thu, 11 Oct 2012 23:47:56 +0000 (16:47 -0700)]
Silence warning in prev commit
Howard Chu [Thu, 11 Oct 2012 19:21:40 +0000 (12:21 -0700)]
Add mdb_copy for backing up a DB environment
Hallvard Furuseth [Wed, 3 Oct 2012 19:13:21 +0000 (21:13 +0200)]
Tweak MDB restrictions
Hallvard Furuseth [Wed, 3 Oct 2012 16:08:56 +0000 (18:08 +0200)]
Fix mdb_open() off-by-one error in maxdbs check
Hallvard Furuseth [Wed, 3 Oct 2012 16:06:29 +0000 (18:06 +0200)]
ITS#7377 Wrap fcntl F_SETLK in EINTR loop too.
This can happen even on local filesystems.
Hallvard Furuseth [Wed, 3 Oct 2012 16:06:29 +0000 (18:06 +0200)]
ITS#7377 Catch ftruncate() error
Hallvard Furuseth [Wed, 3 Oct 2012 16:04:07 +0000 (18:04 +0200)]
ITS#7377 Document caveats/troubeshooting.
Howard Chu [Sun, 30 Sep 2012 17:01:57 +0000 (10:01 -0700)]
Fix prev commit
It only worked if the freelist was already non-empty
Howard Chu [Sun, 30 Sep 2012 14:49:30 +0000 (07:49 -0700)]
Fix mdb_drop
if it was the only operation in a txn, it would be ignored.
Howard Chu [Thu, 27 Sep 2012 12:01:53 +0000 (05:01 -0700)]
ITS#7385, check further for bad splits
More for
48ef27b6f5c804eca6a9d27f8dd2b4ded376f8af
Our definition of "large" data item needs to be smaller to
handle more cases where a bad split is possible
Howard Chu [Wed, 19 Sep 2012 13:15:09 +0000 (06:15 -0700)]
Add MDB_GET_CURRENT
return whatever the cursor is currently pointing at
Howard Chu [Tue, 18 Sep 2012 19:36:54 +0000 (12:36 -0700)]
ITS#7394 ignore WRITEMAP if RDONLY
Howard Chu [Mon, 17 Sep 2012 14:02:41 +0000 (07:02 -0700)]
More for
48ef27b6f5c804eca6a9d27f8dd2b4ded376f8af
page_split with newindex > split_indx
Howard Chu [Mon, 17 Sep 2012 14:01:28 +0000 (07:01 -0700)]
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:15 +0000 (15:42 +0200)]
ITS#7377 Catch MDB user errors.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:15 +0000 (15:42 +0200)]
ITS#7377 Catch MDB setup errors and clean up.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:15 +0000 (15:42 +0200)]
ITS#7364 Limit mdb lock upgrade before sem_unlink.
Do not try shared lock when closing or after error. Track file lock
state to decide. Change meaning of *excl to reflect file lock state.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:15 +0000 (15:42 +0200)]
ITS#7364 Always sem_unlink() in mdb_env_open().
Drop the sem_unlink() error checks, which could prevent the 2nd
unlink. Instead use O_EXCL in sem_open().
This makes "open+close the database" the API for trying to clean
away the old semaphores, if they were left behind by a previous run.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
ITS#7377 Wrap sem_wait & file locks in EINTR loops
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
Remove mdb data races. Use (txnid_t)-1 as "no ID".
Avoid race between numreaders++ and reading numreaders at cleanup. Make
the un-mutexed reset of reader table entry, atomic: Reset mr_pid only.
Instead check mr_pid != 0 in mdb_page_alloc()'s scan for readers.
(txnid_t)-1 as "no ID"-mark avoids a check for mr_txnid != 0.
The scan can stop when seeing an old reader.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
Save pid in MDB_env instead of repeating getpid().
An open MDB environment does not survive or catch fork(),
so repeating getpid() was pointless.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
MDB_WRITEMAP needs no DSYNC descriptor (me_mfd)
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
ITS#7363 Preprocessor namespace cleanup.
Rename USE_POSIX_SEM to MDB_USE_POSIX_SEM.
Separate MDB_FDATASYNC from MDB_USE_POSIX_SEM.
Howard Chu [Mon, 17 Sep 2012 13:35:03 +0000 (06:35 -0700)]
Howard Chu [Mon, 17 Sep 2012 11:41:13 +0000 (04:41 -0700)]
Add mdb_cursor_renew()
Allow cursors on read-only txns to be reused with later txns.
Howard Chu [Mon, 17 Sep 2012 10:48:54 +0000 (03:48 -0700)]
Shared lib should depend on pthread
Howard Chu [Mon, 17 Sep 2012 09:17:25 +0000 (02:17 -0700)]
Add MDB_SET_KEY cursor op
Overwrites the passed in key with the DB's key
Howard Chu [Fri, 14 Sep 2012 13:05:48 +0000 (06:05 -0700)]
ITS#7383 More explicit error codes
Howard Chu [Wed, 12 Sep 2012 17:43:27 +0000 (10:43 -0700)]
ITS#7385 fix mdb_page_split (again)
More for
d793594173bb5367e466134d5fe3366feeb672f6 ITS#7229
Howard Chu [Fri, 7 Sep 2012 20:40:07 +0000 (13:40 -0700)]
Update comments, bump version
Note read-write vs read-only mmap tradeoffs.
Howard Chu [Thu, 6 Sep 2012 18:17:25 +0000 (11:17 -0700)]
msync should default to fully synchronous
Howard Chu [Wed, 5 Sep 2012 15:40:17 +0000 (08:40 -0700)]
Add msync support for WRITEMAP option
Howard Chu [Tue, 4 Sep 2012 17:28:37 +0000 (10:28 -0700)]
Add MDB_WRITEMAP to use a writable mmap
Howard Chu [Tue, 4 Sep 2012 13:02:26 +0000 (06:02 -0700)]
Hallvard Furuseth [Wed, 29 Aug 2012 04:43:35 +0000 (06:43 +0200)]
libmdb: Fix P_INVALID when pgno_t != unsigned long
Hallvard Furuseth [Wed, 29 Aug 2012 04:42:52 +0000 (06:42 +0200)]
libmdb: Check sem_open() against SEM_FAILED.
Howard Chu [Mon, 27 Aug 2012 21:45:16 +0000 (14:45 -0700)]
In mdb_audit, change assert to fprintf
Howard Chu [Sun, 26 Aug 2012 18:05:45 +0000 (11:05 -0700)]
Ondrej Kuznik [Fri, 24 Aug 2012 12:28:36 +0000 (05:28 -0700)]
ITS#7369 mdb_stat: cleanup in case something went wrong.
Howard Chu [Fri, 24 Aug 2012 12:23:55 +0000 (05:23 -0700)]
ITS#7369 abort txn before exiting
Howard Chu [Thu, 23 Aug 2012 06:01:19 +0000 (23:01 -0700)]
ITS#7364 also close the mutexes as needed
Otherwise a process that opens and closes an env more than once
will leak mutex handles. (WIN32/BSD)
Chris Mikkelson [Wed, 22 Aug 2012 21:51:35 +0000 (16:51 -0500)]
ITS#7364 Remove POSIX semaphores when the last user closes the mdb environment.
Chris Mikkelson [Wed, 22 Aug 2012 18:22:19 +0000 (13:22 -0500)]
ITS#7363 Use posix semaphores on apple and bsd systems.
Howard Chu [Thu, 16 Aug 2012 17:25:40 +0000 (10:25 -0700)]
Loop thru all DBs and show their stats
This is quick and dirty, should be integrated into mdb_stat.
Howard Chu [Thu, 2 Aug 2012 22:56:13 +0000 (15:56 -0700)]
Minor cleanup
Howard Chu [Tue, 31 Jul 2012 23:03:48 +0000 (16:03 -0700)]
Write optimization: avoid copying unused portions of pages.
Howard Chu [Thu, 26 Jul 2012 15:42:41 +0000 (08:42 -0700)]
Fix txn_commit error check
Howard Chu [Thu, 26 Jul 2012 14:22:52 +0000 (07:22 -0700)]
Fix
205bb17793e27be67098ccc80ee919165d7cb993
Ignore current cursor pos if it's out of bounds.
Howard Chu [Thu, 26 Jul 2012 13:25:31 +0000 (06:25 -0700)]
Update sub-DBs before updating free list
Howard Chu [Wed, 25 Jul 2012 19:31:08 +0000 (12:31 -0700)]
Append mode should also detect keys that are too small
Howard Chu [Tue, 24 Jul 2012 11:21:40 +0000 (04:21 -0700)]
Fix root split check in mdb_page_split