Hallvard Furuseth [Mon, 15 Oct 2012 15:04:11 +0000 (17:04 +0200)]
mdb_env_open(): Keep mdb_env_set_flags() flags.
Hallvard Furuseth [Mon, 15 Oct 2012 15:03:46 +0000 (17:03 +0200)]
More for mdb_env_copy().
Fix #define _GNU_SOURCE = 1 to match any command line -D_GNU_SOURCE.
.gitignore += mdb_copy.
Hallvard Furuseth [Mon, 15 Oct 2012 15:03:33 +0000 (17:03 +0200)]
Fix last commit.
Add flag MDB_ENV_ACTIVE instead of using the state of me_lfd.
Require MDB_RDONLY for MDB_ROFS. Swap reader/writer mutex lock.
Howard Chu [Sat, 13 Oct 2012 11:05:00 +0000 (04:05 -0700)]
Support read access on read-only media
Howard Chu [Thu, 11 Oct 2012 23:47:56 +0000 (16:47 -0700)]
Silence warning in prev commit
Howard Chu [Thu, 11 Oct 2012 19:21:40 +0000 (12:21 -0700)]
Add mdb_copy for backing up a DB environment
Hallvard Furuseth [Wed, 3 Oct 2012 19:13:21 +0000 (21:13 +0200)]
Tweak MDB restrictions
Hallvard Furuseth [Wed, 3 Oct 2012 16:08:56 +0000 (18:08 +0200)]
Fix mdb_open() off-by-one error in maxdbs check
Hallvard Furuseth [Wed, 3 Oct 2012 16:06:29 +0000 (18:06 +0200)]
ITS#7377 Wrap fcntl F_SETLK in EINTR loop too.
This can happen even on local filesystems.
Hallvard Furuseth [Wed, 3 Oct 2012 16:06:29 +0000 (18:06 +0200)]
ITS#7377 Catch ftruncate() error
Hallvard Furuseth [Wed, 3 Oct 2012 16:04:07 +0000 (18:04 +0200)]
ITS#7377 Document caveats/troubeshooting.
Howard Chu [Sun, 30 Sep 2012 17:01:57 +0000 (10:01 -0700)]
Fix prev commit
It only worked if the freelist was already non-empty
Howard Chu [Sun, 30 Sep 2012 14:49:30 +0000 (07:49 -0700)]
Fix mdb_drop
if it was the only operation in a txn, it would be ignored.
Howard Chu [Thu, 27 Sep 2012 12:01:53 +0000 (05:01 -0700)]
ITS#7385, check further for bad splits
More for
48ef27b6f5c804eca6a9d27f8dd2b4ded376f8af
Our definition of "large" data item needs to be smaller to
handle more cases where a bad split is possible
Howard Chu [Wed, 19 Sep 2012 13:15:09 +0000 (06:15 -0700)]
Add MDB_GET_CURRENT
return whatever the cursor is currently pointing at
Howard Chu [Tue, 18 Sep 2012 19:36:54 +0000 (12:36 -0700)]
ITS#7394 ignore WRITEMAP if RDONLY
Howard Chu [Mon, 17 Sep 2012 14:02:41 +0000 (07:02 -0700)]
More for
48ef27b6f5c804eca6a9d27f8dd2b4ded376f8af
page_split with newindex > split_indx
Howard Chu [Mon, 17 Sep 2012 14:01:28 +0000 (07:01 -0700)]
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:15 +0000 (15:42 +0200)]
ITS#7377 Catch MDB user errors.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:15 +0000 (15:42 +0200)]
ITS#7377 Catch MDB setup errors and clean up.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:15 +0000 (15:42 +0200)]
ITS#7364 Limit mdb lock upgrade before sem_unlink.
Do not try shared lock when closing or after error. Track file lock
state to decide. Change meaning of *excl to reflect file lock state.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:15 +0000 (15:42 +0200)]
ITS#7364 Always sem_unlink() in mdb_env_open().
Drop the sem_unlink() error checks, which could prevent the 2nd
unlink. Instead use O_EXCL in sem_open().
This makes "open+close the database" the API for trying to clean
away the old semaphores, if they were left behind by a previous run.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
ITS#7377 Wrap sem_wait & file locks in EINTR loops
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
Remove mdb data races. Use (txnid_t)-1 as "no ID".
Avoid race between numreaders++ and reading numreaders at cleanup. Make
the un-mutexed reset of reader table entry, atomic: Reset mr_pid only.
Instead check mr_pid != 0 in mdb_page_alloc()'s scan for readers.
(txnid_t)-1 as "no ID"-mark avoids a check for mr_txnid != 0.
The scan can stop when seeing an old reader.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
Save pid in MDB_env instead of repeating getpid().
An open MDB environment does not survive or catch fork(),
so repeating getpid() was pointless.
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
MDB_WRITEMAP needs no DSYNC descriptor (me_mfd)
Hallvard Furuseth [Mon, 17 Sep 2012 13:42:14 +0000 (15:42 +0200)]
ITS#7363 Preprocessor namespace cleanup.
Rename USE_POSIX_SEM to MDB_USE_POSIX_SEM.
Separate MDB_FDATASYNC from MDB_USE_POSIX_SEM.
Howard Chu [Mon, 17 Sep 2012 13:35:03 +0000 (06:35 -0700)]
Howard Chu [Mon, 17 Sep 2012 11:41:13 +0000 (04:41 -0700)]
Add mdb_cursor_renew()
Allow cursors on read-only txns to be reused with later txns.
Howard Chu [Mon, 17 Sep 2012 10:48:54 +0000 (03:48 -0700)]
Shared lib should depend on pthread
Howard Chu [Mon, 17 Sep 2012 09:17:25 +0000 (02:17 -0700)]
Add MDB_SET_KEY cursor op
Overwrites the passed in key with the DB's key
Howard Chu [Fri, 14 Sep 2012 13:05:48 +0000 (06:05 -0700)]
ITS#7383 More explicit error codes
Howard Chu [Wed, 12 Sep 2012 17:43:27 +0000 (10:43 -0700)]
ITS#7385 fix mdb_page_split (again)
More for
d793594173bb5367e466134d5fe3366feeb672f6 ITS#7229
Howard Chu [Fri, 7 Sep 2012 20:40:07 +0000 (13:40 -0700)]
Update comments, bump version
Note read-write vs read-only mmap tradeoffs.
Howard Chu [Thu, 6 Sep 2012 18:17:25 +0000 (11:17 -0700)]
msync should default to fully synchronous
Howard Chu [Wed, 5 Sep 2012 15:40:17 +0000 (08:40 -0700)]
Add msync support for WRITEMAP option
Howard Chu [Tue, 4 Sep 2012 17:28:37 +0000 (10:28 -0700)]
Add MDB_WRITEMAP to use a writable mmap
Howard Chu [Tue, 4 Sep 2012 13:02:26 +0000 (06:02 -0700)]
Hallvard Furuseth [Wed, 29 Aug 2012 04:43:35 +0000 (06:43 +0200)]
libmdb: Fix P_INVALID when pgno_t != unsigned long
Hallvard Furuseth [Wed, 29 Aug 2012 04:42:52 +0000 (06:42 +0200)]
libmdb: Check sem_open() against SEM_FAILED.
Howard Chu [Mon, 27 Aug 2012 21:45:16 +0000 (14:45 -0700)]
In mdb_audit, change assert to fprintf
Howard Chu [Sun, 26 Aug 2012 18:05:45 +0000 (11:05 -0700)]
Ondrej Kuznik [Fri, 24 Aug 2012 12:28:36 +0000 (05:28 -0700)]
ITS#7369 mdb_stat: cleanup in case something went wrong.
Howard Chu [Fri, 24 Aug 2012 12:23:55 +0000 (05:23 -0700)]
ITS#7369 abort txn before exiting
Howard Chu [Thu, 23 Aug 2012 06:01:19 +0000 (23:01 -0700)]
ITS#7364 also close the mutexes as needed
Otherwise a process that opens and closes an env more than once
will leak mutex handles. (WIN32/BSD)
Chris Mikkelson [Wed, 22 Aug 2012 21:51:35 +0000 (16:51 -0500)]
ITS#7364 Remove POSIX semaphores when the last user closes the mdb environment.
Chris Mikkelson [Wed, 22 Aug 2012 18:22:19 +0000 (13:22 -0500)]
ITS#7363 Use posix semaphores on apple and bsd systems.
Howard Chu [Thu, 16 Aug 2012 17:25:40 +0000 (10:25 -0700)]
Loop thru all DBs and show their stats
This is quick and dirty, should be integrated into mdb_stat.
Howard Chu [Thu, 2 Aug 2012 22:56:13 +0000 (15:56 -0700)]
Minor cleanup
Howard Chu [Tue, 31 Jul 2012 23:03:48 +0000 (16:03 -0700)]
Write optimization: avoid copying unused portions of pages.
Howard Chu [Thu, 26 Jul 2012 15:42:41 +0000 (08:42 -0700)]
Fix txn_commit error check
Howard Chu [Thu, 26 Jul 2012 14:22:52 +0000 (07:22 -0700)]
Fix
205bb17793e27be67098ccc80ee919165d7cb993
Ignore current cursor pos if it's out of bounds.
Howard Chu [Thu, 26 Jul 2012 13:25:31 +0000 (06:25 -0700)]
Update sub-DBs before updating free list
Howard Chu [Wed, 25 Jul 2012 19:31:08 +0000 (12:31 -0700)]
Append mode should also detect keys that are too small
Howard Chu [Tue, 24 Jul 2012 11:21:40 +0000 (04:21 -0700)]
Fix root split check in mdb_page_split
Howard Chu [Mon, 23 Jul 2012 11:23:49 +0000 (04:23 -0700)]
Skip unneeded calls to mdb_page_get in mdb_page_search
Howard Chu [Mon, 23 Jul 2012 00:42:36 +0000 (17:42 -0700)]
Leave extra space when creating a fake page
Minimizes the number of memcpy's if additional dups are stored.
Howard Chu [Mon, 23 Jul 2012 00:41:58 +0000 (17:41 -0700)]
Append mode still needs to detect duplicates
Howard Chu [Sat, 21 Jul 2012 13:10:02 +0000 (06:10 -0700)]
Append tweaks, page_split fixes
Append mode now does no key comparisons, input must be in sorted order.
page_split was not updating cursor parents correctly.
Howard Chu [Tue, 17 Jul 2012 11:02:48 +0000 (04:02 -0700)]
Make sure cursor's DB is init'd if STALE.
Howard Chu [Fri, 13 Jul 2012 00:04:05 +0000 (17:04 -0700)]
Don't use env-private copy of DB root nodes.
Just lookup the DB roots as needed. When many DBs are in use,
most of the copies won't be referenced in a given txn, and
there's a bad race condition in the copy routine.
Howard Chu [Fri, 13 Jul 2012 18:57:11 +0000 (11:57 -0700)]
Fix darwin sem_open() names
Must begin with '/'
Howard Chu [Fri, 13 Jul 2012 18:56:29 +0000 (11:56 -0700)]
Howard Chu [Thu, 12 Jul 2012 23:50:27 +0000 (16:50 -0700)]
Windows thread callback support
Howard Chu [Mon, 9 Jul 2012 01:24:11 +0000 (18:24 -0700)]
Drop me_toggle, unused
Howard Chu [Mon, 9 Jul 2012 01:18:12 +0000 (18:18 -0700)]
Revert "libmdb: Do not use the MDB_txbody.mtb_txnid field."
This reverts commit
4a0b30499caab8628efcedb8be5643fead7cd17e.
Howard Chu [Sat, 7 Jul 2012 00:38:03 +0000 (17:38 -0700)]
Fix: avoid direct reference to meta pages
Relevant info should be copied during txn_begin only.
Howard Chu [Fri, 6 Jul 2012 01:11:18 +0000 (18:11 -0700)]
Add MDB_NOMETASYNC env option.
Just a trial. This may not make sense if we decide to split the
meta pages into their own separate file, to allow meta traffic to
reside on a separate spindle.
Howard Chu [Fri, 6 Jul 2012 00:43:29 +0000 (17:43 -0700)]
Tweak
b802bcf7040110e1f0768d9d9ffa88dabd4bd5e7
Clarify prev commit, fix Doxygen comments broken by earlier changes
Howard Chu [Thu, 5 Jul 2012 23:40:46 +0000 (16:40 -0700)]
Partial revert of
13c663f666ac28d7a72cbe644d393fc8d2dd9881
Don't re-use free pages so soon; that leaves us vulnerable to
DB corruption if data syncs successfully but meta doesn't.
Howard Chu [Sat, 2 Jun 2012 12:26:31 +0000 (05:26 -0700)]
Tweak prev commit, use first unused slot
Howard Chu [Sat, 2 Jun 2012 12:24:47 +0000 (05:24 -0700)]
Fix mdb_open, re-use closed dbi slots
Howard Chu [Wed, 9 May 2012 18:24:25 +0000 (11:24 -0700)]
ITS#7266 fix rebalance for LEAF2 pages
Howard Chu [Wed, 9 May 2012 05:17:17 +0000 (22:17 -0700)]
More freelist tweaks, avoid referencing freed page
Howard Chu [Sun, 6 May 2012 02:41:45 +0000 (19:41 -0700)]
Further clarification of mdb_open()
Howard Chu [Wed, 18 Apr 2012 22:59:54 +0000 (15:59 -0700)]
ITS#7251 fix Apple/Windows shared mutex naming
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:14 +0000 (07:34 +0200)]
libmdb: midl.h namespace cleanup.
Prefix ID, NOID, ID2, IDL, ID2L with 'MDB_'.
Also drop bdb from file description.
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:14 +0000 (07:34 +0200)]
libmdb: Simplify pick_meta() from read_meta().
Put one call in #if MDB_DEBUG. Move DPRINTF(Using meta page #) there.
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:14 +0000 (07:34 +0200)]
libmdb: Simplify mdb_default_cmp. No real change.
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:13 +0000 (07:34 +0200)]
libmdb: signed meets unsigned warning cleanup
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:13 +0000 (07:34 +0200)]
libmdb: MDB_FDATASYNC code cleanup.
Define MDB_FDATASYNC() instead of redefining fdatasync().
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:13 +0000 (07:34 +0200)]
libmdb: Add C++ 'extern "C" {}' to *.h
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:13 +0000 (07:34 +0200)]
libmdb: Don't open datafile twice when not needed.
If the database is opened with MDB_RDONLY or MDB_NOSYNC,
instead use the same file descriptor for me_mfd and me_fd.
Also factor out Windows/Unix error handling after open.
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:13 +0000 (07:34 +0200)]
libmdb: Set close-on-exec flag on lockfile
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:13 +0000 (07:34 +0200)]
libmdb: Do not use the MDB_txbody.mtb_txnid field.
Get txn->mt_txnid from the meta page instead. Otherwise it
and txn->mt_toggle could become inconsistent, since read-only
txns fetched both from MDB_txbody without mutex proteciton.
Hallvard Furuseth [Thu, 12 Apr 2012 05:34:13 +0000 (07:34 +0200)]
Fix MDB_DEBUG in last ITS#7299 commit.
Make DPRINTF usable in if-else. Silence signed vs unsigned warning.
Howard Chu [Tue, 10 Apr 2012 21:05:05 +0000 (14:05 -0700)]
ITS#7229 more mdb_page_split tweaks
Also add mdb_debug/mdb_debug_start to toggle debug output at runtime
Howard Chu [Tue, 10 Apr 2012 02:24:47 +0000 (19:24 -0700)]
ITS#7210 fix check for freelist changes
Was being fooled before because the newly malloc'd block had
the same address as the previously freed block.
Howard Chu [Mon, 9 Apr 2012 12:01:37 +0000 (05:01 -0700)]
ITS#7238 doc clarification
Howard Chu [Mon, 9 Apr 2012 11:45:32 +0000 (04:45 -0700)]
ITS#7229 additional tweaks
Should probably compare nsize to pg_size/4 instead...
Howard Chu [Mon, 9 Apr 2012 11:44:08 +0000 (04:44 -0700)]
ITS#7230 check and strip 0 length IDLs from freelist
Howard Chu [Sat, 7 Apr 2012 08:27:38 +0000 (01:27 -0700)]
ITS#7229 more mdb_page_split fixes
Howard Chu [Wed, 4 Apr 2012 12:22:26 +0000 (05:22 -0700)]
ITS#7219 fixup keysize in previous commit
Howard Chu [Tue, 3 Apr 2012 19:24:27 +0000 (12:24 -0700)]
ITS#7219 fix for odd-aligned keys in mdb_update_key()
Howard Chu [Fri, 30 Mar 2012 13:36:02 +0000 (06:36 -0700)]
ITS#7210 fix leak of overflow pages in freelist
Howard Chu [Wed, 28 Mar 2012 17:44:18 +0000 (10:44 -0700)]
Partial revert of prev commit
Sub-DBs were correct before
Howard Chu [Wed, 28 Mar 2012 17:34:35 +0000 (10:34 -0700)]
Fix entry counts in cursor_put
when replacing an existing item
Howard Chu [Wed, 28 Mar 2012 16:33:06 +0000 (09:33 -0700)]
Fix uninit'd xcursor index
Howard Chu [Wed, 28 Mar 2012 16:20:18 +0000 (09:20 -0700)]
Add mfree utility to show the freelist
Howard Chu [Tue, 27 Mar 2012 17:42:22 +0000 (10:42 -0700)]
ITS#7210 additional freelist fixes
Also allow read access to freelist in mdb_cursor_open