Howard Chu [Thu, 29 Aug 2013 02:17:25 +0000 (19:17 -0700)]
Bump version to 0.9.8
Howard Chu [Thu, 29 Aug 2013 02:12:59 +0000 (19:12 -0700)]
Allow mdb_env_set_mapsize() on an open environment
The caller is responsible for making sure no transactions are
active in this process before resizing. This is slightly lighter
weight than doing a full env_close/env_open cycle.
Howard Chu [Tue, 27 Aug 2013 13:28:30 +0000 (06:28 -0700)]
ITS#7672 fix mdb_dbi_flags
Use a txn instead of env argument. Only return persistent flags.
Hallvard Furuseth [Sun, 25 Aug 2013 21:59:58 +0000 (23:59 +0200)]
mdb_page_unspill(): Fix dirty_room in nested txn
Howard Chu [Sat, 24 Aug 2013 19:37:45 +0000 (20:37 +0100)]
Another take on spill_pgs and deletes
Howard Chu [Sat, 24 Aug 2013 18:54:05 +0000 (11:54 -0700)]
Cleanup prev commit
Howard Chu [Sat, 24 Aug 2013 18:48:37 +0000 (19:48 +0100)]
Speedup deletion from spill_pgs
Howard Chu [Fri, 23 Aug 2013 14:36:07 +0000 (07:36 -0700)]
ITS#7671 always return the key for a dup val
Howard Chu [Fri, 23 Aug 2013 04:43:29 +0000 (21:43 -0700)]
ITS#7670 Tweak cursor_del0
Always leave cursor pointing at "next" node, if any.
Find next sibling if we're already at end of current page.
Hallvard Furuseth [Fri, 23 Aug 2013 05:39:04 +0000 (07:39 +0200)]
mdb_page_spill(): Fix nested txn mt_dirty_room
Hallvard Furuseth [Fri, 23 Aug 2013 05:37:17 +0000 (07:37 +0200)]
Simplify last commits
Howard Chu [Fri, 23 Aug 2013 04:29:53 +0000 (21:29 -0700)]
Tweak prev commit
If somehow "need" is larger than the list size, don't
try to fake out the dirty list.
Howard Chu [Fri, 23 Aug 2013 01:51:48 +0000 (18:51 -0700)]
Less aggressive page_spill
spilling all possible pages is a waste because the majority will be
needed again. Just do 1/8th of the dirty list instead.
Howard Chu [Wed, 21 Aug 2013 00:37:53 +0000 (17:37 -0700)]
Update caveats now that mdb_reader_check exists
Howard Chu [Wed, 14 Aug 2013 16:57:25 +0000 (09:57 -0700)]
Silence warnings
Hallvard Furuseth [Wed, 14 Aug 2013 13:21:59 +0000 (15:21 +0200)]
Tweak ITS#7615 fix: Obey strict-aliasing rules.
Do not access a long long as an array of longs.
Howard Chu [Tue, 13 Aug 2013 20:12:47 +0000 (13:12 -0700)]
ITS#7664 better fix
For RDONLY, don't get lockfile until we know datafile exists.
Also, don't try to create a new datafile for me_mfd if someone
deleted it after we got me_fd.
Howard Chu [Tue, 13 Aug 2013 20:05:32 +0000 (13:05 -0700)]
Revert "ITS#7664 delete lockfile if there is no environment"
This reverts commit
8725838409512e999d78ea7b60773ce2f94d588f.
Howard Chu [Tue, 13 Aug 2013 18:34:32 +0000 (11:34 -0700)]
ITS#7664 delete lockfile if there is no environment
Howard Chu [Mon, 12 Aug 2013 19:43:42 +0000 (12:43 -0700)]
Fix uninit'd mem ref in mdb_page_split
Don't compare cursors beyond their depth. (detected by valgrind)
Howard Chu [Mon, 12 Aug 2013 00:25:00 +0000 (17:25 -0700)]
Tweak NODUPDATA flag check
Should allow for the possibility of future flag combos.
Howard Chu [Mon, 12 Aug 2013 00:15:03 +0000 (17:15 -0700)]
Fix obscure MDB_MULTIPLE bug
If a key has a single existing value, and then a put (MDB_MULTIPLE)
is done where the first of the multiple values matches the existing
value, the put would return SUCCESS without writing any of the
values. Fixed to loop to the next value as intended.
Howard Chu [Sat, 10 Aug 2013 21:55:51 +0000 (14:55 -0700)]
Point to mdb.c for docs on macros in Makefile
Hallvard Furuseth [Sat, 10 Aug 2013 11:50:10 +0000 (13:50 +0200)]
Avoid compiling debug args with C90 compilers.
Use double DPRINTF() parenthesis, so varargs macros are not needed.
Parenthesize MDB_DEBUG, in case the user defines it as foo || bar.
Hallvard Furuseth [Sat, 10 Aug 2013 11:23:30 +0000 (13:23 +0200)]
Refuse ops on bad txns
Hallvard Furuseth [Sat, 10 Aug 2013 11:23:30 +0000 (13:23 +0200)]
Replace EINVALs with generalized MDB_INCOMPATIBLE.
Also check xcursor instead of DUPSORT, it's slightly briefer.
John Hewson [Sat, 10 Aug 2013 11:12:42 +0000 (04:12 -0700)]
ITS#7656 fix install target
Salvador Ortiz [Fri, 9 Aug 2013 16:03:28 +0000 (11:03 -0500)]
ITS#7660 Add mdb_txn_env to API
Hallvard Furuseth [Sat, 10 Aug 2013 10:29:45 +0000 (12:29 +0200)]
mdb_txn_commit(): Always commit if MDB_TXN_SPILLS.
Checking dirty_list was insufficient after a spill
with no named databases and no positioned cursors.
Salvador Ortiz [Sat, 10 Aug 2013 07:42:19 +0000 (09:42 +0200)]
ITS#7661 mdb_dbi_flags(): Allow main DBI
Salvador Ortiz [Fri, 9 Aug 2013 15:08:53 +0000 (10:08 -0500)]
mdb_del must ignore data if db not opened with MDB_DUPSORT
Howard Chu [Fri, 9 Aug 2013 11:51:33 +0000 (04:51 -0700)]
Add mdb_env_get_maxkeysize()
Hallvard Furuseth [Fri, 9 Aug 2013 11:05:14 +0000 (13:05 +0200)]
Replace unpredictable EINVAL error returns.
Return EINVAL only for simple programmer errors.
Hallvard Furuseth [Fri, 9 Aug 2013 10:54:42 +0000 (12:54 +0200)]
Re-fix reader-pid code
Hallvard Furuseth [Thu, 8 Aug 2013 17:57:52 +0000 (19:57 +0200)]
mdb_cursorpages_mark: Mark current txn and no more.
Ignore parent txn cursors since it is the current txn's dirty_list
which will be flushed. But check the current txn also when clearing,
since cursors can have pages which are dirty in a parent.
Check !mc_xcursor instead of !MDB_DUPSORT. Equivalent for valid
data, but a bit safer if the sub-DB flags are corrupt.
Hallvard Furuseth [Thu, 8 Aug 2013 17:57:52 +0000 (19:57 +0200)]
Fix mdb_ovpage_free() vs. spill.
Ensure me_pghead has room before removing from spill/dirty list.
Don't return pages to me_pghead in nested txns, use mt_free_pgs.
Hallvard Furuseth [Thu, 8 Aug 2013 17:57:51 +0000 (19:57 +0200)]
Fix page spilling when MDB_WRITEMAP.
mdb_page_spill(): Don't binary-search the unsorted dirty_list.
mdb_page_flush(): Don't overwrite unprocessed dirty_list items.
Hallvard Furuseth [Thu, 8 Aug 2013 17:57:51 +0000 (19:57 +0200)]
Set MDB_TXN_ERROR when inconsistent txn state
Hallvard Furuseth [Thu, 8 Aug 2013 17:54:54 +0000 (19:54 +0200)]
Factor out MDB_env.
Hallvard Furuseth [Thu, 8 Aug 2013 17:43:04 +0000 (19:43 +0200)]
MDB_LOCK_VERSION -> MDB_LOCK_FORMAT.
Pid locking needs a different lockfile-version: MDB_env's with and
without pid locking must not coexist, they can sabotage each other.
Store MDB_LOCK_FORMAT = (version | "use locking" flag) instead.
Hallvard Furuseth [Thu, 8 Aug 2013 17:43:04 +0000 (19:43 +0200)]
Fix mdb_reader_pid().
Treat unexpected errors as "don't know". Invert Pidcheck return
value, so nonzero including error codes = "the process may exist".
On Windows: Catch exited but still existing processes. Handle
undefined PROCESS_QUERY_LIMITED_INFORMATION.
On Unix: don't trust F_GETLK error to leave the input alone,
the fcntl() doc seems unclear.
Howard Chu [Wed, 7 Aug 2013 19:42:46 +0000 (12:42 -0700)]
Use proper printf format on Windows
Hallvard Furuseth [Mon, 5 Aug 2013 08:01:39 +0000 (10:01 +0200)]
Silence warnings
Hallvard Furuseth [Mon, 5 Aug 2013 07:55:57 +0000 (09:55 +0200)]
Tweak comments
Hallvard Furuseth [Mon, 5 Aug 2013 07:55:48 +0000 (09:55 +0200)]
Clarify doc: mdb_copy, nested txns, mdb_drop().
mdb_copy: Does not copy lockfile. Can trigger file growth.
mdb_txn_begin(): Clarify usage restrictions.
mdb_drop(): State what to do rather than what will be done, since
closing the handle could otherwise be read as happening even at failure.
Howard Chu [Wed, 31 Jul 2013 15:09:40 +0000 (08:09 -0700)]
Tweak prev commit again
Make sure errors are propagated from init_meta
Howard Chu [Tue, 30 Jul 2013 20:44:28 +0000 (13:44 -0700)]
Fix typo in Win32 branch
Howard Chu [Tue, 30 Jul 2013 19:47:12 +0000 (12:47 -0700)]
Cleanup prev commit
Loop on copyfd meta write, since pipes may return after partial write.
Howard Chu [Tue, 30 Jul 2013 17:22:12 +0000 (10:22 -0700)]
ITS#7652 fix I/O error checks
partial revert of
d6d2638acc245116b8f091ac425b6700d06c4713 and
26a25df5fcc2fcddae6597a61c1b867fc27c568b
The original code was already tested and working correctly.
Howard Chu [Mon, 29 Jul 2013 00:02:51 +0000 (17:02 -0700)]
Tweak mdb_envinfo numreaders
Return the actual shared reader count when it exists, not
just the current process env's reader count.
Howard Chu [Fri, 26 Jul 2013 17:19:54 +0000 (10:19 -0700)]
ITS#7615 use shorter names for semaphores
NetBSD can only handle up to 14 chars, we were using 21. Now
we encode to 15, and for NetBSD truncate the last char.
Howard Chu [Fri, 19 Jul 2013 16:57:33 +0000 (09:57 -0700)]
Tweak reader_pid check
Check again after acquiring rmutex. Avoids potential issue with
a duplicate pid coming in between initial check and rmutex.
Howard Chu [Fri, 19 Jul 2013 16:55:10 +0000 (09:55 -0700)]
Get pid lock outside of rmutex
Avoid holding rmutex for longer than necessary.
Howard Chu [Thu, 18 Jul 2013 22:24:09 +0000 (15:24 -0700)]
Tweak reader checks
Use mti_numreaders for loop limit, not me_maxreaders.
Howard Chu [Thu, 18 Jul 2013 17:40:21 +0000 (10:40 -0700)]
Add mdb_reader_check()
Howard Chu [Thu, 18 Jul 2013 16:11:09 +0000 (09:11 -0700)]
Split MDB_VERSION to MDB_DATA/MDB_LOCK VERSION
Howard Chu [Thu, 18 Jul 2013 16:00:51 +0000 (09:00 -0700)]
Tweak reader_list
Howard Chu [Thu, 18 Jul 2013 15:33:24 +0000 (08:33 -0700)]
Tweak mdb_stat(1)
Don't obtain reader txn before displaying reader table. Exit
after reader table if no other DB query options were given.
Howard Chu [Thu, 18 Jul 2013 14:41:11 +0000 (07:41 -0700)]
Add mdb_reader_list()
Dump the active slots in the reader table.
Howard Chu [Mon, 15 Jul 2013 17:57:13 +0000 (10:57 -0700)]
Add mdb_dbi_flags()
Retrieve the flags from a DB handle.
Howard Chu [Sun, 14 Jul 2013 23:53:04 +0000 (16:53 -0700)]
Fix child txn dirty_room counts in spill/unspill
Don't count pages twice if they're already accounted in an ancestor txn.
Howard Chu [Sun, 14 Jul 2013 15:28:26 +0000 (08:28 -0700)]
More for stale sub-cursor flags
Same fix for cursor_first/last.
Howard Chu [Sun, 14 Jul 2013 15:20:18 +0000 (08:20 -0700)]
Fix stale sub-cursor C_INIT flag
Whenever we enter cursor_set() the sub-cursor's flag must be
cleared. If the new cursor position has valid subdata it will
be initialized again, if not then the sub-cursor has nothing
to point to.
Howard Chu [Fri, 12 Jul 2013 20:55:18 +0000 (13:55 -0700)]
Tweak comments, defaults should be OK already
Howard Chu [Fri, 12 Jul 2013 20:36:05 +0000 (13:36 -0700)]
Bump version to 0.9.7
Hallvard Furuseth [Fri, 12 Jul 2013 09:30:33 +0000 (11:30 +0200)]
Also set/clear P_KEEP in parent txn's cursors
Howard Chu [Thu, 11 Jul 2013 20:09:47 +0000 (22:09 +0200)]
Spill pages, take 3
Howard Chu [Thu, 11 Jul 2013 20:09:47 +0000 (22:09 +0200)]
Delay touching pages until cursor is positioned.
This avoids unnecessary rewrites of pages that do not change.
(Restructuring for upcoming mdb_page_spill work.)
Hallvard Furuseth [Thu, 11 Jul 2013 20:09:46 +0000 (22:09 +0200)]
Simplify: Always set C_UNTRACK for tracked cursors.
TODO: Rename C_UNTRACK to C_TRACKED. Omitted now for readability.
The current name is because it's lazy: not always set when tracked.
Hallvard Furuseth [Thu, 11 Jul 2013 20:09:46 +0000 (22:09 +0200)]
Save freelist using proper mdb_cursor_put().
(Restructuring for upcoming mdb_page_spill work.)
mdb_freelist_save() can't just Get() the destination, since
mdb_page_spill() may have put the destination in the read-only map.
TODO: Can this new put() modify the freelist, which would break it? The
final iteration's put() can shorten the node, the rest uses MDB_CURRENT.
We could set P_KEEP on dirty freeDB leaves and ovpages, since they are
all about to be modified. But the code in this commit must stay anyway,
if mdb should support dropping a 256G DB. I.e. too big for dirty_list.
Howard Chu [Thu, 11 Jul 2013 20:09:46 +0000 (22:09 +0200)]
Move code out to mdb_page_dirty()
Howard Chu [Thu, 11 Jul 2013 20:09:46 +0000 (22:09 +0200)]
Factor out parent
Howard Chu [Fri, 12 Jul 2013 19:53:35 +0000 (12:53 -0700)]
Fix env_read_header() on Windows
Commit
d6d2638acc245116b8f091ac425b6700d06c4713 broke read
on zero-length files.
Hallvard Furuseth [Wed, 10 Jul 2013 20:11:44 +0000 (22:11 +0200)]
Do not follow uninited cursors' page pointers.
Nor uninited cursors' subcursors' page pointers.
Howard Chu [Wed, 10 Jul 2013 18:03:51 +0000 (11:03 -0700)]
Fix rebalance/cursor adjust
When collapsing root, must also move cursor index down,
not just the page pointer.
Also in mtest, break from NEXT loops on error, otherwise it just
prints the previous key/data again, which looks confusing.
Howard Chu [Wed, 10 Jul 2013 15:49:29 +0000 (08:49 -0700)]
Cursors: Clear C_EOF when clearing C_INITIALIZED
Howard Chu [Tue, 9 Jul 2013 21:21:35 +0000 (14:21 -0700)]
Fixup other cursors after delete op
Hallvard Furuseth [Sun, 7 Jul 2013 15:14:38 +0000 (17:14 +0200)]
ITS#7515 Reject conflicting page versions.
If mdb_page_touch() sees a page in txn's dirty_list, that
is the page version txn's cursors should have. Fail if
the user may be seeing and depending on another version.
Hallvard Furuseth [Sun, 7 Jul 2013 15:13:27 +0000 (17:13 +0200)]
ITS#7515 Fix tracking of parent txn's cursors.
Restore mc_flags and xcursors, they were tracked but not merged.
Simplify: Track parent txn's original cursors after backing them
up, instead of tracking copies and merging them back at commit.
Hallvard Furuseth [Sun, 7 Jul 2013 15:13:27 +0000 (17:13 +0200)]
Simplify MDB_cursor: Drop flags C_ALLOCD,C_SHADOW.
Hallvard Furuseth [Sat, 6 Jul 2013 19:42:45 +0000 (21:42 +0200)]
Silence more uninit warnings
Howard Chu [Tue, 2 Jul 2013 09:23:49 +0000 (02:23 -0700)]
Silence uninit warning in prev commit
Howard Chu [Tue, 2 Jul 2013 09:19:17 +0000 (02:19 -0700)]
Tweaks for MDB_MULTIPLE
Terminate loop on intermediate failures, return count of written items,
document usage.
Howard Chu [Mon, 1 Jul 2013 21:53:29 +0000 (14:53 -0700)]
Howard Chu [Mon, 1 Jul 2013 20:41:23 +0000 (13:41 -0700)]
ITS#7635 fix read txn potential data race
Howard Chu [Sun, 30 Jun 2013 14:40:02 +0000 (07:40 -0700)]
Fix uninit warnings, lseek usage
Hallvard Furuseth [Wed, 26 Jun 2013 16:02:52 +0000 (18:02 +0200)]
Fix alloc/free issues.
Page leak, mdb_page_alloc(). On error, don't shorten me_pghead.
Memleak, mdb_ovpage_free(). Free page or keep it in dirty_list.
Bad MIDL, mdb_midl_need(). Fix midl[-1] (allocated size).
Hallvard Furuseth [Wed, 26 Jun 2013 16:02:48 +0000 (18:02 +0200)]
Factor out some vars, simplify.
Hallvard Furuseth [Wed, 26 Jun 2013 16:02:26 +0000 (18:02 +0200)]
Makefile/user-macro comments.
Hallvard Furuseth [Wed, 26 Jun 2013 16:02:17 +0000 (18:02 +0200)]
Tweak I/O, fix last commit.
Hallvard Furuseth [Sat, 22 Jun 2013 21:15:10 +0000 (23:15 +0200)]
Improve MDB error handling, drop seek calls.
Catch I/O errors. Do nothing between OS call failure and ErrCode().
Do not use errno after non-OS-errors like write() >= 0, which could
give a failure return of success (errno 0) or some irrelevant error
code. Drop seek calls, use pwrite/pread/Windows OVERLAPPED offset.
Hallvard Furuseth [Sat, 22 Jun 2013 21:01:30 +0000 (23:01 +0200)]
Fix Windows I/O.
Don't put a 64-bit filesize in a 32-bit int before shifting
down. Always pass &sizehi to SetFilePointer->maxsize, so
sizelo not is treated a signed distance. Hide unused vars
when _WIN32. Reinitialize OVERLAPPED before reuse.
Hallvard Furuseth [Sat, 22 Jun 2013 20:17:41 +0000 (22:17 +0200)]
Catch more MDB errors. DPRINTF in mdb_env_reset0.
Hallvard Furuseth [Sat, 22 Jun 2013 20:10:43 +0000 (22:10 +0200)]
Tweak MIDLs, catch errors.
Grow midls earlier in order to catch errors earlier. Use
mdb_midl_need() instead of mdb_midl_grow(), then mdb_midl_xappend()
needs no error checks. Factor out mdb_midl_append_range().
Hallvard Furuseth [Sat, 22 Jun 2013 10:30:04 +0000 (12:30 +0200)]
Factor out MDB variables/expressions, cleanup.
mdb_page_malloc(): Take a txn arg instead of a cursor.
Hallvard Furuseth [Sat, 22 Jun 2013 09:56:04 +0000 (11:56 +0200)]
Rearrange MDB dirty page code.
Split out mdb_dpage_free(), mdb_page_flush() and clean up.
Hallvard Furuseth [Thu, 20 Jun 2013 05:41:35 +0000 (07:41 +0200)]
Simplify mdb_page_alloc().
Merge if() branches. Restore retry=500 when MDB_PARANOID, for clarity.
Hallvard Furuseth [Thu, 20 Jun 2013 05:41:35 +0000 (07:41 +0200)]
ITS#7620: Keep empty IDLs. Tweak mdb_page_alloc().
MDB_env.me_pghead: Don't free it when empty. mdb_ovpage_free()
needs it, but cannot allocate it.
mdb_midl_alloc(): Fill in length=0.
mdb_page_alloc(): Also Skip freeDB if txnid<3, instead of <4,
and consistently DPRINTF consumed IDLs.
Howard Chu [Mon, 17 Jun 2013 20:26:11 +0000 (22:26 +0200)]
ITS#7623 Clear P_SUBP on conversion from fake page
Hallvard Furuseth [Thu, 13 Jun 2013 06:58:25 +0000 (08:58 +0200)]
ITS#7515 Nested MDB txns: Inherit txn flags.
Committing a nested txn lost the MDB_TXN_DIRTY flag
in the parent, unless the child had set it too.