Hallvard Furuseth [Sun, 23 Mar 2014 12:06:03 +0000 (13:06 +0100)]
Revert "Update MDB_PANIC description" (thinko).
This reverts commit
6bd34d0534f1c8bfd73deaf70febd3b58a2f9d5a.
Hallvard Furuseth [Sat, 22 Mar 2014 21:27:10 +0000 (22:27 +0100)]
Update MDB_PANIC description
Hallvard Furuseth [Sat, 22 Mar 2014 21:27:10 +0000 (22:27 +0100)]
Tweak mdb_mid2l_insert() call from ITS#7775.
It does not return MDB error codes.
Hallvard Furuseth [Sun, 16 Mar 2014 11:42:43 +0000 (12:42 +0100)]
mtest: Accept an MDB_NOTFOUND case
Hallvard Furuseth [Sun, 16 Mar 2014 11:42:43 +0000 (12:42 +0100)]
Handle more errors. Invalidate txn if needed.
Hallvard Furuseth [Sun, 16 Mar 2014 11:42:42 +0000 (12:42 +0100)]
Don't use non-existent node info in branch/leaf2
Hallvard Furuseth [Sun, 16 Mar 2014 11:42:42 +0000 (12:42 +0100)]
Move code out of mdb_cursor_del0() and mdb_del().
Prepares for the next commits.
Hallvard Furuseth [Sun, 16 Mar 2014 11:42:42 +0000 (12:42 +0100)]
More checks for closed DBIs and invalidated txns.
Factor txn/DBI-checks out to TXN_DBI_EXIST().
mdb_audit(): Skip closed DBIs.
mdb_cursor_renew(), mdb_stat(): Check DBI and txn.
mdb_cursor_count(): Check txn.
mdb_dbi_flags(): Check DBI.
Hallvard Furuseth [Sun, 16 Mar 2014 11:42:42 +0000 (12:42 +0100)]
mdb_page_list(): Support & display all page types
Hallvard Furuseth [Sun, 16 Mar 2014 11:42:42 +0000 (12:42 +0100)]
mdb_txn_renew0(): Fix un-mutexed me_flags update.
Replace env flag MDB_LIVE_READER with field me_live_reader.
Hallvard Furuseth [Sun, 16 Mar 2014 11:42:42 +0000 (12:42 +0100)]
Doxygen fixes
Howard Chu [Sun, 16 Mar 2014 10:55:03 +0000 (03:55 -0700)]
Rename samples
They're meant to be read and studied, not run.
Howard Chu [Thu, 6 Mar 2014 04:40:12 +0000 (20:40 -0800)]
ITS#7815 more for page_split
This case didn't come up in testing. Needs finer grained test.
Howard Chu [Thu, 6 Mar 2014 04:04:42 +0000 (20:04 -0800)]
ITS#7815 fix mdb_page_split
Was leaving cursor pointed at wrong page if the newindx was
in the right sibling page, and the parent was also split.
Hallvard Furuseth [Wed, 19 Feb 2014 20:25:40 +0000 (21:25 +0100)]
mtest*.c, sample-*.c: Add/note error checks.
Hallvard Furuseth [Wed, 19 Feb 2014 20:25:40 +0000 (21:25 +0100)]
mtest[46]: Use different DBs when different flags.
Hallvard Furuseth [Tue, 28 Jan 2014 13:27:27 +0000 (14:27 +0100)]
ITS#7793 mdb_cursor_put(): Fix MDB_CURRENT.
Ignore key, broken by
5bda3565a9bfaa6cd54053faeafcc06da15bc00c
and some older code. Document and clarify MDB_CURRENT usage.
Also affects non-MDB_CURRENT put() with empty data and a key
which matches by the mdb_set_compare function but not by memcmp.
Howard Chu [Wed, 15 Jan 2014 17:09:48 +0000 (09:09 -0800)]
Bump version to 0.9.11
Howard Chu [Wed, 15 Jan 2014 01:12:48 +0000 (17:12 -0800)]
Note changes for LMDB 0.9.11
Hallvard Furuseth [Mon, 13 Jan 2014 19:46:08 +0000 (20:46 +0100)]
Revert "ITS#7772 Reduce spurious sub-page growth."
This reverts commit
75494c08b49c3ef77dce8a00f2bc44c83658c1a7,
which must await a fix to ITS#7771 (DUPSORT cursor tracking).
Howard Chu [Mon, 13 Jan 2014 01:37:04 +0000 (17:37 -0800)]
Make sure mapsize changes take effect on Windows.
Hallvard Furuseth [Mon, 6 Jan 2014 22:21:54 +0000 (23:21 +0100)]
ITS#7775 Drop <assert.h>, drop/replace assert()s.
db_mid2l_insert(): Move assert to mdb.c.
mdb_cursor_set(): Previous assert cannot fail now.
mdb_cursor_put(): Check mc/key and return EINVAL.
mdb_cursor_dbi(): No error return, so just segfault if cursor==NULL.
Howard Chu [Mon, 6 Jan 2014 22:21:44 +0000 (23:21 +0100)]
ITS#7775 Cleanup mdb_node_del()
Hallvard Furuseth [Mon, 6 Jan 2014 22:17:38 +0000 (23:17 +0100)]
ITS#7775 Add mdb_env_set_assert()
Hallvard Furuseth [Mon, 6 Jan 2014 22:17:38 +0000 (23:17 +0100)]
Add mdb_env_<set,get>_userctx()
Hallvard Furuseth [Mon, 6 Jan 2014 22:17:37 +0000 (23:17 +0100)]
ITS#7377 Invalidate txn on page-allocation errors
This should likely be reverted when all callers handle these errors.
Hallvard Furuseth [Sun, 5 Jan 2014 15:37:26 +0000 (16:37 +0100)]
Wrap __func__ in mdb_func_
Hallvard Furuseth [Sun, 5 Jan 2014 15:26:21 +0000 (16:26 +0100)]
Invalidate the txn on corruption/internal errors
Howard Chu [Fri, 3 Jan 2014 20:40:28 +0000 (12:40 -0800)]
ITS#7775 cleanup unnecessary assert()s
Hallvard Furuseth [Tue, 31 Dec 2013 01:16:19 +0000 (02:16 +0100)]
mdb_page_malloc(): Move VGMEMP_ALLOC before memset
Hallvard Furuseth [Sun, 29 Dec 2013 14:58:21 +0000 (15:58 +0100)]
ITS#7771 mdb_cursor_del0(): Fix xcursor tracking.
Hallvard Furuseth [Sun, 29 Dec 2013 14:57:33 +0000 (15:57 +0100)]
sprintf format: pid_t need not be an int.
Hallvard Furuseth [Sun, 29 Dec 2013 14:09:51 +0000 (15:09 +0100)]
Add mdb_dbg_pgno(): (sub)pagenum for debug output.
Hallvard Furuseth [Sun, 29 Dec 2013 14:09:05 +0000 (15:09 +0100)]
ITS#7772 Reduce spurious sub-page growth.
mdb_cursor_put(): Use free sub-page space added by a
previous put(existing DUPSORT item), if any.
Tweak NUMKEYS code to avoid a new signedness warning.
Hallvard Furuseth [Sun, 29 Dec 2013 13:54:17 +0000 (14:54 +0100)]
ITS#7665 Drop lmdb.h doc on non-overlapping flags.
It was inaccurate and misplaced. Verify it in mdb.c instead.
Hallvard Furuseth [Sun, 29 Dec 2013 13:52:30 +0000 (14:52 +0100)]
ITS#7515 has been fixed.
Hallvard Furuseth [Sun, 29 Dec 2013 13:52:30 +0000 (14:52 +0100)]
Drop UNLOCK_MUTEX_R() on mdb_reader_pid() error
Hallvard Furuseth [Sat, 21 Dec 2013 11:31:57 +0000 (12:31 +0100)]
Fix commit "Raise safe max MDB_MAXKEYSIZE."
I.e.
d69d2ce2307e9d0e2167843935c386b05a869c31, it left out some changes.
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:13 +0000 (11:57 +0100)]
Replace MDB_node.mn_offset[], to ease debugging.
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:13 +0000 (11:57 +0100)]
Comment MDB_node, mdb_page_unspill, mdb_update_key
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:13 +0000 (11:57 +0100)]
Fix mdb_reader_list() and its spec.
It and the MDB_msg_func can now return >= 0 for success.
Always return any MDB_msg_func() error result.
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:13 +0000 (11:57 +0100)]
Support dynamic max MDB keysize.
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:13 +0000 (11:57 +0100)]
Raise safe max MDB_MAXKEYSIZE.
Use a sub-DB for DUPSORT item #1/#2 per key if needed: Not a sub-
page too big for a node, nor an overflow page (which not all DUPSORT
code checks for). Move "insert" code, to avoid non-loop goto upwards.
(This is the commit which needs the change to xdata.mv_size in
commit
9d6e4a916367e85ffdf37b1cb7b5cdb7ac0e15b5 "page sizes".)
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:12 +0000 (11:57 +0100)]
Relax MDB_MAXKEYSIZE for existing data.
Handle keys stored by a liblmdb with a bigger MDB_MAXKEYSIZE.
mdb_get/mdb_del(absent key bigger than our MDB_MAXKEYSIZE)
now return MDB_NOTFOUND instead of MDB_BAD_VALSIZE.
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:12 +0000 (11:57 +0100)]
ITS#7589 Fix me_nodemax and related node sizes.
Change me_nodemax to not count the mp_ptrs[] entry. That's mostly
how it was used. Compare node sizes ">" me_nodemax instead of ">=".
The ">=" was a workaround for confusing sizes with and without the
mp_ptrs[] entry, but broke for nodes with size (old me_nodemax-1).
Explicitly make me_nodemax even. An odd value could break the
comparisons. It was even anyway because MDB_MINKEYS == 2.
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:12 +0000 (11:57 +0100)]
ITS#7589 Fix delete(uneven-sized MDB_DUPFIXED data).
Do not give uneven sizes to the sub-page and the node it lives in.
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:12 +0000 (11:57 +0100)]
put(MDB_MULTIPLE): Skip existing item cleanly.
Don't set dkey.mv_size if mdb won't clear it before next iteration.
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:12 +0000 (11:57 +0100)]
Add EVEN(): Micro-optimize "up to multiple of 2".
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:12 +0000 (11:57 +0100)]
Warning cleanup.
Drop unused computed/assigned values.
mdb_page_alloc(): Make "mop_len>0 implies mop!=NULL" provable.
Hallvard Furuseth [Wed, 11 Dec 2013 10:57:12 +0000 (11:57 +0100)]
Clean up mdb_copy.
Support MDB_NOSUBDIR. Catch more errors. Write messages
to stderr, not stdout which the data too may be piped to.
Howard Chu [Thu, 5 Dec 2013 02:42:49 +0000 (18:42 -0800)]
Fix GET_BOTH_RANGE
On near match, return the matched data.
Howard Chu [Sat, 30 Nov 2013 01:03:49 +0000 (17:03 -0800)]
ITS#7756 keysizes were wrong in mdb_update_key
Howard Chu [Tue, 12 Nov 2013 16:10:33 +0000 (08:10 -0800)]
Doc tweak
Measured perf cost of meminit was 7% worst case on an Intel Core2
system. YMMV.
Howard Chu [Tue, 12 Nov 2013 04:15:54 +0000 (20:15 -0800)]
Bump to 0.9.10
Howard Chu [Tue, 12 Nov 2013 04:08:26 +0000 (20:08 -0800)]
Windows/Win64/MSVC tweaks
Silence warnings, fix broken MinGW64 definitions, etc.
Howard Chu [Tue, 12 Nov 2013 00:35:17 +0000 (16:35 -0800)]
Clarify WRITEMAP / NOMEMINIT
Howard Chu [Mon, 11 Nov 2013 23:09:32 +0000 (15:09 -0800)]
Change default mem init behavior
Init malloc'd memory by default, use #MDB_NOMEMINIT to disable.
Bump version to 0.9.10
Hallvard Furuseth [Thu, 7 Nov 2013 22:41:30 +0000 (23:41 +0100)]
Add MDB_CLEANMEM option.
Initialize unused portions of DB pages, for security or to silence
checkers like Purify and Valgrind. Like BDB configure --enable-umrw.
Hallvard Furuseth [Thu, 7 Nov 2013 22:38:15 +0000 (23:38 +0100)]
Fix MDB_MAXKEYSIZE doc: Account for MDB_DUPSORT.
A node may need room for [key, subpage with 2 items] = 3 keys.
Hallvard Furuseth [Thu, 7 Nov 2013 22:38:15 +0000 (23:38 +0100)]
ITS#7713 Handle bigger OS page sizes
Use DB page size = min(32k, OS pagesize). Previous limit was 8k
(MDB_MINKEYS*MDB_PAGESIZE).
Handle DB pagesize < OS pagesize. That's an I/O pessimization,
but transactions remain atomic: Only writing the MDB_meta must
be atomic, and it fits in one OS page.
Don't truncate desired subpage size: Asssign it to a size_t
(mv_size), not an uint16_t (mp_upper).
Hallvard Furuseth [Thu, 7 Nov 2013 22:38:15 +0000 (23:38 +0100)]
mdb_cursor_put(): Factor out code. Kill a label.
Howard Chu [Thu, 7 Nov 2013 17:37:48 +0000 (09:37 -0800)]
Fix prev commit, cursor depth comparison
Howard Chu [Thu, 7 Nov 2013 00:49:25 +0000 (16:49 -0800)]
Fix cursor_del0 cursor fixup
Skip cursors that aren't as deep as current cursor
Howard Chu [Tue, 5 Nov 2013 04:00:55 +0000 (20:00 -0800)]
More for #7589
Howard Chu [Fri, 1 Nov 2013 00:31:15 +0000 (17:31 -0700)]
ITS#7589 another mdb_page_split fix
Tweak split_indx check, go one slot further.
Howard Chu [Thu, 31 Oct 2013 23:01:04 +0000 (16:01 -0700)]
ITS#7736 fix regression in ITS#7733 patch
In
d8eccb353b1be7f07e74149dde9a41bb199d74e8
Howard Chu [Mon, 28 Oct 2013 14:25:54 +0000 (07:25 -0700)]
ITS#7733 check cursor index before cursor_del
David Wilson [Mon, 28 Oct 2013 13:58:04 +0000 (06:58 -0700)]
ITS#7734 Add MDB_NORDAHEAD to CHANGELESS.
Howard Chu [Thu, 24 Oct 2013 22:57:17 +0000 (15:57 -0700)]
Bump to 0.9.9
Howard Chu [Mon, 21 Oct 2013 17:29:38 +0000 (10:29 -0700)]
Add mdb_env_get_fd()
Return the environment's filedescriptor. Useful when the caller
is doing their own locking.
Howard Chu [Mon, 14 Oct 2013 16:47:26 +0000 (09:47 -0700)]
Howard Chu [Sat, 12 Oct 2013 23:20:04 +0000 (16:20 -0700)]
ITS#7725 add MDB_NORDAHEAD flag for env_open
Howard Chu [Sat, 12 Oct 2013 20:40:28 +0000 (13:40 -0700)]
ITS#7589 silence warnings
Howard Chu [Sat, 12 Oct 2013 20:31:09 +0000 (13:31 -0700)]
ITS#7589 Fix typo in prev commit
Howard Chu [Sat, 12 Oct 2013 20:19:33 +0000 (13:19 -0700)]
ITS#7589 Cleanup unneeded vars in prev patch
Howard Chu [Sat, 12 Oct 2013 16:34:40 +0000 (09:34 -0700)]
ITS#7589 avoid wasting space in mdb_page_split
Also, check the split point on branch pages as well as leaf pages.
Howard Chu [Fri, 4 Oct 2013 17:22:14 +0000 (10:22 -0700)]
Add MDB_USE_HASH
There may be other uses for it besides semaphore names.
Howard Chu [Wed, 9 Oct 2013 09:24:45 +0000 (02:24 -0700)]
ITS#7722 fix page_merge for collapsing root
Other cursors being fixed up are not necessarily height 1.
Howard Chu [Tue, 8 Oct 2013 11:43:58 +0000 (04:43 -0700)]
ITS#7718 invalidate cursor when DB gets emptied
Howard Chu [Fri, 4 Oct 2013 09:55:34 +0000 (02:55 -0700)]
Fix mdb_env_set_mapsize()
Don't allow shrinking below minimum size for an already open env.
Howard Chu [Fri, 4 Oct 2013 09:46:38 +0000 (02:46 -0700)]
ITS#7715 always set filesize for WRITEMAP
Hallvard Furuseth [Thu, 3 Oct 2013 22:48:19 +0000 (00:48 +0200)]
Add Caveat: Readers need write access. Whitespace.
Hallvard Furuseth [Thu, 3 Oct 2013 21:59:25 +0000 (23:59 +0200)]
Maintain MDB_cursor.mc_top
Hallvard Furuseth [Thu, 3 Oct 2013 21:59:25 +0000 (23:59 +0200)]
mdb_cursor_sibling(): Fix error result
Hallvard Furuseth [Thu, 3 Oct 2013 21:59:25 +0000 (23:59 +0200)]
mdb_node_add(): Plug page leak when MDB_PAGE_FULL.
Do not fail after mdb_page_new() succeeds.
Hallvard Furuseth [Thu, 3 Oct 2013 21:59:24 +0000 (23:59 +0200)]
Optimize code holding reader mutex
Hallvard Furuseth [Thu, 3 Oct 2013 21:59:24 +0000 (23:59 +0200)]
Set MDB_xcursor DB_DIRTY, clear md_name.
Both were unused and md_name was unmaintained -- except
mdb_cursor_touch(xcursor) would abuse md_name as a key to
touch MAIN_DBI if it could somehow get passed ! DB_DIRTY.
Hallvard Furuseth [Thu, 3 Oct 2013 21:59:24 +0000 (23:59 +0200)]
Set subDB DBI=parent DBI, fix MDB_DUPSORT delete.
xcursor DBIs were parent DBI+1 for debugging. Instead output
-(parent DBI). Fixes a crash in mdb_cursor_del0()'s xcursor
tracking, it forgot to subtract 1 for C_SUB cursors.
Howard Chu [Thu, 3 Oct 2013 17:26:44 +0000 (10:26 -0700)]
s/MDB_NORDLOCK/MDB_NOLOCK/
Leave all lock management to the caller.
Howard Chu [Wed, 2 Oct 2013 06:36:57 +0000 (23:36 -0700)]
More doc for MDB_NORDLOCK
Howard Chu [Tue, 1 Oct 2013 20:16:38 +0000 (13:16 -0700)]
Add MDB_NORDLOCK to omit all reader table usage
Calling app wants to manage its own locking.
Hallvard Furuseth [Mon, 23 Sep 2013 18:21:11 +0000 (20:21 +0200)]
Drop unneeded code.
MDB_txn.mt_toggle: Use (mt_txnid & 1) instead.
Drop error checks which will be repeated.
mdb_cursor_set(): Turn assert into if/return to match the above.
mdb_cursor_del(): 'flags' are now used as bitflags.
Hallvard Furuseth [Mon, 23 Sep 2013 18:21:11 +0000 (20:21 +0200)]
Clean up and simplify mdb_page_search().
Only named DBs can have DB_STALE, and they do not use MDB_PS_MODIFY.
Replace magic key values with flags. Drop duplicated comments at
mdb_page_search_root() vs. mdb_page_search(), and rephrase.
Hallvard Furuseth [Mon, 23 Sep 2013 18:20:42 +0000 (20:20 +0200)]
Tweak MDB_DEBUG output
Hallvard Furuseth [Mon, 23 Sep 2013 18:20:05 +0000 (20:20 +0200)]
Rename SWAP() to avoid conflict with sqlightning.
Happened since sqlightning #includes mdb.c, midl.c.
Hallvard Furuseth [Mon, 23 Sep 2013 18:13:27 +0000 (20:13 +0200)]
mdb_page_unspill(): Rename local vars.
The names have caused bugs, "txn" was treated as
the current transaction.
Hallvard Furuseth [Mon, 23 Sep 2013 18:13:27 +0000 (20:13 +0200)]
ITS#7515 Fix mdb_page_unspill() in nested txn.
Malloc a page in this txn, not in a parent.
Hallvard Furuseth [Mon, 23 Sep 2013 18:13:27 +0000 (20:13 +0200)]
ITS#7515 Fix mdb_txn_commit(nested txn with spills).
Catch malloc error.
Fix hunt for dirty vs spilled pages: Don't leave x at a deleted pageno.
Cleanup: Factor out variables, squash pages already marked for deletion.
Hallvard Furuseth [Mon, 23 Sep 2013 18:13:27 +0000 (20:13 +0200)]
ITS#7515 Fix mt_dirty_room in nested txns.
Fix description & code: Also ignore dirty pages hidden by
spilled pages, as they won't merge into our dirty_list.
Update it in mdb_page_flush() instead of mdb_page_spill().
Hallvard Furuseth [Mon, 23 Sep 2013 18:13:27 +0000 (20:13 +0200)]
ITS#7515 mdb_pages_xkeep(): Careful about xcursors.
Don't prod sub-pages or pages referring to uninitialized xcursors.