Hallvard Furuseth [Sat, 10 Dec 2016 20:42:39 +0000 (21:42 +0100)]
Clean up and comment C_UNTRACK
Don't use it as a "cursor is tracked" hint in mdb_pages_xkeep().
It's been harmless so far, but would break after mdb_cursor_copy().
Checking m0 directly short-circuits better anyway.
Hallvard Furuseth [Sat, 10 Dec 2016 20:11:12 +0000 (21:11 +0100)]
ITS#8355 cleanup
Drop ~C_EOF, pointless after
8c215aa970215a58ee0df458813c0405ad27a6e9
Hallvard Furuseth [Sat, 10 Dec 2016 16:17:28 +0000 (17:17 +0100)]
MDB_VL32 cleanup: MDB_env.mm_mapsize type
Hallvard Furuseth [Sat, 10 Dec 2016 08:16:17 +0000 (09:16 +0100)]
ITS#8542 mdb_dbi_open(): Protect mainDB cursors
Hallvard Furuseth [Thu, 8 Dec 2016 23:03:36 +0000 (00:03 +0100)]
Cleanup: Add flag DB_DUPDATA, drop DB_DIRTY hack
Hallvard Furuseth [Wed, 7 Dec 2016 18:06:11 +0000 (19:06 +0100)]
MDB_CP_COMPACT comments
Hallvard Furuseth [Wed, 7 Dec 2016 18:04:19 +0000 (19:04 +0100)]
doxygen cleanup
Hallvard Furuseth [Wed, 7 Dec 2016 17:55:21 +0000 (18:55 +0100)]
Note functions which must set MDB_TXN_ERROR on failure
Other functions depend on them to do so.
For mdb_node_read(), instead remove such a dependence.
Hallvard Furuseth [Fri, 2 Dec 2016 05:20:29 +0000 (06:20 +0100)]
MDB_VL32 mdb_page_get(): Set MDB_TXN_ERROR on failure.
Hallvard Furuseth [Fri, 2 Dec 2016 04:22:11 +0000 (05:22 +0100)]
Fix mdb_env_open() with MDB_VL32
Init me_rpmutex independently of MDB_NOLOCK.
Plug leaks on mdb_env_open() failure.
Tweak mdb_env_close0() to handle the rearranged mdb_env_open().
Hallvard Furuseth [Thu, 1 Dec 2016 20:17:42 +0000 (21:17 +0100)]
Factor out refreshing sub-page pointers
Howard Chu [Tue, 29 Nov 2016 19:19:45 +0000 (19:19 +0000)]
More for ITS#8406
xcursor fixup depends on init state
Howard Chu [Sat, 12 Nov 2016 23:11:20 +0000 (23:11 +0000)]
More for ITS#8406
Revert excess cursor fixup
Hallvard Furuseth [Tue, 27 Sep 2016 05:03:45 +0000 (07:03 +0200)]
Only set me_mfd if needed. Drop unused read access.
Hallvard Furuseth [Tue, 27 Sep 2016 05:03:42 +0000 (07:03 +0200)]
ITS#8505 Clarify fork() caveat, mdb_env_get_fd(), flock->fcntl.
Hallvard Furuseth [Tue, 27 Sep 2016 05:03:40 +0000 (07:03 +0200)]
ITS#8505 Protect parent from fork()-pthread_exit()
Hallvard Furuseth [Tue, 27 Sep 2016 05:03:38 +0000 (07:03 +0200)]
ITS#8505 Set FD_CLOEXEC for me_mfd,env_copy as well
Hallvard Furuseth [Tue, 27 Sep 2016 05:03:34 +0000 (07:03 +0200)]
Move opening files to mdb_fopen()
No change in functionality.
Hallvard Furuseth [Sat, 17 Sep 2016 19:31:04 +0000 (21:31 +0200)]
Factor filename handling out to mdb_fname_*()
No change in functionality, except needs less mallocing.
Hallvard Furuseth [Tue, 6 Sep 2016 16:12:01 +0000 (18:12 +0200)]
ITS#7992 Tighter utf8_to_utf16(), fix errcodes
The 0xFFFD check seems due to misleading MultiByteToWideChar() doc.
Bad UTF-8 gives 0xFFFD in the output string, not the return value.
Hallvard Furuseth [Tue, 6 Sep 2016 15:56:13 +0000 (17:56 +0200)]
Clean up strange fcntl result check
...and check !MDB_CLOEXEC in an 'if' rather than '#if'
to match its non-zero usage.
Hallvard Furuseth [Tue, 6 Sep 2016 15:48:31 +0000 (17:48 +0200)]
Drop spurious Errcode() call
Hallvard Furuseth [Tue, 6 Sep 2016 15:41:02 +0000 (17:41 +0200)]
ITS#7682 F_NOCACHE: Allow error, skip any O_DIRECT
We can run without F_NOCACHE if it fails. And we do not know
what combining it with O_DIRECT means, if a system has both.
Hallvard Furuseth [Sat, 3 Sep 2016 07:41:30 +0000 (09:41 +0200)]
Silence warning for initializer "mdb_copy my = {0}"
1st struct member was not a scalar.
Howard Chu [Wed, 31 Aug 2016 23:41:35 +0000 (00:41 +0100)]
ITS#8489 reset cursor EOF flag in cursor_set
It usually gets done anyway, but one of the fastpath shortcuts
bypassed this step.
Hallvard Furuseth [Sun, 21 Aug 2016 21:22:18 +0000 (23:22 +0200)]
Clean up MDB_USE_ROBUST #defines
Make explicit and default nonzero equivalent. Parenthesize.
Hallvard Furuseth [Sun, 21 Aug 2016 21:20:22 +0000 (23:20 +0200)]
Fix mdb_page_list() message
Howard Chu [Fri, 19 Aug 2016 16:24:25 +0000 (17:24 +0100)]
ITS#8481 make shared lib suffix overridable
Howard Chu [Thu, 11 Aug 2016 20:30:56 +0000 (21:30 +0100)]
MDB_VL32 plug rpage leak
mdb_cursor_set wasn't unref'ing as intended.
Hallvard Furuseth [Tue, 2 Aug 2016 20:17:54 +0000 (22:17 +0200)]
Comment MDB_page
Hallvard Furuseth [Tue, 2 Aug 2016 20:16:09 +0000 (22:16 +0200)]
Doc fixes: VALID_FLAGS, mm_last_pg, mt_loose_count
Hallvard Furuseth [Tue, 2 Aug 2016 19:04:17 +0000 (21:04 +0200)]
mdb_size comments
Hallvard Furuseth [Tue, 2 Aug 2016 19:02:35 +0000 (21:02 +0200)]
MDB_VL32: Switch to mdb_size_t formats PRIu64 & co
Drop macro Y=MDB_FMT_Y, add Yu/Yd=MDB_PRIy(). Replace
Y"d..." -> Yd"...", Y"u..." -> Yu"..." / MDB_SCNy(u)"...".
Hallvard Furuseth [Sat, 23 Jul 2016 10:45:46 +0000 (12:45 +0200)]
Refactor mdb_page_get()
Hallvard Furuseth [Sat, 23 Jul 2016 10:43:16 +0000 (12:43 +0200)]
MDB_VL32: Match mdb_size_t type with format modifier.
When using format modifier "ll" or "I64", use the matching
type unsigned <long long / __int64> rather than uint64_t.
Hallvard Furuseth [Sat, 23 Jul 2016 10:18:01 +0000 (12:18 +0200)]
Fix MDB_INTEGERKEY doc of integer types
Hallvard Furuseth [Sat, 23 Jul 2016 10:17:04 +0000 (12:17 +0200)]
Use mdb_size_t for line numbers in mdb_load
This matches the mdb_size_t entry counts.
Hallvard Furuseth [Sat, 23 Jul 2016 10:16:20 +0000 (12:16 +0200)]
Fix size_t/formats -> mdb_size_t for MDB_VL32
Hallvard Furuseth [Sat, 23 Jul 2016 10:11:34 +0000 (12:11 +0200)]
Factor some MDB_VL32-related '#if's out to macros
Add MC_OVPG() + MC_SET_OVPG(), NEED_CMP_CLONG(), MDB_CURSOR_UNREF().
Hallvard Furuseth [Sat, 23 Jul 2016 10:08:12 +0000 (12:08 +0200)]
Factor out MDB_SIZE_MAX, MDB_FMT_Y, MDB_FMT_Z
Hallvard Furuseth [Thu, 14 Jul 2016 03:53:21 +0000 (05:53 +0200)]
Add error MDB_PROBLEM, replace some MDB_CORRUPTED
When problem is most likely in txn, not on disk.
Hallvard Furuseth [Wed, 29 Jun 2016 04:25:37 +0000 (06:25 +0200)]
ITS#8209 Tweak previous fixes
Some _aligned_malloc() doc seems to think arg NULL = user error.
Don't know if posix_memalign() pointer is defined after failure.
Hallvard Furuseth [Sat, 25 Jun 2016 05:57:04 +0000 (07:57 +0200)]
ITS#8209 MDB_CP_COMPACT: Handle empty or broken DB
Preserve DB flags (use metapage#1) when main DB is empty.
Fail if metapage root != actual root in output file.
Hallvard Furuseth [Sat, 25 Jun 2016 05:55:34 +0000 (07:55 +0200)]
ITS#8209 MDB_CP_COMPACT fixes
Handle errors. Fix cond_wait condition so mc_new
is the sole control var. Drop specious cond_waits.
Do not look at 'mo' while copythr writes it.
Hallvard Furuseth [Sun, 5 Jun 2016 21:42:44 +0000 (23:42 +0200)]
mdb_env_setup_locks: Plug mutexattr leak on error
Hallvard Furuseth [Fri, 3 Jun 2016 04:11:54 +0000 (06:11 +0200)]
Init "locked" flag for SysV semaphores
Howard Chu [Thu, 2 Jun 2016 20:01:27 +0000 (21:01 +0100)]
ITS#8339 Solaris 10/11 robust mutex fixes
Check for PTHREAD_MUTEX_ROBUST_NP definition (this doesn't work
on Linux/glibc because they used an enum). Zero out mutex before
initing.
Howard Chu [Sat, 14 May 2016 23:44:54 +0000 (00:44 +0100)]
ITS#8424 init cursor in mdb_env_cwalk
Hallvard Furuseth [Tue, 10 May 2016 05:11:44 +0000 (07:11 +0200)]
Comment ovpage code in mdb_cursor_put()
Howard Chu [Tue, 26 Apr 2016 11:52:21 +0000 (12:52 +0100)]
ITS#8412 fix NEXT_DUP after cursor_del
Howard Chu [Mon, 18 Apr 2016 17:07:56 +0000 (18:07 +0100)]
ITS#8406 fix xcursors after cursor_del
Don't leave them uninit'd if they now point at a valid DUP node
Howard Chu [Sat, 9 Apr 2016 19:42:45 +0000 (20:42 +0100)]
mdb_drop optimization
If we know there are no sub-DBs and no overflow pages, skip leaf scan.
Howard Chu [Mon, 28 Mar 2016 00:35:56 +0000 (01:35 +0100)]
ITS#8393 fix MDB_GET_BOTH on non-dup record
Howard Chu [Wed, 17 Feb 2016 17:27:50 +0000 (17:27 +0000)]
MDB_VL32 - increase max write txn size
Howard Chu [Tue, 16 Feb 2016 23:34:27 +0000 (23:34 +0000)]
Tweak MDB_PREV_MULTIPLE for uninit'd cursor
Howard Chu [Mon, 15 Feb 2016 00:07:04 +0000 (00:07 +0000)]
Tweak Win32 errmsg buffer
Howard Chu [Sun, 14 Feb 2016 23:53:05 +0000 (23:53 +0000)]
ITS#8324 Map NTAPI result codes to WIN32 codes
Hallvard Furuseth [Thu, 4 Feb 2016 02:23:13 +0000 (03:23 +0100)]
ITS#7992 Fix memleak in prev change
Howard Chu [Sat, 30 Jan 2016 12:54:32 +0000 (12:54 +0000)]
Happy New Year
Hallvard Furuseth [Thu, 28 Jan 2016 18:45:01 +0000 (19:45 +0100)]
Fix MDB_VL32 mdb_cursor_count()/entrycount types
Howard Chu [Thu, 28 Jan 2016 14:23:02 +0000 (14:23 +0000)]
MDB_VL32 change overflow page scan
Just check the requested page, don't worry about any other pages
Howard Chu [Thu, 28 Jan 2016 04:18:14 +0000 (04:18 +0000)]
ITS#8363 Fix off-by-one in mdb_midl_shrink
Howard Chu [Wed, 27 Jan 2016 11:48:22 +0000 (11:48 +0000)]
MDB_VL32 more for
1ba5adb2ec262405f9207d6015d4f29eea548d25
fix 32bit overflow in set_mapsize
Howard Chu [Tue, 26 Jan 2016 22:13:01 +0000 (22:13 +0000)]
ITS#8324 fix
a6ccef73ed288271f9b5871909d14a2e481c81ae
Removing the WRITEMAP test dropped this into the MDB_VL32 code path,
which was wrong.
Howard Chu [Sun, 24 Jan 2016 11:32:49 +0000 (11:32 +0000)]
ITS#8355 fix subcursors
make sure C_DEL gets reset in subcursor after it moves.
Howard Chu [Sat, 23 Jan 2016 16:51:42 +0000 (16:51 +0000)]
Update WRITEMAP doc
Howard Chu [Wed, 20 Jan 2016 01:30:57 +0000 (01:30 +0000)]
WIN64 needs off_t redefined too
Howard Chu [Sat, 16 Jan 2016 17:11:36 +0000 (17:11 +0000)]
ITS#8324 fix
c4e31434c7773ee54f2ffdeb545e5740f56492a1
Actually, there is no guarantee that Windows will map newly written
data, so we need VirtualAlloc even for non-WRITEMAP.
Hallvard Furuseth [Tue, 12 Jan 2016 22:18:06 +0000 (23:18 +0100)]
lmdb.h Caveats: Reserved vs. actual mem/disk usage
Howard Chu [Tue, 12 Jan 2016 07:13:48 +0000 (07:13 +0000)]
ITS#8346 fix typo in prev commit
Howard Chu [Mon, 11 Jan 2016 20:25:33 +0000 (20:25 +0000)]
ITS#8347 fix off-by-one in VL32 purge
Howard Chu [Mon, 11 Jan 2016 20:17:42 +0000 (20:17 +0000)]
ITS#8346 free last txn0->mt_rpages in env_close
Howard Chu [Thu, 7 Jan 2016 18:28:29 +0000 (18:28 +0000)]
Add MDB_PREV_MULTIPLE
Logical counterpart to GET_MULTIPLE, NEXT_MULTIPLE
Howard Chu [Sat, 2 Jan 2016 12:19:42 +0000 (12:19 +0000)]
Fixups for env_copy with large files
wsize was being truncated to 32bits on Windows.
Only try to write 1GB at a time on Windows64;
larger writes fail with ERROR_WORKING_SET_QUOTA.
Howard Chu [Mon, 28 Dec 2015 20:31:09 +0000 (20:31 +0000)]
MDB_VL32 fix cursor_unref - ignore cursor with empty stack
Howard Chu [Mon, 28 Dec 2015 19:33:51 +0000 (19:33 +0000)]
ITS#8342 MDB_VL32/WIN32 - close file mapping handle in env_close
Howard Chu [Mon, 28 Dec 2015 19:37:42 +0000 (19:37 +0000)]
MDB_VL32 tweak prev commit
Howard Chu [Wed, 23 Dec 2015 23:02:06 +0000 (23:02 +0000)]
MDB_VL32 - prevent mixing with non-VL32 builds
Howard Chu [Tue, 22 Dec 2015 20:24:26 +0000 (20:24 +0000)]
MDB_VL32 - Fix me_rpmutex usage
Howard Chu [Tue, 22 Dec 2015 18:30:41 +0000 (18:30 +0000)]
MDB_RESERVE doc
Add mdb_put text to mdb_cursor_put description for people who
fail to draw logical conclusions.
Howard Chu [Tue, 22 Dec 2015 06:13:25 +0000 (06:13 +0000)]
MDB_VL32 - fix prev commit
Only tweak length for read-only envs
Howard Chu [Tue, 22 Dec 2015 05:20:33 +0000 (05:20 +0000)]
MDB_VL32 - fix for Win32 read-only envs
We can't map with MEM_RESERVE because that requires write access
to the underlying file/section. Mapping with the default (MEM_COMMIT)
requires that we don't map past the end of the file.
Howard Chu [Tue, 22 Dec 2015 04:22:01 +0000 (04:22 +0000)]
MDB_VL32 - track overflow pages too
Otherwise they'll fill up the txn's pageref list when traversing
a DB with lots of overflow records.
Hallvard Furuseth [Sun, 23 Aug 2015 20:00:14 +0000 (22:00 +0200)]
mdb_page_alloc(): Use original snapshot of freeDB.
I can't help thinking this should be safer, and useful somewhere.
Hallvard Furuseth [Sun, 23 Aug 2015 19:10:52 +0000 (21:10 +0200)]
Prep for next commit: C_WRITEMAP, C_ORIG_RDONLY.
No change in behavior.
Hallvard Furuseth [Sun, 23 Aug 2015 18:33:02 +0000 (20:33 +0200)]
Pass cursor to mdb_page_get(), mdb_node_read().
No change in behavior.
Howard Chu [Mon, 21 Dec 2015 02:36:20 +0000 (02:36 +0000)]
ITS#8324 fix for WRITEMAP
We called FlushViewOfFile with (map,mapsize) which worked fine
when we had allocated the entire map already. Now we have to make
sure to only flush as much as was actually written. Add a numpgs
argument to tell how much to flush in env_sync0().
Howard Chu [Sun, 20 Dec 2015 00:22:13 +0000 (00:22 +0000)]
Note that we're prepping for 1.0
Howard Chu [Sat, 19 Dec 2015 23:58:39 +0000 (23:58 +0000)]
MDB_VL32 support 64 bit DBs on 32 bit processors
Howard Chu [Sat, 19 Dec 2015 23:51:31 +0000 (23:51 +0000)]
MDB_VL32 preparation
Use 64 bit types consistently. This keeps database structures the
same size for MDB_VL32 and native 64 bit builds.
Howard Chu [Sat, 19 Dec 2015 22:57:00 +0000 (22:57 +0000)]
Fix typos
Howard Chu [Sat, 19 Dec 2015 22:53:26 +0000 (22:53 +0000)]
Add Getting Started doc
Howard Chu [Tue, 15 Dec 2015 18:45:34 +0000 (18:45 +0000)]
ITS#7992 cleanup
check for utf8_to_utf16 failures
Hallvard Furuseth [Sat, 12 Dec 2015 18:25:06 +0000 (19:25 +0100)]
mdb_dbi_open(): Catch strdup failure
Howard Chu [Tue, 8 Dec 2015 19:35:59 +0000 (19:35 +0000)]
ITS#8324 fix for read-only envs
Howard Chu [Tue, 8 Dec 2015 18:17:24 +0000 (18:17 +0000)]
ITS#8336 fix page_search_root assert on FreeDB
Let "illegal" branch pages thru on the FreeDB - the condition
is only temporary and will be fixed by the time rebalance finishes.
Hallvard Furuseth [Tue, 8 Dec 2015 15:43:55 +0000 (16:43 +0100)]
ITS#8334 Fix MDB_APPENDDUP vs. rewrite(single item)
Howard Chu [Sun, 6 Dec 2015 20:54:23 +0000 (20:54 +0000)]
ITS#8324 additional tweaks
VirtualAlloc is only needed with WRITEMAP. Regular writes
already extend the mmap implictly.
Sebastien Launay [Thu, 3 Dec 2015 18:42:36 +0000 (10:42 -0800)]
ITS#8330 Fix robust mutex detection for glibc 2.10 and 2.11
pthread_mutexattr_setrobust and pthread_mutex_consistent are provided since 2.12 not 2.10:
https://sourceware.org/git/?p=glibc.git;a=commit;h=
402cd98775db1478f64c9b0dbe00664b89eb2773
https://sourceware.org/git/?p=glibc.git;a=commit;h=
78ee21859939ff75ccf8bbe00499b0c462df2e2d
Howard Chu [Tue, 1 Dec 2015 14:10:12 +0000 (14:10 +0000)]
ITS#8169 more Makefile tweaks
Use all the same vars as main OpenLDAP makefiles