Panu Matilainen [Mon, 5 Nov 2012 08:22:50 +0000 (10:22 +0200)]
Update library current + age (ie libs are now 3.1.0)
- We haven't removed or changed any interfaces in a way that would
require full soname bump, only a handful of new interfaces have
been added.
- There aren't actually any new interfaces in librpmbuild or librpmsign
but for sanity and consistency's sake they're all updated...
Panu Matilainen [Mon, 5 Nov 2012 08:09:18 +0000 (10:09 +0200)]
Update translations from Transifex
Panu Matilainen [Thu, 1 Nov 2012 10:09:30 +0000 (12:09 +0200)]
Enable built-in %dirname macro
- This might've clashed with %dir in spec %files at some point as
indicated by the comments, but that should no longer be the case.
Panu Matilainen [Thu, 1 Nov 2012 09:45:44 +0000 (11:45 +0200)]
Rename -debug to -debuginfo to match what everybody uses anyway
- Practically everybody renames the debug information sub-packages
from -debug to -debuginfo, might as well do so upstream already.
Intended to do this like five years ago but kept forgetting...
thierry.vignaud@gmail.com [Fri, 19 Oct 2012 08:08:35 +0000 (10:08 +0200)]
make "canonicalization(...)shrank by one character" error message more usefull
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Tim Landscheidt [Wed, 24 Oct 2012 23:18:57 +0000 (23:18 +0000)]
Don't call "rpm --addsign" if rpmbuild didn't create RPMs.
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Panu Matilainen [Thu, 1 Nov 2012 07:45:15 +0000 (09:45 +0200)]
Hide the strpool-related rpmts/rpmfi/rpmds interfaces out of sight for now
- In the package/transaction related things the strpool is more of
an internal implementation detail than an end-goal in itself, move
string pool related interfaces of rpmts, rpmfi and rpmds to
internal-only APIs for now. The kind interfaces we'll want to eventually
export a) dont exist yet and b) are likely to be something very different.
- The string pool itself remains exported however, its a handy data
structure for all sorts of things and both librpm and librpmbuild
heavily use it already.
Panu Matilainen [Wed, 31 Oct 2012 07:43:38 +0000 (09:43 +0200)]
Use private NSS context if NSS supports it (RhBug:871485)
- Older NSS versions operate on global context, which can cause
all sorts of trouble when an API user tries to use NSS for their
own purposes: eg they might want to use NSS databases which is not
possible once we've initialized NSS with NSS_NoDB_Init(). Further
background on the subject at https://wiki.mozilla.org/NSS_Library_Init
- Use private private NSS context when possible (NSS >= 3.12.5) to
avoid such clashes, but keep support for older versions for now.
Panu Matilainen [Tue, 30 Oct 2012 10:38:20 +0000 (12:38 +0200)]
Handle EINTR in Fread() and Fwrite()
- Not all systems automatically restart signal-interrupted operations,
handle this centrally since its so easy to do. Also related to
RhBug:580974.
Panu Matilainen [Tue, 30 Oct 2012 09:36:56 +0000 (11:36 +0200)]
Handle partial reads and errors in package IO (RhBug:802839, RhBug:580974)
- There are no guarantees Fread() will return all of the requested size:
it can return partial data eg on signals and pipe descriptors. Introduce
a helper function to handle this centrally for all package read IO,
effectively reintroducing timedRead() but without the caveats:
timedRead() did not work on compressed streams, did not handle
eg EINTR correctly and while really being an internal helper,
was exported in the API.
Panu Matilainen [Tue, 30 Oct 2012 07:29:46 +0000 (09:29 +0200)]
Fix missing error on --import on bogus key file (RhBug:869667)
- When the "BEGIN PGP" marker is not found at all, we would silently
exit with success when trying to import utter garbage, such as
rpmkeys --import /bin/bash (not that I consider bash as gargabe ;)
Panu Matilainen [Fri, 26 Oct 2012 09:15:07 +0000 (12:15 +0300)]
Remove transition-time leftover pool unfreezing
- Nobody is freezing our pool so there's no need to unfreeze it either...
Panu Matilainen [Fri, 26 Oct 2012 08:47:31 +0000 (11:47 +0300)]
Eliminate static "misc stuff pool" from rpmfi
- Prior to string pool existence, the static "caches" were kinda
necessary for sharing the relatively static content of user/group
names and file languages, but this is nothing but an unfreeable
block of memory at this point. Just use the same pool as everything
else - whether private or shared.
Panu Matilainen [Fri, 19 Oct 2012 13:20:02 +0000 (16:20 +0300)]
Eliminate copy-paste nibble(), use rnibble() instead
Panu Matilainen [Fri, 19 Oct 2012 12:45:14 +0000 (15:45 +0300)]
Default to NSS crypto though...
- NSS is a big and quirky monster for our needs but that's what we've
been defaulting to for the last years, and then there are those
certifications...
- Also default to external beecrypt even if --with-beecrypt is used
Michael Schroeder [Fri, 19 Oct 2012 12:43:09 +0000 (15:43 +0300)]
Add support for beecrypt as an alternative crypto implementation
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Panu Matilainen [Fri, 19 Oct 2012 10:54:11 +0000 (13:54 +0300)]
Dont bother with digest and link comparisons if size differs, part II
- If link or regular file sizes differ, they cannot possibly be
identical. Saves us from doing bunch of pointless string and
memory comparisons on what's a very busy path - not exactly a huge
with but measurable nevertheless.
Panu Matilainen [Fri, 19 Oct 2012 10:45:04 +0000 (13:45 +0300)]
Dont bother with %config digests on conflicts if size differs
- Files of different size cannot possibly be identical, dont bother
with expensive digest calculation.
Panu Matilainen [Thu, 11 Oct 2012 14:57:10 +0000 (17:57 +0300)]
Fix noarch __isa_* macro filter in installplatform (RhBug:865436)
- The filter wasn't doing what it was supposed to due to extra single
quotes getting inserted, causing "rpmbuild --target noarch foo.spec"
to whine about empty macro bodies. This is a regression introduced
in rpm 4.10, commit
07ec480c180e4005a629242b8f9f8ab640e3e950 to be
precise.
Panu Matilainen [Thu, 11 Oct 2012 12:14:04 +0000 (15:14 +0300)]
Eliminate now unnecessary WITH_SELINUX conditionals in sepolicy plugin
Panu Matilainen [Thu, 11 Oct 2012 12:09:15 +0000 (15:09 +0300)]
Only try compiling sepolicy plugin if selinux is enabled
Panu Matilainen [Thu, 11 Oct 2012 12:04:54 +0000 (15:04 +0300)]
Missing include in string pool
- When compiled without selinux support, stdlib.h doesn't get
included here. Wtf?
Florian Festi [Thu, 11 Oct 2012 10:20:06 +0000 (12:20 +0200)]
Revert "Update man page to reflect that rpm no longer uses MD5 as a default digest algorithm"
This reverts commit
b040b323d6fac32f7f4fa77fd5cb60c7967fc53f.
The change does only affect file digests.
Florian Festi [Wed, 10 Oct 2012 12:59:32 +0000 (14:59 +0200)]
Add rpm --setperms and rpm --setugids to the man page
Florian Festi [Wed, 10 Oct 2012 11:17:09 +0000 (13:17 +0200)]
Update man page to reflect that rpm no longer uses MD5 as a default digest algorithm
Anders F Bjorklund [Wed, 12 Sep 2012 21:26:38 +0000 (23:26 +0200)]
Add lua 5.2 support.
Add compatibility support for both lua-5.1 and lua-5.2,
assuming that the LUA_COMPAT might have been disabled.
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Panu Matilainen [Mon, 8 Oct 2012 06:27:18 +0000 (09:27 +0300)]
Don't bother calculating digests of %ghost %config files
- Calculating digest of %ghost config is useless as there's nothing
to compare the result to. Also we never take backups of %ghost
configuration anyway, so this is a total waste of time. One common
case of %ghost %config is the rpmdb (environment and all) which
can be rather large and calculating digests can take several seconds,
only for the results to be thrown away unused.
- There are some cases where it might be reasonable to back up %ghost
%config (eg if it gets replaced by non-config), but
rpmfiConfigConflictIndex() doesn't have sufficient context to figure
that out. For now, preserve the traditional simple rule: no backups
for ghosts, ever.
Panu Matilainen [Fri, 5 Oct 2012 08:38:26 +0000 (11:38 +0300)]
Use rpmal lookups for already added obsoletions too
- Related to commit
bee348b5d101a0ea6039b56c39e8ddc1cfc09a1a,
use the newly added obsoletes hash to lookup already added
obsoleters. Eliminates the dumb linear lookup and is unsurprisingly
a whole lot faster on larger transactions.
Panu Matilainen [Fri, 5 Oct 2012 08:35:11 +0000 (11:35 +0300)]
Add a hash + lookup function on obsoletes to rpmal
- Similar lazily created hash as provides for fast obsoletes lookups.
This is so similar the provides that creation etc functions should
be unified, but leaving that exercise till later.
Panu Matilainen [Fri, 5 Oct 2012 08:34:22 +0000 (11:34 +0300)]
Rename rpmalProvidesHash to rpmalDepHash
- The same hash type is valid for any ds and we'll be adding more...
Panu Matilainen [Fri, 5 Oct 2012 08:25:49 +0000 (11:25 +0300)]
Avoid changing the file info sets when adding to rpmal file hash
- Use indexed access to the file info sets so we're not mucking
around with somebody elses rpmfi iterator index (currently not an
issue but still...)
Panu Matilainen [Fri, 5 Oct 2012 07:16:44 +0000 (10:16 +0300)]
Avoid changing the dependency sets when looking up from rpmal
- Use indexed access and comparison to avoid mucking around with
eg transaction element dependency sets while the callers are walking
over them.
Panu Matilainen [Fri, 5 Oct 2012 07:13:09 +0000 (10:13 +0300)]
Avoid changing the dependency sets when adding to rpmal hash
- Use indexed access to the dependencies so we're not mucking with
the rpmds iterator index behind anybodys back, this could affect
all sorts of things but miraculously nothing is hitting it atm...
Panu Matilainen [Fri, 5 Oct 2012 07:09:50 +0000 (10:09 +0300)]
Guard against NULL ds in the rpmds indexed getters
- NULL dependency sets are all over the place as NULL ds means
"no dependencies of this kind", whereas for eg rpmfi NULL is an
error.
Panu Matilainen [Fri, 5 Oct 2012 07:07:52 +0000 (10:07 +0300)]
Export the indexed rpmds accessors internally
- Add an internal header for rpmds too to allow adding interfaces we
dont necessarily want to export in the public API, make the indexed
accessors available internally.
Panu Matilainen [Fri, 5 Oct 2012 07:04:04 +0000 (10:04 +0300)]
Add indexed access variants for rpmdsColor() and rpmdsCompare()
- Various places in rpm need random access to the dependency sets,
save and restore on somebody elses "iterator index" just doesn't
cut it. This is merely preliminaries for further changes.
Panu Matilainen [Thu, 4 Oct 2012 12:48:43 +0000 (15:48 +0300)]
Be as lazy as possible wrt rpmal hash creation
- Separate provides and files hash creation, delay both until the last
moment before first valid lookup. In practise, this means the provides
hash is created early due to lookups from rpmtsAddInstallElement(),
but the big bad file hash creation is delayed until the entire
transaction set has been more-or-less populated. Which means we have
a better idea about the necessary hash table size, meaning fewer
hash resizes, resulting in good deal faster execution with no
downsides - if something happened to trigger an early file lookup
it'll all still work, just slower.
Panu Matilainen [Tue, 2 Oct 2012 13:40:45 +0000 (16:40 +0300)]
Use rpmal lookups for finding redundancies in the added set
- Replace the dumb linear search across all elements on each
addition with (filtered) rpmal lookups where possible. rpmal
doesn't (yet) have obsoletes information so for already added
obsoletions we have no choice but to walk the walk.
- As a result, findPos() is hugely faster for large transactions
but rpmal hashes are now generated on the fly for everything
instead of doing it all at once before the actual dependency
checks / ordering, which will cost us something in terms of
hash table resizes.
Panu Matilainen [Tue, 2 Oct 2012 13:17:05 +0000 (16:17 +0300)]
Dont bother with rpmRelocateFileList() if relocations have been specified
- rpmRelocateFileList() doesn't modify anything when no relocations
are to be done, but what it does is not exactly free, unnecessarily
calling it is dumb.
Panu Matilainen [Tue, 2 Oct 2012 09:12:20 +0000 (12:12 +0300)]
Only return non-NULL from rpmalAllSatisfiesDepends() on real matches
- The provides hash lookup can and does return hits that dont actually
satisfy the dependency. Dont bother callers with apparent hits
(ie non-NULL returns) when nothing actually matches the dependency.
Panu Matilainen [Tue, 2 Oct 2012 07:50:10 +0000 (10:50 +0300)]
Export rpmalAllSatisfiesDepend() internally (again)
- For some uses, we need to be more selective about providers...
Panu Matilainen [Tue, 2 Oct 2012 07:37:36 +0000 (10:37 +0300)]
Pass the newly create ts element instead of header to findPos()
- Avoids a little bit of extra work, we already have the relevant
bits of information in the rpmte and grabbing them from there is
cheaper than looking up stuff from headers. Also avoids creating
another copy of the new elements obsoletes dependency set unnecessarily.
Panu Matilainen [Mon, 1 Oct 2012 10:19:59 +0000 (13:19 +0300)]
Eliminate all the now unnecessary fsm->ix save-and-restores
- Now that the relevant places are accepting file index as argument,
we no longer need to save and restore fsm->ix in all the places
dealing with hard links.
Panu Matilainen [Mon, 1 Oct 2012 10:15:30 +0000 (13:15 +0300)]
Pass file index as argument to fsm fsmMapPath()
- Normally this is just the current index, but when writing links
its something else and makes sense as an argument (so we dont
need to save and restore fsm->ix when doing something different)
Panu Matilainen [Mon, 1 Oct 2012 10:01:33 +0000 (13:01 +0300)]
Pass file index as argument to fsm fsmCommit()
- Normally this is just the current index, but when writing links
its something else and makes sense as an argument (so we dont
need to save and restore fsm->ix when doing something different)
Panu Matilainen [Mon, 1 Oct 2012 09:54:15 +0000 (12:54 +0300)]
Pass file index as argument to fsm writeFile()
- Normally this is just the current index, but when writing links
its something else and makes sense as an argument (so we dont
need to save and restore fsm->ix when doing something different)
Panu Matilainen [Mon, 1 Oct 2012 09:30:28 +0000 (12:30 +0300)]
Eliminate current hardlink set from fsm struct
- Return the hard link set from saveHardLink() when ready into a local
variable in the only place that cares: rpmPackageFilesInstall().
Another pointless and hard-to-follow fsm-global state variable
gone...
Panu Matilainen [Mon, 1 Oct 2012 09:16:18 +0000 (12:16 +0300)]
Pinpoint the one place where fsm->li is modified on real purpose
- saveHardLinks() is the only place where fsm->li value matters on
return: during installation its used for "returning" the current
link set when all the links in that set have been seen so they
are ready for creating.
Panu Matilainen [Mon, 1 Oct 2012 09:04:29 +0000 (12:04 +0300)]
Avoid using fsm->li directly when creating pending hard links
- fsmMakeLinks() operates on the current link at hand, doesn't walk
the links or anything... pass the link as an argument from the sole
caller and operate on that.
Panu Matilainen [Mon, 1 Oct 2012 08:53:48 +0000 (11:53 +0300)]
Avoid using/modifying fsm->li when committing hardlinks
- Using a "global" variable for local... more of the same:
fsmCommitLinks() gets called at the end of each round of the install
loop if there are links, it does the actual link discovery on its
own and fsm->li state will get rewritten by saveHardLink() at the
start of the next round.
Panu Matilainen [Mon, 1 Oct 2012 08:47:29 +0000 (11:47 +0300)]
Avoid using/modifying fsm->li when writing links to payload
- Using a "global" variable for local iteration is just... this gets
called once at the end of package build, fsm->li carries no
global state here.
Panu Matilainen [Mon, 1 Oct 2012 08:39:47 +0000 (11:39 +0300)]
Avoid using/modifying fsm->li when freeing hard link sets
- Using a "global" variable for local iteration is...
Panu Matilainen [Mon, 1 Oct 2012 08:37:01 +0000 (11:37 +0300)]
Avoid using/modifying fsm->li on hard link checking
- Using a "global" variable for local iteration is just dumb...
Panu Matilainen [Mon, 1 Oct 2012 08:33:08 +0000 (11:33 +0300)]
Move hardlink saving out of fsmInit()
- Now that we have separate functions for install, erase and build,
handle the hardlink saving locally where it matters.
Panu Matilainen [Mon, 1 Oct 2012 06:54:35 +0000 (09:54 +0300)]
Free hardlink sets centrally in fsmFree()
- Having three places doing the same thing doesn't make a whole
lotta sense...
Panu Matilainen [Mon, 1 Oct 2012 06:23:29 +0000 (09:23 +0300)]
Filter out skipped files on hardlink checking
- Legitimately skipped files (links) must not cause install-errors.
This has always been broken, but the errors were completely ignored
on install prior to rpm 4.10, and now that we're only creating the
first instance of a shared file, secondary arch multilib packages
with hardlinks were causing install failures because of the "missing"
hardlinks here.
- The relevant part here is obviously the XFA_SKIPPING() test, for
which we need the file states. To keep the lines short, grab the
index to a helper variable and replace other fsm->li->filex[i] uses
with that too.
Panu Matilainen [Fri, 28 Sep 2012 09:43:39 +0000 (12:43 +0300)]
Simplify fpLookupSubdir() a bit
- Use string offsets for basename start and end to track the progress,
avoiding +1/-1 adjustments in every damn calculation.
- Reduce the places where new basename is calculated to just one
at the start of the main loop, just adjust the basename start
and end accordingly beforehand.
- This shouldn't change any functionality, just simplify the code
a little bit.
Florian Festi [Fri, 28 Sep 2012 07:23:03 +0000 (10:23 +0300)]
Make string pool strings static in memory
- Use multiple chunks that get allocated as old ones get filled up
instead of reallocating, store direct pointers to the strings in
the id array.
- This prevents nasty surprises when previously retrieved pointer
to a pool string goes invalid underneath us due to somebody
adding a new string, and restores former rpm API behavior:
string pointers retrieved from eg rpmds and rpmfi are valid for
the entire lifetime of those objects.
Panu Matilainen [Thu, 27 Sep 2012 12:56:56 +0000 (15:56 +0300)]
Minimally resurrect fpLookupSubdir() functionality
- fpLookupSubdir() has been broken for a long, long time, probably
because some subtle extra/missing slash issue. It got totally
wrecked by the pool changes though: there are heading and trailing
slashes everywhere and the calculations were off-by-one to every
possible direction because of that. And the previous attempts
(eg commit
566a332c6907a0969ea8f79a4c7a62bca2d1f1b4) makes
even a bigger mess of it.
- This is far messier than it needs to be, but for now going for
minimal resurrection rather than rewrite it all at once. It's also
not quite right either, but it does now actually detect at the conflicts
it was always supposed to.
Panu Matilainen [Thu, 27 Sep 2012 10:11:03 +0000 (13:11 +0300)]
Add doxygen documentation to string pool API
Panu Matilainen [Thu, 27 Sep 2012 10:09:39 +0000 (13:09 +0300)]
Fixup a copy-paste error in rpmdsPool() doxygen markup
Panu Matilainen [Thu, 27 Sep 2012 09:18:39 +0000 (12:18 +0300)]
Fixup string pool prototype argument names to match implementation
- Just a simple s/sidpool/pool/ to match the implementation, "sidpool"
is a leftover from early draft version that somehow made its way
to the master tree.
Panu Matilainen [Thu, 27 Sep 2012 08:43:17 +0000 (11:43 +0300)]
Remove leftover, no longer valid comment
Panu Matilainen [Wed, 26 Sep 2012 09:52:13 +0000 (12:52 +0300)]
Freeze the rpmfc string pools once all additions are done
- Memory use isn't typically that critical during build, but saving
a little bit of it doesn't hurt anyway...
Panu Matilainen [Wed, 26 Sep 2012 09:32:32 +0000 (12:32 +0300)]
Dont waste time with argi for rpmfc file class indexes
- argiAdd() isn't as costly as argvAdd(), but it still involves unnecessary
reallocation on every addition as the number of files is predetermined,
and each file has an associated class index.
- Additionally fixes a mostly harmless glitch introduced in commit
65e616cc9fdd00f523939088fab1070021b9f742: the pool id's are one larger
than the indexes we store in headers, take this into account in the
debug output.
Panu Matilainen [Wed, 26 Sep 2012 09:02:27 +0000 (12:02 +0300)]
Dont waste time with argi for rpmfc file colors
- argiAdd() isn't as costly as argvAdd(), but it still involves unnecessary
reallocation on every addition as the number of files is predetermined,
and each file has an associated color (whether its zero or something else)
Panu Matilainen [Wed, 26 Sep 2012 08:31:41 +0000 (11:31 +0300)]
Dont waste time with argv for rpmfc file names
- argvAdd() gets more and more costly as the number of files increase,
use a plain old string array where one is called for: the size
is predetermined (from another argv) and we're just copying the
strings for internal storage here. We do need to pay a little bit
more attention to details now though.
Panu Matilainen [Wed, 26 Sep 2012 07:26:53 +0000 (10:26 +0300)]
Use a string pool for the build-time file dependency string storage
- The pool is mostly just a dumb storage space for this case, but
insertion to pool is far more efficient than argv search, add,
sort operation. The order does not matter as the actual information
is encoded in the string and then painfully parsed out again. This
could really use a special-purpose data structure for sanity but...
Panu Matilainen [Wed, 26 Sep 2012 06:56:18 +0000 (09:56 +0300)]
Use a string pool to build file class dictionary and indexes
- The string pool is a natural fit for this and much more efficient
when the number of files (and classes) is large.
- This does have an effect on the generated classdict array: it's no
longer sorted, and we no longer always (possibly unnecessarily)
insert an empty string and "directory" in it, but only when they
actually exist. Neither change makes any difference for usage,
unless some 3rd party software is making assumptions about the
classdict contents they shouldn't be making (very unlikely anyway)
Panu Matilainen [Wed, 26 Sep 2012 06:41:30 +0000 (09:41 +0300)]
Cosmetics: properly indent the rpmfc debug foobar
Panu Matilainen [Wed, 26 Sep 2012 05:34:40 +0000 (08:34 +0300)]
Add function to get number of unique strings in the pool
Panu Matilainen [Mon, 24 Sep 2012 08:57:55 +0000 (11:57 +0300)]
Switch back to early added packages rpmal populating
- Basically reverts commit
d10a9941326f7d397428c8ab0b4ba571cfc6c184
which was just a temporary transition-period thing. Moving pointers
dont bother us anymore...
Panu Matilainen [Mon, 24 Sep 2012 08:56:34 +0000 (11:56 +0300)]
Test obsoletions within the transaction set, both ways
- Similar to commit
f7c1eb5ef041b9b06cbb81dac7d7972a602cad44 but
on obsoletions
Panu Matilainen [Mon, 24 Sep 2012 08:51:49 +0000 (11:51 +0300)]
Test different versions in same transaction both ways
- Different versions can happen two ways: older first or older last,
test both cases
- Add verbose output for install to get the warning messages too and
require that matches as well
Panu Matilainen [Fri, 21 Sep 2012 12:49:11 +0000 (15:49 +0300)]
Take advantage of rpmstrPoolStreq() in rpmdsCompare()
- This gives quite a speedup for dependency checking as within
transaction, all the dependency sets come from the same pool
and making this just an integer comparison.
Florian Festi [Tue, 18 Sep 2012 08:55:51 +0000 (10:55 +0200)]
Change poolHash to use internal collision resolution
Panu Matilainen [Wed, 19 Sep 2012 10:01:59 +0000 (13:01 +0300)]
Eliminate strdup() in doLookupId()
- Take advantage of the length-aware pool string->id lookup to avoid
the need to copy and locally modify the canonized dirname. Makes
the code cleaner and a little bit faster too.
- There are further possibilities in this direction, canonDir()
could return an id instead of malloced memory but that doesn't
remove the need for temporary bugg^H^Hffer to clean up the dir.
Panu Matilainen [Wed, 19 Sep 2012 09:50:31 +0000 (12:50 +0300)]
Lift directory name canonicalization into a helper function
- Suddenly it all seems so much clearer...
- Also try to better handle the theoretical realpath() errors: on
failure the buffer contents are undefined and we shouldn't look
at them, free the buf and return NULL.
Panu Matilainen [Wed, 19 Sep 2012 07:42:59 +0000 (10:42 +0300)]
Aargh, stupid thinko in rpmstrPoolStrlen() last id special case
- At the largest id, the end boundary is data, not offset size... doh
Panu Matilainen [Tue, 18 Sep 2012 16:04:40 +0000 (19:04 +0300)]
Unbreak fingerprinting when called twice for the same transaction set
- Fixes regression (yum updates barf with conflicts) from commit
3b492620fb35a21d05b975e31130dc071f6fd8e2 which doesn't affect rpm
itself but breaks some API users like yum that first run a
test-transaction and if that succeeds, the same transaction is run
for real. In that case we need to redo the whole fingerprint dance
from scratch: throw away any former results and redo.
- Blaming Fedora 18 alpha preparations for not noticing this earlier:
there have been no updates to F16/F17 in almost a week, effectively
disabling a part of the regular rpm development QA :P
Panu Matilainen [Tue, 18 Sep 2012 07:40:18 +0000 (10:40 +0300)]
Oops, refactoring error in commit
327701572ff912413ec9564d29d946b4ab21a9f3
- sizeof() doesn't work very well for malloced arrays... not that it
matters a whole lot, this isn't a path that will ever be executed
afaict.
Panu Matilainen [Tue, 18 Sep 2012 07:03:13 +0000 (10:03 +0300)]
Ensure fingerprint subdirs have heading slashes
- Similar to commit
ad99f4e84a5fea05edca59257b6e00d91d6c8a08 but at the
other end of the name: not all subdirs will exist in the pool already
but none of them exists without the heading slash. Saves another handful
of megs on larger transactions.
Panu Matilainen [Tue, 18 Sep 2012 06:39:38 +0000 (09:39 +0300)]
Ensure directories have trailing slashes in doLookupId()
- In the old days, stripping trailing slashes made sense as its useless
as it is and stripping it saved a bit of memory, but now with the
directories in the pool things are different: while not all paths
we'll see here necessarily exist in the pool already, NONE of them
exists without the trailing slash, so stripping it out wastes gobs
of memory. Simplifies the code a bit and saves several megs of memory
on larger transactions, what's not to like?
Panu Matilainen [Tue, 18 Sep 2012 04:29:43 +0000 (07:29 +0300)]
Further doLookupId() cleanup
- Refer to dirName when we haven't yet cleaned it up to clarify uses,
only initialize the const cleaned dir name pointer after we've
successfully cleaned it up. Rename cleanDirName to cdn to keep it
sweet and short, and suggest the relation to cdnl and cdnbuf.
Panu Matilainen [Tue, 18 Sep 2012 03:56:46 +0000 (06:56 +0300)]
Elimiate one of the umphteen directory variables in doLookupId()
- Use dynamically allocated cdnbuf in the non-absolute path case (when
exactly that occurs is another question) too instead of the on-stack
dir[], making the two paths a bit more similar.
Panu Matilainen [Tue, 18 Sep 2012 03:30:59 +0000 (06:30 +0300)]
Avoid redundant ds -> id -> string lookups in rpmalAllFileSatisfiesDepend()
- We need to grab the dependency string in rpmalAllSatisfiesDepend()
already to see whether its a file dependency or not, just pass that
to the file deps to avoid redundant work.
Panu Matilainen [Tue, 18 Sep 2012 03:27:57 +0000 (06:27 +0300)]
Dont bother with separate notifications on multiple file dep matches
Panu Matilainen [Tue, 18 Sep 2012 03:26:34 +0000 (06:26 +0300)]
Avoid the string copy in rpmalAllFileSatisfiesDepend() now that we can
Panu Matilainen [Tue, 18 Sep 2012 03:01:43 +0000 (06:01 +0300)]
Make pool string->id operations properly length-aware
- Allow looking up and inserting partial key strings, this is useful
in various cases where previously a local copy was needed for
\0-terminating the key in the caller.
- Take advantage of rstrlenhash() in rpmstrPoolId(), previously the
length was only interesting when adding so we wasted a strlen()
on every call when the string was already in the pool.
Panu Matilainen [Tue, 18 Sep 2012 01:46:10 +0000 (04:46 +0300)]
Add length aware variant(s) of string hashing
- Being able to hash partial strings is needed for allowing string pool
to operate on partial strings...
Panu Matilainen [Tue, 18 Sep 2012 01:34:38 +0000 (04:34 +0300)]
Enhanced string hash to permit calculating string length on the same call
- String hashing needs to walk the entire string anyhow, might as well
take advantage of this and have it return the string length to avoid
having to separately call strlen() in the cases where this matters.
- Move the implementation into rpmstrpool.c for inlining possibilities,
rstrhash() is now just a wrapper to rstrlenhash(). The generic
hash implementation could not take advantage of this anyway really.
Panu Matilainen [Tue, 18 Sep 2012 01:15:56 +0000 (04:15 +0300)]
Dont assume \0 terminated strings in rpmstrPoolPut()
- Before this, the slen argument was only good for avoiding an extra
strlen() but being able to handle shove and lookup partial strings
without local copy+modify in callers is handy, this is one of
the prerequisites for that.
Panu Matilainen [Mon, 17 Sep 2012 12:52:59 +0000 (15:52 +0300)]
Use pool id's for hash table key, lookup strings from pool as needed
- The pool itself can address its contents by id alone, storing
pointers to the strings only hurts as reallocation moving the
data blob requires rehashing the whole thing needlessly.
- We now store just the key id in the hash buckets, and lookup the
actual string for comparison from the pool. This avoids the
need to rehash on realloc and saves memory too, and this is one of
the biggest reasons for wanting a separate hash implementation for
the string pool. Incidentally, this is how libsolv does it too.
- Individual bucket allocation becomes rather wasteful now: a bucket
stores a single integer, and a single pointer to the next bucket,
a pointer which can be twice the size of the key data it holds.
Further tuning and cleaning up after the marriage of these two
datatypes left after the honeymoon is over...
Panu Matilainen [Mon, 17 Sep 2012 12:32:36 +0000 (15:32 +0300)]
Move the string pool struct definition earlier so we can reference it...
Panu Matilainen [Mon, 17 Sep 2012 12:18:01 +0000 (15:18 +0300)]
Inline poolHashfindEntry() into GetEntry(), nothing else needs it
Panu Matilainen [Mon, 17 Sep 2012 12:01:46 +0000 (15:01 +0300)]
Eliminate key comparison and hash function vectors from poolHash
- As the pool is hardwired to single hash type, these dont make
any sense here and the extra indirection will only hurt performance.
Panu Matilainen [Mon, 17 Sep 2012 11:48:21 +0000 (14:48 +0300)]
More poolHash multiple data-value cleanups
- The only data associated with a pool key is a single id, we dont need
an array for that
- Change poolHash get-entry return the id directly instead of pointer array
Panu Matilainen [Mon, 17 Sep 2012 11:43:43 +0000 (14:43 +0300)]
Eliminate redundant data counting from poolHash
- There's a strict 1:1 relation between keys and data in the string
pool, this keeping count of data is pointless.
Panu Matilainen [Mon, 17 Sep 2012 11:30:55 +0000 (14:30 +0300)]
Eliminate unnecessary key and data free-functionality from poolHash
Panu Matilainen [Mon, 17 Sep 2012 11:20:30 +0000 (14:20 +0300)]
Pull a private hash-implementation copy to string pool
- The string pool is more specialized a data structure to be efficiently
handled with the generic hash table implementation in rpmhash.[CH]
and really requires quite a different approach.
- For starters, import a private copy generated roughly with:
gcc -E -DHASHTYPE=poolHash \
-DHTKEYTYPE="const char *" -DHTDATATYPE=rpmsid rpmhash.C
...and clean it up a bit: eliminate unused functions (except for
stats which we'll want to keep for debug purposes), make remaining
functions static and overall tidy up from the mess 'gcc -E' created.
Lots of redundant fluff here still, to be cleaned up gradually...
- This doesn't change anything at all, but opens up the playground
for tuning the pool hash implementation in ways the generic version
could not (at least sanely) be.