Jim Schutt [Fri, 30 Nov 2012 16:15:25 +0000 (09:15 -0700)]
libceph: for chooseleaf rules, retry CRUSH map descent from root if leaf is failed
Add libceph support for a new CRUSH tunable recently added to Ceph servers.
Consider the CRUSH rule
step chooseleaf firstn 0 type <node_type>
This rule means that <n> replicas will be chosen in a manner such that
each chosen leaf's branch will contain a unique instance of <node_type>.
When an object is re-replicated after a leaf failure, if the CRUSH map uses
a chooseleaf rule the remapped replica ends up under the <node_type> bucket
that held the failed leaf. This causes uneven data distribution across the
storage cluster, to the point that when all the leaves but one fail under a
particular <node_type> bucket, that remaining leaf holds all the data from
its failed peers.
This behavior also limits the number of peers that can participate in the
re-replication of the data held by the failed leaf, which increases the
time required to re-replicate after a failure.
For a chooseleaf CRUSH rule, the tree descent has two steps: call them the
inner and outer descents.
If the tree descent down to <node_type> is the outer descent, and the descent
from <node_type> down to a leaf is the inner descent, the issue is that a
down leaf is detected on the inner descent, so only the inner descent is
retried.
In order to disperse re-replicated data as widely as possible across a
storage cluster after a failure, we want to retry the outer descent. So,
fix up crush_choose() to allow the inner descent to return immediately on
choosing a failed leaf. Wire this up as a new CRUSH tunable.
Note that after this change, for a chooseleaf rule, if the primary OSD
in a placement group has failed, choosing a replacement may result in
one of the other OSDs in the PG colliding with the new primary. This
requires that OSD's data for that PG to need moving as well. This
seems unavoidable but should be relatively rare.
This corresponds to ceph.git commit
88f218181a9e6d2292e2697fc93797d0f6d6e5dc.
Signed-off-by: Jim Schutt <jaschut@sandia.gov>
Reviewed-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Fri, 4 Jan 2013 07:30:10 +0000 (15:30 +0800)]
ceph: check mds_wanted for imported cap
The MDS may have incorrect wanted caps after importing caps. So the
client should check the value mds has and send cap update if necessary.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Fri, 4 Jan 2013 06:45:18 +0000 (14:45 +0800)]
ceph: allocate cap_release message when receiving cap import
When client wants to release an imported cap, it's possible there
is no reserved cap_release message in corresponding mds session.
so __queue_cap_release causes kernel panic.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Fri, 4 Jan 2013 06:37:57 +0000 (14:37 +0800)]
ceph: allow revoking duplicated caps issued by non-auth MDS
Allow revoking duplicated caps issued by non-auth MDS if these caps
are also issued by auth MDS.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Fri, 4 Jan 2013 06:28:07 +0000 (14:28 +0800)]
ceph: move dirty inode to migrating list when clearing auth caps
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Fri, 30 Nov 2012 05:49:51 +0000 (13:49 +0800)]
ceph: re-calculate truncate_size for strip object
Otherwise osd may truncate the object to larger size.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Sam Lang [Fri, 28 Dec 2012 17:56:46 +0000 (09:56 -0800)]
ceph: Check for created flag in response from mds
The mds now sends back a created inode if the create request
performed the create. If the file already existed, no inode is
returned in the reply. This allows ceph to set the created flag
in atomic_open so that permissions are properly checked in the case
that the file wasn't created by the create call to the mds.
To ensure compability with previous kernels, a feature for sending
back the inode in the create reply was added, so that the mds will
only send back the inode if the client indicates it supports the
feature.
Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Sam Lang [Wed, 19 Dec 2012 19:44:23 +0000 (09:44 -1000)]
ceph: Check for err on mds request in atomic_open
The error returned by ceph_mdsc_do_request includes errors sending the
request, errors on timeout, or any errors coming from the mds. If
ceph_mdsc_do_request returns an error, the reply struct will most likely
be bogus. We need to bail out and propogate the error instead of
overwriting it.
Signed-off-by: Sam Lang <sam.lang@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Sage Weil [Fri, 28 Dec 2012 02:27:04 +0000 (20:27 -0600)]
libceph: fix protocol feature mismatch failure path
We should not set con->state to CLOSED here; that happens in
ceph_fault() in the caller, where it first asserts that the state
is not yet CLOSED. Avoids a BUG when the features don't match.
Since the fail_protocol() has become a trivial wrapper, replace
calls to it with direct calls to reset_connection().
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Alex Elder [Wed, 26 Dec 2012 16:43:57 +0000 (10:43 -0600)]
libceph: WARN, don't BUG on unexpected connection states
A number of assertions in the ceph messenger are implemented with
BUG_ON(), killing the system if connection's state doesn't match
what's expected. At this point our state model is (evidently) not
well understood enough for these assertions to trigger a BUG().
Convert all BUG_ON(con->state...) calls to be WARN_ON(con->state...)
so we learn about these issues without killing the machine.
We now recognize that a connection fault can occur due to a socket
closure at any time, regardless of the state of the connection. So
there is really nothing we can assert about the state of the
connection at that point so eliminate that assertion.
Reported-by: Ugis <ugis22@gmail.com>
Tested-by: Ugis <ugis22@gmail.com>
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Wed, 26 Dec 2012 20:31:40 +0000 (14:31 -0600)]
libceph: always reset osds when kicking
When ceph_osdc_handle_map() is called to process a new osd map,
kick_requests() is called to ensure all affected requests are
updated if necessary to reflect changes in the osd map. This
happens in two cases: whenever an incremental map update is
processed; and when a full map update (or the last one if there is
more than one) gets processed.
In the former case, the kick_requests() call is followed immediately
by a call to reset_changed_osds() to ensure any connections to osds
affected by the map change are reset. But for full map updates
this isn't done.
Both cases should be doing this osd reset.
Rather than duplicating the reset_changed_osds() call, move it into
the end of kick_requests().
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Wed, 19 Dec 2012 21:52:36 +0000 (15:52 -0600)]
libceph: move linger requests sooner in kick_requests()
The kick_requests() function is called by ceph_osdc_handle_map()
when an osd map change has been indicated. Its purpose is to
re-queue any request whose target osd is different from what it
was when it was originally sent.
It is structured as two loops, one for incomplete but registered
requests, and a second for handling completed linger requests.
As a special case, in the first loop if a request marked to linger
has not yet completed, it is moved from the request list to the
linger list. This is as a quick and dirty way to have the second
loop handle sending the request along with all the other linger
requests.
Because of the way it's done now, however, this quick and dirty
solution can result in these incomplete linger requests never
getting re-sent as desired. The problem lies in the fact that
the second loop only arranges for a linger request to be sent
if it appears its target osd has changed. This is the proper
handling for *completed* linger requests (it avoids issuing
the same linger request twice to the same osd).
But although the linger requests added to the list in the first loop
may have been sent, they have not yet completed, so they need to be
re-sent regardless of whether their target osd has changed.
The first required fix is we need to avoid calling __map_request()
on any incomplete linger request. Otherwise the subsequent
__map_request() call in the second loop will find the target osd
has not changed and will therefore not re-send the request.
Second, we need to be sure that a sent but incomplete linger request
gets re-sent. If the target osd is the same with the new osd map as
it was when the request was originally sent, this won't happen.
This can be fixed through careful handling when we move these
requests from the request list to the linger list, by unregistering
the request *before* it is registered as a linger request. This
works because a side-effect of unregistering the request is to make
the request's r_osd pointer be NULL, and *that* will ensure the
second loop actually re-sends the linger request.
Processing of such a request is done at that point, so continue with
the next one once it's been moved.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Fri, 16 Nov 2012 15:29:16 +0000 (09:29 -0600)]
rbd: get rid of rbd_{get,put}_dev()
The functions rbd_get_dev() and rbd_put_dev() are trivial wrappers
that add no value, and their existence suggests they may do more
than what they do.
Get rid of them.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Alex Elder [Thu, 6 Dec 2012 13:22:04 +0000 (07:22 -0600)]
libceph: register request before unregister linger
In kick_requests(), we need to register the request before we
unregister the linger request. Otherwise the unregister will
reset the request's osd pointer to NULL.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Mon, 17 Dec 2012 18:23:48 +0000 (12:23 -0600)]
libceph: don't use rb_init_node() in ceph_osdc_alloc_request()
The red-black node in the ceph osd request structure is initialized
in ceph_osdc_alloc_request() using rbd_init_node(). We do need to
initialize this, because in __unregister_request() we call
RB_EMPTY_NODE(), which expects the node it's checking to have
been initialized. But rb_init_node() is apparently overkill, and
may in fact be on its way out. So use RB_CLEAR_NODE() instead.
For a little more background, see this commit:
4c199a93 rbtree: empty nodes have no color"
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Mon, 17 Dec 2012 18:23:48 +0000 (12:23 -0600)]
libceph: init event->node in ceph_osdc_create_event()
The red-black node node in the ceph osd event structure is not
initialized in create_osdc_create_event(). Because this node can
be the subject of a RB_EMPTY_NODE() call later on, we should ensure
the node is initialized properly for that.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Thu, 6 Dec 2012 13:22:04 +0000 (07:22 -0600)]
libceph: init osd->o_node in create_osd()
The red-black node node in the ceph osd structure is not initialized
in create_osd(). Because this node can be the subject of a
RB_EMPTY_NODE() call later on, we should ensure the node is
initialized properly for that. Add a call to RB_CLEAR_NODE()
initialize it.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Fri, 14 Dec 2012 22:47:41 +0000 (16:47 -0600)]
libceph: report connection fault with warning
When a connection's socket disconnects, or if there's a protocol
error of some kind on the connection, a fault is signaled and
the connection is reset (closed and reopened, basically). We
currently get an error message on the log whenever this occurs.
A ceph connection will attempt to reestablish a socket connection
repeatedly if a fault occurs. This means that these error messages
will get repeatedly added to the log, which is undesirable.
Change the error message to be a warning, so they don't get
logged by default.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Sat, 8 Dec 2012 01:50:07 +0000 (19:50 -0600)]
libceph: socket can close in any connection state
A connection's socket can close for any reason, independent of the
state of the connection (and without irrespective of the connection
mutex). As a result, the connectino can be in pretty much any state
at the time its socket is closed.
Handle those other cases at the top of con_work(). Pull this whole
block of code into a separate function to reduce the clutter.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Thu, 1 Nov 2012 13:39:26 +0000 (08:39 -0500)]
rbd: don't use ENOTSUPP
ENOTSUPP is not a standard errno (it shows up as "Unknown error 524"
in an error message). This is what was getting produced when the
the local rbd code does not implement features required by a
discovered rbd image.
Change the error code returned in this case to ENXIO.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Thu, 6 Dec 2012 15:37:23 +0000 (09:37 -0600)]
rbd: remove linger unconditionally
In __unregister_linger_request(), the request is being removed
from the osd client's req_linger list only when the request
has a non-null osd pointer. It should be done whether or not
the request currently has an osd.
This is most likely a non-issue because I believe the request
will always have an osd when this function is called.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Fri, 9 Nov 2012 21:05:54 +0000 (15:05 -0600)]
rbd: get rid of RBD_MAX_SEG_NAME_LEN
RBD_MAX_SEG_NAME_LEN represents the maximum length of an rbd object
name (i.e., one of the objects providing storage backing an rbd
image).
Another symbol, MAX_OBJ_NAME_SIZE, is used in the osd client code to
define the maximum length of any object name in an osd request.
Right now they disagree, with RBD_MAX_SEG_NAME_LEN being too big.
There's no real benefit at this point to defining the rbd object
name length limit separate from any other object name, so just
get rid of RBD_MAX_SEG_NAME_LEN and use MAX_OBJ_NAME_SIZE in its
place.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Fri, 7 Dec 2012 15:57:58 +0000 (09:57 -0600)]
libceph: avoid using freed osd in __kick_osd_requests()
If an osd has no requests and no linger requests, __reset_osd()
will just remove it with a call to __remove_osd(). That drops
a reference to the osd, and therefore the osd may have been free
by the time __reset_osd() returns. That function offers no
indication this may have occurred, and as a result the osd will
continue to be used even when it's no longer valid.
Change__reset_osd() so it returns an error (ENODEV) when it
deletes the osd being reset. And change __kick_osd_requests() so it
returns immediately (before referencing osd again) if __reset_osd()
returns *any* error.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Thu, 29 Nov 2012 14:37:03 +0000 (08:37 -0600)]
ceph: don't reference req after put
In __unregister_request(), there is a call to list_del_init()
referencing a request that was the subject of a call to
ceph_osdc_put_request() on the previous line. This is not
safe, because the request structure could have been freed
by the time we reach the list_del_init().
Fix this by reversing the order of these lines.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-off-by: Sage Weil <sage@inktank.com>
Alex Elder [Fri, 16 Nov 2012 15:29:16 +0000 (09:29 -0600)]
rbd: do not allow remove of mounted-on image
There is no check in rbd_remove() to see if anybody holds open the
image being removed. That's not cool.
Add a simple open count that goes up and down with opens and closes
(releases) of the device, and don't allow an rbd image to be removed
if the count is non-zero.
Protect the updates of the open count value with ctl_mutex to ensure
the underlying rbd device doesn't get removed while concurrently
being opened.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
David Zafman [Tue, 4 Dec 2012 03:14:05 +0000 (19:14 -0800)]
libceph: Unlock unprocessed pages in start_read() error path
Function start_read() can get an error before processing all pages.
It must not only release the remaining pages, but unlock them too.
This fixes http://tracker.newdream.net/issues/3370
Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Yan, Zheng [Mon, 19 Nov 2012 02:49:09 +0000 (10:49 +0800)]
ceph: call handle_cap_grant() for cap import message
If client sends cap message that requests new max size during
exporting caps, the exporting MDS will drop the message quietly.
So the client may wait for the reply that updates the max size
forever. call handle_cap_grant() for cap import message can
avoid this issue.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Mon, 19 Nov 2012 02:49:08 +0000 (10:49 +0800)]
ceph: Fix __ceph_do_pending_vmtruncate
we should set i_truncate_pending to 0 after page cache is truncated
to i_truncate_size
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Mon, 19 Nov 2012 02:49:07 +0000 (10:49 +0800)]
ceph: Don't add dirty inode to dirty list if caps is in migration
Add dirty inode to cap_dirty_migrating list instead, this can avoid
ceph_flush_dirty_caps() entering infinite loop.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Mon, 19 Nov 2012 02:49:06 +0000 (10:49 +0800)]
ceph: Fix infinite loop in __wake_requests
__wake_requests() will enter infinite loop if we use it to wake
requests in the session->s_waiting list. __wake_requests() deletes
requests from the list and __do_request() adds requests back to
the list.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Mon, 19 Nov 2012 02:49:04 +0000 (10:49 +0800)]
ceph: Don't update i_max_size when handling non-auth cap
The cap from non-auth mds doesn't have a meaningful max_size value.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
Joe Perches [Thu, 29 Nov 2012 14:37:03 +0000 (08:37 -0600)]
bdi_register: add __printf verification, fix arg mismatch
__printf is useful to verify format and arguments.
Signed-off-by: Joe Perches <joe@perches.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Sage Weil [Wed, 28 Nov 2012 20:28:24 +0000 (12:28 -0800)]
libceph: remove 'osdtimeout' option
This would reset a connection with any OSD that had an outstanding
request that was taking more than N seconds. The idea was that if the
OSD was buggy, the client could compensate by resending the request.
In reality, this only served to hide server bugs, and we haven't
actually seen such a bug in quite a while. Moreover, the userspace
client code never did this.
More importantly, often the request is taking a long time because the
OSD is trying to recover, or overloaded, and killing the connection
and retrying would only make the situation worse by giving the OSD
more work to do.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Cyril Roelandt [Tue, 20 Nov 2012 16:23:07 +0000 (10:23 -0600)]
ceph: fix dentry reference leak in ceph_encode_fh().
dput() was not called in the error path.
Signed-off-by: Cyril Roelandt <tipecaml@gmail.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Sage Weil [Mon, 5 Nov 2012 19:07:23 +0000 (11:07 -0800)]
ceph: Fix i_size update race
ceph_aio_write() has an optimization that marks cap EPH_CAP_FILE_WR
dirty before data is copied to page cache and inode size is updated.
If ceph_check_caps() flushes the dirty cap before the inode size is
updated, MDS can miss the new inode size. The fix is move
ceph_{get,put}_cap_refs() into ceph_write_{begin,end}() and call
__ceph_mark_dirty_caps() after inode size is updated.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Sat, 3 Nov 2012 02:32:37 +0000 (10:32 +0800)]
ceph: Hold caps_list_lock when adjusting caps_{use, total}_count
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
Alex Elder [Wed, 31 Oct 2012 00:40:33 +0000 (19:40 -0500)]
rbd: get additional info in parent spec
When a layered rbd image has a parent, that parent is identified
only by its pool id, image id, and snapshot id. Images that have
been mapped also record *names* for those three id's.
Add code to look up these names for parent images so they match
mapped images more closely. Skip doing this for an image if it
already has its pool name defined (this will be the case for images
mapped by the user).
It is possible that an the name of a parent image can't be
determined, even if the image id is valid. If this occurs it
does not preclude correct operation, so don't treat this as
an error.
On the other hand, defined pools will always have both an id and a
name. And any snapshot of an image identified as a parent for a
clone image will exist, and will have a name (if not it indicates
some other internal error). So treat failure to get these bits
of information as errors.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Wed, 31 Oct 2012 00:40:33 +0000 (19:40 -0500)]
libceph: define ceph_pg_pool_name_by_id()
Define and export function ceph_pg_pool_name_by_id() to supply
the name of a pg pool whose id is given. This will be used by
the next patch.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:42 +0000 (23:34 -0500)]
rbd: get parent spec for version 2 images
Add support for getting the the information identifying the parent
image for rbd images that have them. The child image holds a
reference to its parent image specification structure. Create a new
entry "parent" in /sys/bus/rbd/image/N/ to report the identifying
information for the parent image, if any.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Wed, 31 Oct 2012 00:40:33 +0000 (19:40 -0500)]
rbd: allow null image name
Format 2 parent images are partially identified by their image id,
but it may not be possible to determine their image name. The name
is not strictly needed for correct operation, so we won't be
treating it as an error if we don't know it. Handle this case
gracefully in rbd_name_show().
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Wed, 31 Oct 2012 00:40:33 +0000 (19:40 -0500)]
rbd: allow null image name
We will know the image id for format 2 parent images, but won't
initially know its image name. Avoid making the query for an image
id in rbd_dev_image_id() if it's already known.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Tue, 30 Oct 2012 20:47:17 +0000 (15:47 -0500)]
rbd: encapsulate last part of probe
Group the activities that now take place after an rbd_dev_probe()
call into a single function, and move the call to that function
into rbd_dev_probe() itself.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:42 +0000 (23:34 -0500)]
rbd: define rbd_dev_{create,destroy}() helpers
Encapsulate the creation/initialization and destruction of rbd
device structures. The rbd_client and the rbd_spec structures
provided on creation hold references whose ownership is transferred
to the new rbd_device structure.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:42 +0000 (23:34 -0500)]
rbd: consolidate rbd_dev init in rbd_add()
Group the allocation and initialization of fields of the rbd device
structure created in rbd_add(). Move the grouped code down later in
the function, just prior to the call to rbd_dev_probe(). This is
for the most part simple code movement.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:42 +0000 (23:34 -0500)]
rbd: don't pass rbd_dev to rbd_get_client()
The only reason rbd_dev is passed to rbd_get_client() is so its
rbd_client field can get assigned. Instead, just return the
rbd_client pointer as a result and have the caller do the
assignment.
Change rbd_put_client() so it takes an rbd_client structure,
so follows the more typical symmetry with rbd_get_client().
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:42 +0000 (23:34 -0500)]
rbd: fill rbd_spec in rbd_add_parse_args()
Pass the address of an rbd_spec structure to rbd_add_parse_args().
Use it to hold the information defining the rbd image to be mapped
in an rbd_add() call.
Use the result in the caller to initialize the rbd_dev->id field.
This means rbd_dev is no longer needed in rbd_add_parse_args(),
so get rid of it.
Now that this transformation of rbd_add_parse_args() is complete,
correct and expand on the its header documentation to reflect the
new reality.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 22:25:24 +0000 (17:25 -0500)]
rbd: add reference counting to rbd_spec
With layered images we'll share rbd_spec structures, so add a
reference count to it. It neatens up some code also.
A silly get/put pair is added to the alloc routine just to avoid
"defined but not used" warnings. It will go away soon.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: define image specification structure
Group the fields that uniquely specify an rbd image into a new
reference-counted rbd_spec structure. This structure will be used
to describe the desired image when mapping an image, and when
probing parent images in layered rbd devices. Replace the set of
fields in the rbd device structure with a pointer to a dynamically
allocated rbd_spec.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: have rbd_add_parse_args() return error
Change the interface to rbd_add_parse_args() so it returns an
error code rather than a pointer. Return the ceph_options result
via a pointer whose address is passed as an argument.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: pass and populate rbd_options structure
Have the caller pass the address of an rbd_options structure to
rbd_add_parse_args(), to be initialized with the information
gleaned as a result of the parse.
I know, this is another near-reversal of a recent change...
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: remove snap_name arg from rbd_add_parse_args()
The snapshot name returned by rbd_add_parse_args() just gets saved
in the rbd_dev eventually. So just do that inside that function and
do away with the snap_name argument, both in rbd_add_parse_args()
and rbd_dev_set_mapping().
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: remove options args from rbd_add_parse_args()
They "options" argument to rbd_add_parse_args() (and it's partner
options_size) is now only needed within the function, so there's no
need to have the caller allocate and pass the options buffer. Just
allocate the options buffer within the function using dup_token().
Also distinguish between failures due to failed memory allocation
and failing because a required argument was missing.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: get rid of snap_name_len
The value returned in the "snap_name_len" argument to
rbd_add_parse_args() is never actually used, so get rid of it.
The snap_name_len recorded in rbd_dev_v2_snap_name() is not
useful either, so get rid of that too.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: do all argument parsing in one place
This patch makes rbd_add_parse_args() be the single place all
argument parsing occurs for an image map request:
- Move the ceph_parse_options() call into that function
- Use local variables rather than parameters to hold the list
of monitor addresses supplied
- Rather than returning it, pass the snapshot name (and its
length) back via parameters
- Have the function return a ceph_options structure pointer
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: move ceph_parse_options() call up
Move option parsing out of rbd_get_client() and into its caller.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 22:25:23 +0000 (17:25 -0500)]
rbd: rename snap_exists field
A Boolean field "snap_exists" in an rbd mapping is used to indicate
whether a mapped snapshot has been removed from an image's snapshot
context, to stop sending requests for that snapshot as soon as we
know it's gone.
Generalize the interpretation of this field so it applies to
non-snapshot (i.e. "head") mappings. That is, define its value
to be false until the mapping has been set, and then define it to be
true for both snapshot mappings or head mappings.
Rename the field "exists" to reflect the broader interpretation.
The rbd_mapping structure is on its way out, so move the field
back into the rbd_device structure.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: move snap info out of rbd_mapping struct
Moving the snap_id and snap_name fields into the separate
rbd_mapping structure was misguided. (And in time, perhaps
we'll do away with that structure altogether...)
Move these fields back into struct rbd_device.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:41 +0000 (23:34 -0500)]
rbd: make pool_id a 64 bit value
If a format 2 image has a parent, its pool id will be specified
using a 64-bit value. Change the pool id we save for an image to
match that.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 26 Oct 2012 04:34:40 +0000 (23:34 -0500)]
rbd: remove snapshots on error in rbd_add()
If rbd_dev_snaps_update() has ever been called for an rbd device
structure there could be snapshot structures on its snaps list.
In rbd_add(), this function is called but a subsequent error
path neglected to clean up any of these snapshots.
Add a call to rbd_remove_all_snaps() in the appropriate spot to
remedy this. Change a couple of error labels to be a little
clearer while there.
Drop the leading underscores from the function name; there's nothing
special about that function that they might signify. As suggested
in review, the leading underscores in __rbd_remove_snap_dev() have
been removed as well.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Sun, 21 Oct 2012 03:17:27 +0000 (22:17 -0500)]
rbd: simplify rbd_rq_fn()
When processing a request, rbd_rq_fn() makes clones of the bio's in
the request's bio chain and submits the results to osd's to be
satisfied. If a request bio straddles the boundary between objects
backing the rbd image, it must be represented by two cloned bio's,
one for the first part (at the end of one object) and one for the
second (at the beginning of the next object).
This has been handled by a function bio_chain_clone(), which
includes an interface only a mother could love, and which has
been found to have other problems.
This patch defines two new fairly generic bio functions (one which
replaces bio_chain_clone()) to help out the situation, and then
revises rbd_rq_fn() to make use of them.
First, bio_clone_range() clones a portion of a single bio, starting
at a given offset within the bio and including only as many bytes
as requested. As a convenience, a request to clone the entire bio
is passed directly to bio_clone().
Second, bio_chain_clone_range() performs a similar function,
producing a chain of cloned bio's covering a sub-range of the
source chain. No bio_pair structures are used, and if successful
the result will represent exactly the specified range.
Using bio_chain_clone_range() makes bio_rq_fn() a little easier
to understand, because it avoids the need to pass very much
state information between consecutive calls. By avoiding the need
to track a bio_pair structure, it also eliminates the problem
described here: http://tracker.newdream.net/issues/2933
Note that a block request (and therefore the complete length of
a bio chain processed in rbd_rq_fn()) is an unsigned int, while
the result of rbd_segment_length() is u64. This change makes
this range trunctation explicit, and trips a bug if the the
segment boundary is too far off.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Sage Weil [Mon, 29 Oct 2012 18:01:42 +0000 (11:01 -0700)]
libceph: fix osdmap decode error paths
Ensure that we set the err value correctly so that we do not pass a 0
value to ERR_PTR and confuse the calling code. (In particular,
osd_client.c handle_map() will BUG(!newmap)).
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Alex Elder [Mon, 22 Oct 2012 16:31:27 +0000 (11:31 -0500)]
rbd: kill rbd_device->rbd_opts
The rbd_device structure has an embedded rbd_options structure.
Such a structure is needed to work with the generic ceph argument
parsing code, but there's no need to keep it around once argument
parsing is done.
Use a local variable to hold the rbd options used in parsing in
rbd_get_client(), and just transfer its content (it's just a
read_only flag) into the field in the rbd_mapping sub-structure
that requires that information.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Alex Elder [Sun, 21 Oct 2012 03:17:27 +0000 (22:17 -0500)]
rbd: simplify rbd_merge_bvec()
The aim of this patch is to make what's going on rbd_merge_bvec() a
bit more obvious than it was before. This was an issue when a
recent btrfs bug led us to question whether the merge function was
working correctly.
Use "obj" rather than "chunk" to indicate the units whose boundaries
we care about we call (rados) "objects".
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Alex Elder [Tue, 3 Jul 2012 21:01:19 +0000 (16:01 -0500)]
rbd: increase maximum snapshot name length
Change RBD_MAX_SNAP_NAME_LEN to be based on NAME_MAX. That is a
practical limit for the length of a snapshot name (based on the
presence of a directory using the name under /sys/bus/rbd to
represent the snapshot).
The /sys entry is created by prefixing it with "snap_"; define that
prefix symbolically, and take its length into account in defining
the snapshot name length limit.
Enforce the limit in rbd_add_parse_args(). Also delete a dout()
call in that function that was not meant to be committed.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Sun, 21 Oct 2012 03:17:27 +0000 (22:17 -0500)]
rbd: verify rbd image order value
This adds a verification that an rbd image's object order is
within the upper and lower bounds supported by this implementation.
It must be at least 9 (SECTOR_SHIFT), because the Linux bio system
assumes that minimum granularity.
It also must be less than 32 (at the moment anyway) because there
exist spots in the code that store the size of a "segment" (object
backing an rbd image) in a signed int variable, which can be 32 bits
including the sign. We should be able to relax this limit once
we've verified the code uses 64-bit types where needed.
Note that the CLI tool already limits the order to the range 12-25.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Thu, 11 Oct 2012 01:59:29 +0000 (18:59 -0700)]
rbd: consolidate rbd_do_op() calls
The two calls to rbd_do_op() from rbd_rq_fn() differ only in the
value passed for the snapshot id and the snapshot context.
For reads the snapshot always comes from the mapping, and for writes
the snapshot id is always CEPH_NOSNAP.
The snapshot context is always null for reads. For writes, the
snapshot context always comes from the rbd header, but it is
acquired under protection of header semaphore and could change
thereafter, so we can't simply use what's available inside
rbd_do_op().
Eliminate the snapid parameter from rbd_do_op(), and set it
based on the I/O direction inside that function instead. Always
pass the snapshot context acquired in the caller, but reset it
to a null pointer inside rbd_do_op() if the operation is a read.
As a result, there is no difference in the read and write calls
to rbd_do_op() made in rbd_rq_fn(), so just call it unconditionally.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Thu, 11 Oct 2012 01:59:29 +0000 (18:59 -0700)]
rbd: drop rbd_do_op() opcode and flags
The only callers of rbd_do_op() are in rbd_rq_fn(), where call one
is used for writes and the other used for reads. The request passed
to rbd_do_op() already encodes the I/O direction, and that
information can be used inside the function to set the opcode and
flags value (rather than passing them in as arguments).
So get rid of the opcode and flags arguments to rbd_do_op().
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Thu, 11 Oct 2012 01:59:29 +0000 (18:59 -0700)]
rbd: kill rbd_req_{read,write}()
Both rbd_req_read() and rbd_req_write() are simple wrapper routines
for rbd_do_op(), and each is only called once. Replace each wrapper
call with a direct call to rbd_do_op(), and get rid of the wrapper
functions.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Mon, 22 Oct 2012 16:31:26 +0000 (11:31 -0500)]
rbd: fix read-only option name
The name of the "read-only" mapping option was inadvertently changed
in this commit:
f84344f3 rbd: separate mapping info in rbd_dev
Revert that hunk to return it to what it should be.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Thu, 11 Oct 2012 04:19:13 +0000 (21:19 -0700)]
rbd: zero return code in rbd_dev_image_id()
When rbd_dev_probe() calls rbd_dev_image_id() it expects to get
a 0 return code if successful, but it is getting a positive value.
The reason is that rbd_dev_image_id() returns the value it gets from
rbd_req_sync_exec(), which returns the number of bytes read in as a
result of the request. (This ultimately comes from
ceph_copy_from_page_vector() in rbd_req_sync_op()).
Force the return value to 0 when successful in rbd_dev_image_id().
Do the same in rbd_dev_v2_object_prefix().
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Reviewed-by: Dan Mick <dan.mick@inktank.com>
Alex Elder [Thu, 11 Oct 2012 04:19:13 +0000 (21:19 -0700)]
rbd: fix bug in rbd_dev_id_put()
In rbd_dev_id_put(), there's a loop that's intended to determine
the maximum device id in use. But it isn't doing that at all,
the effect of how it's written is to simply use the just-put id
number, which ignores whole purpose of this function.
Fix the bug.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
David Zafman [Thu, 25 Oct 2012 17:23:46 +0000 (10:23 -0700)]
ceph: Fix NULL ptr crash in strlen()
set_request_path_attr() checks for NULL ptr before calling strlen()
This fixes http://tracker.newdream.net/issues/3404
Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Sage Weil [Thu, 25 Oct 2012 15:49:41 +0000 (08:49 -0700)]
libceph: avoid NULL kref_put from NULL alloc_msg return
The ceph_on_in_msg_alloc() method calls the ->alloc_msg() helper which
may return NULL. It also drops con->mutex while it allocates a message,
which means that the connection state may change (e.g., get closed). If
that happens, we clean up and bail out. Avoid calling ceph_msg_put() on
a NULL return value and triggering a crash.
This was observed when an ->alloc_msg() call races with a timeout that
resends a zillion messages and resets the connection, and ->alloc_msg()
returns NULL (because the request was resent to another target).
Fixes http://tracker.newdream.net/issues/3342
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Alex Elder <elder@inktank.com>
David Zafman [Thu, 18 Oct 2012 21:01:43 +0000 (14:01 -0700)]
ceph: fix dentry reference leak in encode_fh()
Call to d_find_alias() needs a corresponding dput()
This fixes http://tracker.newdream.net/issues/3271
Signed-off-by: David Zafman <david.zafman@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Fri, 31 Aug 2012 22:29:55 +0000 (17:29 -0500)]
rbd: activate v2 image support
Now that v2 images support is fully implemented, have
rbd_dev_v2_probe() return 0 to indicate a successful probe.
(Note that an image that implements layering will fail
the probe early because of the feature chekc.)
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Tue, 9 Oct 2012 20:50:17 +0000 (13:50 -0700)]
rbd: implement feature checks
Version 2 images have two sets of feature bit fields. The first
indicates features possibly used by the image. The second indicates
features that the client *must* support in order to use the image.
When an image (or snapshot) is first examined, we need to make sure
that the local implementation supports the image's required
features. If not, fail the probe for the image.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 31 Aug 2012 22:29:55 +0000 (17:29 -0500)]
rbd: define rbd_dev_v2_refresh()
Define a new function rbd_dev_v2_refresh() to update/refresh the
snapshot context for a format version 2 rbd image. This function
will update anything that is not fixed for the life of an rbd
image--at the moment this is mainly the snapshot context and (for
a base mapping) the size.
Update rbd_refresh_header() so it selects which function to use
based on the image format.
Rename __rbd_refresh_header() to be rbd_dev_v1_refresh()
to be consistent with the naming of its version 2 counterpart.
Similarly rename rbd_refresh_header() to be rbd_dev_refresh().
Unrelated--we use rbd_image_format_valid() here. Delete the other
use of it, which was primarily put in place to ensure that function
was referenced at the time it was defined.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Tue, 9 Oct 2012 20:50:17 +0000 (13:50 -0700)]
rbd: define rbd_update_mapping_size()
Encapsulate the code that handles updating the size of a mapping
after an rbd image has been refreshed. This is done in anticipation
of the next patch, which will make this common code for format 1 and
2 images.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Tue, 9 Oct 2012 03:37:30 +0000 (20:37 -0700)]
rbd: define common queue_con_delay()
This patch defines a single function, queue_con_delay() to call
queue_delayed_work() for a connection. It basically generalizes
what was previously queue_con() by adding the delay argument.
queue_con() is now a simple helper that passes 0 for its delay.
queue_con_delay() returns 0 if it queued work or an errno if it
did not for some reason.
If con_work() finds the BACKOFF flag set for a connection, it now
calls queue_con_delay() to handle arranging to start again after a
delay.
Note about connection reference counts: con_work() only ever gets
called as a work item function. At the time that work is scheduled,
a reference to the connection is acquired, and the corresponding
con_work() call is then responsible for dropping that reference
before it returns.
Previously, the backoff handling inside con_work() silently handed
off its reference to delayed work it scheduled. Now that
queue_con_delay() is used, a new reference is acquired for the
newly-scheduled work, and the original reference is dropped by the
con->ops->put() call at the end of the function.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Tue, 9 Oct 2012 03:37:30 +0000 (20:37 -0700)]
rbd: let con_work() handle backoff
Both ceph_fault() and con_work() include handling for imposing a
delay before doing further processing on a faulted connection.
The latter is used only if ceph_fault() is unable to.
Instead, just let con_work() always be responsible for implementing
the delay. After setting up the delay value, set the BACKOFF flag
on the connection unconditionally and call queue_con() to ensure
con_work() will get called to handle it.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Tue, 9 Oct 2012 03:37:30 +0000 (20:37 -0700)]
rbd: reset BACKOFF if unable to re-queue
If ceph_fault() is unable to queue work after a delay, it sets the
BACKOFF connection flag so con_work() will attempt to do so.
In con_work(), when BACKOFF is set, if queue_delayed_work() doesn't
result in newly-queued work, it simply ignores this condition and
proceeds as if no backoff delay were desired. There are two
problems with this--one of which is a bug.
The first problem is simply that the intended behavior is to back
off, and if we aren't able queue the work item to run after a delay
we're not doing that.
The only reason queue_delayed_work() won't queue work is if the
provided work item is already queued. In the messenger, this
means that con_work() is already scheduled to be run again. So
if we simply set the BACKOFF flag again when this occurs, we know
the next con_work() call will again attempt to hold off activity
on the connection until after the delay.
The second problem--the bug--is a leak of a reference count. If
queue_delayed_work() returns 0 in con_work(), con->ops->put() drops
the connection reference held on entry to con_work(). However,
processing is (was) allowed to continue, and at the end of the
function a second con->ops->put() is called.
This patch fixes both problems.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Alex Elder [Tue, 2 Oct 2012 15:25:51 +0000 (10:25 -0500)]
ceph: avoid 32-bit page index overflow
A pgoff_t is defined (by default) to have type (unsigned long). On
architectures such as i686 that's a 32-bit type. The ceph address
space code was attempting to produce 64 bit offsets by shifting a
page's index by PAGE_CACHE_SHIFT, but the result was not what was
desired because the shift occurred before the result got promoted
to 64 bits.
Fix this by converting all uses of page->index used in this way to
use the page_offset() macro, which ensures the 64-bit result has the
intended value.
This fixes http://tracker.newdream.net/issues/3112
Reported-by: Mohamed Pakkeer <pakkeer.mohideen@realimage.com>
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Sage Weil <sage@inktank.com>
Sage Weil [Tue, 25 Sep 2012 04:04:57 +0000 (21:04 -0700)]
ceph: return EIO on invalid layout on GET_DATALOC ioctl
If the user calls GET_DATALOC on a file with an invalid (e.g.,
zeroed) layout, return EIO to userland.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Sage Weil [Tue, 25 Sep 2012 04:02:47 +0000 (21:02 -0700)]
rbd: BUG on invalid layout
This shouldn't actually be possible because the layout struct is
constructed from the RBD header and validated then.
[elder@inktank.com: converted BUG() call to equivalent rbd_assert()]
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Sage Weil [Tue, 25 Sep 2012 04:01:02 +0000 (21:01 -0700)]
ceph: propagate layout error on osd request creation
If we are creating an osd request and get an invalid layout, return
an EINVAL to the caller. We switch up the return to have an error
code instead of NULL implying -ENOMEM.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Sage Weil [Tue, 25 Sep 2012 03:59:48 +0000 (20:59 -0700)]
libceph: check for invalid mapping
If we encounter an invalid (e.g., zeroed) mapping, return an error
and avoid a divide by zero.
Signed-off-by: Sage Weil <sage@inktank.com>
Reviewed-by: Alex Elder <elder@inktank.com>
Wei Yongjun [Fri, 28 Sep 2012 04:59:16 +0000 (12:59 +0800)]
ceph: convert to use le32_add_cpu()
Convert cpu_to_le32(le32_to_cpu(E1) + E2) to use le32_add_cpu().
dpatch engine is used to auto generate this patch.
(https://github.com/weiyj/dpatch)
Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: Sage Weil <sage@inktank.com>
Yan, Zheng [Thu, 20 Sep 2012 09:42:25 +0000 (17:42 +0800)]
ceph: Fix oops when handling mdsmap that decreases max_mds
When i >= newmap->m_max_mds, ceph_mdsmap_get_addr(newmap, i) return
NULL. Passing NULL to memcmp() triggers oops.
Signed-off-by: Yan, Zheng <zheng.z.yan@intel.com>
Signed-off-by: Sage Weil <sage@inktank.com>
Alex Elder [Tue, 3 Jul 2012 21:01:19 +0000 (16:01 -0500)]
rbd: update remaining header fields for v2
There are three fields that are not yet updated for format 2 rbd
image headers: the version of the header object; the encryption
type; and the compression type. There is no interface defined for
fetching the latter two, so just initialize them explicitly to 0 for
now.
Change rbd_dev_v2_snap_context() so the caller can be supplied the
version for the header object.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Tue, 3 Jul 2012 21:01:19 +0000 (16:01 -0500)]
rbd: get snapshot name for a v2 image
Define rbd_dev_v2_snap_name() to fetch the name for a particular
snapshot in a format 2 rbd image.
Define rbd_dev_v2_snap_info() to to be a wrapper for getting the
name, size, and features for a particular snapshot, using an
interface that matches the equivalent function for version 1 images.
Define rbd_dev_snap_info() wrapper function and use it to call the
appropriate function for getting the snapshot name, size, and
features, dependent on the rbd image format.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Tue, 3 Jul 2012 21:01:19 +0000 (16:01 -0500)]
rbd: get the snapshot context for a v2 image
Fetch the snapshot context for an rbd format 2 image by calling
the "get_snapcontext" method on its header object.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Tue, 3 Jul 2012 21:01:19 +0000 (16:01 -0500)]
rbd: get image features for a v2 image
The features values for an rbd format 2 image are fetched from the
server using a "get_features" method. The same method is used for
getting the features for a snapshot, so structure this addition with
a generic helper routine that can get this information for either.
The server will provide two 64-bit feature masks, one representing
the features potentially in use for this image (or its snapshot),
and one representing features that must be supported by the client
in order to work with the image.
For the time being, neither of these is really used so we keep
things simple and just record the first feature vector. Once we
start using these feature masks, what we record and what we expose
to the user will most likely change.
Signed-off-by: Alex Elder <elder@inktank.com>
Alex Elder [Tue, 3 Jul 2012 21:01:19 +0000 (16:01 -0500)]
rbd: get the object prefix for a v2 rbd image
The object prefix of an rbd format 2 image is fetched from the
server using a "get_object_prefix" method.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Tue, 3 Jul 2012 21:01:19 +0000 (16:01 -0500)]
rbd: add code to get the size of a v2 rbd image
The size of an rbd format 2 image is fetched from the server using a
"get_size" method. The same method is used for getting the size of
a snapshot, so structure this addition with a generic helper routine
that we can get this information for either.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Wed, 11 Jul 2012 01:30:11 +0000 (20:30 -0500)]
rbd: lay out header probe infrastructure
This defines a new function rbd_dev_probe() as a top-level
function for populating detailed information about an rbd device.
It first checks for the existence of a format 2 rbd image id object.
If it exists, the image is assumed to be a format 2 rbd image, and
another function rbd_dev_v2() is called to finish populating
header data for that image. If it does not exist, it is assumed to
be an old (format 1) rbd image, and calls a similar function
rbd_dev_v1() to populate its header information.
A new field, rbd_dev->format, is defined to record which version
of the rbd image format the device represents. For a valid mapped
rbd device it will have one of two values, 1 or 2.
So far, the format 2 images are not really supported; this is
laying out the infrastructure for fleshing out that support.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Tue, 3 Jul 2012 21:01:19 +0000 (16:01 -0500)]
rbd: encapsulate code that gets snapshot info
Create a function that encapsulates looking up the name, size and
features related to a given snapshot, which is indicated by its
index in an rbd device's snapshot context array of snapshot ids.
This interface will be used to hide differences between the format 1
and format 2 images.
At the moment this (looking up the name anyway) is slightly less
efficient than what's done currently, but we may be able to optimize
this a bit later on by cacheing the last lookup if it proves to be a
problem.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Sat, 14 Jul 2012 01:35:12 +0000 (20:35 -0500)]
rbd: add an rbd features field
Record the features values for each rbd image and each of its
snapshots. This is really something that only becomes meaningful
for version 2 images, so this is just putting in place code
that will form common infrastructure.
It may be useful to expand the sysfs entries--and therefore the
information we maintain--for the image and for each snapshot.
But I'm going to hold off doing that until we start making
active use of the feature bits.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Wed, 11 Jul 2012 01:30:11 +0000 (20:30 -0500)]
rbd: don't use index in __rbd_add_snap_dev()
Pass the snapshot id and snapshot size rather than an index
to __rbd_add_snap_dev() to specify values for a new snapshot.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Fri, 10 Aug 2012 20:12:10 +0000 (13:12 -0700)]
rbd: kill create_snap sysfs entry
Josh proposed the following change, and I don't think I could
explain it any better than he did:
From: Josh Durgin <josh.durgin@inktank.com>
Date: Tue, 24 Jul 2012 14:22:11 -0700
To: ceph-devel <ceph-devel@vger.kernel.org>
Message-ID: <
500F1203.9050605@inktank.com>
Right now the kernel still has one piece of rbd management
duplicated from the rbd command line tool: snapshot creation.
There's nothing special about snapshot creation that makes it
advantageous to do from the kernel, so I'd like to remove the
create_snap sysfs interface. That is,
/sys/bus/rbd/devices/<id>/create_snap
would be removed.
Does anyone rely on the sysfs interface for creating rbd
snapshots? If so, how hard would it be to replace with:
rbd snap create pool/image@snap
Is there any benefit to the sysfs interface that I'm missing?
Josh
This patch implements this proposal, removing the code that
implements the "snap_create" sysfs interface for rbd images.
As a result, quite a lot of other supporting code goes away.
Suggested-by: Josh Durgin <josh.durgin@inktank.com>
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>
Alex Elder [Wed, 11 Jul 2012 01:30:11 +0000 (20:30 -0500)]
rbd: define rbd_dev_image_id()
New format 2 rbd images are permanently identified by a unique image
id. Each rbd image also has a name, but the name can be changed.
A format 2 rbd image will have an object--whose name is based on the
image name--which maps an image's name to its image id.
Create a new function rbd_dev_image_id() that checks for the
existence of the image id object, and if it's found, records the
image id in the rbd_device structure.
Create a new rbd device attribute (/sys/bus/rbd/<num>/image_id) that
makes this information available.
Signed-off-by: Alex Elder <elder@inktank.com>
Reviewed-by: Josh Durgin <josh.durgin@inktank.com>