on exports which may be served either via TLS or in plain
text unless the client insists on TLS.
- ### Status
-
- This functionality has not yet been implemented by the reference
- implementation, but was implemented by qemu and subsequently
- by other users, so has been moved out of the "experimental" section.
+ ## Block size constraints
+
+ During transmission phase, several operations are constrained by the
+ export size sent by the final `NBD_OPT_EXPORT_NAME` or `NBD_OPT_GO`,
+ as well as by three block size constraints defined here (minimum,
+ preferred, and maximum).
+
+ If a client can honour server block size constraints (as set out below
+ and under `NBD_INFO_BLOCK_SIZE`), it SHOULD announce this during the
+ handshake phase by using `NBD_OPT_GO` (and `NBD_OPT_INFO` if used) with
+ an `NBD_INFO_BLOCK_SIZE` information request, and MUST use `NBD_OPT_GO`
+ rather than `NBD_OPT_EXPORT_NAME` (except in the case of a fallback
+ where the server did not support `NBD_OPT_INFO` or `NBD_OPT_GO`).
+
+ A server with block size constraints other than the default SHOULD
+ advertise the block size constraints during handshake phase via
+ `NBD_INFO_BLOCK_SIZE` in response to `NBD_OPT_INFO` or `NBD_OPT_GO`,
+ and MUST do so unless it has agreed on block size constraints via out
+ of band means.
+
+ Some servers are able to make optimizations, such as opening files
+ with `O_DIRECT`, if they know that the client will obey a particular
+ minimum block size, where it must fall back to safer but slower code
+ if the client might send unaligned requests. For that reason, if a
+ client issues an `NBD_OPT_GO` including an `NBD_INFO_BLOCK_SIZE`
+ information request, it MUST abide by the block size constraints it
+ receives. Clients MAY issue `NBD_OPT_INFO` with `NBD_INFO_BLOCK_SIZE` to
+ learn the server's constraints without committing to them.
+
+ If block size constraints have not been advertised or agreed on externally,
+ then a client SHOULD assume a default minimum block size of 1, a preferred
+ block size of 2^12 (4,096), and a maximum block size of the smaller of
+ the export size or 0xffffffff (effectively unlimited). A server that
+ wants to enforce block sizes other than the defaults specified here
+ MAY refuse to go into transmission phase with a client that uses
+ `NBD_OPT_EXPORT_NAME` (via a hard disconnect) or which fails to use
+ `NBD_INFO_BLOCK_SIZE` with `NBD_OPT_GO` (where the server uses
+ `NBD_REP_ERR_BLOCK_SIZE_REQD`), although a server SHOULD permit such
+ clients if block size constraints are the default or can be agreed on
+ externally. When allowing such clients, the server MUST cleanly error
+ commands that fall outside block size constraints without corrupting
+ data; even so, this may limit interoperability.
+
+ A client MAY choose to operate as if tighter block size constraints had
+ been specified (for example, even when the server advertises the default
+ minimum block size of 1, a client may safely use a minimum block size
+ of 2^9 (512), a preferred block size of 2^16 (65,536), and a maximum
+ block size of 2^25 (33,554,432)). Notwithstanding any maximum block
+ size advertised, either the server or the client MAY initiate a hard
+ disconnect if the size of a request or a reply is large enough to be
+ deemed a denial of service attack.
+
+ The minimum block size represents the smallest addressable length and
+ alignment within the export, although writing to an area that small
+ may require the server to use a less-efficient read-modify-write
+ action. If advertised, this value MUST be a power of 2, MUST NOT be
+ larger than 2^16 (65,536), and MAY be as small as 1 for an export
+ backed by a regular file, although the values of 2^9 (512) or 2^12
+ (4,096) are more typical for an export backed by a block device. If a
+ server advertises a minimum block size, the advertised export size
+ SHOULD be an integer multiple of that block size, since otherwise, the
+ client would be unable to access the final few bytes of the export.
+
+ The preferred block size represents the minimum size at which aligned
+ requests will have efficient I/O, avoiding behaviour such as
+ read-modify-write. If advertised, this MUST be a power of 2 at least
+ as large as the smaller of the minimum block size and 2^12 (4,096),
+ although larger values (such as the minimum granularity of a hole) are
+ also appropriate. The preferred block size MAY be larger than the
+ export size, in which case the client is unable to utilize the
+ preferred block size for that export. The server MAY advertise an
+ export size that is not an integer multiple of the preferred block
+ size.
+
+ The maximum block size represents the maximum length that the server
+ is willing to handle in one request. If advertised, it MUST be either
+ an integer multiple of the minimum block size or the value 0xffffffff
+ for no inherent limit, MUST be at least as large as the smaller of the
+ preferred block size or export size, and SHOULD be at least 2^25
+ (33,554,432) if the export is that large, but MAY be something other
+ than a power of 2. For convenience, the server MAY advertise a
+ maximum block size that is larger than the export size, although in
+ that case, the client MUST treat the export size as the effective
+ maximum block size (as further constrained by a non-zero offset).
+
+ Where a transmission request can have a non-zero *offset* and/or
+ *length* (such as `NBD_CMD_READ`, `NBD_CMD_WRITE`, or `NBD_CMD_TRIM`),
+ the client MUST ensure that *offset* and *length* are integer
+ multiples of any advertised minimum block size, and SHOULD use integer
+ multiples of any advertised preferred block size where possible. For
+ those requests, the client MUST NOT use a *length* larger than any
+ advertised maximum block size or which, when added to *offset*, would
+ exceed the export size. The server SHOULD report an `EINVAL` error if
+ the client's request is not aligned to advertised minimum block size
+ boundaries, or is larger than the advertised maximum block size,
+ although the server MAY instead initiate a hard disconnect if a large
+ *length* could be deemed as a denial of service attack.
+## Metadata querying
+
+It is often helpful for the client to be able to query the status
+of a range of blocks. The nature of the status that can be
+queried is in part implementation dependent. For instance,
+the status might represent:
+
+* in a sparse storage format, whether the relevant blocks are
+ actually present on the backing device for the export; or
+
+* whether the relevant blocks are 'dirty'; some storage formats
+ and operations over such formats express a concept of data dirtiness.
+ Whether the operation is block device mirroring, incremental block
+ device backup or any other operation with a concept of data dirtiness,
+ they all share a need to provide a list of ranges that this
+ particular operation treats as dirty.
+
+To provide such classes of information, the NBD protocol has a generic
+framework for querying metadata; however, its use must first be
+negotiated, and one or more metadata contexts must be selected.
+
+The procedure works as follows:
+
+- First, during negotiation, if the client wishes to query metadata
+ during transmission, the client MUST select one or more metadata
+ contexts with the `NBD_OPT_SET_META_CONTEXT` command. If needed, the client
+ can use `NBD_OPT_LIST_META_CONTEXT` to list contexts that the server
+ supports.
+- During transmission, a client can then indicate interest in metadata
+ for a given region by way of the `NBD_CMD_BLOCK_STATUS` command, where
+ *offset* and *length* indicate the area of interest. The server MUST
+ then respond with the requested information, for all contexts which
+ were selected during negotiation. For every metadata context, the
+ server sends one set of extent chunks, where the sizes of the
+ extents MUST be less than or equal to the length as specified in the
+ request. Each extent comes with a *flags* field, the semantics of
+ which are defined by the metadata context.
+- A server MUST reply to `NBD_CMD_BLOCK_STATUS` with a structured reply
+ of type `NBD_REPLY_TYPE_BLOCK_STATUS`.
+
+A client MUST NOT use `NBD_CMD_BLOCK_STATUS` unless it selected a
+non-zero number of metadata contexts during negotiation. Servers SHOULD
+reply to clients sending `NBD_CMD_BLOCK_STATUS` without
+selecting metadata contexts with `EINVAL`.
+
+The reply to the `NBD_CMD_BLOCK_STATUS` request MUST be sent as a
+structured reply; this implies that in order to use metadata querying,
+structured replies MUST be negotiated first.
+
+Metadata contexts are identified by their names. The name MUST
+consist of a namespace, followed by a colon, followed by a leaf-name.
+The namespace must consist entirely of printable non-whitespace
+UTF-8 characters other than colons, and be non-empty. The entire name
+(namespace, colon, and leaf-name) MUST follow the restrictions for
+strings as laid out earlier in this document.
+
+Namespaces MUST be consist of one of the following:
+- `base`, for metadata contexts defined by this document;
+- `nbd-server`, for metadata contexts defined by the
+ implementation that accompanies this document (none
+ currently);
+- `x-*`, where `*` can be replaced by an arbitrary string not
+ containing colons, for local experiments. This SHOULD NOT be
+ used by metadata contexts that are expected to be widely used.
+- A third-party namespace from the list below.
+
+Third-party implementations can register additional namespaces by simple
+request to the mailing-list. The following additional third-party namespaces
+are currently registered:
+* (none)
+
+Save in respect of the `base:` namespace described below, this specification
+requires no specific semantics of metadata contexts, except that all the
+information they provide MUST be representable within the flags field as
+defined for `NBD_REPLY_TYPE_BLOCK_STATUS`. Likewise, save in respect of
+the `base:` namespace, the syntax of query strings is not specified by this
+document.
+
+Server implementations SHOULD ensure the syntax for query strings they
+support and semantics for resulting metadata context is documented similarly
+to this document.
+
+### The `base:` metadata namespace
+
+This standard defines exactly one metadata context; it is called
+`base:allocation`, and it provides information on the basic allocation
+status of extents (that is, whether they are allocated at all in a
+sparse file context).
+
+The query string within the `base:` metadata context can take
+one of two forms:
+* `base:` - in which case all metadata contexts within `base:` are
+ listed; or
+* `base:[leaf-name]` where `[leaf-name]` is a context leaf-name
+ that exists within the `base` namespace (currently just
+ `base:allocation`).
+
+#### `base:allocation` metadata context
+
+The `base:allocation` metadata context is the basic "allocated at all"
+metadata context. If an extent is marked with `NBD_STATE_HOLE` at that
+context, this means that the given extent is not allocated in the
+backend storage, and that writing to the extent MAY result in the `ENOSPC`
+error. This supports sparse file semantics on the server side.
+If a server supports the `base:allocation` metadata context, then writing
+to an extent which has `NBD_STATE_HOLE` clear MUST NOT fail with `ENOSPC`
+unless for reasons specified in the definition of another context.
+
+It defines the following flags for the flags field:
+
+- `NBD_STATE_HOLE` (bit 0): if set, the block represents a hole (and
+ future writes to that area may cause fragmentation or encounter an
+ `ENOSPC` error); if clear, the block is allocated or the server could
+ not otherwise determine its status. Note that the use of
+ `NBD_CMD_TRIM` is related to this status, but that the server MAY
+ report a hole even where `NBD_CMD_TRIM` has not been requested, and
+ also that a server MAY report that the block is allocated even where
+ `NBD_CMD_TRIM` has been requested.
+- `NBD_STATE_ZERO` (bit 1): if set, the block contents read as all
+ zeroes; if clear, the block contents are not known. Note that the use
+ of `NBD_CMD_WRITE_ZEROES` is related to this status, but that the
+ server MAY report zeroes even where `NBD_CMD_WRITE_ZEROES` has not been
+ requested, and also that a server MAY report unknown content even
+ where `NBD_CMD_WRITE_ZEROES` has been requested.
+
+It is not an error for a server to report that a region of the
+export has both `NBD_STATE_HOLE` set and `NBD_STATE_ZERO` clear. The
+contents of such an area are undefined, and a client
+reading such an area should make no assumption as to its contents
+or stability.
+
+For the `base:allocation` context, the remainder of the flags field is
+reserved. Servers SHOULD set it to all-zero; clients MUST ignore unknown
+flags.
+
## Values
This section describes the value and meaning of constants (other than
particular client request, this field is defined to be a string
suitable for direct display to a human being.
-* `NBD_REP_INFO` (3)
+- `NBD_REP_INFO` (3)
- Defined by the experimental `INFO` [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-info/doc/proto.md).
+ A detailed description about an aspect of an export. The response
+ to `NBD_OPT_INFO` and `NBD_OPT_GO` includes zero or more of these
+ messages prior to a final error reply, or at least one before an
+ `NBD_REP_ACK` reply indicating success. The server MUST send an
+ `NBD_INFO_EXPORT` information type at some point before sending an
+ `NBD_REP_ACK`, so that `NBD_OPT_GO` can provide a superset of the
+ information given in response to `NBD_OPT_EXPORT_NAME`; all other
+ information types are optional. A particular information type
+ SHOULD only appear once for a given export unless documented
+ otherwise.
+
+ A client MUST NOT rely on any particular ordering amongst the
+ `NBD_OPT_INFO` replies, and MUST ignore information types that it
+ does not recognize.
+
+ The acceptable values for the header *length* field are determined
+ by the information type, and includes the 2 bytes for the type
+ designator, in the following general layout:
+
+ - 16 bits, information type (e.g. `NBD_INFO_EXPORT`)
+ - *length - 2* bytes, information payload
+
+ The following information types are defined:
+
+ * `NBD_INFO_EXPORT` (0)
+
+ Mandatory information before a successful completion of
+ `NBD_OPT_INFO` or `NBD_OPT_GO`. Describes the same information
+ that is sent in response to the older `NBD_OPT_EXPORT_NAME`,
+ except that there are no trailing zeroes whether or not
+ `NBD_FLAG_C_NO_ZEROES` was negotiated. *length* MUST be 12, and
+ the reply payload is interpreted as follows:
+
+ - 16 bits, `NBD_INFO_EXPORT`
+ - 64 bits, size of the export in bytes (unsigned)
+ - 16 bits, transmission flags
+
+ * `NBD_INFO_NAME` (1)
+
+ Represents the server's canonical name of the export. The name
+ MAY differ from the name presented in the client's option
+ request, and the information item MAY be omitted if the client
+ option request already used the canonical name. This
+ information type represents the same name that would appear in
+ the name portion of an `NBD_REP_SERVER` in response to
+ `NBD_OPT_LIST`. The *length* MUST be at least 2, and the reply
+ payload is interpreted as:
+
+ - 16 bits, `NBD_INFO_NAME`
+ - String: name of the export, *length - 2* bytes
+
+ * `NBD_INFO_DESCRIPTION` (2)
+
+ A description of the export, suitable for direct display to the
+ human being. This information type represents the same optional
+ description that may appear after the name portion of an
+ `NBD_REP_SERVER` in response to `NBD_OPT_LIST`. The *length*
+ MUST be at least 2, and the reply payload is interpreted as:
+
+ - 16 bits, `NBD_INFO_DESCRIPTION`
+ - String: description of the export, *length - 2* bytes
+
+ * `NBD_INFO_BLOCK_SIZE` (3)
+
+ Represents the server's advertised block size constraints; see the
+ "Block size constraints" section for more details on what these
+ values represent, and on constraints on their values. The server
+ MUST send this info if it is requested and it intends to enforce
+ block size constraints other than the defaults. After
+ sending this information in response to an `NBD_OPT_GO` in which
+ the client specifically requested `NBD_INFO_BLOCK_SIZE`, the server
+ can legitimately assume that any client that continues the session
+ will support the block size constraints supplied (note that this
+ assumption cannot be made solely on the basis of an `NBD_OPT_INFO`
+ with an `NBD_INFO_BLOCK_SIZE` request, or an `NBD_OPT_GO` without
+ an explicit `NBD_INFO_BLOCK_SIZE` request). The *length* MUST be 14,
+ and the reply payload is interpreted as:
+
+ - 16 bits, `NBD_INFO_BLOCK_SIZE`
+ - 32 bits, minimum block size
+ - 32 bits, preferred block size
+ - 32 bits, maximum block size
+- `NBD_REP_META_CONTEXT` (4)
+
+ A description of a metadata context. Data:
+
+ - 32 bits, NBD metadata context ID.
+ - String, name of the metadata context. This is not required to be
+ a human-readable string, but it MUST be valid UTF-8 data.
+
There are a number of error reply types, all of which are denoted by
having bit 31 set. All error replies MAY have some data set, in which
case that data is an error message string suitable for display to the user.
* `NBD_REP_ERR_UNKNOWN` (2^31 + 6)
- Defined by the experimental `INFO` [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-info/doc/proto.md).
+ The requested export is not available.
-* `NBD_REP_ERR_SHUTDOWN` (2^32 + 7)
+* `NBD_REP_ERR_SHUTDOWN` (2^31 + 7)
The server is unwilling to continue negotiation as it is in the
process of being shut down.
-* `NBD_REP_ERR_BLOCK_SIZE_REQD` (2^32 + 8)
+* `NBD_REP_ERR_BLOCK_SIZE_REQD` (2^31 + 8)
- Defined by the experimental `INFO` [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-info/doc/proto.md).
+ The server is unwilling to enter transmission phase for a given
+ export unless the client first acknowledges (via
+ `NBD_INFO_BLOCK_SIZE`) that it will obey non-default block sizing
+ requirements.
+* `NBD_REP_ERR_TOO_BIG` (2^31 + 9)
+
+ The request or the reply is too large to process.
+
### Transmission phase
#### Flag fields
if `NBD_FLAG_SEND_TRIM` was not set in the transmission flags field.
The server MUST support the use of this flag if it advertises
`NBD_FLAG_SEND_WRITE_ZEROES`.
- - bit 2, `NBD_CMD_FLAG_DF`; the "don't fragment" flag, valid during `NBD_CMD_READ`.
- SHOULD be set to 1 if the client requires the server to send at most one
- content chunk in reply. MUST NOT be set unless the transmission
- flags include `NBD_FLAG_SEND_DF`. Use of this flag MAY trigger an
- `EOVERFLOW` error chunk, if the request length is too large.
+ - bit 2, `NBD_CMD_FLAG_DF`; the "don't fragment" flag, valid during
+ `NBD_CMD_READ`. SHOULD be set to 1 if the client requires the
+ server to send at most one content chunk in reply. MUST NOT be set
+ unless the transmission flags include `NBD_FLAG_SEND_DF`. Use of
+ this flag MAY trigger an `EOVERFLOW` error chunk, if the request
+ length is too large.
+- bit 3, `NBD_CMD_FLAG_REQ_ONE`; valid during `NBD_CMD_BLOCK_STATUS`. If
+ set, the client is interested in only one extent per metadata
+ context. If this flag is present, the server MUST NOT send metadata
+ on more than one extent in the reply. Client implementors should note
+ that using this flag on multiple contiguous requests is likely to be
+ inefficient.
##### Structured reply flags
* `NBD_CMD_BLOCK_STATUS` (7)
- Defined by the experimental `BLOCK_STATUS`
- [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-blockstatus/doc/proto.md).
+ A block status query request. Length and offset define the range of
+ interest.
+
+ A client MUST NOT send `NBD_CMD_BLOCK_STATUS` unless
+ within the negotiation phase it sent `NBD_OPT_SET_META_CONTEXT`
+ at least once, and the final time it was sent, it referred
+ to the export name that was finally selected, the server
+ responded without an error, and returned at least one metadata
+ context. This in turn requires the client to
+ first negotiate structured replies. For a successful return, the
+ server MUST use a structured reply, containing at least one chunk of
+ type `NBD_REPLY_TYPE_BLOCK_STATUS`, where the status field of each
+ descriptor is determined by the flags field as defined by the
+ metadata context.
+
+ The list of block status descriptors within the
+ `NBD_REPLY_TYPE_BLOCK_STATUS` chunk represent consecutive portions
+ of the file starting from specified *offset*, and the sum of the
+ *length* fields of each descriptor MUST not be greater than the
+ overall *length* of the request. This means that the server MAY
+ return less data than required. However the server MUST return at
+ least one status descriptor. The server SHOULD use different
+ *status* values between consecutive descriptors, and SHOULD use
+ descriptor lengths that are an integer multiple of 512 bytes where
+ possible (the first and last descriptor of an unaligned query being
+ the most obvious places for an exception). The status flags are
+ intentionally defined so that a server MAY always safely report a
+ status of 0 for any block, although the server SHOULD return
+ additional status values when they can be easily detected.
+
+ If an error occurs, the server SHOULD set the appropriate error
+ code in the error field of an error chunk. However, if the error
+ does not involve invalid usage (such as a request beyond the bounds
+ of the file), a server MAY reply with a single block status
+ descriptor with *length* matching the requested length, rather than
+ reporting the error; in this case the context MAY mandate the
+ status returned.
+
+ A client MAY initiate a hard disconnect if it detects that the
+ server has sent an invalid chunk. The server SHOULD return `EINVAL`
+ if it receives a `NBD_CMD_BLOCK_STATUS` request including one or
+ more sectors beyond the size of the device.
+ * `NBD_CMD_RESIZE` (8)
+
+ Defined by the experimental `RESIZE`
+ [extension](https://github.com/NetworkBlockDevice/nbd/blob/extension-resize/doc/proto.md).
+
* Other requests
Some third-party implementations may require additional protocol
projects / platform / upstream / nbd.git / commitdiff