sdk/emulator/qemu.git
8 years agoMerge remote-tracking branch 'remotes/kraxel/tags/pull-vga-20160720-1' into staging
Peter Maydell [Wed, 20 Jul 2016 18:41:20 +0000 (19:41 +0100)]
Merge remote-tracking branch 'remotes/kraxel/tags/pull-vga-20160720-1' into staging

qxl: fix qxl_set_dirty call in qxl_dirty_one_surface

# gpg: Signature made Wed 20 Jul 2016 12:28:01 BST
# gpg:                using RSA key 0x4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/pull-vga-20160720-1:
  qxl: fix qxl_set_dirty call in qxl_dirty_one_surface

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging
Peter Maydell [Wed, 20 Jul 2016 17:52:10 +0000 (18:52 +0100)]
Merge remote-tracking branch 'remotes/famz/tags/docker-pull-request' into staging

# gpg: Signature made Wed 20 Jul 2016 12:19:56 BST
# gpg:                using RSA key 0xCA35624C6A9171C6
# gpg: Good signature from "Fam Zheng <famz@redhat.com>"
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 5003 7CB7 9706 0F76 F021  AD56 CA35 624C 6A91 71C6

* remotes/famz/tags/docker-pull-request:
  docker: pass EXECUTABLE to build script
  docker: Don't start a container that doesn't exist
  docker: Add "images" subcommand to docker.py
  docker: Fix exit code if $CMD failed
  docker: More sensible run script
  tests/docker/docker.py: add update operation
  tests/docker/dockerfiles: new debian-bootstrap.docker
  tests/docker/docker.py: check and run .pre script
  tests/docker/docker.py: support --include-executable
  tests/docker/docker.py: docker_dir outside build

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/cody/tags/block-pull-request' into staging
Peter Maydell [Wed, 20 Jul 2016 16:05:35 +0000 (17:05 +0100)]
Merge remote-tracking branch 'remotes/cody/tags/block-pull-request' into staging

# gpg: Signature made Wed 20 Jul 2016 01:18:39 BST
# gpg:                using RSA key 0xBDBE7B27C0DE3057
# gpg: Good signature from "Jeffrey Cody <jcody@redhat.com>"
# gpg:                 aka "Jeffrey Cody <jeff@codyprime.org>"
# gpg:                 aka "Jeffrey Cody <codyprime@gmail.com>"
# Primary key fingerprint: 9957 4B4D 3474 90E7 9D98  D624 BDBE 7B27 C0DE 3057

* remotes/cody/tags/block-pull-request:
  block/gluster: add support for multiple gluster servers
  block/gluster: using new qapi schema
  block/gluster: deprecate rdma support
  block/gluster: code cleanup
  block/gluster: rename [server, volname, image] -> [host, volume, path]
  mirror: fix request throttling in drive-mirror
  mirror: improve performance of mirroring of empty disk
  mirror: efficiently zero out target
  mirror: optimize dirty bitmap filling in mirror_run a bit
  block: remove extra condition in bdrv_can_write_zeroes_with_unmap
  mirror: create mirror_dirty_init helper for mirror_run
  mirror: create mirror_throttle helper
  mirror: make sectors_in_flight int64_t
  dirty-bitmap: operate with int64_t amount

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-07-19' into staging
Peter Maydell [Wed, 20 Jul 2016 13:34:08 +0000 (14:34 +0100)]
Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2016-07-19' into staging

QAPI patches for 2016-07-19

# gpg: Signature made Tue 19 Jul 2016 19:35:27 BST
# gpg:                using RSA key 0x3870B400EB918653
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>"
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>"
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2016-07-19:
  net: Use correct type for bool flag
  qapi: Change Netdev into a flat union
  block: Simplify drive-mirror
  block: Simplify block_set_io_throttle
  qapi: Implement boxed types for commands/events
  qapi: Plumb in 'boxed' to qapi generator lower levels
  qapi-event: Simplify visit of non-implicit data
  qapi: Drop useless gen_err_check()
  qapi: Add type.is_empty() helper
  qapi: Hide tag_name data member of variants
  qapi: Special case c_name() for empty type
  qapi: Require all branches of flat union enum to be covered
  net: use Netdev instead of NetClientOptions in client init
  qapi: change QmpInputVisitor to QSLIST
  qapi: change QmpOutputVisitor to QSLIST

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20160719' into...
Peter Maydell [Wed, 20 Jul 2016 11:48:18 +0000 (12:48 +0100)]
Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20160719' into staging

target-arm queue:
 * fix two minor Coverity complaints

# gpg: Signature made Tue 19 Jul 2016 18:02:34 BST
# gpg:                using RSA key 0x3C2525ED14360CDE
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>"
# gpg:                 aka "Peter Maydell <pmaydell@gmail.com>"
# gpg:                 aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>"
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83  15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-target-arm-20160719:
  arm_gicv3: Add assert()s to tell Coverity that offsets are aligned
  target-arm: Fix unreachable code in gicv3_class_name()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agodocker: pass EXECUTABLE to build script
Alex Bennée [Tue, 19 Jul 2016 13:20:45 +0000 (21:20 +0800)]
docker: pass EXECUTABLE to build script

To build a docker image with which needs qemu linux-user emulation we
need to pass --include-executable to the build script. Using the same
mechanism as for other container controls we enable the option is
EXECUTABLE is set on the make command line e.g:

    make docker-image-debian-bootstrap V=1 J=9 DEB_ARCH=armhf \
        DEB_TYPE=stable EXECUTABLE=./arm-linux-user/qemu-arm

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1468934445-32183-11-git-send-email-famz@redhat.com
Signed-off-by: Fam Zheng <famz@redhat.com>
8 years agodocker: Don't start a container that doesn't exist
Fam Zheng [Tue, 19 Jul 2016 13:20:44 +0000 (21:20 +0800)]
docker: Don't start a container that doesn't exist

Image building targets are dependencies of test running targets, so when
a docker image doesn't exist, it means it's skipped (due to dependency
checks in pre script). Therefore, skip the test too.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1468934445-32183-10-git-send-email-famz@redhat.com

8 years agodocker: Add "images" subcommand to docker.py
Fam Zheng [Tue, 19 Jul 2016 13:20:43 +0000 (21:20 +0800)]
docker: Add "images" subcommand to docker.py

This is a wrapper for the 'docker images' command.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1468934445-32183-9-git-send-email-famz@redhat.com

8 years agodocker: Fix exit code if $CMD failed
Fam Zheng [Tue, 19 Jul 2016 13:20:42 +0000 (21:20 +0800)]
docker: Fix exit code if $CMD failed

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1468934445-32183-8-git-send-email-famz@redhat.com

8 years agodocker: More sensible run script
Fam Zheng [Tue, 19 Jul 2016 13:20:41 +0000 (21:20 +0800)]
docker: More sensible run script

It is very easy to figure out current directory and bash option from the
execution, so do less in the Makefile invocation command line, and
figure both options in the script.

This makes the next patch easier.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1468934445-32183-7-git-send-email-famz@redhat.com

8 years agotests/docker/docker.py: add update operation
Alex Bennée [Tue, 19 Jul 2016 13:20:40 +0000 (21:20 +0800)]
tests/docker/docker.py: add update operation

This adds a new operation to the docker script to allow updating of
binaries in an existing container. This is because it would be
inefficient to re-build the whole container just for an update to the
QEMU binary.

To update the executable run:

    ./tests/docker/docker.py update \
        debian:armhf ./arm-linux-user/qemu-arm

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1468934445-32183-6-git-send-email-famz@redhat.com
Signed-off-by: Fam Zheng <famz@redhat.com>
8 years agotests/docker/dockerfiles: new debian-bootstrap.docker
Alex Bennée [Tue, 19 Jul 2016 13:20:39 +0000 (21:20 +0800)]
tests/docker/dockerfiles: new debian-bootstrap.docker

Together with the debian-bootstrap.pre script can now build an arbitrary
architecture of Debian using debootstrap. This allows debootstrap to set
up its first stage before the container is built.

To build a container you need a command line like:

  DEB_ARCH=armhf DEB_TYPE=testing \
    ./tests/docker/docker.py build \
    --include-executable=arm-linux-user/qemu-arm debian:armhf \
    ./tests/docker/dockerfiles/debian-bootstrap.docker

Although a number of non-debian systems package the debootstrap script
it is fairly portable in itself. Assuming we have some sort of fakeroot
implementation we can just clone the upstream repository and use the
script from there.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1468934445-32183-5-git-send-email-famz@redhat.com
Signed-off-by: Fam Zheng <famz@redhat.com>
8 years agotests/docker/docker.py: check and run .pre script
Alex Bennée [Tue, 19 Jul 2016 13:20:38 +0000 (21:20 +0800)]
tests/docker/docker.py: check and run .pre script

The docker script will now search for an associated $dockerfile.pre
script which gets run in the same build context as the dockerfile will
be. This is to support pre-seeding the build context before running the
docker build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1468934445-32183-4-git-send-email-famz@redhat.com
Signed-off-by: Fam Zheng <famz@redhat.com>
8 years agotests/docker/docker.py: support --include-executable
Alex Bennée [Tue, 19 Jul 2016 13:20:37 +0000 (21:20 +0800)]
tests/docker/docker.py: support --include-executable

When passed the path to a binary we copy it and any linked libraries (if
it is dynamically linked) into the docker build context. These can then
be included by a dockerfile with the line:

  # Copy all of context into container
  ADD . /

This is mainly intended for setting up foreign architecture docker
images which use qemu-$arch to do cross-architecture linux-user
execution. It also relies on the host and guest file-system following
reasonable multi-arch layouts so the copied libraries don't clash with
the guest ones.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1468934445-32183-3-git-send-email-famz@redhat.com
Signed-off-by: Fam Zheng <famz@redhat.com>
8 years agotests/docker/docker.py: docker_dir outside build
Alex Bennée [Tue, 19 Jul 2016 13:20:36 +0000 (21:20 +0800)]
tests/docker/docker.py: docker_dir outside build

Instead of letting the build_image create the temporary working dir we
move the creation to the build command. This is preparation for the
later patches where additional files can be added to the build context
before the build step is run.

We also ensure we remove the build context after we are done (mkdtemp
doesn't do this automatically for you).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1468934445-32183-2-git-send-email-famz@redhat.com
Signed-off-by: Fam Zheng <famz@redhat.com>
8 years agoMerge remote-tracking branch 'remotes/riku/tags/pull-linux-user-20160719-2' into...
Peter Maydell [Wed, 20 Jul 2016 10:32:12 +0000 (11:32 +0100)]
Merge remote-tracking branch 'remotes/riku/tags/pull-linux-user-20160719-2' into staging

linux-user fixes before 2.7 freeze, fix commit message

# gpg: Signature made Tue 19 Jul 2016 14:18:54 BST
# gpg:                using RSA key 0xB44890DEDE3C9BC0
# gpg: Good signature from "Riku Voipio <riku.voipio@iki.fi>"
# gpg:                 aka "Riku Voipio <riku.voipio@linaro.org>"
# Primary key fingerprint: FF82 03C8 C391 98AE 0581  41EF B448 90DE DE3C 9BC0

* remotes/riku/tags/pull-linux-user-20160719-2:
  linux-user: AArch64 has sync_file_range, not sync_file_range2
  linux-user: Fix type for SIOCATMARK ioctl
  linux-user: define missing sparc syscalls
  linux-user: Fix terminal control ioctls
  linux-user: Add some new blk ioctls
  linux-user: Handle short lengths in host_to_target_sockaddr()
  linux-user: Forget about synchronous signal once it is delivered
  linux-user: Correct type for LOOP_GET_STATUS{,64} ioctls
  linux-user: Correct type for BLKSSZGET
  linux-user: Add loop control ioctls
  linux-user: Check sigsetsize argument to syscalls
  linux-user: add nested netlink types
  linux-user: convert sockaddr_ll from host to target
  linux-user: add fd_trans helper in do_recvfrom()
  linux-user: fix netlink memory corruption
  linux-user: fd_trans_*_data() returns the length

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoqxl: fix qxl_set_dirty call in qxl_dirty_one_surface
Gerd Hoffmann [Wed, 13 Jul 2016 12:33:06 +0000 (14:33 +0200)]
qxl: fix qxl_set_dirty call in qxl_dirty_one_surface

qxl_set_dirty() expects start and end as range specification.
qxl_dirty_one_surface passes 'size' instead of 'offset + size' as end
parameter.  Fix that.  Also use uint64_t everywhere while being at it.

Bug was added by "e25139b qxl: set only off-screen surfaces dirty instead
of the whole vram" and carried forward unnoticed by "5cdc402 qxl: fix
surface migration".

Reported-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-id: 1468413187-22071-1-git-send-email-kraxel@redhat.com

8 years agoblock/gluster: add support for multiple gluster servers
Prasanna Kumar Kalever [Tue, 19 Jul 2016 16:57:33 +0000 (22:27 +0530)]
block/gluster: add support for multiple gluster servers

This patch adds a way to specify multiple volfile servers to the gluster
block backend of QEMU with tcp|rdma transport types and their port numbers.

Problem:

Currently VM Image on gluster volume is specified like this:

file=gluster[+tcp]://host[:port]/testvol/a.img

Say we have three hosts in a trusted pool with replica 3 volume in action.
When the host mentioned in the command above goes down for some reason,
the other two hosts are still available. But there's currently no way
to tell QEMU about them.

Solution:

New way of specifying VM Image on gluster volume with volfile servers:
(We still support old syntax to maintain backward compatibility)

Basic command line syntax looks like:

Pattern I:
 -drive driver=gluster,
        volume=testvol,path=/path/a.raw,[debug=N,]
        server.0.type=tcp,
        server.0.host=1.2.3.4,
        server.0.port=24007,
        server.1.type=unix,
        server.1.socket=/path/socketfile

Pattern II:
 'json:{"driver":"qcow2","file":{"driver":"gluster",
       "volume":"testvol","path":"/path/a.qcow2",["debug":N,]
       "server":[{hostinfo_1}, ...{hostinfo_N}]}}'

   driver      => 'gluster' (protocol name)
   volume      => name of gluster volume where our VM image resides
   path        => absolute path of image in gluster volume
  [debug]      => libgfapi loglevel [(0 - 9) default 4 -> Error]

  {hostinfo}   => {{type:"tcp",host:"1.2.3.4"[,port=24007]},
                   {type:"unix",socket:"/path/sockfile"}}

   type        => transport type used to connect to gluster management daemon,
                  it can be tcp|unix
   host        => host address (hostname/ipv4/ipv6 addresses/socket path)
   port        => port number on which glusterd is listening.
   socket      => path to socket file

Examples:
1.
 -drive driver=qcow2,file.driver=gluster,
        file.volume=testvol,file.path=/path/a.qcow2,file.debug=9,
        file.server.0.type=tcp,
        file.server.0.host=1.2.3.4,
        file.server.0.port=24007,
        file.server.1.type=unix,
        file.server.1.socket=/var/run/glusterd.socket
2.
  'json:{"driver":"qcow2","file":{"driver":"gluster","volume":"testvol",
         "path":"/path/a.qcow2","debug":9,"server":
         [{"type":"tcp","host":"1.2.3.4","port":"24007"},
          {"type":"unix","socket":"/var/run/glusterd.socket"}
         ]}}'

This patch gives a mechanism to provide all the server addresses, which are in
replica set, so in case host1 is down VM can still boot from any of the
active hosts.

This is equivalent to the backup-volfile-servers option supported by
mount.glusterfs (FUSE way of mounting gluster volume)

credits: sincere thanks to all the supporters

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-id: 1468947453-5433-6-git-send-email-prasanna.kalever@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agoblock/gluster: using new qapi schema
Prasanna Kumar Kalever [Tue, 19 Jul 2016 16:57:32 +0000 (22:27 +0530)]
block/gluster: using new qapi schema

this patch adds 'GlusterServer' related schema in qapi/block-core.json

[Jeff: minor fix-ups of comments and formatting, per patch reviews]

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1468947453-5433-5-git-send-email-prasanna.kalever@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agoblock/gluster: deprecate rdma support
Prasanna Kumar Kalever [Tue, 19 Jul 2016 16:57:31 +0000 (22:27 +0530)]
block/gluster: deprecate rdma support

gluster volfile server fetch happens through unix and/or tcp, it doesn't
support volfile fetch over rdma. The rdma code may actually mislead,
so to make sure things do not break, for now we fallback to tcp when requested
for rdma, with a warning.

If you are wondering how this worked all these days, its the gluster libgfapi
code which handles anything other than unix transport as socket/tcp, sad but
true.

Also gluster doesn't support ipv6 addresses, removing the ipv6 related
comments/docs section

[Jeff: Minor grammatical fixes in comments and commit message, per
review comments]

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 1468947453-5433-4-git-send-email-prasanna.kalever@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agoblock/gluster: code cleanup
Prasanna Kumar Kalever [Tue, 19 Jul 2016 16:57:30 +0000 (22:27 +0530)]
block/gluster: code cleanup

unified coding styles of multiline function arguments and other error functions
moved random declarations of structures and other list variables

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 1468947453-5433-3-git-send-email-prasanna.kalever@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agoblock/gluster: rename [server, volname, image] -> [host, volume, path]
Prasanna Kumar Kalever [Tue, 19 Jul 2016 16:57:29 +0000 (22:27 +0530)]
block/gluster: rename [server, volname, image] -> [host, volume, path]

A future patch will add support for multiple gluster servers. Existing
terminology is a bit unusual in relation to what names are used by
other networked devices, and doesn't map very well to the terminology
we expect to use for multiple servers.  Therefore, rename the following
options:
'server'  -> 'host'
'image'   -> 'path'
'volname' -> 'volume'

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 1468947453-5433-2-git-send-email-prasanna.kalever@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agomirror: fix request throttling in drive-mirror
Denis V. Lunev [Wed, 22 Jun 2016 12:35:27 +0000 (15:35 +0300)]
mirror: fix request throttling in drive-mirror

There are 2 deficiencies here:
- mirror_iteration could start several requests inside. Thus we could
  simply have more in_flight requests than MAX_IN_FLIGHT.
- keeping this in mind throttling in mirror_run which is checking
  s->in_flight == MAX_IN_FLIGHT is wrong.

The patch adds the check and throttling into mirror_iteration and fixes
the check in mirror_run() to be sure.

Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 1466598927-5990-1-git-send-email-den@openvz.org
CC: Jeff Cody <jcody@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
(cherry picked from commit e648dc95c28fbca12e67be26a1fc4b9a0676c3fe)

8 years agomirror: improve performance of mirroring of empty disk
Denis V. Lunev [Thu, 14 Jul 2016 13:33:29 +0000 (16:33 +0300)]
mirror: improve performance of mirroring of empty disk

We should not take into account zero blocks for delay calculations.
They are not read and thus IO throttling is not required. In the
other case VM migration with 16 Tb QCOW2 disk with 4 Gb of data takes
days.

Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1468503209-19498-9-git-send-email-den@openvz.org
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Jeff Cody <jcody@redhat.com>
CC: Eric Blake <eblake@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agomirror: efficiently zero out target
Denis V. Lunev [Thu, 14 Jul 2016 13:33:28 +0000 (16:33 +0300)]
mirror: efficiently zero out target

With a bdrv_co_write_zeroes method on a target BDS and when this method
is working as indicated by the bdrv_can_write_zeroes_with_unmap(), zeroes
will not be placed into the wire. Thus the target could be very efficiently
zeroed out. This should be done with the largest chunk possible.

Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy<vsementsov@virtuozzo.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1468503209-19498-8-git-send-email-den@openvz.org
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Jeff Cody <jcody@redhat.com>
CC: Eric Blake <eblake@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agomirror: optimize dirty bitmap filling in mirror_run a bit
Denis V. Lunev [Thu, 14 Jul 2016 13:33:27 +0000 (16:33 +0300)]
mirror: optimize dirty bitmap filling in mirror_run a bit

There is no need to scan allocation tables if we have mark_all_dirty flag
set. Just mark it all dirty.

Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy<vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1468503209-19498-7-git-send-email-den@openvz.org
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Jeff Cody <jcody@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agoblock: remove extra condition in bdrv_can_write_zeroes_with_unmap
Denis V. Lunev [Thu, 14 Jul 2016 13:33:26 +0000 (16:33 +0300)]
block: remove extra condition in bdrv_can_write_zeroes_with_unmap

All .bdrv_co_write_zeroes callbacks nowadays work perfectly even
with backing store attached. If future new callbacks would be unable to do
that - they have a chance to block this in bdrv_get_info().

Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1468503209-19498-6-git-send-email-den@openvz.org
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Jeff Cody <jcody@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agomirror: create mirror_dirty_init helper for mirror_run
Denis V. Lunev [Thu, 14 Jul 2016 13:33:25 +0000 (16:33 +0300)]
mirror: create mirror_dirty_init helper for mirror_run

The code inside the helper will be extended in the next patch. mirror_run
itself is overbloated at the moment.

Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy<vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1468503209-19498-5-git-send-email-den@openvz.org
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Jeff Cody <jcody@redhat.com>
CC: Eric Blake <eblake@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agomirror: create mirror_throttle helper
Denis V. Lunev [Thu, 14 Jul 2016 13:33:24 +0000 (16:33 +0300)]
mirror: create mirror_throttle helper

The patch also places last_pause_ns from stack in mirror_run into
MirrorBlockJob structure. This helper will be useful in next patches.

Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1468503209-19498-4-git-send-email-den@openvz.org
CC: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
CC: Eric Blake <eblake@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Fam Zheng <famz@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Jeff Cody <jcody@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agomirror: make sectors_in_flight int64_t
Denis V. Lunev [Thu, 14 Jul 2016 13:33:23 +0000 (16:33 +0300)]
mirror: make sectors_in_flight int64_t

We keep here the sum of int fields. Thus this could easily overflow,
especially when we will start sending big requests in next patches.

Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1468503209-19498-3-git-send-email-den@openvz.org
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Jeff Cody <jcody@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agodirty-bitmap: operate with int64_t amount
Denis V. Lunev [Thu, 14 Jul 2016 13:33:22 +0000 (16:33 +0300)]
dirty-bitmap: operate with int64_t amount

Underlying HBitmap operates even with uint64_t. Thus this change is safe.
This would be useful f.e. to mark entire bitmap dirty in one call.

Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1468503209-19498-2-git-send-email-den@openvz.org
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Jeff Cody <jcody@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
8 years agonet: Use correct type for bool flag
Eric Blake [Thu, 14 Jul 2016 03:50:24 +0000 (21:50 -0600)]
net: Use correct type for bool flag

is_netdev is only used as a bool, so make it one.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-14-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: Change Netdev into a flat union
Eric Blake [Thu, 14 Jul 2016 03:50:23 +0000 (21:50 -0600)]
qapi: Change Netdev into a flat union

This is a mostly-mechanical conversion that creates a new flat
union 'Netdev' QAPI type that covers all the branches of the
former 'NetClientOptions' simple union, where the branches are
now listed in a new 'NetClientDriver' enum rather than generated
from the simple union.  The existence of a flat union has no
change to the command line syntax accepted for new code, and
will make it possible for a future patch to switch the QMP
command to parse a boxed union for no change to valid QMP; but
it does have some ripple effect on the C code when dealing with
the new types.

While making the conversion, note that the 'NetLegacy' type
remains unchanged: it applies only to legacy command line options,
and will not be ported to QMP, so it should remain a wrapper
around a simple union; to avoid confusion, the type named
'NetClientOptions' is now gone, and we introduce 'NetLegacyOptions'
in its place.  Then, in the C code, we convert from NetLegacy to
Netdev as soon as possible, so that the bulk of the net stack
only has to deal with one QAPI type, not two.  Note that since
the old legacy code always rejected 'hubport', we can just omit
that branch from the new 'NetLegacyOptions' simple union.

Based on an idea originally by Zoltán Kővágó <DirtY.iCE.hu@gmail.com>:
Message-Id: <01a527fbf1a5de880091f98cf011616a78adeeee.1441627176.git.DirtY.iCE.hu@gmail.com>
although the sed script in that patch no longer applies due to
other changes in the tree since then, and I also did some manual
cleanups (such as fixing whitespace to keep checkpatch happy).

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-13-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Fixup from Eric squashed in]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoarm_gicv3: Add assert()s to tell Coverity that offsets are aligned
Peter Maydell [Mon, 11 Jul 2016 18:22:52 +0000 (19:22 +0100)]
arm_gicv3: Add assert()s to tell Coverity that offsets are aligned

Coverity complains that the GICR_IPRIORITYR case in gicv3_readl()
can overflow an array, because it doesn't know that the offsets
passed to that function must be word aligned. Add some assert()s
which hopefully tell Coverity that this isn't possible.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 1468261372-17508-1-git-send-email-peter.maydell@linaro.org

8 years agotarget-arm: Fix unreachable code in gicv3_class_name()
Peter Maydell [Mon, 11 Jul 2016 18:09:12 +0000 (19:09 +0100)]
target-arm: Fix unreachable code in gicv3_class_name()

Coverity complains that the exit() in gicv3_class_name()
can be unreachable, because if TARGET_AARCH64 is defined
then all code paths return before reaching it. Move the
exit() up to the error_report() that it belongs with.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Shannon Zhao <shannon.zhao@linaro.org>
Message-id: 1468260552-8400-1-git-send-email-peter.maydell@linaro.org

8 years agodisas: Fix ATTRIBUTE_UNUSED define clash with ALSA headers
Peter Maydell [Tue, 19 Jul 2016 14:04:36 +0000 (15:04 +0100)]
disas: Fix ATTRIBUTE_UNUSED define clash with ALSA headers

disas/bfd.h defines ATTRIBUTE_UNUSED, but unfortunately the
ALSA system headers also define this macro, which means that
you can get a compilation failure if building with ALSA and
any files happen to include the alsa headers before bfd.h
rather than the other way around.

This is unfortunate namespace pollution by the ALSA headers but
we can work around it. Add an #ifndef guard to bfd.h and remove
the unnecessary extra definition in disas/arm.c to fix this.

Reported-by: BALATON Zoltan <balaton@eik.bme.hu>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1468937076-21503-1-git-send-email-peter.maydell@linaro.org

8 years agoMerge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging
Peter Maydell [Tue, 19 Jul 2016 14:08:05 +0000 (15:08 +0100)]
Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* two old patches from prospective GSoC students
* i386 -kernel device tree support
* Coverity fix
* memory usage improvement from Peter
* checkpatch fix
* g_path_get_dirname cleanup
* caching of block status for iSCSI

# gpg: Signature made Tue 19 Jul 2016 07:43:41 BST
# gpg:                using RSA key 0xBFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
  target-i386: Remove redundant HF_SOFTMMU_MASK
  block/iscsi: allow caching of the allocation map
  block/iscsi: fix rounding in iscsi_allocationmap_set
  Move README to markdown
  cpu-exec: Move down some declarations in cpu_exec()
  exec: avoid realloc in phys_map_node_reserve
  checkpatch: consider git extended headers valid patches
  megasas: remove useless check for cmd->frame
  compiler: never omit assertions if using a static analysis tool
  hw/i386: add device tree support
  Changed malloc to g_malloc, free to g_free in bsd-user/qemu.h
  use g_path_get_dirname instead of dirname

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/stsquad/tags/pull-travis-20160718-1' into staging
Peter Maydell [Tue, 19 Jul 2016 13:41:41 +0000 (14:41 +0100)]
Merge remote-tracking branch 'remotes/stsquad/tags/pull-travis-20160718-1' into staging

Make IRC a little less noisy

# gpg: Signature made Mon 18 Jul 2016 16:42:57 BST
# gpg:                using RSA key 0xFBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>"
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-travis-20160718-1:
  .travis.yml: Disable IRC build status updates from forks

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agolinux-user: AArch64 has sync_file_range, not sync_file_range2
Peter Maydell [Fri, 15 Jul 2016 16:28:06 +0000 (17:28 +0100)]
linux-user: AArch64 has sync_file_range, not sync_file_range2

The AArch64 Linux ABI syscall 84 is sync_file_range, not
sync_file_range2 (in the kernel it uses the asm-generic
headers and does not define __ARCH_WANT_SYNC_FILE_RANGE2).
Update our TARGET_NR_* definitions accordingly.

This fixes the sync_file_range syscall which otherwise
gets its arguments in the wrong order.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: Fix type for SIOCATMARK ioctl
Peter Maydell [Fri, 15 Jul 2016 11:09:31 +0000 (12:09 +0100)]
linux-user: Fix type for SIOCATMARK ioctl

The SIOCATMARK ioctl takes an argument which should be a
pointer to an integer where the kernel will write the result.
We were incorrectly declaring it as TYPE_NULL which would mean
it would always fail (with EFAULT) when it should succeed.
Correct the type.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: define missing sparc syscalls
Laurent Vivier [Fri, 8 Jul 2016 19:47:08 +0000 (21:47 +0200)]
linux-user: define missing sparc syscalls

NR_lookup_dcookie, NR_fadvise64, NR_fadvise64_64

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: Fix terminal control ioctls
Timothy Pearson [Sun, 19 Jun 2016 00:15:35 +0000 (19:15 -0500)]
linux-user: Fix terminal control ioctls

TIOCGPTN and related terminal control ioctls were not converted to the guest ioctl format on x86_64 targets. Convert these ioctls to enable terminal functionality on x86_64 guests.

Signed-off-by: Timothy Pearson <tpearson@raptorengineering.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/kraxel/tags/pull-vnc-20160719-1' into staging
Peter Maydell [Tue, 19 Jul 2016 13:07:27 +0000 (14:07 +0100)]
Merge remote-tracking branch 'remotes/kraxel/tags/pull-vnc-20160719-1' into staging

vnc: bugfixes for -rc0

# gpg: Signature made Tue 19 Jul 2016 08:27:05 BST
# gpg:                using RSA key 0x4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>"
# gpg:                 aka "Gerd Hoffmann <gerd@kraxel.org>"
# gpg:                 aka "Gerd Hoffmann (private) <kraxel@gmail.com>"
# Primary key fingerprint: A032 8CFF B93A 17A7 9901  FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/pull-vnc-20160719-1:
  vnc-tight: fix regression with libxenstore
  vnc-enc-tight: fix off-by-one bug
  vnc: make sure we finish disconnect

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/mcayland/tags/qemu-openbios-signed' into staging
Peter Maydell [Tue, 19 Jul 2016 12:41:31 +0000 (13:41 +0100)]
Merge remote-tracking branch 'remotes/mcayland/tags/qemu-openbios-signed' into staging

Update OpenBIOS images

# gpg: Signature made Tue 19 Jul 2016 07:42:30 BST
# gpg:                using RSA key 0x5BC2C56FAE0F321F
# gpg: Good signature from "Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>"
# Primary key fingerprint: CC62 1AB9 8E82 200D 915C  C9C4 5BC2 C56F AE0F 321F

* remotes/mcayland/tags/qemu-openbios-signed:
  Update OpenBIOS images to e79bca6 built from submodule.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agolinux-user: Add some new blk ioctls
Peter Maydell [Fri, 1 Jul 2016 18:46:04 +0000 (19:46 +0100)]
linux-user: Add some new blk ioctls

Add some new blk ioctls (these are 0x12,119 through
to 0x12,127). Several of these are used by mke2fs; this silences
the warnings:

mke2fs 1.42.12 (29-Aug-2014)
Unsupported ioctl: cmd=0x127b
Unsupported ioctl: cmd=0x127a
warning: Unable to get device geometry for /dev/loop5
Unsupported ioctl: cmd=0x127c
Unsupported ioctl: cmd=0x127c
Unsupported ioctl: cmd=0x1277

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: Handle short lengths in host_to_target_sockaddr()
Peter Maydell [Thu, 7 Jul 2016 14:44:43 +0000 (15:44 +0100)]
linux-user: Handle short lengths in host_to_target_sockaddr()

If userspace specifies a short buffer for a target sockaddr,
the kernel will only copy in as much as it has space for
(or none at all if the length is zero) -- see the kernel
move_addr_to_user() function. Mimic this in QEMU's
host_to_target_sockaddr() routine.

In particular, this fixes a segfault running the LTP
recvfrom01 test, where the guest makes a recvfrom()
call with a bad buffer pointer and other parameters which
cause the kernel to set the addrlen to zero; because we
did not skip the attempt to swap the sa_family field we
segfaulted on the bad address.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: Forget about synchronous signal once it is delivered
Peter Maydell [Wed, 6 Jul 2016 14:09:29 +0000 (15:09 +0100)]
linux-user: Forget about synchronous signal once it is delivered

Commit 655ed67c2a248cf which switched synchronous signals to
benig recorded in ts->sync_signal rather than in a queue
with every other signal had a bug: we failed to clear
the flag indicating that a synchronous signal was pending
when we delivered it. This meant that we would take the signal
again and again every time the guest made a syscall.
(This is a bug introduced in my refactoring of Timothy Baldwin's
original code.)

Fix this by passing in the struct emulated_sigtable* to
handle_pending_signal(), so that we clear the pending flag
in the ts->sync_signal struct when handling a synchronous signal.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: Correct type for LOOP_GET_STATUS{,64} ioctls
Peter Maydell [Tue, 5 Jul 2016 15:36:29 +0000 (16:36 +0100)]
linux-user: Correct type for LOOP_GET_STATUS{,64} ioctls

The LOOP_GET_STATUS and LOOP_GET_STATUS64 ioctls were incorrectly
defined as IOC_W rather than IOC_R, which meant we weren't
correctly copying the information back from the kernel to the guest.
The loop_info64 structure definition was also missing a member
and using the wrong type for several 32-bit fields.

In particular, this meant that "kpartx -d image.img" didn't work
and "losetup -a" behaved strangely. Correct the ioctl type definitions.

Reported-by: Chanho Park <chanho61.park@samsung.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: Correct type for BLKSSZGET
Peter Maydell [Mon, 4 Jul 2016 16:06:18 +0000 (17:06 +0100)]
linux-user: Correct type for BLKSSZGET

The BLKSSZGET ioctl takes an argument which is a pointer to an int.
We were incorrectly declaring it to take a pointer to a long, which
meant that we would incorrectly write to memory which we should not
if the guest is a 64-bit architecture.

In particular, kpartx uses this ioctl to write to an int on the
stack, which tends to result in it crashing immediately.

Reported-by: Chanho Park <chanho61.park@samsung.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: Add loop control ioctls
Peter Maydell [Mon, 4 Jul 2016 16:06:17 +0000 (17:06 +0100)]
linux-user: Add loop control ioctls

Add support for the /dev/loop-control ioctls:
 LOOP_CTL_ADD
 LOOP_CTL_REMOVE
 LOOP_CTL_GET_FREE

[RV: fixed to apply to new header guards]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: Check sigsetsize argument to syscalls
Peter Maydell [Thu, 30 Jun 2016 13:23:24 +0000 (14:23 +0100)]
linux-user: Check sigsetsize argument to syscalls

Many syscalls which take a sigset_t argument also take an argument
giving the size of the sigset_t.  The kernel insists that this
matches its idea of the type size and fails EINVAL if it is not.
Implement this logic in QEMU.  (This mostly just means some LTP test
cases which check error cases now pass.)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
8 years agolinux-user: add nested netlink types
Laurent Vivier [Mon, 27 Jun 2016 16:54:30 +0000 (18:54 +0200)]
linux-user: add nested netlink types

Nested types are used by the kernel to send link information and
protocol properties.

We can see following errors with "ip link show":

Unimplemented nested type 26
Unimplemented nested type 26
Unimplemented nested type 18
Unimplemented nested type 26
Unimplemented nested type 18
Unimplemented nested type 26

This patch implements nested types 18 (IFLA_LINKINFO) and
26 (IFLA_AF_SPEC).

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: convert sockaddr_ll from host to target
Laurent Vivier [Sun, 26 Jun 2016 22:18:22 +0000 (00:18 +0200)]
linux-user: convert sockaddr_ll from host to target

As we convert sockaddr for AF_PACKET family for sendto() (target to
host) we need also to convert this for getsockname() (host to target).

arping uses getsockname() to get the the interface address and uses
this address with sendto().

Tested with:

    /sbin/arping -D -q -c2 -I eno1 192.168.122.88

...
getsockname(3, {sa_family=AF_PACKET, proto=0x806, if2,
pkttype=PACKET_HOST, addr(6)={1, 10c37b6b9a76}, [18]) = 0
...
sendto(3, "..." 28, 0,
       {sa_family=AF_PACKET, proto=0x806, if2, pkttype=PACKET_HOST,
       addr(6)={1, ffffffffffff}, 20) = 28
...

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: add fd_trans helper in do_recvfrom()
Laurent Vivier [Tue, 21 Jun 2016 17:51:15 +0000 (19:51 +0200)]
linux-user: add fd_trans helper in do_recvfrom()

Fix passwd using netlink audit.

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: fix netlink memory corruption
Laurent Vivier [Tue, 21 Jun 2016 17:51:14 +0000 (19:51 +0200)]
linux-user: fix netlink memory corruption

Netlink is byte-swapping data in the guest memory (it's bad).

It's ok when the data come from the host as they are generated by the
host.

But it doesn't work when data come from the guest: the guest can
try to reuse these data whereas they have been byte-swapped.

This is what happens in glibc:

glibc generates a sequence number in nlh.nlmsg_seq and calls
sendto() with this nlh. In sendto(), we byte-swap nlmsg.seq.

Later, after the recvmsg(), glibc compares nlh.nlmsg_seq with
sequence number given in return, and of course it fails (hangs),
because nlh.nlmsg_seq is not valid anymore.

The involved code in glibc is:

sysdeps/unix/sysv/linux/check_pf.c:make_request()
...
  req.nlh.nlmsg_seq = time (NULL);
...
  if (TEMP_FAILURE_RETRY (__sendto (fd, (void *) &req, sizeof (req), 0,
                                    (struct sockaddr *) &nladdr,
                                    sizeof (nladdr))) < 0)
<here req.nlh.nlmsg_seq has been byte-swapped>
...
  do
    {
...
      ssize_t read_len = TEMP_FAILURE_RETRY (__recvmsg (fd, &msg, 0));
...
      struct nlmsghdr *nlmh;
      for (nlmh = (struct nlmsghdr *) buf;
           NLMSG_OK (nlmh, (size_t) read_len);
           nlmh = (struct nlmsghdr *) NLMSG_NEXT (nlmh, read_len))
        {
<we compare nlmh->nlmsg_seq with corrupted req.nlh.nlmsg_seq>
          if (nladdr.nl_pid != 0 || (pid_t) nlmh->nlmsg_pid != pid
              || nlmh->nlmsg_seq != req.nlh.nlmsg_seq)
            continue;
...
          else if (nlmh->nlmsg_type == NLMSG_DONE)
            /* We found the end, leave the loop.  */
            done = true;
        }
    }
  while (! done);

As we have a continue on "nlmh->nlmsg_seq != req.nlh.nlmsg_seq",
"done" cannot be set to "true" and we have an infinite loop.

It's why commands like "apt-get update" or "dnf update hangs".

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agolinux-user: fd_trans_*_data() returns the length
Laurent Vivier [Tue, 21 Jun 2016 17:51:13 +0000 (19:51 +0200)]
linux-user: fd_trans_*_data() returns the length

fd_trans_target_to_host_data() and fd_trans_host_to_target_data() must
return the length of processed data.

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Riku Voipio <riku.voipio@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging
Peter Maydell [Tue, 19 Jul 2016 12:00:35 +0000 (13:00 +0100)]
Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging

# gpg: Signature made Tue 19 Jul 2016 03:33:40 BST
# gpg:                using RSA key 0xEF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F  3562 EF04 965B 398D 6211

* remotes/jasowang/tags/net-pull-request:
  e1000e: fix building without CONFIG_VMXNET3_PCI
  MAINTAINERS: release Scott from being a rocker maintainer
  tap: fix memory leak on failure to create a multiqueue tap device
  net: fix incorrect argument to iov_to_buf
  net: fix incorrect access to pointer
  e1000e: fix incorrect access to pointer

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoblock: Simplify drive-mirror
Eric Blake [Thu, 14 Jul 2016 22:37:58 +0000 (16:37 -0600)]
block: Simplify drive-mirror

Now that we can support boxed commands, use it to greatly
reduce the number of parameters (and likelihood of getting
out of sync) when adjusting drive-mirror parameters.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <1468535878-3760-1-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoblock: Simplify block_set_io_throttle
Eric Blake [Thu, 14 Jul 2016 03:50:21 +0000 (21:50 -0600)]
block: Simplify block_set_io_throttle

Now that we can support boxed commands, use it to greatly
reduce the number of parameters (and likelihood of getting
out of sync) when adjusting throttle parameters.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <1468468228-27827-11-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: Implement boxed types for commands/events
Eric Blake [Thu, 14 Jul 2016 03:50:20 +0000 (21:50 -0600)]
qapi: Implement boxed types for commands/events

Turn on the ability to pass command and event arguments in
a single boxed parameter, which must name a non-empty type
(although the type can be a struct with all optional members).
For structs, it makes it possible to pass a single qapi type
instead of a breakout of all struct members (useful if the
arguments are already in a struct or if the number of members
is large); for other complex types, it is now possible to use
a union or alternate as the data for a command or event.

The empty type may be technically feasible if needed down the
road, but it's easier to forbid it now and relax things to allow
it later, than it is to allow it now and have to special case
how the generated 'q_empty' type is handled (see commit 7ce106a9
for reasons why nothing is generated for the empty type).  An
alternate type is never considered empty, but now that a boxed
type can be either an object or an alternate, we have to provide
a trivial QAPISchemaAlternateType.is_empty().  The new call to
arg_type.is_empty() during QAPISchemaCommand.check() requires
that we first check the type in question; but there is no chance
of introducing a cycle since objects do not refer back to commands.

We still have a split in syntax checking between ad-hoc parsing
up front (merely validates that 'boxed' has a sane value) and
during .check() methods (if 'boxed' is set, then 'data' must name
a non-empty user-defined type).

Generated code is unchanged, as long as no client uses the
new feature.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-10-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Test files renamed to *-boxed-*]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: Plumb in 'boxed' to qapi generator lower levels
Eric Blake [Thu, 14 Jul 2016 03:50:19 +0000 (21:50 -0600)]
qapi: Plumb in 'boxed' to qapi generator lower levels

The next patch will add support for passing a qapi union type
as the 'data' of a command.  But to do that, the user function
for implementing the command, as called by the generated
marshal command, must take the corresponding C struct as a
single boxed pointer, rather than a breakdown into one
parameter per member.  Even without a union, being able to use
a C struct rather than a list of parameters can make it much
easier to handle coding with QAPI.

This patch adds the internal plumbing of a 'boxed' flag
associated with each command and event.  In several cases,
this means adding indentation, with one new dead branch and
the remaining branch being the original code more deeply
nested; this was done so that the new implementation in the
next patch is easier to review without also being mixed with
indentation changes.

For this patch, no behavior or generated output changes, other
than the testsuite outputting the value of the new flag
(always False for now).

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-9-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Identifier box renamed to boxed in two places]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi-event: Simplify visit of non-implicit data
Eric Blake [Thu, 14 Jul 2016 03:50:18 +0000 (21:50 -0600)]
qapi-event: Simplify visit of non-implicit data

Commit 7ce106a9 documented why we don't generated a visit_type_FOO()
for implicit types; and therefore events with an anonymous type for
'data' have to open-code a visit.  Note that the open-coded visit in
qapi-event.c is slightly different from what is done in
qapi-visit.c for normal types, in part because we don't have to
check for *obj being NULL or free things on error.  But where the
type is not implicit, it is nicer to reuse the normal visit instead
of open-coding a duplicate.

At the moment, the only event with a non-implicit 'data' is in the
testsuite, where test-qapi-event.c changes as follows:

|@@ -155,6 +155,7 @@ void qapi_event_send___org_qemu_x_event(
|     __org_qemu_x_Struct param = {
|         __org_qemu_x_member1, (char *)__org_qemu_x_member2, has_q_wchar_t, q_wchar_t
|     };
|+    __org_qemu_x_Struct *arg = &param;
|
|     emit = qmp_event_get_func_emit();
|     if (!emit) {
|@@ -164,16 +165,7 @@ void qapi_event_send___org_qemu_x_event(
|     qmp = qmp_event_build_dict("__ORG.QEMU_X-EVENT");
|
|     v = qmp_output_visitor_new(&obj);
|-
|-    visit_start_struct(v, "__ORG.QEMU_X-EVENT", NULL, 0, &err);
|-    if (err) {
|-        goto out;
|-    }
|-    visit_type___org_qemu_x_Struct_members(v, &param, &err);
|-    if (!err) {
|-    if (!err) {
|-        visit_check_struct(v, &err);
|-    }
|-    visit_end_struct(v, NULL);
|+    visit_type___org_qemu_x_Struct(v, "__ORG.QEMU_X-EVENT", &arg, &err);
|     if (err) {
|         goto out;
|     }

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-8-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: Drop useless gen_err_check()
Eric Blake [Thu, 14 Jul 2016 03:50:17 +0000 (21:50 -0600)]
qapi: Drop useless gen_err_check()

Ever since commit 12f254f removed the last parameterization
of gen_err_check(), it no longer makes sense to hide the three
lines of generated C code behind a macro call. Just inline it
into the remaining users.

No change to generated code.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-7-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: Add type.is_empty() helper
Eric Blake [Thu, 14 Jul 2016 03:50:16 +0000 (21:50 -0600)]
qapi: Add type.is_empty() helper

In the near future, we want to lift our artificial restriction of
no variants at the top level of an event, at which point the
currently open-coded check for empty members will become
insufficient.  Factor it out into a new helper method is_empty()
now, and future-proof it by checking variants, too, along with an
assert that it is not used prior to the completion of .check().
Update places that were checking for (non-)empty .members to use
the new helper.

All of the current callers assert that there are no variants (either
directly, or by qapi.py asserting that base types have no variants),
so this is not a semantic change.

No change to generated code.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-6-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: Hide tag_name data member of variants
Eric Blake [Thu, 14 Jul 2016 03:50:15 +0000 (21:50 -0600)]
qapi: Hide tag_name data member of variants

Clean up the only remaining external use of the tag_name field of
QAPISchemaObjectTypeVariants, by explicitly listing the generated
'type' tag for all variants in the testsuite (you can still tell
simple unions by the -wrapper types).  Then we can mark the
tag_name field as private by adding a leading underscore to prevent
any further use.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-5-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: Special case c_name() for empty type
Eric Blake [Thu, 14 Jul 2016 03:50:14 +0000 (21:50 -0600)]
qapi: Special case c_name() for empty type

Commit 7ce106a rendered QAPISchemaObjectType.c_name() redundant,
since it now does nothing more than delegate to its superclass.
However, rather than deleting it, we can restore part of the
assertion that was removed in that commit, to prove that we never
emit the empty type directly in generated code, but rather
special-case it as a built-in that makes other aspects of code
generation easier to reason about.

Reported-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-4-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: Require all branches of flat union enum to be covered
Eric Blake [Thu, 14 Jul 2016 03:50:13 +0000 (21:50 -0600)]
qapi: Require all branches of flat union enum to be covered

We were previously enforcing that all flat union branches were
found in the corresponding enum, but not that all enum values
were covered by branches.  The resulting generated code would
abort() if the user passes the uncovered enum value.

We don't automatically treat non-present branches in a flat
union as empty types, for symmetry with simple unions (there,
the enum type is generated from the list of all branches, so
there is no way to omit a branch but still have it be part of
the union).

A later patch will add shorthand so that branches that are empty
in flat unions can be declared as 'branch':{} instead of
'branch':'Empty', to avoid the need for an otherwise useless
explicit empty type.  [Such shorthand for simple unions is a bit
harder to justify, since we would still have to generate a
wrapper type that parses 'data':{}, rather than truly being an
empty branch with no additional siblings to the 'type' member.]

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-3-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agonet: use Netdev instead of NetClientOptions in client init
Kővágó, Zoltán [Thu, 14 Jul 2016 03:50:12 +0000 (21:50 -0600)]
net: use Netdev instead of NetClientOptions in client init

This way we no longer need NetClientOptions and can convert Netdev
into a flat union.

Signed-off-by: Kővágó, Zoltán <DirtY.iCE.hu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <93ffdfed7054529635e6acb935150d95dc173a12.1441627176.git.DirtY.iCE.hu@gmail.com>

[rework net_client_init1() to pass Netdev by copying from NetdevLegacy,
rather than merging the two types - which means that we still need
NetClientOptions after all.  Rebase to qapi changes. The bulk of the
patch is mechanical, replacing 'opts' by 'netdev->opts', while
net_client_init1() takes care of converting between legacy and modern
types.]

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1468468228-27827-2-git-send-email-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: change QmpInputVisitor to QSLIST
Paolo Bonzini [Thu, 7 Jul 2016 15:53:18 +0000 (17:53 +0200)]
qapi: change QmpInputVisitor to QSLIST

This saves a lot of memory compared to a statically-sized array,
or at least 24kb could be considered a lot on an Atari ST.
It also makes the code more similar to QmpOutputVisitor.

This removes the limit on the depth of a QObject that can be processed
into a QAPI tree.  This is not a problem because QObjects can be
considered trusted; the text received on the QMP wire is untrusted
input, but the JSON parser already takes pains to limit the QObject tree
it creates.  We don't need the QMP input visitor to limit it again.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1467906798-5312-3-git-send-email-pbonzini@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message typo fixed]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoqapi: change QmpOutputVisitor to QSLIST
Paolo Bonzini [Thu, 7 Jul 2016 15:53:17 +0000 (17:53 +0200)]
qapi: change QmpOutputVisitor to QSLIST

This saves a little memory compared to the doubly-linked QTAILQ.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1467906798-5312-2-git-send-email-pbonzini@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Comment tweaked to avoid long line]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
8 years agoMerge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging
Peter Maydell [Tue, 19 Jul 2016 10:47:06 +0000 (11:47 +0100)]
Merge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging

# gpg: Signature made Mon 18 Jul 2016 23:53:15 BST
# gpg:                using RSA key 0x7DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>"
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/ide-pull-request:
  block: ignore flush requests when storage is clean
  tests: in IDE and AHCI tests perform DMA write before flushing
  ide: set retry_unit for PIO and FLUSH requests
  ide: refactor retry_unit set and clear into separate function

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' into staging
Peter Maydell [Tue, 19 Jul 2016 09:54:49 +0000 (10:54 +0100)]
Merge remote-tracking branch 'remotes/stefanha/tags/tracing-pull-request' into staging

# gpg: Signature made Mon 18 Jul 2016 22:59:55 BST
# gpg:                using RSA key 0x9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/tracing-pull-request:
  trace: Add QAPI/QMP interfaces to query and control per-vCPU tracing state
  trace: Allow event name pattern in "info trace-events"
  trace: Conditionally trace events based on their per-vCPU state
  trace: Add per-vCPU tracing states for events with the 'vcpu' property
  trace: Cosmetic changes on fast-path tracing
  disas: Remove unused macro '_'
  trace: Identify events with the 'vcpu' property
  trace: [bsd-user] Commandline arguments to control tracing
  trace: [linux-user] Commandline arguments to control tracing

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoMerge remote-tracking branch 'remotes/awilliam/tags/vfio-update-20160718.0' into...
Peter Maydell [Tue, 19 Jul 2016 08:02:05 +0000 (09:02 +0100)]
Merge remote-tracking branch 'remotes/awilliam/tags/vfio-update-20160718.0' into staging

VFIO update 2016-07-18

One fix for 2.7-rc0 which hides the ARI extended capability, fixing
multifunction support in PCIe configurations where the assigned device
function topology does not match the host (Alex Williamson)

# gpg: Signature made Mon 18 Jul 2016 18:02:27 BST
# gpg:                using RSA key 0x239B9B6E3BB08B22
# gpg: Good signature from "Alex Williamson <alex.williamson@redhat.com>"
# gpg:                 aka "Alex Williamson <alex@shazbot.org>"
# gpg:                 aka "Alex Williamson <alwillia@redhat.com>"
# gpg:                 aka "Alex Williamson <alex.l.williamson@gmail.com>"
# Primary key fingerprint: 42F6 C04E 540B D1A9 9E7B  8A90 239B 9B6E 3BB0 8B22

* remotes/awilliam/tags/vfio-update-20160718.0:
  vfio/pci: Hide ARI capability

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agoUpdate OpenBIOS images to e79bca6 built from submodule.
Mark Cave-Ayland [Tue, 19 Jul 2016 06:41:52 +0000 (07:41 +0100)]
Update OpenBIOS images to e79bca6 built from submodule.

Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
8 years agotarget-i386: Remove redundant HF_SOFTMMU_MASK
Sergey Fedorov [Fri, 15 Jul 2016 17:58:45 +0000 (20:58 +0300)]
target-i386: Remove redundant HF_SOFTMMU_MASK

'HF_SOFTMMU_MASK' is only set when 'CONFIG_SOFTMMU' is defined. So
there's no need in this flag: test 'CONFIG_SOFTMMU' instead.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Sergey Fedorov <serge.fdrv@gmail.com>
Signed-off-by: Sergey Fedorov <sergey.fedorov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20160715175852.30749-6-sergey.fedorov@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
8 years agoblock/iscsi: allow caching of the allocation map
Peter Lieven [Mon, 18 Jul 2016 08:52:20 +0000 (10:52 +0200)]
block/iscsi: allow caching of the allocation map

until now the allocation map was used only as a hint if a cluster
is allocated or not. If a block was not allocated (or Qemu had
no info about the allocation status) a get_block_status call was
issued to check the allocation status and possibly avoid
a subsequent read of unallocated sectors. If a block known to be
allocated the get_block_status call was omitted. In the other case
a get_block_status call was issued before every read to avoid
the necessity for a consistent allocation map. To avoid the
potential overhead of calling get_block_status for each and
every read request this took only place for the bigger requests.

This patch enhances this mechanism to cache the allocation
status and avoid calling get_block_status for blocks where
the allocation status has been queried before. This allows
for bypassing the read request even for smaller requests and
additionally omits calling get_block_status for known to be
unallocated blocks.

Signed-off-by: Peter Lieven <pl@kamp.de>
Message-Id: <1468831940-15556-3-git-send-email-pl@kamp.de>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
8 years agoblock/iscsi: fix rounding in iscsi_allocationmap_set
Peter Lieven [Mon, 18 Jul 2016 08:52:19 +0000 (10:52 +0200)]
block/iscsi: fix rounding in iscsi_allocationmap_set

when setting clusters as alloacted the boundaries have
to be expanded. As Paolo pointed out the calculation of
the number of clusters is wrong:

Suppose cluster_sectors is 2, sector_num = 1, nb_sectors = 6:

In the "mark allocated" case, you want to set 0..8, i.e.
cluster_num=0, nb_clusters=4.

   0--.--2--.--4--.--6--.--8
   <--|_________________|-->  (<--> = expanded)

Instead you are setting nb_clusters=3, so that 6..8 is not marked.

   0--.--2--.--4--.--6--.--8
   <--|______________|!!!     (! = wrong)

Cc: qemu-stable@nongnu.org
Reported-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Peter Lieven <pl@kamp.de>
Message-Id: <1468831940-15556-2-git-send-email-pl@kamp.de>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
8 years agoMove README to markdown
Pranith Kumar [Fri, 15 Jul 2016 04:31:11 +0000 (00:31 -0400)]
Move README to markdown

Move the README file to markdown so that it makes the github page look
prettier. I know that github repo is a mirror and not the official
repo, but I think it doesn't hurt to have it in markdown format.

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Message-Id: <20160715043111.29007-1-bobby.prani@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
8 years agoblock: ignore flush requests when storage is clean
Evgeny Yakovlev [Mon, 18 Jul 2016 19:39:52 +0000 (22:39 +0300)]
block: ignore flush requests when storage is clean

Some guests (win2008 server for example) do a lot of unnecessary
flushing when underlying media has not changed. This adds additional
overhead on host when calling fsync/fdatasync.

This change introduces a write generation scheme in BlockDriverState.
Current write generation is checked against last flushed generation to
avoid unnessesary flushes.

The problem with excessive flushing was found by a performance test
which does parallel directory tree creation (from 2 processes).
Results improved from 0.424 loops/sec to 0.432 loops/sec.
Each loop creates 10^3 directories with 10 files in each.

This affected some blkdebug testcases that were expecting error logs from
failure-injected flushes which are now skipped entirely
(tests 026 071 089).

This also affects the performance of block jobs and thus BLOCK_JOB_READY
events for driver-mirror and active block-commit commands now arrives
faster, before QMP send successfully returns to caller (tests 141 144).

Signed-off-by: Evgeny Yakovlev <eyakovlev@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1468870792-7411-5-git-send-email-den@openvz.org
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Fam Zheng <famz@redhat.com>
CC: John Snow <jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
8 years agotests: in IDE and AHCI tests perform DMA write before flushing
Evgeny Yakovlev [Mon, 18 Jul 2016 19:39:51 +0000 (22:39 +0300)]
tests: in IDE and AHCI tests perform DMA write before flushing

Due to changes in flush behaviour clean disks stopped generating
flush_to_disk events and IDE and AHCI tests that test flush commands
started to fail.

This change adds additional DMA writes to affected tests before sending
flush commands so that bdrv_flush actually generates flush_to_disk event.

Signed-off-by: Evgeny Yakovlev <eyakovlev@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1468870792-7411-4-git-send-email-den@openvz.org
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Fam Zheng <famz@redhat.com>
CC: John Snow <jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
8 years agoide: set retry_unit for PIO and FLUSH requests
Evgeny Yakovlev [Mon, 18 Jul 2016 19:39:50 +0000 (22:39 +0300)]
ide: set retry_unit for PIO and FLUSH requests

The following sequence of tests discovered a problem in IDE emulation:
1. Send DMA write to IDE device 0
2. Send CMD_FLUSH_CACHE to same IDE device which will be failed by block
layer using blkdebug script in tests/ide-test:test_retry_flush

When doing DMA request ide/core.c will set s->retry_unit to s->unit in
ide_start_dma. When dma completes ide_set_inactive sets retry_unit to -1.
After that ide_flush_cache runs and fails thanks to blkdebug.
ide_flush_cb calls ide_handle_rw_error which asserts that s->retry_unit
== s->unit. But s->retry_unit is still -1 after previous DMA completion
and flush does not use anything related to retry.

This patch restricts retry unit assertion only to ops that actually use
retry logic.

Signed-off-by: Evgeny Yakovlev <eyakovlev@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1468870792-7411-3-git-send-email-den@openvz.org
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Fam Zheng <famz@redhat.com>
CC: John Snow <jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
8 years agoide: refactor retry_unit set and clear into separate function
Evgeny Yakovlev [Mon, 18 Jul 2016 19:39:49 +0000 (22:39 +0300)]
ide: refactor retry_unit set and clear into separate function

Code to set and clear state associated with retry in moved into
ide_set_retry and ide_clear_retry to make adding retry setups easier.

Signed-off-by: Evgeny Yakovlev <eyakovlev@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1468870792-7411-2-git-send-email-den@openvz.org
CC: Kevin Wolf <kwolf@redhat.com>
CC: Max Reitz <mreitz@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Fam Zheng <famz@redhat.com>
CC: John Snow <jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
8 years agotrace: Add QAPI/QMP interfaces to query and control per-vCPU tracing state
Lluís Vilanova [Mon, 11 Jul 2016 10:53:57 +0000 (12:53 +0200)]
trace: Add QAPI/QMP interfaces to query and control per-vCPU tracing state

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agotrace: Allow event name pattern in "info trace-events"
Lluís Vilanova [Mon, 11 Jul 2016 10:53:51 +0000 (12:53 +0200)]
trace: Allow event name pattern in "info trace-events"

Homogenizes the command capabilities with QMP.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agotrace: Conditionally trace events based on their per-vCPU state
Lluís Vilanova [Mon, 11 Jul 2016 10:53:46 +0000 (12:53 +0200)]
trace: Conditionally trace events based on their per-vCPU state

Events with the 'vcpu' property are conditionally emitted according to
their per-vCPU state. Other events are emitted normally based on their
global tracing state.

Note that the per-vCPU condition check applies to all tracing backends.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agotrace: Add per-vCPU tracing states for events with the 'vcpu' property
Lluís Vilanova [Mon, 11 Jul 2016 10:53:41 +0000 (12:53 +0200)]
trace: Add per-vCPU tracing states for events with the 'vcpu' property

Each vCPU gets a 'trace_dstate' bitmap to control the per-vCPU dynamic
tracing state of events with the 'vcpu' property.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agotrace: Cosmetic changes on fast-path tracing
Lluís Vilanova [Mon, 11 Jul 2016 10:53:35 +0000 (12:53 +0200)]
trace: Cosmetic changes on fast-path tracing

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agodisas: Remove unused macro '_'
Lluís Vilanova [Mon, 11 Jul 2016 10:53:30 +0000 (12:53 +0200)]
disas: Remove unused macro '_'

Eliminates a future compilation error when UI code includes the tracing
headers (indirectly pulling "disas/bfd.h" through "qom/cpu.h") and
GLib's i18n '_' macro.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agotrace: Identify events with the 'vcpu' property
Lluís Vilanova [Mon, 11 Jul 2016 10:53:24 +0000 (12:53 +0200)]
trace: Identify events with the 'vcpu' property

A new event attribute 'cpu_id' is added to have a separate ID
space ('TRACE_VCPU_*') for all events with the 'vcpu' property.

These are later used to identify which events are enabled on each vCPU.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agotrace: [bsd-user] Commandline arguments to control tracing
Lluís Vilanova [Fri, 15 Jul 2016 17:08:43 +0000 (19:08 +0200)]
trace: [bsd-user] Commandline arguments to control tracing

[Changed const char *trace_file to char *trace_file since it's a
heap-allocated string that needs to be freed.  This type is also
returned by trace_opt_parse() and used in vl.c.

Also fixed coding style on for(;;) and else statement as suggested by
Eric Blake <eblake@redhat.com> since the patch modifies these lines or
close enough.
--Stefan]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Message-id: 146860252322.30668.18276041739086338328.stgit@fimbulvetr.bsc.es
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agotrace: [linux-user] Commandline arguments to control tracing
Lluís Vilanova [Fri, 15 Jul 2016 17:08:38 +0000 (19:08 +0200)]
trace: [linux-user] Commandline arguments to control tracing

[Changed const char *trace_file to char *trace_file since it's a
heap-allocated string that needs to be freed.  This type is also
returned by trace_opt_parse() and used in vl.c.
--Stefan]

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Message-id: 146860251784.30668.17339867835129075077.stgit@fimbulvetr.bsc.es
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agoMerge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging
Peter Maydell [Mon, 18 Jul 2016 17:13:01 +0000 (18:13 +0100)]
Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Mon 18 Jul 2016 17:58:27 BST
# gpg:                using RSA key 0x9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  MAINTAINERS: Add include/block/aio.h to block I/O path section
  virtio-blk: dataplane cleanup
  checkpatch: consider git extended headers valid patches
  aio-posix: remove useless parameter
  linux-aio: prevent submitting more than MAX_EVENTS
  aio_ctx_check: follow CODING_STYLE
  linux-aio: share one LinuxAioState within an AioContext
  spec/parallels: fix a mistake

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
8 years agovfio/pci: Hide ARI capability
Alex Williamson [Mon, 18 Jul 2016 16:55:17 +0000 (10:55 -0600)]
vfio/pci: Hide ARI capability

QEMU supports ARI on downstream ports and assigned devices may support
ARI in their extended capabilities.  The endpoint ARI capability
specifies the next function, such that the OS doesn't need to walk
each possible function, however this next function is relative to the
host, not the guest.  This leads to device discovery issues when we
combine separate functions into virtual multi-function packages in a
guest.  For example, SR-IOV VFs are not enumerated by simply probing
the function address space, therefore the ARI next-function field is
zero.  When we combine multiple VFs together as a multi-function
device in the guest, the guest OS identifies ARI is enabled, relies on
this next-function field, and stops looking for additional function
after the first is found.

Long term we should expose the ARI capability to the guest to enable
configurations with more than 8 functions per slot, but this requires
additional QEMU PCI infrastructure to manage the next-function field
for multiple, otherwise independent devices.  In the short term,
hiding this capability allows equivalent functionality to what we
currently have on non-express chipsets.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
8 years ago.travis.yml: Disable IRC build status updates from forks
Pranith Kumar [Mon, 27 Jun 2016 18:13:22 +0000 (14:13 -0400)]
.travis.yml: Disable IRC build status updates from forks

We want the travis build bot to post notifications on IRC only for the
master qemu repository and not the various forks/branches of
others. Currently there is no direct option to restrict the updates to
one repository. This is being worked upon by the developers and
tracked in https://github.com/travis-ci/travis-ci/issues/1094.

Until such time, we can use the workaround as posted in
ref. https://github.com/facebook/flow/pull/1822.

This basically creates an ecrypted string which decrypts to qemu IRC
channel only on "qemu/qemu" repo and not on the forks. This enables
the build bot to notify the IRC only for the main repo.

Signed-off-by: Pranith Kumar <bobby.prani@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
8 years agoMAINTAINERS: Add include/block/aio.h to block I/O path section
Fam Zheng [Mon, 18 Jul 2016 07:19:47 +0000 (15:19 +0800)]
MAINTAINERS: Add include/block/aio.h to block I/O path section

This file is actually the header for async.c and aio-*.c., so add it to
the same section.

Suggested-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1468826387-10473-1-git-send-email-famz@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agovirtio-blk: dataplane cleanup
Cao jin [Mon, 18 Jul 2016 04:05:49 +0000 (12:05 +0800)]
virtio-blk: dataplane cleanup

No need duplicate the judgment, there is one in function entry.

Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Max Reitz <mreitz@redhat.com>
Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 1468814749-14510-1-git-send-email-caoj.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agocheckpatch: consider git extended headers valid patches
Stefan Hajnoczi [Fri, 15 Jul 2016 09:46:54 +0000 (10:46 +0100)]
checkpatch: consider git extended headers valid patches

Renames look like this with git-diff(1) when diff.renames = true is set:

  diff --git a/a b/b
  similarity index 100%
  rename from a
  rename to b

This raises the "Does not appear to be a unified-diff format patch"
error because checkpatch.pl only considers a diff valid if it contains
at least one "@@" hunk.

This patch accepts renames and copies too so that checkpatch.pl exits
successfully when a diff only renames/copies files.  The git diff
extended header format is described on the git-diff(1) man page.

Reported-by: Colin Lord <clord@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1468576014-28788-1-git-send-email-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agoaio-posix: remove useless parameter
Cao jin [Fri, 15 Jul 2016 10:28:44 +0000 (18:28 +0800)]
aio-posix: remove useless parameter

Parameter **errp of aio_context_setup() is useless, remove it
and clean up the related code.

Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Fam Zheng <famz@redhat.com>
Cc: Eric Blake <eblake@redhat.com>
Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1468578524-23433-1-git-send-email-caoj.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agolinux-aio: prevent submitting more than MAX_EVENTS
Roman Pen [Wed, 13 Jul 2016 13:03:24 +0000 (15:03 +0200)]
linux-aio: prevent submitting more than MAX_EVENTS

Invoking io_setup(MAX_EVENTS) we ask kernel to create ring buffer for us
with specified number of events.  But kernel ring buffer allocation logic
is a bit tricky (ring buffer is page size aligned + some percpu allocation
are required) so eventually more than requested events number is allocated.

From a userspace side we have to follow the convention and should not try
to io_submit() more or logic, which consumes completed events, should be
changed accordingly.  The pitfall is in the following sequence:

    MAX_EVENTS = 128
    io_setup(MAX_EVENTS)

    io_submit(MAX_EVENTS)
    io_submit(MAX_EVENTS)

    /* now 256 events are in-flight */

    io_getevents(MAX_EVENTS) = 128

    /* we can handle only 128 events at once, to be sure
     * that nothing is pended the io_getevents(MAX_EVENTS)
     * call must be invoked once more or hang will happen. */

To prevent the hang or reiteration of io_getevents() call this patch
restricts the number of in-flights, which is now limited to MAX_EVENTS.

Signed-off-by: Roman Pen <roman.penyaev@profitbricks.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1468415004-31755-1-git-send-email-roman.penyaev@profitbricks.com
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: qemu-devel@nongnu.org
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
8 years agoaio_ctx_check: follow CODING_STYLE
Cao jin [Thu, 14 Jul 2016 13:10:43 +0000 (21:10 +0800)]
aio_ctx_check: follow CODING_STYLE

replace tab with spaces

Signed-off-by: Cao jin <caoj.fnst@cn.fujitsu.com>
Message-id: 1468501843-14927-1-git-send-email-caoj.fnst@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>