Paolo Bonzini | 7c9e527 | 2017-08-21 18:58:56 +0200 | [diff] [blame] | 1 | ====================================== |
| 2 | Persistent reservation managers |
| 3 | ====================================== |
| 4 | |
| 5 | SCSI persistent Reservations allow restricting access to block devices |
| 6 | to specific initiators in a shared storage setup. When implementing |
| 7 | clustering of virtual machines, it is a common requirement for virtual |
| 8 | machines to send persistent reservation SCSI commands. However, |
| 9 | the operating system restricts sending these commands to unprivileged |
| 10 | programs because incorrect usage can disrupt regular operation of the |
| 11 | storage fabric. |
| 12 | |
| 13 | For this reason, QEMU's SCSI passthrough devices, ``scsi-block`` |
| 14 | and ``scsi-generic`` (both are only available on Linux) can delegate |
| 15 | implementation of persistent reservations to a separate object, |
| 16 | the "persistent reservation manager". Only PERSISTENT RESERVE OUT and |
| 17 | PERSISTENT RESERVE IN commands are passed to the persistent reservation |
| 18 | manager object; other commands are processed by QEMU as usual. |
| 19 | |
| 20 | ----------------------------------------- |
| 21 | Defining a persistent reservation manager |
| 22 | ----------------------------------------- |
| 23 | |
| 24 | A persistent reservation manager is an instance of a subclass of the |
| 25 | "pr-manager" QOM class. |
| 26 | |
| 27 | Right now only one subclass is defined, ``pr-manager-helper``, which |
| 28 | forwards the commands to an external privileged helper program |
| 29 | over Unix sockets. The helper program only allows sending persistent |
| 30 | reservation commands to devices for which QEMU has a file descriptor, |
| 31 | so that QEMU will not be able to effect persistent reservations |
| 32 | unless it has access to both the socket and the device. |
| 33 | |
| 34 | ``pr-manager-helper`` has a single string property, ``path``, which |
| 35 | accepts the path to the helper program's Unix socket. For example, |
| 36 | the following command line defines a ``pr-manager-helper`` object and |
| 37 | attaches it to a SCSI passthrough device:: |
| 38 | |
| 39 | $ qemu-system-x86_64 |
| 40 | -device virtio-scsi \ |
| 41 | -object pr-manager-helper,id=helper0,path=/var/run/qemu-pr-helper.sock |
| 42 | -drive if=none,id=hd,driver=raw,file.filename=/dev/sdb,file.pr-manager=helper0 |
| 43 | -device scsi-block,drive=hd |
| 44 | |
| 45 | Alternatively, using ``-blockdev``:: |
| 46 | |
| 47 | $ qemu-system-x86_64 |
| 48 | -device virtio-scsi \ |
| 49 | -object pr-manager-helper,id=helper0,path=/var/run/qemu-pr-helper.sock |
| 50 | -blockdev node-name=hd,driver=raw,file.driver=host_device,file.filename=/dev/sdb,file.pr-manager=helper0 |
| 51 | -device scsi-block,drive=hd |
Paolo Bonzini | b855f8d | 2017-08-22 06:50:18 +0200 | [diff] [blame] | 52 | |
| 53 | ---------------------------------- |
| 54 | Invoking :program:`qemu-pr-helper` |
| 55 | ---------------------------------- |
| 56 | |
| 57 | QEMU provides an implementation of the persistent reservation helper, |
| 58 | called :program:`qemu-pr-helper`. The helper should be started as a |
| 59 | system service and supports the following option: |
| 60 | |
| 61 | -d, --daemon run in the background |
| 62 | -q, --quiet decrease verbosity |
Paolo Bonzini | fe8fc5a | 2017-08-22 06:50:55 +0200 | [diff] [blame] | 63 | -v, --verbose increase verbosity |
Paolo Bonzini | b855f8d | 2017-08-22 06:50:18 +0200 | [diff] [blame] | 64 | -f, --pidfile=path PID file when running as a daemon |
| 65 | -k, --socket=path path to the socket |
| 66 | -T, --trace=trace-opts tracing options |
| 67 | |
| 68 | By default, the socket and PID file are placed in the runtime state |
| 69 | directory, for example :file:`/var/run/qemu-pr-helper.sock` and |
| 70 | :file:`/var/run/qemu-pr-helper.pid`. The PID file is not created |
| 71 | unless :option:`-d` is passed too. |
| 72 | |
| 73 | :program:`qemu-pr-helper` can also use the systemd socket activation |
| 74 | protocol. In this case, the systemd socket unit should specify a |
| 75 | Unix stream socket, like this:: |
| 76 | |
| 77 | [Socket] |
| 78 | ListenStream=/var/run/qemu-pr-helper.sock |
| 79 | |
| 80 | After connecting to the socket, :program:`qemu-pr-helper`` can optionally drop |
| 81 | root privileges, except for those capabilities that are needed for |
| 82 | its operation. To do this, add the following options: |
| 83 | |
| 84 | -u, --user=user user to drop privileges to |
| 85 | -g, --group=group group to drop privileges to |
Paolo Bonzini | fe8fc5a | 2017-08-22 06:50:55 +0200 | [diff] [blame] | 86 | |
| 87 | --------------------------------------------- |
| 88 | Multipath devices and persistent reservations |
| 89 | --------------------------------------------- |
| 90 | |
| 91 | Proper support of persistent reservation for multipath devices requires |
| 92 | communication with the multipath daemon, so that the reservation is |
| 93 | registered and applied when a path is newly discovered or becomes online |
| 94 | again. :command:`qemu-pr-helper` can do this if the ``libmpathpersist`` |
| 95 | library was available on the system at build time. |
| 96 | |
| 97 | As of August 2017, a reservation key must be specified in ``multipath.conf`` |
| 98 | for ``multipathd`` to check for persistent reservation for newly |
| 99 | discovered paths or reinstated paths. The attribute can be added |
| 100 | to the ``defaults`` section or the ``multipaths`` section; for example:: |
| 101 | |
| 102 | multipaths { |
| 103 | multipath { |
| 104 | wwid XXXXXXXXXXXXXXXX |
| 105 | alias yellow |
| 106 | reservation_key 0x123abc |
| 107 | } |
| 108 | } |
| 109 | |
| 110 | Linking :program:`qemu-pr-helper` to ``libmpathpersist`` does not impede |
| 111 | its usage on regular SCSI devices. |