=======================
Virtio device migration
=======================

Copyright 2015 IBM Corp.

This work is licensed under the terms of the GNU GPL, version 2 or later.  See
the COPYING file in the top-level directory.

Saving and restoring the state of virtio devices is a bit of a twisty maze,
for several reasons:

- state is distributed between several parts:

  - virtio core, for common fields like features, number of queues, ...

  - virtio transport (pci, ccw, ...), for the different proxy devices and
    transport specific state (msix vectors, indicators, ...)

  - virtio device (net, blk, ...), for the different device types and their
    state (mac address, request queue, ...)

- most fields are saved via the stream interface; subsequently, subsections
  have been added to make cross-version migration possible

This file attempts to document the current procedure and point out some
caveats.

Save state procedure
====================

::

  virtio core               virtio transport          virtio device
  -----------               ----------------          -------------

                                                      save() function registered
                                                      via VMState wrapper on
                                                      device class
  virtio_save()                                       <----------
               ------>      save_config()
                            - save proxy device
                            - save transport-specific
                              device fields
  - save common device
    fields
  - save common virtqueue
    fields
               ------>      save_queue()
                            - save transport-specific
                              virtqueue fields
               ------>                               save_device()
                                                     - save device-specific
                                                       fields
  - save subsections
    - device endianness,
      if changed from
      default endianness
    - 64 bit features, if
      any high feature bit
      is set
    - virtio-1 virtqueue
      fields, if VERSION_1
      is set

Load state procedure
====================

::

  virtio core               virtio transport          virtio device
  -----------               ----------------          -------------

                                                      load() function registered
                                                      via VMState wrapper on
                                                      device class
  virtio_load()                                       <----------
               ------>      load_config()
                            - load proxy device
                            - load transport-specific
                              device fields
  - load common device
    fields
  - load common virtqueue
    fields
               ------>      load_queue()
                            - load transport-specific
                              virtqueue fields
  - notify guest
               ------>                               load_device()
                                                     - load device-specific
                                                       fields
  - load subsections
    - device endianness
    - 64 bit features
    - virtio-1 virtqueue
      fields
  - sanitize endianness
  - sanitize features
  - virtqueue index sanity
    check
                                                     - feature-dependent setup

Implications of this setup
==========================

Devices need to be careful in their state processing during load: The
load_device() procedure is invoked by the core before subsections have
been loaded. Any code that depends on information transmitted in subsections
therefore has to be invoked in the device's load() function _after_
virtio_load() returned (like e.g. code depending on features).

Any extension of the state being migrated should be done in subsections
added to the core for compatibility reasons. If transport or device specific
state is added, core needs to invoke a callback from the new subsection.