Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 1 | The memory API |
| 2 | ============== |
| 3 | |
| 4 | The memory API models the memory and I/O buses and controllers of a QEMU |
| 5 | machine. It attempts to allow modelling of: |
| 6 | |
| 7 | - ordinary RAM |
| 8 | - memory-mapped I/O (MMIO) |
| 9 | - memory controllers that can dynamically reroute physical memory regions |
Ademar de Souza Reis Jr | 69ddaf6 | 2011-12-05 16:54:14 -0300 | [diff] [blame] | 10 | to different destinations |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 11 | |
| 12 | The memory model provides support for |
| 13 | |
| 14 | - tracking RAM changes by the guest |
| 15 | - setting up coalesced memory for kvm |
| 16 | - setting up ioeventfd regions for kvm |
| 17 | |
Paolo Bonzini | 2d40178 | 2013-05-06 18:23:38 +0200 | [diff] [blame] | 18 | Memory is modelled as an acyclic graph of MemoryRegion objects. Sinks |
| 19 | (leaves) are RAM and MMIO regions, while other nodes represent |
| 20 | buses, memory controllers, and memory regions that have been rerouted. |
| 21 | |
| 22 | In addition to MemoryRegion objects, the memory API provides AddressSpace |
| 23 | objects for every root and possibly for intermediate MemoryRegions too. |
| 24 | These represent memory as seen from the CPU or a device's viewpoint. |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 25 | |
| 26 | Types of regions |
| 27 | ---------------- |
| 28 | |
Peter Maydell | 5056c0c | 2016-01-28 18:54:57 +0000 | [diff] [blame] | 29 | There are multiple types of memory regions (all represented by a single C type |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 30 | MemoryRegion): |
| 31 | |
| 32 | - RAM: a RAM region is simply a range of host memory that can be made available |
| 33 | to the guest. |
Peter Maydell | 5056c0c | 2016-01-28 18:54:57 +0000 | [diff] [blame] | 34 | You typically initialize these with memory_region_init_ram(). Some special |
| 35 | purposes require the variants memory_region_init_resizeable_ram(), |
| 36 | memory_region_init_ram_from_file(), or memory_region_init_ram_ptr(). |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 37 | |
| 38 | - MMIO: a range of guest memory that is implemented by host callbacks; |
| 39 | each read or write causes a callback to be called on the host. |
Cao jin | 0c52a80 | 2016-04-01 18:47:57 +0800 | [diff] [blame] | 40 | You initialize these with memory_region_init_io(), passing it a |
| 41 | MemoryRegionOps structure describing the callbacks. |
Peter Maydell | 5056c0c | 2016-01-28 18:54:57 +0000 | [diff] [blame] | 42 | |
| 43 | - ROM: a ROM memory region works like RAM for reads (directly accessing |
Peter Maydell | a1777f7 | 2016-07-04 13:06:35 +0100 | [diff] [blame] | 44 | a region of host memory), and forbids writes. You initialize these with |
| 45 | memory_region_init_rom(). |
| 46 | |
| 47 | - ROM device: a ROM device memory region works like RAM for reads |
| 48 | (directly accessing a region of host memory), but like MMIO for |
| 49 | writes (invoking a callback). You initialize these with |
| 50 | memory_region_init_rom_device(). |
Peter Maydell | 5056c0c | 2016-01-28 18:54:57 +0000 | [diff] [blame] | 51 | |
| 52 | - IOMMU region: an IOMMU region translates addresses of accesses made to it |
| 53 | and forwards them to some other target memory region. As the name suggests, |
| 54 | these are only needed for modelling an IOMMU, not for simple devices. |
| 55 | You initialize these with memory_region_init_iommu(). |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 56 | |
| 57 | - container: a container simply includes other memory regions, each at |
| 58 | a different offset. Containers are useful for grouping several regions |
| 59 | into one unit. For example, a PCI BAR may be composed of a RAM region |
| 60 | and an MMIO region. |
| 61 | |
| 62 | A container's subregions are usually non-overlapping. In some cases it is |
| 63 | useful to have overlapping regions; for example a memory controller that |
| 64 | can overlay a subregion of RAM with MMIO or ROM, or a PCI controller |
| 65 | that does not prevent card from claiming overlapping BARs. |
| 66 | |
Peter Maydell | 5056c0c | 2016-01-28 18:54:57 +0000 | [diff] [blame] | 67 | You initialize a pure container with memory_region_init(). |
| 68 | |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 69 | - alias: a subsection of another region. Aliases allow a region to be |
| 70 | split apart into discontiguous regions. Examples of uses are memory banks |
| 71 | used when the guest address space is smaller than the amount of RAM |
| 72 | addressed, or a memory controller that splits main memory to expose a "PCI |
| 73 | hole". Aliases may point to any type of region, including other aliases, |
| 74 | but an alias may not point back to itself, directly or indirectly. |
Peter Maydell | 5056c0c | 2016-01-28 18:54:57 +0000 | [diff] [blame] | 75 | You initialize these with memory_region_init_alias(). |
| 76 | |
| 77 | - reservation region: a reservation region is primarily for debugging. |
| 78 | It claims I/O space that is not supposed to be handled by QEMU itself. |
| 79 | The typical use is to track parts of the address space which will be |
| 80 | handled by the host kernel when KVM is enabled. |
| 81 | You initialize these with memory_region_init_reservation(), or by |
| 82 | passing a NULL callback parameter to memory_region_init_io(). |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 83 | |
Peter Maydell | 6f1ce94 | 2013-10-15 15:42:34 +0100 | [diff] [blame] | 84 | It is valid to add subregions to a region which is not a pure container |
| 85 | (that is, to an MMIO, RAM or ROM region). This means that the region |
| 86 | will act like a container, except that any addresses within the container's |
| 87 | region which are not claimed by any subregion are handled by the |
| 88 | container itself (ie by its MMIO callbacks or RAM backing). However |
| 89 | it is generally possible to achieve the same effect with a pure container |
| 90 | one of whose subregions is a low priority "background" region covering |
| 91 | the whole address range; this is often clearer and is preferred. |
| 92 | Subregions cannot be added to an alias region. |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 93 | |
Peter Maydell | 2286468 | 2017-07-07 15:42:57 +0100 | [diff] [blame] | 94 | Migration |
| 95 | --------- |
| 96 | |
| 97 | Where the memory region is backed by host memory (RAM, ROM and |
| 98 | ROM device memory region types), this host memory needs to be |
| 99 | copied to the destination on migration. These APIs which allocate |
| 100 | the host memory for you will also register the memory so it is |
| 101 | migrated: |
| 102 | - memory_region_init_ram() |
| 103 | - memory_region_init_rom() |
| 104 | - memory_region_init_rom_device() |
| 105 | |
| 106 | For most devices and boards this is the correct thing. If you |
| 107 | have a special case where you need to manage the migration of |
| 108 | the backing memory yourself, you can call the functions: |
| 109 | - memory_region_init_ram_nomigrate() |
| 110 | - memory_region_init_rom_nomigrate() |
| 111 | - memory_region_init_rom_device_nomigrate() |
| 112 | which only initialize the MemoryRegion and leave handling |
| 113 | migration to the caller. |
| 114 | |
| 115 | The functions: |
| 116 | - memory_region_init_resizeable_ram() |
| 117 | - memory_region_init_ram_from_file() |
| 118 | - memory_region_init_ram_from_fd() |
| 119 | - memory_region_init_ram_ptr() |
| 120 | - memory_region_init_ram_device_ptr() |
| 121 | are for special cases only, and so they do not automatically |
| 122 | register the backing memory for migration; the caller must |
| 123 | manage migration if necessary. |
| 124 | |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 125 | Region names |
| 126 | ------------ |
| 127 | |
| 128 | Regions are assigned names by the constructor. For most regions these are |
| 129 | only used for debugging purposes, but RAM regions also use the name to identify |
| 130 | live migration sections. This means that RAM region names need to have ABI |
| 131 | stability. |
| 132 | |
| 133 | Region lifecycle |
| 134 | ---------------- |
| 135 | |
Paolo Bonzini | 8b5c216 | 2015-02-13 13:42:03 +0100 | [diff] [blame] | 136 | A region is created by one of the memory_region_init*() functions and |
| 137 | attached to an object, which acts as its owner or parent. QEMU ensures |
| 138 | that the owner object remains alive as long as the region is visible to |
| 139 | the guest, or as long as the region is in use by a virtual CPU or another |
| 140 | device. For example, the owner object will not die between an |
| 141 | address_space_map operation and the corresponding address_space_unmap. |
Paolo Bonzini | d8d9581 | 2014-06-11 12:42:01 +0200 | [diff] [blame] | 142 | |
Paolo Bonzini | 8b5c216 | 2015-02-13 13:42:03 +0100 | [diff] [blame] | 143 | After creation, a region can be added to an address space or a |
| 144 | container with memory_region_add_subregion(), and removed using |
| 145 | memory_region_del_subregion(). |
Paolo Bonzini | d8d9581 | 2014-06-11 12:42:01 +0200 | [diff] [blame] | 146 | |
Paolo Bonzini | 8b5c216 | 2015-02-13 13:42:03 +0100 | [diff] [blame] | 147 | Various region attributes (read-only, dirty logging, coalesced mmio, |
| 148 | ioeventfd) can be changed during the region lifecycle. They take effect |
| 149 | as soon as the region is made visible. This can be immediately, later, |
| 150 | or never. |
| 151 | |
| 152 | Destruction of a memory region happens automatically when the owner |
| 153 | object dies. |
| 154 | |
| 155 | If however the memory region is part of a dynamically allocated data |
| 156 | structure, you should call object_unparent() to destroy the memory region |
| 157 | before the data structure is freed. For an example see VFIOMSIXInfo |
| 158 | and VFIOQuirk in hw/vfio/pci.c. |
| 159 | |
| 160 | You must not destroy a memory region as long as it may be in use by a |
| 161 | device or CPU. In order to do this, as a general rule do not create or |
| 162 | destroy memory regions dynamically during a device's lifetime, and only |
| 163 | call object_unparent() in the memory region owner's instance_finalize |
| 164 | callback. The dynamically allocated data structure that contains the |
| 165 | memory region then should obviously be freed in the instance_finalize |
| 166 | callback as well. |
| 167 | |
| 168 | If you break this rule, the following situation can happen: |
| 169 | |
| 170 | - the memory region's owner had a reference taken via memory_region_ref |
| 171 | (for example by address_space_map) |
| 172 | |
| 173 | - the region is unparented, and has no owner anymore |
| 174 | |
| 175 | - when address_space_unmap is called, the reference to the memory region's |
| 176 | owner is leaked. |
| 177 | |
| 178 | |
| 179 | There is an exception to the above rule: it is okay to call |
| 180 | object_unparent at any time for an alias or a container region. It is |
| 181 | therefore also okay to create or destroy alias and container regions |
| 182 | dynamically during a device's lifetime. |
| 183 | |
| 184 | This exceptional usage is valid because aliases and containers only help |
| 185 | QEMU building the guest's memory map; they are never accessed directly. |
| 186 | memory_region_ref and memory_region_unref are never called on aliases |
| 187 | or containers, and the above situation then cannot happen. Exploiting |
| 188 | this exception is rarely necessary, and therefore it is discouraged, |
| 189 | but nevertheless it is used in a few places. |
| 190 | |
| 191 | For regions that "have no owner" (NULL is passed at creation time), the |
| 192 | machine object is actually used as the owner. Since instance_finalize is |
| 193 | never called for the machine object, you must never call object_unparent |
| 194 | on regions that have no owner, unless they are aliases or containers. |
| 195 | |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 196 | |
| 197 | Overlapping regions and priority |
| 198 | -------------------------------- |
| 199 | Usually, regions may not overlap each other; a memory address decodes into |
| 200 | exactly one target. In some cases it is useful to allow regions to overlap, |
| 201 | and sometimes to control which of an overlapping regions is visible to the |
| 202 | guest. This is done with memory_region_add_subregion_overlap(), which |
| 203 | allows the region to overlap any other region in the same container, and |
| 204 | specifies a priority that allows the core to decide which of two regions at |
| 205 | the same address are visible (highest wins). |
Marcel Apfelbaum | 8002ccd | 2013-09-16 11:21:15 +0300 | [diff] [blame] | 206 | Priority values are signed, and the default value is zero. This means that |
| 207 | you can use memory_region_add_subregion_overlap() both to specify a region |
| 208 | that must sit 'above' any others (with a positive priority) and also a |
| 209 | background region that sits 'below' others (with a negative priority). |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 210 | |
Peter Maydell | 6f1ce94 | 2013-10-15 15:42:34 +0100 | [diff] [blame] | 211 | If the higher priority region in an overlap is a container or alias, then |
| 212 | the lower priority region will appear in any "holes" that the higher priority |
| 213 | region has left by not mapping subregions to that area of its address range. |
| 214 | (This applies recursively -- if the subregions are themselves containers or |
| 215 | aliases that leave holes then the lower priority region will appear in these |
| 216 | holes too.) |
| 217 | |
| 218 | For example, suppose we have a container A of size 0x8000 with two subregions |
xiaoqiang zhao | 8210f5f | 2016-02-26 16:40:51 +0800 | [diff] [blame] | 219 | B and C. B is a container mapped at 0x2000, size 0x4000, priority 2; C is |
| 220 | an MMIO region mapped at 0x0, size 0x6000, priority 1. B currently has two |
Peter Maydell | 6f1ce94 | 2013-10-15 15:42:34 +0100 | [diff] [blame] | 221 | of its own subregions: D of size 0x1000 at offset 0 and E of size 0x1000 at |
| 222 | offset 0x2000. As a diagram: |
| 223 | |
Wei Jiangang | b3f3fde | 2016-03-22 17:45:54 +0800 | [diff] [blame] | 224 | 0 1000 2000 3000 4000 5000 6000 7000 8000 |
| 225 | |------|------|------|------|------|------|------|------| |
| 226 | A: [ ] |
Peter Maydell | 6f1ce94 | 2013-10-15 15:42:34 +0100 | [diff] [blame] | 227 | C: [CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC] |
| 228 | B: [ ] |
| 229 | D: [DDDDD] |
| 230 | E: [EEEEE] |
| 231 | |
| 232 | The regions that will be seen within this address range then are: |
| 233 | [CCCCCCCCCCCC][DDDDD][CCCCC][EEEEE][CCCCC] |
| 234 | |
| 235 | Since B has higher priority than C, its subregions appear in the flat map |
| 236 | even where they overlap with C. In ranges where B has not mapped anything |
| 237 | C's region appears. |
| 238 | |
| 239 | If B had provided its own MMIO operations (ie it was not a pure container) |
| 240 | then these would be used for any addresses in its range not handled by |
| 241 | D or E, and the result would be: |
| 242 | [CCCCCCCCCCCC][DDDDD][BBBBB][EEEEE][BBBBB] |
| 243 | |
| 244 | Priority values are local to a container, because the priorities of two |
| 245 | regions are only compared when they are both children of the same container. |
| 246 | This means that the device in charge of the container (typically modelling |
| 247 | a bus or a memory controller) can use them to manage the interaction of |
| 248 | its child regions without any side effects on other parts of the system. |
| 249 | In the example above, the priorities of D and E are unimportant because |
| 250 | they do not overlap each other. It is the relative priority of B and C |
| 251 | that causes D and E to appear on top of C: D and E's priorities are never |
| 252 | compared against the priority of C. |
| 253 | |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 254 | Visibility |
| 255 | ---------- |
| 256 | The memory core uses the following rules to select a memory region when the |
| 257 | guest accesses an address: |
| 258 | |
| 259 | - all direct subregions of the root region are matched against the address, in |
| 260 | descending priority order |
| 261 | - if the address lies outside the region offset/size, the subregion is |
| 262 | discarded |
Peter Maydell | 6f1ce94 | 2013-10-15 15:42:34 +0100 | [diff] [blame] | 263 | - if the subregion is a leaf (RAM or MMIO), the search terminates, returning |
| 264 | this leaf region |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 265 | - if the subregion is a container, the same algorithm is used within the |
| 266 | subregion (after the address is adjusted by the subregion offset) |
Peter Maydell | 6f1ce94 | 2013-10-15 15:42:34 +0100 | [diff] [blame] | 267 | - if the subregion is an alias, the search is continued at the alias target |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 268 | (after the address is adjusted by the subregion offset and alias offset) |
Peter Maydell | 6f1ce94 | 2013-10-15 15:42:34 +0100 | [diff] [blame] | 269 | - if a recursive search within a container or alias subregion does not |
| 270 | find a match (because of a "hole" in the container's coverage of its |
| 271 | address range), then if this is a container with its own MMIO or RAM |
| 272 | backing the search terminates, returning the container itself. Otherwise |
| 273 | we continue with the next subregion in priority order |
| 274 | - if none of the subregions match the address then the search terminates |
| 275 | with no match found |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 276 | |
| 277 | Example memory map |
| 278 | ------------------ |
| 279 | |
| 280 | system_memory: container@0-2^48-1 |
| 281 | | |
| 282 | +---- lomem: alias@0-0xdfffffff ---> #ram (0-0xdfffffff) |
| 283 | | |
| 284 | +---- himem: alias@0x100000000-0x11fffffff ---> #ram (0xe0000000-0xffffffff) |
| 285 | | |
Wei Jiangang | b3f3fde | 2016-03-22 17:45:54 +0800 | [diff] [blame] | 286 | +---- vga-window: alias@0xa0000-0xbffff ---> #pci (0xa0000-0xbffff) |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 287 | | (prio 1) |
| 288 | | |
| 289 | +---- pci-hole: alias@0xe0000000-0xffffffff ---> #pci (0xe0000000-0xffffffff) |
| 290 | |
| 291 | pci (0-2^32-1) |
| 292 | | |
| 293 | +--- vga-area: container@0xa0000-0xbffff |
| 294 | | | |
| 295 | | +--- alias@0x00000-0x7fff ---> #vram (0x010000-0x017fff) |
| 296 | | | |
| 297 | | +--- alias@0x08000-0xffff ---> #vram (0x020000-0x027fff) |
| 298 | | |
| 299 | +---- vram: ram@0xe1000000-0xe1ffffff |
| 300 | | |
| 301 | +---- vga-mmio: mmio@0xe2000000-0xe200ffff |
| 302 | |
| 303 | ram: ram@0x00000000-0xffffffff |
| 304 | |
Ademar de Souza Reis Jr | 69ddaf6 | 2011-12-05 16:54:14 -0300 | [diff] [blame] | 305 | This is a (simplified) PC memory map. The 4GB RAM block is mapped into the |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 306 | system address space via two aliases: "lomem" is a 1:1 mapping of the first |
| 307 | 3.5GB; "himem" maps the last 0.5GB at address 4GB. This leaves 0.5GB for the |
| 308 | so-called PCI hole, that allows a 32-bit PCI bus to exist in a system with |
| 309 | 4GB of memory. |
| 310 | |
| 311 | The memory controller diverts addresses in the range 640K-768K to the PCI |
Avi Kivity | 7075ba3 | 2011-08-08 19:58:50 +0300 | [diff] [blame] | 312 | address space. This is modelled using the "vga-window" alias, mapped at a |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 313 | higher priority so it obscures the RAM at the same addresses. The vga window |
| 314 | can be removed by programming the memory controller; this is modelled by |
| 315 | removing the alias and exposing the RAM underneath. |
| 316 | |
| 317 | The pci address space is not a direct child of the system address space, since |
| 318 | we only want parts of it to be visible (we accomplish this using aliases). |
| 319 | It has two subregions: vga-area models the legacy vga window and is occupied |
| 320 | by two 32K memory banks pointing at two sections of the framebuffer. |
| 321 | In addition the vram is mapped as a BAR at address e1000000, and an additional |
| 322 | BAR containing MMIO registers is mapped after it. |
| 323 | |
| 324 | Note that if the guest maps a BAR outside the PCI hole, it would not be |
| 325 | visible as the pci-hole alias clips it to a 0.5GB range. |
| 326 | |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 327 | MMIO Operations |
| 328 | --------------- |
| 329 | |
| 330 | MMIO regions are provided with ->read() and ->write() callbacks; in addition |
| 331 | various constraints can be supplied to control how these callbacks are called: |
| 332 | |
| 333 | - .valid.min_access_size, .valid.max_access_size define the access sizes |
| 334 | (in bytes) which the device accepts; accesses outside this range will |
| 335 | have device and bus specific behaviour (ignored, or machine check) |
Peter Maydell | ef00bda | 2016-02-26 16:40:51 +0800 | [diff] [blame] | 336 | - .valid.unaligned specifies that the *device being modelled* supports |
| 337 | unaligned accesses; if false, unaligned accesses will invoke the |
| 338 | appropriate bus or CPU specific behaviour. |
Avi Kivity | 9d3a473 | 2011-07-26 14:26:00 +0300 | [diff] [blame] | 339 | - .impl.min_access_size, .impl.max_access_size define the access sizes |
| 340 | (in bytes) supported by the *implementation*; other access sizes will be |
| 341 | emulated using the ones available. For example a 4-byte write will be |
Ademar de Souza Reis Jr | 69ddaf6 | 2011-12-05 16:54:14 -0300 | [diff] [blame] | 342 | emulated using four 1-byte writes, if .impl.max_access_size = 1. |
Fam Zheng | edc1ba7 | 2014-05-05 15:53:41 +0800 | [diff] [blame] | 343 | - .impl.unaligned specifies that the *implementation* supports unaligned |
| 344 | accesses; if false, unaligned accesses will be emulated by two aligned |
| 345 | accesses. |
Peter Maydell | ef00bda | 2016-02-26 16:40:51 +0800 | [diff] [blame] | 346 | - .old_mmio eases the porting of code that was formerly using |
Fam Zheng | edc1ba7 | 2014-05-05 15:53:41 +0800 | [diff] [blame] | 347 | cpu_register_io_memory(). It should not be used in new code. |