blob: 8ed810f8b9a8c0128e90f1b2e91d28c44c20d666 [file] [log] [blame]
Avi Kivity9d3a4732011-07-26 14:26:00 +03001The memory API
2==============
3
4The memory API models the memory and I/O buses and controllers of a QEMU
5machine. It attempts to allow modelling of:
6
7 - ordinary RAM
8 - memory-mapped I/O (MMIO)
9 - memory controllers that can dynamically reroute physical memory regions
Ademar de Souza Reis Jr69ddaf62011-12-05 16:54:14 -030010 to different destinations
Avi Kivity9d3a4732011-07-26 14:26:00 +030011
12The memory model provides support for
13
14 - tracking RAM changes by the guest
15 - setting up coalesced memory for kvm
16 - setting up ioeventfd regions for kvm
17
Paolo Bonzini2d401782013-05-06 18:23:38 +020018Memory is modelled as an acyclic graph of MemoryRegion objects. Sinks
19(leaves) are RAM and MMIO regions, while other nodes represent
20buses, memory controllers, and memory regions that have been rerouted.
21
22In addition to MemoryRegion objects, the memory API provides AddressSpace
23objects for every root and possibly for intermediate MemoryRegions too.
24These represent memory as seen from the CPU or a device's viewpoint.
Avi Kivity9d3a4732011-07-26 14:26:00 +030025
26Types of regions
27----------------
28
Peter Maydell5056c0c2016-01-28 18:54:57 +000029There are multiple types of memory regions (all represented by a single C type
Avi Kivity9d3a4732011-07-26 14:26:00 +030030MemoryRegion):
31
32- RAM: a RAM region is simply a range of host memory that can be made available
33 to the guest.
Peter Maydell5056c0c2016-01-28 18:54:57 +000034 You typically initialize these with memory_region_init_ram(). Some special
35 purposes require the variants memory_region_init_resizeable_ram(),
36 memory_region_init_ram_from_file(), or memory_region_init_ram_ptr().
Avi Kivity9d3a4732011-07-26 14:26:00 +030037
38- MMIO: a range of guest memory that is implemented by host callbacks;
39 each read or write causes a callback to be called on the host.
Cao jin0c52a802016-04-01 18:47:57 +080040 You initialize these with memory_region_init_io(), passing it a
41 MemoryRegionOps structure describing the callbacks.
Peter Maydell5056c0c2016-01-28 18:54:57 +000042
43- ROM: a ROM memory region works like RAM for reads (directly accessing
Peter Maydella1777f72016-07-04 13:06:35 +010044 a region of host memory), and forbids writes. You initialize these with
45 memory_region_init_rom().
46
47- ROM device: a ROM device memory region works like RAM for reads
48 (directly accessing a region of host memory), but like MMIO for
49 writes (invoking a callback). You initialize these with
50 memory_region_init_rom_device().
Peter Maydell5056c0c2016-01-28 18:54:57 +000051
52- IOMMU region: an IOMMU region translates addresses of accesses made to it
53 and forwards them to some other target memory region. As the name suggests,
54 these are only needed for modelling an IOMMU, not for simple devices.
55 You initialize these with memory_region_init_iommu().
Avi Kivity9d3a4732011-07-26 14:26:00 +030056
57- container: a container simply includes other memory regions, each at
58 a different offset. Containers are useful for grouping several regions
59 into one unit. For example, a PCI BAR may be composed of a RAM region
60 and an MMIO region.
61
62 A container's subregions are usually non-overlapping. In some cases it is
63 useful to have overlapping regions; for example a memory controller that
64 can overlay a subregion of RAM with MMIO or ROM, or a PCI controller
65 that does not prevent card from claiming overlapping BARs.
66
Peter Maydell5056c0c2016-01-28 18:54:57 +000067 You initialize a pure container with memory_region_init().
68
Avi Kivity9d3a4732011-07-26 14:26:00 +030069- alias: a subsection of another region. Aliases allow a region to be
70 split apart into discontiguous regions. Examples of uses are memory banks
71 used when the guest address space is smaller than the amount of RAM
72 addressed, or a memory controller that splits main memory to expose a "PCI
73 hole". Aliases may point to any type of region, including other aliases,
74 but an alias may not point back to itself, directly or indirectly.
Peter Maydell5056c0c2016-01-28 18:54:57 +000075 You initialize these with memory_region_init_alias().
76
77- reservation region: a reservation region is primarily for debugging.
78 It claims I/O space that is not supposed to be handled by QEMU itself.
79 The typical use is to track parts of the address space which will be
80 handled by the host kernel when KVM is enabled.
81 You initialize these with memory_region_init_reservation(), or by
82 passing a NULL callback parameter to memory_region_init_io().
Avi Kivity9d3a4732011-07-26 14:26:00 +030083
Peter Maydell6f1ce942013-10-15 15:42:34 +010084It is valid to add subregions to a region which is not a pure container
85(that is, to an MMIO, RAM or ROM region). This means that the region
86will act like a container, except that any addresses within the container's
87region which are not claimed by any subregion are handled by the
88container itself (ie by its MMIO callbacks or RAM backing). However
89it is generally possible to achieve the same effect with a pure container
90one of whose subregions is a low priority "background" region covering
91the whole address range; this is often clearer and is preferred.
92Subregions cannot be added to an alias region.
Avi Kivity9d3a4732011-07-26 14:26:00 +030093
Peter Maydell22864682017-07-07 15:42:57 +010094Migration
95---------
96
97Where the memory region is backed by host memory (RAM, ROM and
98ROM device memory region types), this host memory needs to be
99copied to the destination on migration. These APIs which allocate
100the host memory for you will also register the memory so it is
101migrated:
102 - memory_region_init_ram()
103 - memory_region_init_rom()
104 - memory_region_init_rom_device()
105
106For most devices and boards this is the correct thing. If you
107have a special case where you need to manage the migration of
108the backing memory yourself, you can call the functions:
109 - memory_region_init_ram_nomigrate()
110 - memory_region_init_rom_nomigrate()
111 - memory_region_init_rom_device_nomigrate()
112which only initialize the MemoryRegion and leave handling
113migration to the caller.
114
115The functions:
116 - memory_region_init_resizeable_ram()
117 - memory_region_init_ram_from_file()
118 - memory_region_init_ram_from_fd()
119 - memory_region_init_ram_ptr()
120 - memory_region_init_ram_device_ptr()
121are for special cases only, and so they do not automatically
122register the backing memory for migration; the caller must
123manage migration if necessary.
124
Avi Kivity9d3a4732011-07-26 14:26:00 +0300125Region names
126------------
127
128Regions are assigned names by the constructor. For most regions these are
129only used for debugging purposes, but RAM regions also use the name to identify
130live migration sections. This means that RAM region names need to have ABI
131stability.
132
133Region lifecycle
134----------------
135
Paolo Bonzini8b5c2162015-02-13 13:42:03 +0100136A region is created by one of the memory_region_init*() functions and
137attached to an object, which acts as its owner or parent. QEMU ensures
138that the owner object remains alive as long as the region is visible to
139the guest, or as long as the region is in use by a virtual CPU or another
140device. For example, the owner object will not die between an
141address_space_map operation and the corresponding address_space_unmap.
Paolo Bonzinid8d95812014-06-11 12:42:01 +0200142
Paolo Bonzini8b5c2162015-02-13 13:42:03 +0100143After creation, a region can be added to an address space or a
144container with memory_region_add_subregion(), and removed using
145memory_region_del_subregion().
Paolo Bonzinid8d95812014-06-11 12:42:01 +0200146
Paolo Bonzini8b5c2162015-02-13 13:42:03 +0100147Various region attributes (read-only, dirty logging, coalesced mmio,
148ioeventfd) can be changed during the region lifecycle. They take effect
149as soon as the region is made visible. This can be immediately, later,
150or never.
151
152Destruction of a memory region happens automatically when the owner
153object dies.
154
155If however the memory region is part of a dynamically allocated data
156structure, you should call object_unparent() to destroy the memory region
157before the data structure is freed. For an example see VFIOMSIXInfo
158and VFIOQuirk in hw/vfio/pci.c.
159
160You must not destroy a memory region as long as it may be in use by a
161device or CPU. In order to do this, as a general rule do not create or
162destroy memory regions dynamically during a device's lifetime, and only
163call object_unparent() in the memory region owner's instance_finalize
164callback. The dynamically allocated data structure that contains the
165memory region then should obviously be freed in the instance_finalize
166callback as well.
167
168If you break this rule, the following situation can happen:
169
170- the memory region's owner had a reference taken via memory_region_ref
171 (for example by address_space_map)
172
173- the region is unparented, and has no owner anymore
174
175- when address_space_unmap is called, the reference to the memory region's
176 owner is leaked.
177
178
179There is an exception to the above rule: it is okay to call
180object_unparent at any time for an alias or a container region. It is
181therefore also okay to create or destroy alias and container regions
182dynamically during a device's lifetime.
183
184This exceptional usage is valid because aliases and containers only help
185QEMU building the guest's memory map; they are never accessed directly.
186memory_region_ref and memory_region_unref are never called on aliases
187or containers, and the above situation then cannot happen. Exploiting
188this exception is rarely necessary, and therefore it is discouraged,
189but nevertheless it is used in a few places.
190
191For regions that "have no owner" (NULL is passed at creation time), the
192machine object is actually used as the owner. Since instance_finalize is
193never called for the machine object, you must never call object_unparent
194on regions that have no owner, unless they are aliases or containers.
195
Avi Kivity9d3a4732011-07-26 14:26:00 +0300196
197Overlapping regions and priority
198--------------------------------
199Usually, regions may not overlap each other; a memory address decodes into
200exactly one target. In some cases it is useful to allow regions to overlap,
201and sometimes to control which of an overlapping regions is visible to the
202guest. This is done with memory_region_add_subregion_overlap(), which
203allows the region to overlap any other region in the same container, and
204specifies a priority that allows the core to decide which of two regions at
205the same address are visible (highest wins).
Marcel Apfelbaum8002ccd2013-09-16 11:21:15 +0300206Priority values are signed, and the default value is zero. This means that
207you can use memory_region_add_subregion_overlap() both to specify a region
208that must sit 'above' any others (with a positive priority) and also a
209background region that sits 'below' others (with a negative priority).
Avi Kivity9d3a4732011-07-26 14:26:00 +0300210
Peter Maydell6f1ce942013-10-15 15:42:34 +0100211If the higher priority region in an overlap is a container or alias, then
212the lower priority region will appear in any "holes" that the higher priority
213region has left by not mapping subregions to that area of its address range.
214(This applies recursively -- if the subregions are themselves containers or
215aliases that leave holes then the lower priority region will appear in these
216holes too.)
217
218For example, suppose we have a container A of size 0x8000 with two subregions
xiaoqiang zhao8210f5f2016-02-26 16:40:51 +0800219B and C. B is a container mapped at 0x2000, size 0x4000, priority 2; C is
220an MMIO region mapped at 0x0, size 0x6000, priority 1. B currently has two
Peter Maydell6f1ce942013-10-15 15:42:34 +0100221of its own subregions: D of size 0x1000 at offset 0 and E of size 0x1000 at
222offset 0x2000. As a diagram:
223
Wei Jiangangb3f3fde2016-03-22 17:45:54 +0800224 0 1000 2000 3000 4000 5000 6000 7000 8000
225 |------|------|------|------|------|------|------|------|
226 A: [ ]
Peter Maydell6f1ce942013-10-15 15:42:34 +0100227 C: [CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC]
228 B: [ ]
229 D: [DDDDD]
230 E: [EEEEE]
231
232The regions that will be seen within this address range then are:
233 [CCCCCCCCCCCC][DDDDD][CCCCC][EEEEE][CCCCC]
234
235Since B has higher priority than C, its subregions appear in the flat map
236even where they overlap with C. In ranges where B has not mapped anything
237C's region appears.
238
239If B had provided its own MMIO operations (ie it was not a pure container)
240then these would be used for any addresses in its range not handled by
241D or E, and the result would be:
242 [CCCCCCCCCCCC][DDDDD][BBBBB][EEEEE][BBBBB]
243
244Priority values are local to a container, because the priorities of two
245regions are only compared when they are both children of the same container.
246This means that the device in charge of the container (typically modelling
247a bus or a memory controller) can use them to manage the interaction of
248its child regions without any side effects on other parts of the system.
249In the example above, the priorities of D and E are unimportant because
250they do not overlap each other. It is the relative priority of B and C
251that causes D and E to appear on top of C: D and E's priorities are never
252compared against the priority of C.
253
Avi Kivity9d3a4732011-07-26 14:26:00 +0300254Visibility
255----------
256The memory core uses the following rules to select a memory region when the
257guest accesses an address:
258
259- all direct subregions of the root region are matched against the address, in
260 descending priority order
261 - if the address lies outside the region offset/size, the subregion is
262 discarded
Peter Maydell6f1ce942013-10-15 15:42:34 +0100263 - if the subregion is a leaf (RAM or MMIO), the search terminates, returning
264 this leaf region
Avi Kivity9d3a4732011-07-26 14:26:00 +0300265 - if the subregion is a container, the same algorithm is used within the
266 subregion (after the address is adjusted by the subregion offset)
Peter Maydell6f1ce942013-10-15 15:42:34 +0100267 - if the subregion is an alias, the search is continued at the alias target
Avi Kivity9d3a4732011-07-26 14:26:00 +0300268 (after the address is adjusted by the subregion offset and alias offset)
Peter Maydell6f1ce942013-10-15 15:42:34 +0100269 - if a recursive search within a container or alias subregion does not
270 find a match (because of a "hole" in the container's coverage of its
271 address range), then if this is a container with its own MMIO or RAM
272 backing the search terminates, returning the container itself. Otherwise
273 we continue with the next subregion in priority order
274- if none of the subregions match the address then the search terminates
275 with no match found
Avi Kivity9d3a4732011-07-26 14:26:00 +0300276
277Example memory map
278------------------
279
280system_memory: container@0-2^48-1
281 |
282 +---- lomem: alias@0-0xdfffffff ---> #ram (0-0xdfffffff)
283 |
284 +---- himem: alias@0x100000000-0x11fffffff ---> #ram (0xe0000000-0xffffffff)
285 |
Wei Jiangangb3f3fde2016-03-22 17:45:54 +0800286 +---- vga-window: alias@0xa0000-0xbffff ---> #pci (0xa0000-0xbffff)
Avi Kivity9d3a4732011-07-26 14:26:00 +0300287 | (prio 1)
288 |
289 +---- pci-hole: alias@0xe0000000-0xffffffff ---> #pci (0xe0000000-0xffffffff)
290
291pci (0-2^32-1)
292 |
293 +--- vga-area: container@0xa0000-0xbffff
294 | |
295 | +--- alias@0x00000-0x7fff ---> #vram (0x010000-0x017fff)
296 | |
297 | +--- alias@0x08000-0xffff ---> #vram (0x020000-0x027fff)
298 |
299 +---- vram: ram@0xe1000000-0xe1ffffff
300 |
301 +---- vga-mmio: mmio@0xe2000000-0xe200ffff
302
303ram: ram@0x00000000-0xffffffff
304
Ademar de Souza Reis Jr69ddaf62011-12-05 16:54:14 -0300305This is a (simplified) PC memory map. The 4GB RAM block is mapped into the
Avi Kivity9d3a4732011-07-26 14:26:00 +0300306system address space via two aliases: "lomem" is a 1:1 mapping of the first
3073.5GB; "himem" maps the last 0.5GB at address 4GB. This leaves 0.5GB for the
308so-called PCI hole, that allows a 32-bit PCI bus to exist in a system with
3094GB of memory.
310
311The memory controller diverts addresses in the range 640K-768K to the PCI
Avi Kivity7075ba32011-08-08 19:58:50 +0300312address space. This is modelled using the "vga-window" alias, mapped at a
Avi Kivity9d3a4732011-07-26 14:26:00 +0300313higher priority so it obscures the RAM at the same addresses. The vga window
314can be removed by programming the memory controller; this is modelled by
315removing the alias and exposing the RAM underneath.
316
317The pci address space is not a direct child of the system address space, since
318we only want parts of it to be visible (we accomplish this using aliases).
319It has two subregions: vga-area models the legacy vga window and is occupied
320by two 32K memory banks pointing at two sections of the framebuffer.
321In addition the vram is mapped as a BAR at address e1000000, and an additional
322BAR containing MMIO registers is mapped after it.
323
324Note that if the guest maps a BAR outside the PCI hole, it would not be
325visible as the pci-hole alias clips it to a 0.5GB range.
326
Avi Kivity9d3a4732011-07-26 14:26:00 +0300327MMIO Operations
328---------------
329
330MMIO regions are provided with ->read() and ->write() callbacks; in addition
331various constraints can be supplied to control how these callbacks are called:
332
333 - .valid.min_access_size, .valid.max_access_size define the access sizes
334 (in bytes) which the device accepts; accesses outside this range will
335 have device and bus specific behaviour (ignored, or machine check)
Peter Maydellef00bda2016-02-26 16:40:51 +0800336 - .valid.unaligned specifies that the *device being modelled* supports
337 unaligned accesses; if false, unaligned accesses will invoke the
338 appropriate bus or CPU specific behaviour.
Avi Kivity9d3a4732011-07-26 14:26:00 +0300339 - .impl.min_access_size, .impl.max_access_size define the access sizes
340 (in bytes) supported by the *implementation*; other access sizes will be
341 emulated using the ones available. For example a 4-byte write will be
Ademar de Souza Reis Jr69ddaf62011-12-05 16:54:14 -0300342 emulated using four 1-byte writes, if .impl.max_access_size = 1.
Fam Zhengedc1ba72014-05-05 15:53:41 +0800343 - .impl.unaligned specifies that the *implementation* supports unaligned
344 accesses; if false, unaligned accesses will be emulated by two aligned
345 accesses.
Peter Maydellef00bda2016-02-26 16:40:51 +0800346 - .old_mmio eases the porting of code that was formerly using
Fam Zhengedc1ba72014-05-05 15:53:41 +0800347 cpu_register_io_memory(). It should not be used in new code.