| = How to use the QAPI code generator = |
| |
| QAPI is a native C API within QEMU which provides management-level |
| functionality to internal/external users. For external |
| users/processes, this interface is made available by a JSON-based |
| QEMU Monitor protocol that is provided by the QMP server. |
| |
| To map QMP-defined interfaces to the native C QAPI implementations, |
| a JSON-based schema is used to define types and function |
| signatures, and a set of scripts is used to generate types/signatures, |
| and marshaling/dispatch code. The QEMU Guest Agent also uses these |
| scripts, paired with a separate schema, to generate |
| marshaling/dispatch code for the guest agent server running in the |
| guest. |
| |
| This document will describe how the schemas, scripts, and resulting |
| code are used. |
| |
| |
| == QMP/Guest agent schema == |
| |
| This file defines the types, commands, and events used by QMP. It should |
| fully describe the interface used by QMP. |
| |
| This file is designed to be loosely based on JSON although it's technically |
| executable Python. While dictionaries are used, they are parsed as |
| OrderedDicts so that ordering is preserved. |
| |
| There are two basic syntaxes used, type definitions and command definitions. |
| |
| The first syntax defines a type and is represented by a dictionary. There are |
| three kinds of user-defined types that are supported: complex types, |
| enumeration types and union types. |
| |
| Generally speaking, types definitions should always use CamelCase for the type |
| names. Command names should be all lower case with words separated by a hyphen. |
| |
| |
| === Includes === |
| |
| The QAPI schema definitions can be modularized using the 'include' directive: |
| |
| { 'include': 'path/to/file.json'} |
| |
| The directive is evaluated recursively, and include paths are relative to the |
| file using the directive. Multiple includes of the same file are safe. |
| |
| |
| === Complex types === |
| |
| A complex type is a dictionary containing a single key whose value is a |
| dictionary. This corresponds to a struct in C or an Object in JSON. An |
| example of a complex type is: |
| |
| { 'type': 'MyType', |
| 'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } } |
| |
| The use of '*' as a prefix to the name means the member is optional. |
| |
| The default initialization value of an optional argument should not be changed |
| between versions of QEMU unless the new default maintains backward |
| compatibility to the user-visible behavior of the old default. |
| |
| With proper documentation, this policy still allows some flexibility; for |
| example, documenting that a default of 0 picks an optimal buffer size allows |
| one release to declare the optimal size at 512 while another release declares |
| the optimal size at 4096 - the user-visible behavior is not the bytes used by |
| the buffer, but the fact that the buffer was optimal size. |
| |
| On input structures (only mentioned in the 'data' side of a command), changing |
| from mandatory to optional is safe (older clients will supply the option, and |
| newer clients can benefit from the default); changing from optional to |
| mandatory is backwards incompatible (older clients may be omitting the option, |
| and must continue to work). |
| |
| On output structures (only mentioned in the 'returns' side of a command), |
| changing from mandatory to optional is in general unsafe (older clients may be |
| expecting the field, and could crash if it is missing), although it can be done |
| if the only way that the optional argument will be omitted is when it is |
| triggered by the presence of a new input flag to the command that older clients |
| don't know to send. Changing from optional to mandatory is safe. |
| |
| A structure that is used in both input and output of various commands |
| must consider the backwards compatibility constraints of both directions |
| of use. |
| |
| A complex type definition can specify another complex type as its base. |
| In this case, the fields of the base type are included as top-level fields |
| of the new complex type's dictionary in the QMP wire format. An example |
| definition is: |
| |
| { 'type': 'BlockdevOptionsGenericFormat', 'data': { 'file': 'str' } } |
| { 'type': 'BlockdevOptionsGenericCOWFormat', |
| 'base': 'BlockdevOptionsGenericFormat', |
| 'data': { '*backing': 'str' } } |
| |
| An example BlockdevOptionsGenericCOWFormat object on the wire could use |
| both fields like this: |
| |
| { "file": "/some/place/my-image", |
| "backing": "/some/place/my-backing-file" } |
| |
| === Enumeration types === |
| |
| An enumeration type is a dictionary containing a single key whose value is a |
| list of strings. An example enumeration is: |
| |
| { 'enum': 'MyEnum', 'data': [ 'value1', 'value2', 'value3' ] } |
| |
| === Union types === |
| |
| Union types are used to let the user choose between several different data |
| types. A union type is defined using a dictionary as explained in the |
| following paragraphs. |
| |
| |
| A simple union type defines a mapping from discriminator values to data types |
| like in this example: |
| |
| { 'type': 'FileOptions', 'data': { 'filename': 'str' } } |
| { 'type': 'Qcow2Options', |
| 'data': { 'backing-file': 'str', 'lazy-refcounts': 'bool' } } |
| |
| { 'union': 'BlockdevOptions', |
| 'data': { 'file': 'FileOptions', |
| 'qcow2': 'Qcow2Options' } } |
| |
| In the QMP wire format, a simple union is represented by a dictionary that |
| contains the 'type' field as a discriminator, and a 'data' field that is of the |
| specified data type corresponding to the discriminator value: |
| |
| { "type": "qcow2", "data" : { "backing-file": "/some/place/my-image", |
| "lazy-refcounts": true } } |
| |
| |
| A union definition can specify a complex type as its base. In this case, the |
| fields of the complex type are included as top-level fields of the union |
| dictionary in the QMP wire format. An example definition is: |
| |
| { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } } |
| { 'union': 'BlockdevOptions', |
| 'base': 'BlockdevCommonOptions', |
| 'data': { 'raw': 'RawOptions', |
| 'qcow2': 'Qcow2Options' } } |
| |
| And it looks like this on the wire: |
| |
| { "type": "qcow2", |
| "readonly": false, |
| "data" : { "backing-file": "/some/place/my-image", |
| "lazy-refcounts": true } } |
| |
| |
| Flat union types avoid the nesting on the wire. They are used whenever a |
| specific field of the base type is declared as the discriminator ('type' is |
| then no longer generated). The discriminator must be of enumeration type. |
| The above example can then be modified as follows: |
| |
| { 'enum': 'BlockdevDriver', 'data': [ 'raw', 'qcow2' ] } |
| { 'type': 'BlockdevCommonOptions', |
| 'data': { 'driver': 'BlockdevDriver', 'readonly': 'bool' } } |
| { 'union': 'BlockdevOptions', |
| 'base': 'BlockdevCommonOptions', |
| 'discriminator': 'driver', |
| 'data': { 'raw': 'RawOptions', |
| 'qcow2': 'Qcow2Options' } } |
| |
| Resulting in this JSON object: |
| |
| { "driver": "qcow2", |
| "readonly": false, |
| "backing-file": "/some/place/my-image", |
| "lazy-refcounts": true } |
| |
| |
| A special type of unions are anonymous unions. They don't form a dictionary in |
| the wire format but allow the direct use of different types in their place. As |
| they aren't structured, they don't have any explicit discriminator but use |
| the (QObject) data type of their value as an implicit discriminator. This means |
| that they are restricted to using only one discriminator value per QObject |
| type. For example, you cannot have two different complex types in an anonymous |
| union, or two different integer types. |
| |
| Anonymous unions are declared using an empty dictionary as their discriminator. |
| The discriminator values never appear on the wire, they are only used in the |
| generated C code. Anonymous unions cannot have a base type. |
| |
| { 'union': 'BlockRef', |
| 'discriminator': {}, |
| 'data': { 'definition': 'BlockdevOptions', |
| 'reference': 'str' } } |
| |
| This example allows using both of the following example objects: |
| |
| { "file": "my_existing_block_device_id" } |
| { "file": { "driver": "file", |
| "readonly": false, |
| "filename": "/tmp/mydisk.qcow2" } } |
| |
| |
| === Commands === |
| |
| Commands are defined by using a list containing three members. The first |
| member is the command name, the second member is a dictionary containing |
| arguments, and the third member is the return type. |
| |
| An example command is: |
| |
| { 'command': 'my-command', |
| 'data': { 'arg1': 'str', '*arg2': 'str' }, |
| 'returns': 'str' } |
| |
| === Events === |
| |
| Events are defined with the keyword 'event'. When 'data' is also specified, |
| additional info will be included in the event. Finally there will be C API |
| generated in qapi-event.h; when called by QEMU code, a message with timestamp |
| will be emitted on the wire. If timestamp is -1, it means failure to retrieve |
| host time. |
| |
| An example event is: |
| |
| { 'event': 'EVENT_C', |
| 'data': { '*a': 'int', 'b': 'str' } } |
| |
| Resulting in this JSON object: |
| |
| { "event": "EVENT_C", |
| "data": { "b": "test string" }, |
| "timestamp": { "seconds": 1267020223, "microseconds": 435656 } } |
| |
| |
| == Code generation == |
| |
| Schemas are fed into 3 scripts to generate all the code/files that, paired |
| with the core QAPI libraries, comprise everything required to take JSON |
| commands read in by a QMP/guest agent server, unmarshal the arguments into |
| the underlying C types, call into the corresponding C function, and map the |
| response back to a QMP/guest agent response to be returned to the user. |
| |
| As an example, we'll use the following schema, which describes a single |
| complex user-defined type (which will produce a C struct, along with a list |
| node structure that can be used to chain together a list of such types in |
| case we want to accept/return a list of this type with a command), and a |
| command which takes that type as a parameter and returns the same type: |
| |
| $ cat example-schema.json |
| { 'type': 'UserDefOne', |
| 'data': { 'integer': 'int', 'string': 'str' } } |
| |
| { 'command': 'my-command', |
| 'data': {'arg1': 'UserDefOne'}, |
| 'returns': 'UserDefOne' } |
| |
| { 'event': 'MY_EVENT' } |
| |
| === scripts/qapi-types.py === |
| |
| Used to generate the C types defined by a schema. The following files are |
| created: |
| |
| $(prefix)qapi-types.h - C types corresponding to types defined in |
| the schema you pass in |
| $(prefix)qapi-types.c - Cleanup functions for the above C types |
| |
| The $(prefix) is an optional parameter used as a namespace to keep the |
| generated code from one schema/code-generation separated from others so code |
| can be generated/used from multiple schemas without clobbering previously |
| created code. |
| |
| Example: |
| |
| $ python scripts/qapi-types.py --output-dir="qapi-generated" \ |
| --prefix="example-" --input-file=example-schema.json |
| $ cat qapi-generated/example-qapi-types.c |
| [Uninteresting stuff omitted...] |
| |
| void qapi_free_UserDefOneList(UserDefOneList *obj) |
| { |
| QapiDeallocVisitor *md; |
| Visitor *v; |
| |
| if (!obj) { |
| return; |
| } |
| |
| md = qapi_dealloc_visitor_new(); |
| v = qapi_dealloc_get_visitor(md); |
| visit_type_UserDefOneList(v, &obj, NULL, NULL); |
| qapi_dealloc_visitor_cleanup(md); |
| } |
| |
| void qapi_free_UserDefOne(UserDefOne *obj) |
| { |
| QapiDeallocVisitor *md; |
| Visitor *v; |
| |
| if (!obj) { |
| return; |
| } |
| |
| md = qapi_dealloc_visitor_new(); |
| v = qapi_dealloc_get_visitor(md); |
| visit_type_UserDefOne(v, &obj, NULL, NULL); |
| qapi_dealloc_visitor_cleanup(md); |
| } |
| |
| $ cat qapi-generated/example-qapi-types.h |
| [Uninteresting stuff omitted...] |
| |
| #ifndef EXAMPLE_QAPI_TYPES_H |
| #define EXAMPLE_QAPI_TYPES_H |
| |
| [Builtin types omitted...] |
| |
| typedef struct UserDefOne UserDefOne; |
| |
| typedef struct UserDefOneList |
| { |
| union { |
| UserDefOne *value; |
| uint64_t padding; |
| }; |
| struct UserDefOneList *next; |
| } UserDefOneList; |
| |
| [Functions on builtin types omitted...] |
| |
| struct UserDefOne |
| { |
| int64_t integer; |
| char *string; |
| }; |
| |
| void qapi_free_UserDefOneList(UserDefOneList *obj); |
| void qapi_free_UserDefOne(UserDefOne *obj); |
| |
| #endif |
| |
| === scripts/qapi-visit.py === |
| |
| Used to generate the visitor functions used to walk through and convert |
| a QObject (as provided by QMP) to a native C data structure and |
| vice-versa, as well as the visitor function used to dealloc a complex |
| schema-defined C type. |
| |
| The following files are generated: |
| |
| $(prefix)qapi-visit.c: visitor function for a particular C type, used |
| to automagically convert QObjects into the |
| corresponding C type and vice-versa, as well |
| as for deallocating memory for an existing C |
| type |
| |
| $(prefix)qapi-visit.h: declarations for previously mentioned visitor |
| functions |
| |
| Example: |
| |
| $ python scripts/qapi-visit.py --output-dir="qapi-generated" |
| --prefix="example-" --input-file=example-schema.json |
| $ cat qapi-generated/example-qapi-visit.c |
| [Uninteresting stuff omitted...] |
| |
| static void visit_type_UserDefOne_fields(Visitor *m, UserDefOne **obj, Error **errp) |
| { |
| Error *err = NULL; |
| visit_type_int(m, &(*obj)->integer, "integer", &err); |
| if (err) { |
| goto out; |
| } |
| visit_type_str(m, &(*obj)->string, "string", &err); |
| if (err) { |
| goto out; |
| } |
| |
| out: |
| error_propagate(errp, err); |
| } |
| |
| void visit_type_UserDefOne(Visitor *m, UserDefOne **obj, const char *name, Error **errp) |
| { |
| Error *err = NULL; |
| |
| visit_start_struct(m, (void **)obj, "UserDefOne", name, sizeof(UserDefOne), &err); |
| if (!err) { |
| if (*obj) { |
| visit_type_UserDefOne_fields(m, obj, errp); |
| } |
| visit_end_struct(m, &err); |
| } |
| error_propagate(errp, err); |
| } |
| |
| void visit_type_UserDefOneList(Visitor *m, UserDefOneList **obj, const char *name, Error **errp) |
| { |
| Error *err = NULL; |
| GenericList *i, **prev; |
| |
| visit_start_list(m, name, &err); |
| if (err) { |
| goto out; |
| } |
| |
| for (prev = (GenericList **)obj; |
| !err && (i = visit_next_list(m, prev, &err)) != NULL; |
| prev = &i) { |
| UserDefOneList *native_i = (UserDefOneList *)i; |
| visit_type_UserDefOne(m, &native_i->value, NULL, &err); |
| } |
| |
| error_propagate(errp, err); |
| err = NULL; |
| visit_end_list(m, &err); |
| out: |
| error_propagate(errp, err); |
| } |
| $ python scripts/qapi-commands.py --output-dir="qapi-generated" \ |
| --prefix="example-" --input-file=example-schema.json |
| $ cat qapi-generated/example-qapi-visit.h |
| [Uninteresting stuff omitted...] |
| |
| #ifndef EXAMPLE_QAPI_VISIT_H |
| #define EXAMPLE_QAPI_VISIT_H |
| |
| [Visitors for builtin types omitted...] |
| |
| void visit_type_UserDefOne(Visitor *m, UserDefOne **obj, const char *name, Error **errp); |
| void visit_type_UserDefOneList(Visitor *m, UserDefOneList **obj, const char *name, Error **errp); |
| |
| #endif |
| |
| === scripts/qapi-commands.py === |
| |
| Used to generate the marshaling/dispatch functions for the commands defined |
| in the schema. The following files are generated: |
| |
| $(prefix)qmp-marshal.c: command marshal/dispatch functions for each |
| QMP command defined in the schema. Functions |
| generated by qapi-visit.py are used to |
| convert QObjects received from the wire into |
| function parameters, and uses the same |
| visitor functions to convert native C return |
| values to QObjects from transmission back |
| over the wire. |
| |
| $(prefix)qmp-commands.h: Function prototypes for the QMP commands |
| specified in the schema. |
| |
| Example: |
| |
| $ python scripts/qapi-commands.py --output-dir="qapi-generated" |
| --prefix="example-" --input-file=example-schema.json |
| $ cat qapi-generated/example-qmp-marshal.c |
| [Uninteresting stuff omitted...] |
| |
| static void qmp_marshal_output_my_command(UserDefOne *ret_in, QObject **ret_out, Error **errp) |
| { |
| Error *local_err = NULL; |
| QmpOutputVisitor *mo = qmp_output_visitor_new(); |
| QapiDeallocVisitor *md; |
| Visitor *v; |
| |
| v = qmp_output_get_visitor(mo); |
| visit_type_UserDefOne(v, &ret_in, "unused", &local_err); |
| if (local_err) { |
| goto out; |
| } |
| *ret_out = qmp_output_get_qobject(mo); |
| |
| out: |
| error_propagate(errp, local_err); |
| qmp_output_visitor_cleanup(mo); |
| md = qapi_dealloc_visitor_new(); |
| v = qapi_dealloc_get_visitor(md); |
| visit_type_UserDefOne(v, &ret_in, "unused", NULL); |
| qapi_dealloc_visitor_cleanup(md); |
| } |
| |
| static void qmp_marshal_input_my_command(QDict *args, QObject **ret, Error **errp) |
| { |
| Error *local_err = NULL; |
| UserDefOne *retval = NULL; |
| QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args)); |
| QapiDeallocVisitor *md; |
| Visitor *v; |
| UserDefOne *arg1 = NULL; |
| |
| v = qmp_input_get_visitor(mi); |
| visit_type_UserDefOne(v, &arg1, "arg1", &local_err); |
| if (local_err) { |
| goto out; |
| } |
| |
| retval = qmp_my_command(arg1, &local_err); |
| if (local_err) { |
| goto out; |
| } |
| |
| qmp_marshal_output_my_command(retval, ret, &local_err); |
| |
| out: |
| error_propagate(errp, local_err); |
| qmp_input_visitor_cleanup(mi); |
| md = qapi_dealloc_visitor_new(); |
| v = qapi_dealloc_get_visitor(md); |
| visit_type_UserDefOne(v, &arg1, "arg1", NULL); |
| qapi_dealloc_visitor_cleanup(md); |
| return; |
| } |
| |
| static void qmp_init_marshal(void) |
| { |
| qmp_register_command("my-command", qmp_marshal_input_my_command, QCO_NO_OPTIONS); |
| } |
| |
| qapi_init(qmp_init_marshal); |
| $ cat qapi-generated/example-qmp-commands.h |
| [Uninteresting stuff omitted...] |
| |
| #ifndef EXAMPLE_QMP_COMMANDS_H |
| #define EXAMPLE_QMP_COMMANDS_H |
| |
| #include "example-qapi-types.h" |
| #include "qapi/qmp/qdict.h" |
| #include "qapi/error.h" |
| |
| UserDefOne *qmp_my_command(UserDefOne *arg1, Error **errp); |
| |
| #endif |
| |
| === scripts/qapi-event.py === |
| |
| Used to generate the event-related C code defined by a schema. The |
| following files are created: |
| |
| $(prefix)qapi-event.h - Function prototypes for each event type, plus an |
| enumeration of all event names |
| $(prefix)qapi-event.c - Implementation of functions to send an event |
| |
| Example: |
| |
| $ python scripts/qapi-event.py --output-dir="qapi-generated" |
| --prefix="example-" --input-file=example-schema.json |
| $ cat qapi-generated/example-qapi-event.c |
| [Uninteresting stuff omitted...] |
| |
| void qapi_event_send_my_event(Error **errp) |
| { |
| QDict *qmp; |
| Error *local_err = NULL; |
| QMPEventFuncEmit emit; |
| emit = qmp_event_get_func_emit(); |
| if (!emit) { |
| return; |
| } |
| |
| qmp = qmp_event_build_dict("MY_EVENT"); |
| |
| emit(EXAMPLE_QAPI_EVENT_MY_EVENT, qmp, &local_err); |
| |
| error_propagate(errp, local_err); |
| QDECREF(qmp); |
| } |
| |
| const char *EXAMPLE_QAPIEvent_lookup[] = { |
| "MY_EVENT", |
| NULL, |
| }; |
| $ cat qapi-generated/example-qapi-event.h |
| [Uninteresting stuff omitted...] |
| |
| #ifndef EXAMPLE_QAPI_EVENT_H |
| #define EXAMPLE_QAPI_EVENT_H |
| |
| #include "qapi/error.h" |
| #include "qapi/qmp/qdict.h" |
| #include "example-qapi-types.h" |
| |
| |
| void qapi_event_send_my_event(Error **errp); |
| |
| extern const char *EXAMPLE_QAPIEvent_lookup[]; |
| typedef enum EXAMPLE_QAPIEvent |
| { |
| EXAMPLE_QAPI_EVENT_MY_EVENT = 0, |
| EXAMPLE_QAPI_EVENT_MAX = 1, |
| } EXAMPLE_QAPIEvent; |
| |
| #endif |