qapi-gen: New common driver for code and doc generators

Whenever qapi-schema.json changes, we run six programs eleven times to
update eleven files.  Similar for qga/qapi-schema.json.  This is
silly.  Replace the six programs by a single program that spits out
all eleven files.

The programs become modules in new Python package qapi, along with the
helper library.  This requires moving them to scripts/qapi/.  While
moving them, consistently drop executable mode bits.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20180211093607.27351-9-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
[eblake: move change to one-line 'blurb' earlier in series, mention mode
bit change as intentional, update qapi-code-gen.txt to match actual
generated events.c file]
Signed-off-by: Eric Blake <eblake@redhat.com>
This commit is contained in:
Markus Armbruster 2018-02-26 13:48:58 -06:00 committed by Eric Blake
parent 26df4e7fab
commit fb0bc835e5
16 changed files with 191 additions and 277 deletions

View file

@ -899,12 +899,13 @@ the names of built-in types. Clients should examine member
== Code generation ==
Schemas are fed into five scripts to generate all the code/files that,
paired with the core QAPI libraries, comprise everything required to
take JSON commands read in by a Client JSON Protocol server, unmarshal
the arguments into the underlying C types, call into the corresponding
C function, map the response back to a Client JSON Protocol response
to be returned to the user, and introspect the commands.
The QAPI code generator qapi-gen.py generates code and documentation
from the schema. Together with the core QAPI libraries, this code
provides everything required to take JSON commands read in by a Client
JSON Protocol server, unmarshal the arguments into the underlying C
types, call into the corresponding C function, map the response back
to a Client JSON Protocol response to be returned to the user, and
introspect the commands.
As an example, we'll use the following schema, which describes a
single complex user-defined type, along with command which takes a
@ -922,18 +923,23 @@ qmp_my_command(); everything else is produced by the generator.
{ 'event': 'MY_EVENT' }
We run qapi-gen.py like this:
$ python scripts/qapi-gen.py --output-dir="qapi-generated" \
--prefix="example-" example-schema.json
For a more thorough look at generated code, the testsuite includes
tests/qapi-schema/qapi-schema-tests.json that covers more examples of
what the generator will accept, and compiles the resulting C code as
part of 'make check-unit'.
=== scripts/qapi-types.py ===
=== Code generated for QAPI types ===
Used to generate the C types defined by a schema, along with
supporting code. The following files are created:
The following files are created:
$(prefix)qapi-types.h - C types corresponding to types defined in
the schema you pass in
the schema
$(prefix)qapi-types.c - Cleanup functions for the above C types
The $(prefix) is an optional parameter used as a namespace to keep the
@ -943,8 +949,6 @@ created code.
Example:
$ python scripts/qapi-types.py --output-dir="qapi-generated" \
--prefix="example-" example-schema.json
$ cat qapi-generated/example-qapi-types.h
[Uninteresting stuff omitted...]
@ -1008,28 +1012,26 @@ Example:
visit_free(v);
}
=== scripts/qapi-visit.py ===
=== Code generated for visiting QAPI types ===
Used to generate the visitor functions used to walk through and
convert between a native QAPI C data structure and some other format
(such as QObject); the generated functions are named visit_type_FOO()
and visit_type_FOO_members().
These are the visitor functions used to walk through and convert
between a native QAPI C data structure and some other format (such as
QObject); the generated functions are named visit_type_FOO() and
visit_type_FOO_members().
The following files are generated:
$(prefix)qapi-visit.c: visitor function for a particular C type, used
$(prefix)qapi-visit.c: Visitor function for a particular C type, used
to automagically convert QObjects into the
corresponding C type and vice-versa, as well
as for deallocating memory for an existing C
type
$(prefix)qapi-visit.h: declarations for previously mentioned visitor
$(prefix)qapi-visit.h: Declarations for previously mentioned visitor
functions
Example:
$ python scripts/qapi-visit.py --output-dir="qapi-generated"
--prefix="example-" example-schema.json
$ cat qapi-generated/example-qapi-visit.h
[Uninteresting stuff omitted...]
@ -1137,30 +1139,22 @@ Example:
error_propagate(errp, err);
}
=== scripts/qapi-commands.py ===
=== Code generated for commands ===
Used to generate the marshaling/dispatch functions for the commands
defined in the schema. The generated code implements
qmp_marshal_COMMAND() (registered automatically), and declares
qmp_COMMAND() that the user must implement. The following files are
generated:
These are the marshaling/dispatch functions for the commands defined
in the schema. The generated code provides qmp_marshal_COMMAND(), and
declares qmp_COMMAND() that the user must implement.
$(prefix)qmp-marshal.c: command marshal/dispatch functions for each
QMP command defined in the schema. Functions
generated by qapi-visit.py are used to
convert QObjects received from the wire into
function parameters, and uses the same
visitor functions to convert native C return
values to QObjects from transmission back
over the wire.
The following files are generated:
$(prefix)qmp-marshal.c: Command marshal/dispatch functions for each
QMP command defined in the schema
$(prefix)qmp-commands.h: Function prototypes for the QMP commands
specified in the schema.
specified in the schema
Example:
$ python scripts/qapi-commands.py --output-dir="qapi-generated"
--prefix="example-" example-schema.json
$ cat qapi-generated/example-qmp-commands.h
[Uninteresting stuff omitted...]
@ -1242,20 +1236,20 @@ Example:
qmp_marshal_my_command, QCO_NO_OPTIONS);
}
=== scripts/qapi-event.py ===
=== Code generated for events ===
Used to generate the event-related C code defined by a schema, with
implementations for qapi_event_send_FOO(). The following files are
created:
This is the code related to events defined in the schema, providing
qapi_event_send_EVENT().
The following files are created:
$(prefix)qapi-event.h - Function prototypes for each event type, plus an
enumeration of all event names
$(prefix)qapi-event.c - Implementation of functions to send an event
Example:
$ python scripts/qapi-event.py --output-dir="qapi-generated"
--prefix="example-" example-schema.json
$ cat qapi-generated/example-qapi-event.h
[Uninteresting stuff omitted...]
@ -1301,24 +1295,24 @@ Example:
QDECREF(qmp);
}
const char *const example_QAPIEvent_lookup[] = {
[EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT",
[EXAMPLE_QAPI_EVENT__MAX] = NULL,
const QEnumLookup example_QAPIEvent_lookup = {
.array = (const char *const[]) {
[EXAMPLE_QAPI_EVENT_MY_EVENT] = "MY_EVENT",
},
.size = EXAMPLE_QAPI_EVENT__MAX
};
=== scripts/qapi-introspect.py ===
=== Code generated for introspection ===
Used to generate the introspection C code for a schema. The following
files are created:
The following files are created:
$(prefix)qmp-introspect.c - Defines a string holding a JSON
description of the schema.
$(prefix)qmp-introspect.h - Declares the above string.
description of the schema
$(prefix)qmp-introspect.h - Declares the above string
Example:
$ python scripts/qapi-introspect.py --output-dir="qapi-generated"
--prefix="example-" example-schema.json
$ cat qapi-generated/example-qmp-introspect.h
[Uninteresting stuff omitted...]