Sphinx < 4.1 handles cross-references ... differently. Factor out and
isolate the compatibility goop we need to make cross references work
properly in old versions of Sphinx.
Yes, it's ugly. Yes, it works. No, I don't want to talk about
it.
Understand that this patch exists because of the overflowing love in my
heart.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-30-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Improve the general look and feel of generated QAPI docs.
Attempt to limit line lengths to offer a more comfortable measure on
maximized windows, and improve some margin and spacing for field lists.
Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-29-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This commit, finally, adds cross-referencing support to various field
lists; modeled tightly after Sphinx's own Python domain code.
Cross-referencing support is added to type names provided to :arg:,
:memb:, :returns: and :choice:.
:feat:, :error: and :value:, which do not take type names, do not
support this syntax.
The general syntax is simple:
:arg TypeName ArgName: Lorem Ipsum ...
The domain will transform TypeName into :qapi:type:`TypeName` in this
basic case, and also apply the ``literal`` decoration to indicate that
this is a type cross-reference.
For optional arguments, the special "?" suffix is used. Because "*" has
special meaning in rST that would cause parsing errors, we elect to use
"?" instead. The special syntax processing strips this character from
the end of any type name argument and will append ", optional" to the
rendered output, applying the cross-reference only to the actual type
name.
The intent here is that the actual syntax in doc-blocks need not change;
but e.g. qapidoc.py will need to process and transform "@arg foo lorem
ipsum" into ":arg type? foo: lorem ipsum" based on the schema
information. Therefore, nobody should ever actually witness this
intermediate syntax unless they are writing manual documentation or the
doc transmogrifier breaks.
For array arguments, type names can similarly be surrounded by "[]",
which are stripped off and then re-appended outside of the
cross-reference.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-28-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Normally, Sphinx will silently fall back to its standard field list
processing if it doesn't match one of your defined fields. A lot of the
time, that's not what we want - we want to be warned if we goof
something up.
For instance, the canonical argument field list form is:
:arg type name: descr
This form is captured by Sphinx and transformed so that the field label
will become "Arguments:". It's possible to omit the type name and descr
and still have it be processed correctly. However, if you omit the type
name, Sphinx no longer recognizes it:
:arg: this is not recognized.
This will turn into an arbitrary field list entry whose label is "Arg:",
and it otherwise silently fails. You may also see failures for doing
things like using :values: instead of :value:, or :errors: instead of
:error:, and so on. It's also case sensitive, and easy to trip up.
Add a validator that guarantees all field list entries that are the
direct child of an ObjectDescription use only recognized forms of field
lists, and emit a warning (treated as error by default in most build
configurations) whenever we detect one that is goofed up.
However, there's still benefit to allowing arbitrary fields -- they are
after all not a Sphinx invention, but perfectly normal docutils
syntax. Create an allow list for known spellings we don't mind letting
through, but warn against anything else.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-27-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Add a special :ifcond: option that allows us to annotate the
definition-level conditionals.
The syntax of the argument is currently undefined, but it's possible we
can apply better formatting in the future. Currently, we just display
the ifcond string as preformatted text.
Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-26-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Although "unstable" is a feature (and *will* appear in the features
list), add a special :unstable: option to generate an eye-catch that
makes this information very hard to miss.
The forthcoming Transmogrifier in qapidoc.py will add this option
whenever it detects that the features list attached to a definition
contains the "unstable" entry.
Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-25-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Although "deprecated" is a feature (and *will* appear in the features
list), add a special :deprecated: option to generate an eye-catch that
makes this information very hard to miss.
The forthcoming Transmogrifier in qapidoc.py will add this option
whenever it detects that the features list attached to a definition
contains the "deprecated" entry.
P.S., I outsourced the CSS ;)
Signed-off-by: Harmonie Snow <harmonie@gmail.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-24-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Adds the .. qapi:object:: directive, object, and :qapi:obj:`name`
cross-referencing role. This directive is meant to document both structs
and unions.
As per usual, QAPI cross-referencing for types in the member field list
will be added in a forthcoming commit.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-23-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Adds the .. qapi:event:: directive, object, and :qapi:event:`name`
cross-referencing role.
Adds the :memb type name: field list syntax for documenting event data
members. As this syntax and phrasing will be shared with Structs and
Unions as well, add the field list definition to a shared abstract
class.
As per usual, QAPI cross-referencing for types in the member field list
will be added in a forthcoming commit.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-22-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Add the .. qapi:alternate:: directive, object, and qapi:alt:`name`
cross-reference role.
Add the "Alternatives:" field list for describing alternate choices. Like
other field lists that reference QAPI types, a forthcoming commit will
add cross-referencing support to this field.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-21-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Add the .. qapi:enum:: directive, object, and :qapi:enum:`name`
cross-reference role.
Add the :value name: field list for documenting Enum values.
Of note, also introduce a new "type" role that is intended to be used by
other QAPI object directives to cross-reference arbitrary QAPI type
names, but will exclude commands, events, and modules from
consideration.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-20-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Add "Return:" field list syntax to QAPI Commands.
Like "Arguments:" and "Errors:", the type name isn't currently processed
for cross-referencing, but this will be addressed in a forthcoming
commit.
The syntax of the new field is:
:return TypeName: description
description cont'd
This patch adds "Return" as a GroupedField, which means that multiple
return values can be annotated - this is only done because Sphinx does
not support mandatory type arguments to Ungrouped fields. Because we
want to cross-reference this type information later, we want to make the
type argument mandatory. As a result, you can technically add multiple
:return: fields, though I'm not aware of any circumstance in which you'd
need or want to. Recommendation: "Don't do that, then." The forthcoming
QAPIDoc transmogrifier does not, in fact, ever "do that".
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-19-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
``:error: descr`` can now be used to document error conditions. The
format of the description is not defined here; so the ability to name
specific types is left to the document writer.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-18-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Add support for Features field lists. There is no QAPI-specific
functionality here, but this could be changed if desired (if we wanted
the feature names to link somewhere, for instance.)
This feature list doesn't have any restrictions, so it can be used to
document object-wide features or per-member features as deemed
appropriate. It's essentially free-form text.
The syntax for this field is:
:feat name: description
description cont'd
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-17-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This adds special rendering for Sphinx's typed info field lists.
This patch does not add any QAPI-aware markup, rendering, or
cross-referencing for the type names, yet. That feature requires a
subclass to TypedField which will happen in its own commit quite a bit
later in this series; after all the basic fields and objects have been
established first.
The syntax for this field is:
:arg type name: description
description cont'd
You can omit the type or the description. You should not omit the name;
if you do so, it degenerates into a "normal field list" entry, and
probably isn't what you want.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-16-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Add a little special markup for registering "Since:" information. Adding
it as an option instead of generic content lets us hoist the information
into the Signature bar, optionally put it in the index, etc.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-15-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This commit adds a stubbed version of QAPICommand that utilizes the
QAPIObject class, the qapi:command directive, the :qapi:cmd:
cross-reference role, and the "command" object type in the QAPI object
registry.
They don't do anything *particularly* interesting yet, but that will
come in forthcoming commits.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-14-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This patch adds another abstract class that describes "a QAPI
thingie". The main difference here is that this class will be generating
visible documentation, unlike the QAPIDescription class.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-13-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This adds the qapi:module directive, which just notes the current module
being documented and performs a nested parse of the content block, if
present.
This code is based pretty heavily on Sphinx's PyModule directive, but
with unnecessary features excised.
For example:
.. qapi:module:: block-core
Hello, and welcome to block-core!
=================================
lorem ipsum, dolor sit amet ...
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-12-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This class is a generic, top-level directive for documenting some kind
of QAPI thingamajig that we expect to go into the Index. This class
doesn't do much by itself, and it isn't yet associated with any
particular directive.
handle_signature(), _object_hierarchy_parts() and _toc_entry_name() are
defined in the base class. get_index_text() and add_target_and_index()
are new methods defined here; they are based heavily on the layout and
format of the Python domain's general object class.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-11-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Sphinx prior to v4.0 uses different classes for rendering elements of
documentation objects; add some compatibility classes to use the right
node classes conditionally.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-10-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Add domain-specific cross-reference syntax. As of this commit, that
means new :qapi:any:`block-core` referencing syntax.
The :any: role will find anything registered to the QAPI domain,
including modules, commands, events, etc.
Creating the cross-references is powered by the QAPIXRefRole class;
resolving them is handled by QAPIDomain.resolve_xref().
QAPIXrefRole is based heavily on Sphinx's own PyXrefRole, with
modifications necessary for QAPI features.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-9-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Add the ability to resolve cross-references using the `any`
cross-reference syntax. Adding QAPI-specific cross-reference roles will
be added in a forthcoming commit, and will share the same find_obj()
helper.
(There's less code needed for the generic cross-reference resolver, so
it comes first in this series.)
Once again, this code is based very heavily on sphinx.domains.python.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-8-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Use the QAPI object registry to generate a special index just for QAPI
definitions. The index can show entries both by definition type and all
together, alphabetically.
The index can be linked from anywhere in the QEMU manual by using the
reference `qapi-index`.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-7-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This is the first step towards QAPI domain cross-references and a QAPI
reference index.
This patch just creates the object registry, and updates the
merge_domaindata stub method now that we have actual data we may need to
merge.
Note that how to handle merge conflict resolution is unhandled, as the
Sphinx python domain itself does not handle it either. I do not know how
to intentionally trigger it, so I've left an assertion instead if it
should ever come up ...
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-6-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Create a compat module that handles sphinx cross-version compatibility
issues. For the inaugural function, add a nested_parse_with_titles()
helper that handles differences in line number tracking for nested
directive body parsing.
Spoilers: there are more cross-version hacks to come throughout the
series.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-5-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
A Sphinx domain is a collection of directive and role extensions meant
to facilitate the documentation of a specific language. For instance,
Sphinx ships with "python" and "cpp" domains. This patch introduces a
stub for the "qapi" language domain.
Please see https://www.sphinx-doc.org/en/master/usage/domains/index.html
for more information.
This stub doesn't really do anything yet, we'll get to it brick-by-brick
in the forthcoming commits to keep the series breezy and the git history
informative.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-ID: <20250311034303.75779-4-jsnow@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
iQEzBAABCAAdFiEEIV1G9IJGaJ7HfzVi7wSWWzmNYhEFAmfO1zkACgkQ7wSWWzmN
YhET+wf+PkaGeFTNUrOtWpl35fSMKlmOVbb1fkPfuhVBmeY2Vh1EIN3OjqnzdV0F
wxpuk+wwmFiuV1n6RNuMHQ0nz1mhgsSlZh93N5rArC/PUr3iViaT0cb82RjwxhaI
RODBhhy7V9WxEhT9hR8sCP2ky2mrKgcYbjiIEw+IvFZOVQa58rMr2h/cbAb/iH4l
7T9Wba03JBqOS6qgzSFZOMxvqnYdVjhqXN8M6W9ngRJOjPEAkTB6Evwep6anRjcM
mCUOgkf2sgQwKve8pYAeTMkzXFctvTc/qCU4ZbN8XcoKVVxe2jllGQqdOpMskPEf
slOuINeW5M0K5gyjsb/huqcOTfDI2A==
=/Y0+
-----END PGP SIGNATURE-----
Merge tag 'net-pull-request' of https://github.com/jasowang/qemu into staging
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEIV1G9IJGaJ7HfzVi7wSWWzmNYhEFAmfO1zkACgkQ7wSWWzmN
# YhET+wf+PkaGeFTNUrOtWpl35fSMKlmOVbb1fkPfuhVBmeY2Vh1EIN3OjqnzdV0F
# wxpuk+wwmFiuV1n6RNuMHQ0nz1mhgsSlZh93N5rArC/PUr3iViaT0cb82RjwxhaI
# RODBhhy7V9WxEhT9hR8sCP2ky2mrKgcYbjiIEw+IvFZOVQa58rMr2h/cbAb/iH4l
# 7T9Wba03JBqOS6qgzSFZOMxvqnYdVjhqXN8M6W9ngRJOjPEAkTB6Evwep6anRjcM
# mCUOgkf2sgQwKve8pYAeTMkzXFctvTc/qCU4ZbN8XcoKVVxe2jllGQqdOpMskPEf
# slOuINeW5M0K5gyjsb/huqcOTfDI2A==
# =/Y0+
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 10 Mar 2025 20:12:41 HKT
# gpg: using RSA key 215D46F48246689EC77F3562EF04965B398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>" [full]
# Primary key fingerprint: 215D 46F4 8246 689E C77F 3562 EF04 965B 398D 6211
* tag 'net-pull-request' of https://github.com/jasowang/qemu:
tap-linux: Open ipvtap and macvtap
Revert "hw/net/net_tx_pkt: Fix overrun in update_sctp_checksum()"
util/iov: Do not assert offset is in iov
net: move backend cleanup to NIC cleanup
net: parameterize the removing client from nc list
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
- update and expand aarch64 GPU tests
- fix build dependence for plugins
- update libvirt-ci to vulkan-tools
- allow plugin tests to run on non-POSIX systems
- tweak test/vm times
- mark test-vma as linux only
- various compiler fixes for tcg tests
- add gitlab build unit tracker
- error out early on stalled RME tests
- compile core plugin code once
- update MAINTAINERS
-----BEGIN PGP SIGNATURE-----
iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmfOwjcACgkQ+9DbCVqe
KkRkcwgAlRTflCqYlZxdlo4CiOSXaHAFr8yfwWq138LJQRQH530JZnz1lZtxTbEM
pXT7ixnuJQDMybCQJmvUlK5UTUkZhGS3VuAR1VeM2J8/3VXYzf5sFjZ7yko9mA8S
2FX8vdfbko8/J31+lKccA0tpbHyi2AbMR+mO8xj6KZQoePwmHoRmhgH7p7LE35YO
8ytaOjMwTKF5fReVK+tlcrVJHFMdGsGNwtsnB2FhhVjI56fQqyM5hGXfOING2Fx3
iZH3rjzfDB4SWbBqaRsMgH9RXjuB9Eo4v0Qs5ve5SjDyzRJk+/CbbBJ4oRt9hurJ
bA+CYZuNLGBf8Z/mUeYMavA7rxT5rw==
=qobU
-----END PGP SIGNATURE-----
Merge tag 'pull-10.0-for-softfreeze-100325-3' of https://gitlab.com/stsquad/qemu into staging
functional and tcg tests, plugins and MAINTAINERS
- update and expand aarch64 GPU tests
- fix build dependence for plugins
- update libvirt-ci to vulkan-tools
- allow plugin tests to run on non-POSIX systems
- tweak test/vm times
- mark test-vma as linux only
- various compiler fixes for tcg tests
- add gitlab build unit tracker
- error out early on stalled RME tests
- compile core plugin code once
- update MAINTAINERS
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmfOwjcACgkQ+9DbCVqe
# KkRkcwgAlRTflCqYlZxdlo4CiOSXaHAFr8yfwWq138LJQRQH530JZnz1lZtxTbEM
# pXT7ixnuJQDMybCQJmvUlK5UTUkZhGS3VuAR1VeM2J8/3VXYzf5sFjZ7yko9mA8S
# 2FX8vdfbko8/J31+lKccA0tpbHyi2AbMR+mO8xj6KZQoePwmHoRmhgH7p7LE35YO
# 8ytaOjMwTKF5fReVK+tlcrVJHFMdGsGNwtsnB2FhhVjI56fQqyM5hGXfOING2Fx3
# iZH3rjzfDB4SWbBqaRsMgH9RXjuB9Eo4v0Qs5ve5SjDyzRJk+/CbbBJ4oRt9hurJ
# bA+CYZuNLGBf8Z/mUeYMavA7rxT5rw==
# =qobU
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 10 Mar 2025 18:43:03 HKT
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* tag 'pull-10.0-for-softfreeze-100325-3' of https://gitlab.com/stsquad/qemu: (31 commits)
MAINTAINERS: remove widely sanctioned entities
plugins/core: make a single build unit
plugins/api: build only once
plugins/api: split out time control helpers
plugins/api: split out the vaddr/hwaddr helpers
plugins/api: split out binary path/start/end/entry code
plugins/loader: compile loader only once
plugins/plugin.h: include queue.h
plugins/api: clean-up the includes
include/qemu: plugin-memory.h doesn't need cpu-defs.h
plugins/loader: populate target_name with target_name()
plugins/api: use qemu_target_page_mask() to get value
tests/functional: add boot error detection for RME tests
gitlab: add a new build_unit job to track build size
tests/tcg: Suppress compiler false-positive warning on sha1.c
tests/tcg: enable -fwrapv for test-i386-bmi
tests/tcg: fix constraints in test-i386-adcox
tests/tcg: add message to _Static_assert in test-avx
tests/tcg: mark test-vma as a linux-only test
tests/vm: bump timeout for shutdown
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Address an error in RDMA-based migration by ensuring RDMA is prioritized
when saving pages in `ram_save_target_page()`.
Previously, the RDMA protocol's page-saving step was placed after other
protocols due to a refactoring in commit bc38dc2f5f. This led to migration
failures characterized by unknown control messages and state loading errors
destination:
(qemu) qemu-system-x86_64: Unknown control message QEMU FILE
qemu-system-x86_64: error while loading state section id 1(ram)
qemu-system-x86_64: load of migration failed: Operation not permitted
source:
(qemu) qemu-system-x86_64: RDMA is in an error state waiting migration to abort!
qemu-system-x86_64: failed to save SaveStateEntry with id(name): 1(ram): -1
qemu-system-x86_64: rdma migration: recv polling control error!
qemu-system-x86_64: warning: Early error. Sending error.
qemu-system-x86_64: warning: rdma migration: send polling control error
RDMA migration implemented its own protocol/method to send pages to
destination side, hand over to RDMA first to prevent pages being saved by
other protocol.
Fixes: bc38dc2f5f ("migration: refactor ram_save_target_page functions")
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
Message-ID: <20250305062825.772629-2-lizhijian@fujitsu.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
Unlike cpr-reboot mode, cpr-transfer mode cannot save volatile ram blocks
in the migration stream file and recreate them later, because the physical
memory for the blocks is pinned and registered for vfio. Add a blocker
for volatile ram blocks.
Also add a blocker for RAM_GUEST_MEMFD. Preserving guest_memfd may be
sufficient for CPR, but it has not been tested yet.
Signed-off-by: Steve Sistare <steven.sistare@oracle.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-ID: <1740667681-257312-1-git-send-email-steven.sistare@oracle.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
On the incoming migration side, QEMU uses a coroutine to load all the VM
states. Inside, it may reference MigrationState on global states like
migration capabilities, parameters, error state, shared mutexes and more.
However there's nothing yet to make sure MigrationState won't get
destroyed (e.g. after migration_shutdown()). Meanwhile there's also no API
available to remove the incoming coroutine in migration_shutdown(),
avoiding it to access the freed elements.
There's a bug report showing this can happen and crash dest QEMU when
migration is cancelled on source.
When it happens, the dest main thread is trying to cleanup everything:
#0 qemu_aio_coroutine_enter
#1 aio_dispatch_handler
#2 aio_poll
#3 monitor_cleanup
#4 qemu_cleanup
#5 qemu_default_main
Then it found the migration incoming coroutine, schedule it (even after
migration_shutdown()), causing crash:
#0 __pthread_kill_implementation
#1 __pthread_kill_internal
#2 __GI_raise
#3 __GI_abort
#4 __assert_fail_base
#5 __assert_fail
#6 qemu_mutex_lock_impl
#7 qemu_lockable_mutex_lock
#8 qemu_lockable_lock
#9 qemu_lockable_auto_lock
#10 migrate_set_error
#11 process_incoming_migration_co
#12 coroutine_trampoline
To fix it, take a refcount after an incoming setup is properly done when
qmp_migrate_incoming() succeeded the 1st time. As it's during a QMP
handler which needs BQL, it means the main loop is still alive (without
going into cleanups, which also needs BQL).
Releasing the refcount now only until the incoming migration coroutine
finished or failed. Hence the refcount is valid for both (1) setup phase
of incoming ports, mostly IO watches (e.g. qio_channel_add_watch_full()),
and (2) the incoming coroutine itself (process_incoming_migration_co()).
Note that we can't unref in migration_incoming_state_destroy(), because
both qmp_xen_load_devices_state() and load_snapshot() will use it without
an incoming migration. Those hold BQL so they're not prone to this issue.
PS: I suspect nobody uses Xen's command at all, as it didn't register yank,
hence AFAIU the command should crash on master when trying to unregister
yank in migration_incoming_state_destroy().. but that's another story.
Also note that in some incoming failure cases we may not always unref the
MigrationState refcount, which is a trade-off to keep things simple. We
could make it accurate, but it can be an overkill. Some examples:
- Unlike most of the rest protocols, socket_start_incoming_migration()
may create net listener after incoming port setup sucessfully.
It means we can't unref in migration_channel_process_incoming() as a
generic path because socket protocol might keep using MigrationState.
- For either socket or file, multiple IO watches might be created, it
means logically each IO watch needs to take one refcount for
MigrationState so as to be 100% accurate on ownership of refcount taken.
In general, we at least need per-protocol handling to make it accurate,
which can be an overkill if we know incoming failed after all. Add a short
comment to explain that when taking the refcount in qmp_migrate_incoming().
Bugzilla: https://issues.redhat.com/browse/RHEL-69775
Tested-by: Yan Fu <yafu@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Message-ID: <20250220132459.512610-1-peterx@redhat.com>
Signed-off-by: Fabiano Rosas <farosas@suse.de>
On IOREQ_TYPE_INVALIDATE we need to invalidate the mapcache for regular
mappings. Since recently we started reusing the mapcache also to keep
track of grants mappings. However, there is no need to remove grant
mappings on IOREQ_TYPE_INVALIDATE requests, we shouldn't do that. So
remove the function call.
Fixes: 9ecdd4bf08 (xen: mapcache: Add support for grant mappings)
Cc: qemu-stable@nongnu.org
Reported-by: Olaf Hering <olaf@aepfle.de>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@amd.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@amd.com>
Reviewed-by: Anthony PERARD <anthony.perard@vates.tech>
Message-Id: <20250206194915.3357743-2-edgar.iglesias@gmail.com>
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
Block devices don't work in PV Grub (0.9x) if there is no mode specified. It
complains: "Error ENOENT when reading the mode"
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Message-Id: <20250207143724.30792-2-dwmw2@infradead.org>
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
In PVH dom0, when passthrough a device to domU, QEMU code
xen_pt_realize->xc_physdev_map_pirq wants to use gsi, but in current codes
the gsi number is got from file /sys/bus/pci/devices/<sbdf>/irq, that is
wrong, because irq is not equal with gsi, they are in different spaces, so
pirq mapping fails.
To solve above problem, use new interface of Xen, xc_pcidev_get_gsi to get
gsi and use xc_physdev_map_pirq_gsi to map pirq when dom0 is PVH.
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
Signed-off-by: Huang Rui <ray.huang@amd.com>
Signed-off-by: Jiqian Chen <Jiqian.Chen@amd.com>
Acked-by: Anthony PERARD <anthony@xenproject.org>
Reviewed-by: Stewart Hildebrand <stewart.hildebrand@amd.com>
Message-Id: <20241106061418.3655304-1-Jiqian.Chen@amd.com>
Signed-off-by: Anthony PERARD <anthony.perard@vates.tech>
The following organisations appear on the US sanctions list:
Yadro: https://sanctionssearch.ofac.treas.gov/Details.aspx?id=41125
ISPRAS: https://sanctionssearch.ofac.treas.gov/Details.aspx?id=50890
As a result maintainers interacting with such entities would face
legal risk in a number of jurisdictions. To reduce the risk of
inadvertent non-compliance remove entries from these organisations
from the MAINTAINERS file.
Mark the pcf8574 system as orphaned until someone volunteers to step
up as a maintainer. Add myself as a second reviewer to record/replay
so I can help with what odd fixes I can.
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-32-alex.bennee@linaro.org>
Trim through the includes and remove everything not needed for the
core. Only include tcg-op-common.h to remove the need to
TARGET_LONG_BITS and move the build unit into the common set.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-31-alex.bennee@linaro.org>
Now all the softmmu/user-mode stuff has been split out we can build
this compilation unit only once.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-30-alex.bennee@linaro.org>
These are only usable in system mode where we control the timer. For
user-mode make them NOPs.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-29-alex.bennee@linaro.org>
These only work for system-mode and are NOPs for user-mode.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-28-alex.bennee@linaro.org>
To move the main api.c to a single build compilation object we need to
start splitting out user and system specific code. As we need to grob
around host headers we move these particular helpers into the *-user
mode directories.
The binary/start/end/entry helpers are all NOPs for system mode.
While using the plugin-api.c.inc trick means we build for both
linux-user and bsd-user the BSD user-mode command line is still
missing -plugin. This can be enabled once we have reliable check-tcg
tests working for the BSDs.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-27-alex.bennee@linaro.org>
There is very little in loader that is different between builds save
for a tiny user/system mode difference in the plugin_info structure.
Create two new files, user and system to hold mode specific helpers
and move loader into common_ss.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-26-alex.bennee@linaro.org>
Headers should bring in what they need so don't rely on getting
queue.h by side effects. This will help with clean-ups in the
following patches.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-25-alex.bennee@linaro.org>
Thanks to re-factoring and clean-up work (especially to exec-all) we
no longer need such broad headers for the api.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-24-alex.bennee@linaro.org>
hwaddr is a fixed size on all builds.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-23-alex.bennee@linaro.org>
We have a function we can call for this, lets not rely on macros that
stop us building once.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-22-alex.bennee@linaro.org>
Requiring TARGET_PAGE_MASK to be defined gets in the way of building
this unit once. qemu_target_page_mask() will tell us what it is.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-21-alex.bennee@linaro.org>
It was identified that those tests randomly fail with a synchronous
exception at boot (reported by EDK2).
While we solve this problem, report failure immediately so tests don't
timeout in CI.
Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20250303185745.2504842-1-pierrick.bouvier@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20250304222439.2035603-20-alex.bennee@linaro.org>
