docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS

It doesn't make sense to have two classes of flaky tests. While it may
take the constrained environment of CI to trigger failures easily it
doesn't mean they don't occasionally happen on developer machines. As
CI is the gating factor to passing there is no point developers
running the tests locally anyway unless they are trying to fix things.

While we are at it update the language in the docs to discourage the
QEMU_TEST_FLAKY_TESTS becoming a permanent solution.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-3-alex.bennee@linaro.org>
This commit is contained in:
Alex Bennée 2023-12-01 09:36:28 +00:00
parent 5dcf6334ae
commit 9b45cc9931
13 changed files with 71 additions and 44 deletions

View file

@ -15,7 +15,7 @@ import logging
import time
from avocado import skip
from avocado import skipIf
from avocado import skipUnless
from avocado import skipUnless
from avocado_qemu import wait_for_console_pattern
from avocado.utils import archive
@ -82,6 +82,7 @@ class ReplayKernelBase(LinuxKernelTest):
class ReplayKernelNormal(ReplayKernelBase):
# See https://gitlab.com/qemu-project/qemu/-/issues/2010
@skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test sometimes gets stuck')
def test_x86_64_pc(self):
"""
@ -179,7 +180,8 @@ class ReplayKernelNormal(ReplayKernelBase):
self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=1)
@skipIf(os.getenv('GITLAB_CI'), 'Running on GitLab')
@skipUnless(os.getenv('QEMU_TEST_FLAKY_TESTS'), 'Test is unstable on GitLab')
def test_arm_cubieboard_initrd(self):
"""
:avocado: tags=arch:arm