The Next LTS¶
The Long Term Stability releases of Avocado are the result of the accumulated changes on regular (non-LTS) releases.
This section tracks the changes introduced on each regular (non-LTS) Avocado release, and gives a sneak preview of what will make into the next LTS release.
When compared to the last LTS (52.x), the main changes to be introduced by the next LTS version are:
- A new loader implementation, that reuses (and resembles) the YAML input used for the varianter yaml_to_mux plugin. It allows the definition of test suite based on a YAML file, including different variants for different tests. For more information refer to YAML Loader (yaml_loader).
- A better handling of interruption related signals, such as
SIGTERM. Avocado will now try harder to not leave test processes that don’t respond to those signals, and will itself behave better when it receives them. For a complete description refer to Signal Handlers.
- The output generated by tests on
stderrare now properly prefixed with
job.log. The prefix is not applied in the case of
$test_result/stderrfiles, as one would expect.
- Test writers will get better protection against mistakes when trying
avocado.core.test.Test“properties”. Some of those were previously implemented using
avocado.utils.data_structures.LazyProperty()which did not prevent test writers from overwriting them.
- Avocado can now run list and run standard Python unittests, that is,
tests written in Python that use the
- Improvements in the serialization of TestIDs allow test result directories to be properly stored and accessed on Windows based filesystems.
- The complete output of tests, that is the combination of
STDERRis now also recorded in the test result directory as a file named
- Support for listing and running golang tests has been introduced. Avocado can now discover tests written in Go, and if Go is properly installed, Avocado can run them.
- The support for test data files has been improved to support more
specific sources of data. For instance, when a test file used to
contain more than one test, all of them shared the same
datadirproperty value, thus the same directory which contained data files. Now, tests should use the newly introduced
get_data()API, which will attempt to locate data files specific to the variant (if used), test name, and finally file name. For more information, please refer to the section Accessing test data files.
- The output check feature will now use the to the most specific data
source location available, which is a consequence of the switch to
the use of the
get_data()API discussed previously. This means that two tests in a single file can generate different output, generate different
- When the output check feature finds a mismatch between expected and actual output, will now produce a unified diff of those, instead of printing out their full content. This makes it a lot easier to read the logs and quickly spot the differences and possibly the failure cause(s).
- Sysinfo collection can now be enabled on a test level basis.
avocado.core.utils.vmimagelibrary now allows users to expand the builtin list of image providers. If you have a local cache of public images, or your own images, you can quickly and easily register your own providers and thus use your images on your tests.
- Avocado can record the output generated from a test, which can then
be used to determine if the test passed or failed. This feature is
commonly known as “output check”. Traditionally, users would choose
to record the output from
STDERRinto separate streams, which would be saved into different files. Some tests suites actually put all content of
STDERRtogether, and unless we record them together, it’d be impossible to record them in the right order. This version introduces the
--output-check-recordoption, which does exactly that: it records both
STDERRinto a single stream and into a single file (named
outputin the test results, and
output.expectedin the test data directory).
- A new varianter plugin has been introduced, based on PICT. PICT is a “Pair Wise” combinatorial tool, that can generate optimal combination of parameters to tests, so that (by default) at least a unique pair of parameter values will be tested at once.
- A new (optional) plugin is avaiable, the “result uploader”. It allows job results to be copied over to a centralized results server at the end of job execution. Please refer to Results Upload Plugin for more information.
avocado.Test.default_parametersmechanism for setting default parameters on tests has been removed. This was introduced quite early in the Avocado development, and allowed users to set a dictionary at the class level with keys/values that would serve as default parameter values. The recommended approach now, is to just provide default values when calling the
self.params.getwithin a test method, such as
self.paramshas been removed. It used to allow users to use a syntax such as
self.params.keywhen attempting to access the value for key
key. The supported syntax is
self.params.get("key")to achieve the same thing.
avocado.utils.vmimagelibrary now contains support for Avocado’s own JeOS (“Just Enough Operating System”) image. A nice addition given the fact that it’s the default image used in Avocado-VT and the latest version is available in the following architectures: x86_64, aarch64, ppc64, ppc64le and s390x.
- The installation of Avocado from sources has improved and moved
towards a more “Pythonic” approach. Installation of files in
“non-Pythonic locations” such as
/etcare no longer attempted by the Python
setup.pycode. Configuration files, for instance, are now considered package data files of the
avocadopackage. The end result is that installation from source works fine outside virtual environments (in addition to installations inside virtual environments). For instance, the locations of
/usr/libexec(libexec) files changed to live within the pkg_data (eg.
/usr/lib/python2.7/site-packages/avocado/etc) by default in order to not to modify files outside the package dir, which allows user installation and also the distribution of wheel packages. GNU/Linux distributions might still modify this to better follow their conventions (eg. for RPM the original locations are used). Please refer to the output of the
avocado configcommand to see the configuration files that are actively being used on your installation.
- A new plugin enables users to list and execute tests based on the GLib test framework. This plugin allows individual tests inside a single binary to be listed and executed.
- Users of the YAML test loader have now access to a few special keys that can tweak test attributes, including adding prefixes to test names. This allows users to easily differentiate among execution of the same test, but executed different configurations. For more information, look for “special keys” in the YAML Loader plugin documentation.
- Users can now dump variants to a (JSON) file, and also reuse a
previously created file in their future jobs execution. This allows
users to avoid recomputing the variants on every job, which might
bring significant speed ups in job execution or simply better
control of the variants used during a job. Also notice that even
when users do not manually dump a variants file to a specific
location, Avocado will automatically save a suitable file at
jobdata/variants.jsonas part of a Job results directory structure.
- SIMPLE tests were limited to returning PASS, FAIL and WARN statuses. Now SIMPLE tests can now also return SKIP status. At the same time, SIMPLE tests were previously limited in how they would flag a WARN or SKIP from the underlying executable. This is now configurable by means of regular expressions.
avocado.utils.processhas seen a number of changes related to how it handles data from the executed processes. In a nutshell, process output (on both
stderr) is now considered binary data. Users that need to deal with text instead, should use the newly added
avocado.utils.process.CmdResult.stderr_text, which are convenience properties that will attempt to decode the
stderrdata into a string-like type using the encoding set, and if none is set, falling back to the Python default encoding. This change of behavior was needed to accommodate Python’s 2 and Python’s 3 differences in bytes and string-like types and handling.
- The TAP result format plugin received improvements, including support for reporting Avocado tests with CANCEL status as SKIP (which is the closest status available in the TAP specification), and providing more visible warning information in the form of comments when Avocado tests finish with WARN status (while maintaining the test as a PASS, since TAP doesn’t define a WARN status).
- Redundant (and deprecated) fields in the test sections of the JSON
result output were removed. Now, instead of
idcarrying the same information, only
- The exception raised by the utility functions in
avocado.utils.memoryhas been renamed from
avocado.utils.memory.MemError. The reason is that
MemoryErroris a Python standard exception, that is intended to be used on different situations.
- Added possibility to limit the amount of characters embedded as
“system-out” in the xunit output plugin (
xunitresult plugin can now limit the amount of output generated by individual tests that will make into the XML based output file. This is intended for situations where tests can generate prohibitive amounts of output that can render the file too large to be reused elsewhere (such as imported by Jenkins).
- SIMPLE tests can also finish with
WARNstatus, depending on the output produced, and the Avocado test runner configuration. It now supports patterns that span across multiple lines. For more information, refer to SIMPLE Tests Status.
- Simple bytes and “unicode strings” utility functions have been added
avocado.utils.astring, and can be used by extension and test writers that need consistent results across Python major versions.
avocado.Test.srcdirattribute has been removed, and with it, the
AVOCADO_TEST_SRCDIRenvironment variable set by Avocado. This was done after a deprecation period, so tests should have been modified by now to make use of the
avocado.Test.datadirattribute has been removed, and with it, the
AVOCADO_TEST_DATADIRenvironment variable set by Avocado. This was done after a deprecation period, so tests should have been modified by now to make use of the
avocado.utils.cpu.set_cpuidle_state()function now takes a boolean value for its
disableparameter (while still allowing the previous integer (0/1) values to be used). The goal is to have a more Pythonic interface, and to drop support legacy integer (0/1) use in the upcoming releases.
avocado.utils.astring.ENCODINGis a new addition, and holds the encoding used on many other Avocado utilities. If your test needs to convert between binary data and text, we recommend you use it as the default encoding (unless your test knows better).
avocado.utils.astring.to_text()now supports setting the error handler. This means that when a perfect decoding is not possible, users can choose how to handle it, like, for example, ignoring the offending characters.
- When running a process by means of the
avocado.utils.processmodule utilities, the output of such a process is captured and can be logged in a
output) file. The logging is now more resilient to decode errors, and will use the
replaceerror handler by default. Please note that the downside is that this may produce different content in those files, from what was actually output by the processes if decoding error conditions happen.
avocado.utils.astring.tabular_output()will now properly strip trailing whitespace from lines that don’t contain data for all “columns”. This is also reflected in the (tabular) output of commands such as
avocado list -v.
- Including test logs in TAP plugin is disabled by default and can
be enabled using
- Switched the FileLoader discovery to :avocado: recursive by default. All tags enable, disable and recursive are still available and might help fine-tuning the class visibility.