69.0 LTS
The Avocado team is proud to present another LTS (Long Term Stability) release: Avocado 69.0, AKA “The King’s Choice”, is now available!
LTS Release
For more information on what a LTS release means, please read RFC: Long Term Stability.
Upgrading from 52.x to 69.0
Upgrading Installations
Avocado is available on a number of different repositories and installation methods. You can find the complete details in Installing Avocado. After looking at your installation options, please consider the following highlights about the changes in the Avocado installation:
Avocado fully supports both Python 2 and 3, and both can even be installed simultaneously. When using RPM packages, if you ask to have
python-avocado
installed, it will be provided by the Python 2 based package. If you want a Python 3 based version you must use thepython3-avocado
package. The same is true for plugins, which have apython2-avocado-plugins
orpython3-avocado-plugins
prefix.Avocado can now be properly installed without super user privileges. Previously one would see an error such as
could not create '/etc/avocado': Permission denied
when trying to do a source or PIP based installation.When installing Avocado on Python “venvs”, the user’s base data directory is now within the venv. If you had content outside the venv, such as results or tests directories, please make sure that you either configure your data directories on the
[datadir.paths]
section of your configuration file, or move the data over.
Porting Tests (Test API compatibility)
If you’re migration from the previous LTS version, these are the changes on the Test API that most likely will affect your test.
Note
Between non-LTS releases, the Avocado Test APIs receive a lot of effort to be kept as stable as possible. When that’s not possible, a deprecation strategy is applied and breakage can occur. For guaranteed stability across longer periods of time, LTS releases such as this one should be used.
Support for default test parameters, given via the class level
default_params
dictionary has been removed. If your test contains a snippet similar to:default_params = {'param1': 'value1', 'param2': 'value2'} def test(self): value1 = self.params.get('param1') value2 = self.params.get('param2')
It should be rewritten to look like this:
def test(self): value1 = self.params.get('param1', default='value1') value2 = self.params.get('param2', default='value2')
Support for getting parameters using the
self.params.key
syntax has been removed. If your test contains a snippet similar to:def test(self): value1 = self.params.key1
It should be rewritten to look like this:
def test(self): value1 = self.params.get('key1')
Support for the
datadir
test class attribute has been removed in favor of theget_data()
method. If your test contains a snippet similar to:def test(self): data = os.path.join(self.datadir, 'data')
It should be rewritten to look like this:
def test(self): data = self.get_data('data')
Support for for
srcdir
test class attribute has been removed in favor of theworkdir
attribute. If your test contains a snippet similar to:def test(self): compiled = os.path.join(self.srcdir, 'binary')
It should be rewritten to look like this:
def test(self): compiled = os.path.join(self.workdir, 'binary')
The
:avocado: enable
and:avocado: recursive
tags may not be necessary anymore, given that “recursive” is now the default loader behavior. If you test contains:def test(self): """ :avocado: enable """
Or:
def test(self): """ :avocado: recursive """
Consider removing the tags completely, and check if the default loader behavior is sufficient with:
$ avocado list your-test-file.py
Support for the
skip
method has been removed from theavocado.Test
class. If your test contains a snippet similar to:def test(self): if not condition(): self.skip("condition not suitable to keep test running")
It should be rewritten to look like this:
def test(self): if not condition(): self.cancel("condition not suitable to keep test running")
Porting Tests (Utility API compatibility)
The changes in the utility APIs (those that live under the
avocado.utils
namespace are too many to present porting
suggestion. Please refer to the Utility APIs
section for a comprehensive list of changes, including new features
your test may be able to leverage.
Changes from previous LTS
Note
This is not a collection of all changes encompassing all releases from 52.0 to 69.0. This list contains changes that are relevant to users of 52.0, when evaluating an upgrade to 69.0.
When compared to the last LTS (version 52.1), the main changes introduced by this versions are:
Test Writers
Test APIs
Test writers will get better protection against mistakes when trying to overwrite
avocado.core.test.Test
“properties”. Some of those were previously implemented usingavocado.utils.data_structures.LazyProperty()
which did not prevent test writers from overwriting them.The
avocado.Test.default_parameters
mechanism for setting default parameters on tests has been removed. This was introduced quite early in the Avocado development, and allowed users to set a dictionary at the class level with keys/values that would serve as default parameter values. The recommended approach now, is to just provide default values when calling theself.params.get
within a test method, such asself.params.get("key", default="default_value_for_key")
.The
__getattr__
interface forself.params
has been removed. It used to allow users to use a syntax such asself.params.key
when attempting to access the value for keykey
. The supported syntax isself.params.get("key")
to achieve the same thing.The support for test data files has been improved to support more specific sources of data. For instance, when a test file used to contain more than one test, all of them shared the same
datadir
property value, thus the same directory which contained data files. Now, tests should use the newly introducedget_data()
API, which will attempt to locate data files specific to the variant (if used), test name, and finally file name. For more information, please refer to the section Accessing test data files.The
avocado.Test.srcdir
attribute has been removed, and with it, theAVOCADO_TEST_SRCDIR
environment variable set by Avocado. Tests should have been modified by now to make use of theavocado.Test.workdir
instead.The
avocado.Test.datadir
attribute has been removed, and with it, theAVOCADO_TEST_DATADIR
environment variable set by Avocado. Tests should now to make use of theavocado.Test.get_data()
instead.Switched the FileLoader discovery to :avocado: recursive by default. All tags enable, disable and recursive are still available and might help fine-tuning the class visibility.
The deprecated
skip
method, previously part of theavocado.Test
API, has been removed. To skip a test, you can still use theavocado.skip()
,avocado.skipIf()
andavocado.skipUnless()
decorators.The
Avocado Test class
now exposes thetags
to the test. The test may use that information, for instance, to decide on default behavior.The Avocado test loader, which does not load or execute Python source code that may contain tests for security reasons, now operates in a way much more similar to the standard Python object inheritance model. Before, classes containing tests that would not directly inherit from
avocado.Test
would require a docstring statement (either:avocado: enable
or:avocado: recursive
). This is not necessary for most users anymore, as the recursive detection is now the default behavior.
Utility APIs
The
avocado.utils.archive
module now supports the handling of gzip files that are not compressed tarballs.avocado.utils.astring.ENCODING
is a new addition, and holds the encoding used on many other Avocado utilities. If your test needs to convert between binary data and text, we recommend you use it as the default encoding (unless your test knows better).avocado.utils.astring.to_text()
now supports setting the error handler. This means that when a perfect decoding is not possible, users can choose how to handle it, like, for example, ignoring the offending characters.The
avocado.utils.astring.tabular_output()
will now properly strip trailing whitespace from lines that don’t contain data for all “columns”. This is also reflected in the (tabular) output of commands such asavocado list -v
.Simple bytes and “unicode strings” utility functions have been added to
avocado.utils.astring
, and can be used by extension and test writers that need consistent results across Python major versions.The
avocado.utils.cpu.set_cpuidle_state()
function now takes a boolean value for itsdisable
parameter (while still allowing the previous integer (0/1) values to be used). The goal is to have a more Pythonic interface, and to drop support legacy integer (0/1) use in the upcoming releases.The
avocado.utils.cpu
functions, such asavocado.utils.cpu.cpu_oneline_list()
now support the S390X architecture.The
avocado.utils.distro
module has dropped the probe that depended on the Python standard libraryplatform.dist()
. The reason is theplatform.dist()
has been deprecated since Python 2.6, and has been removed on the upcoming Python 3.8.The
avocado.utils.distro
module introduced a probe for the Ubuntu distros.The
avocado.core.utils.vmimage
library now allows users to expand the builtin list of image providers. If you have a local cache of public images, or your own images, you can quickly and easily register your own providers and thus use your images on your tests.The
avocado.utils.vmimage
library now contains support for Avocado’s own JeOS (“Just Enough Operating System”) image. A nice addition given the fact that it’s the default image used in Avocado-VT and the latest version is available in the following architectures: x86_64, aarch64, ppc64, ppc64le and s390x.The
avocado.utils.vmimage
library got a provider implementation for OpenSUSE. The limitation is that it tracks the general releases, and not the rolling releases (called Tumbleweed).The
avocado.utils.vmimage.get()
function now provides a directory in which to put the snapshot file, which is usually discarded. Previously, the snapshot file would always be kept in the cache directory, resulting in its pollution.The exception raised by the utility functions in
avocado.utils.memory
has been renamed fromMemoryError
and becameavocado.utils.memory.MemError
. The reason is thatMemoryError
is a Python standard exception, that is intended to be used on different situations.When running a process by means of the
avocado.utils.process
module utilities, the output of such a process is captured and can be logged in astdout
/stderr
(or combinedoutput
) file. The logging is now more resilient to decode errors, and will use thereplace
error handler by default. Please note that the downside is that this may produce different content in those files, from what was actually output by the processes if decoding error conditions happen.The
avocado.utils.process
has seen a number of changes related to how it handles data from the executed processes. In a nutshell, process output (on bothstdout
andstderr
) is now considered binary data. Users that need to deal with text instead, should use the newly addedavocado.utils.process.CmdResult.stdout_text
andavocado.utils.process.CmdResult.stderr_text
, which are convenience properties that will attempt to decode thestdout
orstderr
data into a string-like type using the encoding set, and if none is set, falling back to the Python default encoding. This change of behavior was needed to accommodate Python’s 2 and Python’s 3 differences in bytes and string-like types and handling.The
avocado.utils.process
library now contains helper functions similar to the Python 2commands.getstatusoutput()
andcommands.getoutput()
which can be of help to people porting code from Python 2 to Python 3.New
avocado.utils.process.get_parent_pid()
andavocado.utils.process.get_owner_id()
process related functionsThe
avocado.utils.kernel
library now supports setting the URL that will be used to fetch the Linux kernel from, and can also build installable packages on supported distributions (such as.deb
packages on Ubuntu).The
avocado.utils.iso9660
module gained a pycdlib based backend, which is very capable, and pure Python ISO9660 library. This allows us to have a workingavocado.utils.iso9660
backend on environments in which other backends may not be easily installable.The
avocado.utils.iso9660.iso9660()
function gained a capabilities mechanism, in which users may request a backend that implement a given set of features.The
avocado.utils.iso9660
module, gained “create” and “write” capabilities, currently implemented on the pycdlib based backend. This allows users of theavocado.utils.iso9660
module to create ISO images programmatically - a task that was previously done by runningmkisofs
and similar tools.The
avocado.utils.download
module, and the various utility functions that use it, will have extended logging, including the file size, time stamp information, etc.A brand new module,
avocado.utils.cloudinit
, that aides in the creation of ISO files containing configuration for the virtual machines compatible with cloudinit. Besides authentication credentials, it’s also possible to define a “phone home” address, which is complemented by a simple phone home server implementation. On top of that, a very easy to use function to wait on the phone home is available asavocado.utils.cloudinit.wait_for_phone_home()
.A new utility library,
avocado.utils.ssh
, has been introduced. It’s a simple wrapper around the OpenSSH client utilities (your regular/usr/bin/ssh
) and allows a connection/session to be easily established, and commands to be executed on the remote endpoint using that previously established connection.The
avocado.utils.cloudinit
module now adds support for instances to be configured to allowroot
logins and authentication configuration via SSH keys.New
avocado.utils.disk.get_disk_blocksize()
andavocado.utils.disk.get_disks()
disk related utilities.A new network related utility function,
avocado.utils.network.PortTracker
was ported from Avocado-Virt, given the perceived general value in a variety of tests.A new memory utility utility,
avocado.utils.memory.MemInfo
, and its ready to use instanceavocado.utils.memory.meminfo
, allows easy access to most memory related information on Linux systems.A number of improvements to the
avocado.utils.lv_utils
module now allows users to choose if they want or not to use ramdisks, and allows for a more concise experience when creating Thin Provisioning LVs.New utility function in the
avocado.utils.genio
that allows for easy matching of patterns in files. Seeavocado.utils.is_pattern_in_file()
for more information.New utility functions are available to deal with filesystems, such as
avocado.utils.disk.get_available_filesystems()
andavocado.utils.disk.get_filesystem_type()
.The
avocado.utils.process.kill_process_tree()
now supports waiting a given timeout, and returns the PIDs of all process that had signals delivered to.The
avocado.utils.network.is_port_free()
utility function now supports IPv6 in addition to IPv4, as well as UDP in addition to TCP.A new
avocado.utils.cpu.get_pid_cpus()
utility function allows one to get all the CPUs being used by a given process and its threads.The
avocado.utils.process
module now exposes thetimeout
parameter to users of theavocado.utils.process.SubProcess
class. It allows users to define a timeout, and the type of signal that will be used to attempt to kill the process after the timeout is reached.
Users
Passing parameters to tests is now possible directly on the Avocado command line, without the use of any varianter plugin. In fact, when using variants, these parameters are (currently) ignored. To pass one parameter to a test, use
-p NAME=VAL
, and repeat it for other parameters.The test filtering mechanism using tags now support “key:val” assignments for further categorization. See Python unittest Compatibility Limitations And Caveats for more details.
The output generated by tests on
stdout
andstderr
are now properly prefixed with[stdout]
and[stderr]
in thejob.log
. The prefix is not applied in the case of$test_result/stdout
and$test_result/stderr
files, as one would expect.The installation of Avocado from sources has improved and moved towards a more “Pythonic” approach. Installation of files in “non-Pythonic locations” such as
/etc
are no longer attempted by the Pythonsetup.py
code. Configuration files, for instance, are now considered package data files of theavocado
package. The end result is that installation from source works fine outside virtual environments (in addition to installations inside virtual environments). For instance, the locations of/etc
(config) and/usr/libexec
(libexec) files changed to live within the pkg_data (eg./usr/lib/python2.7/site-packages/avocado/etc
) by default in order to not to modify files outside the package dir, which allows user installation and also the distribution of wheel packages. GNU/Linux distributions might still modify this to better follow their conventions (eg. for RPM the original locations are used). Please refer to the output of theavocado config
command to see the configuration files that are actively being used on your installation.SIMPLE tests were limited to returning PASS, FAIL and WARN statuses. Now SIMPLE tests can now also return SKIP status. At the same time, SIMPLE tests were previously limited in how they would flag a WARN or SKIP from the underlying executable. This is now configurable by means of regular expressions.
Sysinfo collection can now be enabled on a test level basis.
Avocado can record the output generated from a test, which can then be used to determine if the test passed or failed. This feature is commonly known as “output check”. Traditionally, users would choose to record the output from
STDOUT
and/orSTDERR
into separate streams, which would be saved into different files. Some tests suites actually put all content ofSTDOUT
andSTDERR
together, and unless we record them together, it’d be impossible to record them in the right order. This version introduces thecombined
option to--output-check-record
option, which does exactly that: it records bothSTDOUT
andSTDERR
into a single stream and into a single file (namedoutput
in the test results, andoutput.expected
in the test data directory).The complete output of tests, that is the combination of
STDOUT
andSTDERR
is now also recorded in the test result directory as a file namedoutput
.When the output check feature finds a mismatch between expected and actual output, will now produce a unified diff of those, instead of printing out their full content. This makes it a lot easier to read the logs and quickly spot the differences and possibly the failure cause(s).
The output check feature will now use the to the most specific data source location available, which is a consequence of the switch to the use of the
get_data()
API discussed previously. This means that two tests in a single file can generate different output, generate differentstdout.expected
orstderr.expected
.SIMPLE <test_type_simple> tests can also finish with
SKIP
ORWARN
status, depending on the output produced, and the Avocado test runner configuration. It now supports patterns that span across multiple lines. For more information, refer to test_type_simple_status.A better handling of interruption related signals, such as
SIGINT
andSIGTERM
. Avocado will now try harder to not leave test processes that don’t respond to those signals, and will itself behave better when it receives them. For a complete description refer to signal_handlers.Improvements in the serialization of TestIDs allow test result directories to be properly stored and accessed on Windows based filesystems.
The deprecated
jobdata/urls
link tojobdata/test_references
has been removed.The
avocado
command line argument parser is now invoked before plugins are initialized, which allows the use of--config
with configuration file that influence plugin behavior.The test log now contains a number of metadata about the test, under the heading
Test metadata:
. You’ll find information such as the test file name (if one exists), itsworkdir
and itsteststmpdir
if one is set.The test runner will now log the test initialization (look for
INIT
in your test logs) in addition to the already existing start of test execution (logged asSTART
).The test profilers, which are defined by default in
/etc/avocado/sysinfo/profilers
, are now executed without a backing shell. While Avocado doesn’t ship with examples of shell commands as profilers, or suggests users to do so, it may be that some users could be using that functionality. If that’s the case, it will now be necessary to write a script that wraps you previous shell command. The reason for doing so, was to fix a bug that could leave profiler processes after the test had already finished.The Human UI plugin, will now show the “reason” behind test failures, cancellations and others right along the test result status. This hopefully will give more information to users without requiring them to resort to logs every single time.
When installing and using Avocado in a Python virtual environment, the ubiquitous “venvs”, the base data directory now respects the virtual environment. If you have are using the default data directory outside of a venv, please be aware that the updated
Avocado packages are now available in binary “wheel” format on PyPI. This brings faster, more convenient and reliable installs via
pip
. Previously, the source-only tarballs would require the source to be built on the target system, but the wheel package install is mostly an unpack of the already compiled files.The legacy options
--filter-only
,--filter-out
and--multiplex
have now been removed. Please adjust your usage, replacing those options with--mux-filter-only
,--mux-filter-out
and--mux-yaml
respectively.The location of the Avocado configuration files can now be influenced by third parties by means of a new plugin.
The configuration files that have been effectively parsed are now displayed as part of
avocado config
command output.
Output Plugins
Including test logs in TAP plugin is disabled by default and can be enabled using
--tap-include-logs
.The TAP result format plugin received improvements, including support for reporting Avocado tests with CANCEL status as SKIP (which is the closest status available in the TAP specification), and providing more visible warning information in the form of comments when Avocado tests finish with WARN status (while maintaining the test as a PASS, since TAP doesn’t define a WARN status).
A new (optional) plugin is available, the “result uploader”. It allows job results to be copied over to a centralized results server at the end of job execution. Please refer to Results Upload Plugin for more information.
Added possibility to limit the amount of characters embedded as “system-out” in the xunit output plugin (
--xunit-max-test-log-chars XX
).The
xunit
result plugin can now limit the amount of output generated by individual tests that will make into the XML based output file. This is intended for situations where tests can generate prohibitive amounts of output that can render the file too large to be reused elsewhere (such as imported by Jenkins).The xunit output now names the job after the Avocado job results directory. This should make the correlation of results displayed in UIs such as Jenkins and the complete Avocado results much easier.
The xUnit plugin now should produce output that is more compatible with other implementations, specifically newer Jenkin’s as well as Ant and Maven. The specific change was to format the time field with 3 decimal places.
Redundant (and deprecated) fields in the test sections of the JSON result output were removed. Now, instead of
url
,test
andid
carrying the same information, onlyid
remains.
Test Loader Plugins
A new loader implementation, that reuses (and resembles) the YAML input used for the varianter yaml_to_mux plugin. It allows the definition of test suite based on a YAML file, including different variants for different tests. For more information refer to yaml_loader.
Users of the YAML test loader have now access to a few special keys that can tweak test attributes, including adding prefixes to test names. This allows users to easily differentiate among execution of the same test, but executed different configurations. For more information, look for “special keys” in the YAML Loader plugin documentation yaml_loader.
A new plugin glib-plugin enables users to list and execute tests based on the GLib test framework. This plugin allows individual tests inside a single binary to be listed and executed.
Avocado can now run list and run standard Python unittests, that is, tests written in Python that use the
unittest
library alone.Support for listing and running golang tests has been introduced. Avocado can now discover tests written in Go, and if Go is properly installed, Avocado can run them.
Varianter Plugins
A new varianter plugin has been introduced, based on PICT. PICT is a “Pair Wise” combinatorial tool, that can generate optimal combination of parameters to tests, so that (by default) at least a unique pair of parameter values will be tested at once.
A new varianter plugin, the CIT Varianter Plugin. This plugin implements a “Pair-Wise”, also known as “Combinatorial Independent Testing” algorithm, in pure Python. This exciting new functionality is provided thanks to a collaboration with the Czech Technical University in Prague.
Users can now dump variants to a (JSON) file, and also reuse a previously created file in their future jobs execution. This allows users to avoid recomputing the variants on every job, which might bring significant speed ups in job execution or simply better control of the variants used during a job. Also notice that even when users do not manually dump a variants file to a specific location, Avocado will automatically save a suitable file at
jobdata/variants.json
as part of a Job results directory structure. The feature has been isolated into a varianter implementation calledjson_variants
, that you can see withavocado plugins
.
Test Runner Plugins
The command line options
--filter-by-tags
and--filter-by-tags-include-empty
are now white listed for the remote runner plugin.The remote runner plugin will now respect
~/.ssh/config
configuration.
Complete list of changes
For a complete list of changes between the last LTS release (52.1) and this release, please check out the Avocado commit changelog.