Welcome to Avocado
Avocado is a set of tools and libraries to help with automated testing.
One can call it a test framework with benefits. Native tests are written in
Python and they follow the unittest
pattern, but any executable can
serve as a test.
How does it work?
You should first experience Avocado by using the test runner, that is, the command line tool that will conveniently run your tests and collect their results.
To do so, please run avocado
with the run
sub-command followed by a
test reference, which could be either a path to the file, or a recognizable
name:
$ avocado run /bin/true
JOB ID : e0134e010afa18b55d93276ac2a790dc38db7948
JOB LOG : $HOME/avocado/job-results/job-2023-09-06T10.55-e0134e0/job.log
(1/1) /bin/true: STARTED
(1/1) /bin/true: PASS (0.02 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML : $HOME/avocado/job-results/job-2023-09-06T10.55-e0134e0/results.html
JOB TIME : 1.52 s
You probably noticed that we used /bin/true
as a test, and in accordance
with our expectations, it passed! These are known as exec-test, but there
is also another type of test, which we call instrumented tests.
Tip
See more at the Test types section on the Avocado User’s Guide.
Why should I use it?
Multiple result formats
A regular run of Avocado will present the test results on standard output, a nice and colored report useful for human beings. But results for machines can also be generated.
Check the job-results folder ($HOME/avocado/job-results/latest/
) to see the
outputs.
Currently we support, out of box, the following output formats:
xUnit: an XML format that contains test results in a structured form, and are used by other test automation projects, such as jenkins.
JSON: a widely used data exchange format. The JSON Avocado plugin outputs job information, similarly to the xunit output plugin.
TAP: Provides the basic TAP (Test Anything Protocol) results, currently in v12. Unlike most existing Avocado machine readable outputs this one is streamlined (per test results).
Note
You can see the results of the latest job inside the folder
$HOME/avocado/job-results/latest/
. You can also specify at the command line
the options --xunit
, --json
or --tap
followed by a filename.
Avocado will write the output on the specified filename.
When it comes to outputs, Avocado is very flexible. You can check the various output plugins. If you need something more sophisticated, visit our plugins section.
Sysinfo data collector
Avocado comes with a sysinfo plugin, which automatically gathers some system information per each job or even between tests. This is very helpful when trying to identify the cause of a test failure.
Check out the files stored at $HOME/avocado/job-results/latest/sysinfo/
:
$ ls $HOME/avocado/job-results/latest/sysinfo/pre/
'brctl show' hostname modules
cmdline 'ifconfig -a' mounts
cpuinfo installed_packages 'numactl --hardware show'
current_clocksource interrupts partitions
'df -mP' 'ip link' scaling_governor
dmesg 'ld --version' 'uname -a'
dmidecode lscpu uptime
'fdisk -l' 'lspci -vvnn' version
'gcc --version' meminfo
For more information about sysinfo collector, please consult the Avocado User’s Guide.
Job Replay and Job Diff
In order to reproduce a given job using the same data, one can use the
replay
subcommand, informing the hash id from the original job to be
replayed. The hash id can be partial, as long as the provided part corresponds
to the initial characters of the original job id and it is also unique enough.
Or, instead of the job id, you can use the string latest and Avocado will
replay the latest job executed.
Example:
$ avocado replay 825b86
JOB ID : 55a0d10132c02b8cc87deb2b480bfd8abbd956c3
SRC JOB ID : 825b860b0c2f6ec48953c638432e3e323f8d7cad
JOB LOG : $HOME/avocado/job-results/job-2016-01-11T16.18-55a0d10/job.log
(1/2) /bin/true: PASS (0.01 s)
(2/2) /bin/false: FAIL (0.01 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 1 | SKIP 0 | WARN 0 | INTERRUPT 0
JOB TIME : 0.11 s
JOB HTML : $HOME/avocado/job-results/job-2016-01-11T16.18-55a0d10/html/results.html
Avocado Diff plugin allows users to easily compare several aspects of two given jobs. The basic usage is:
$ avocado diff 7025aaba 384b949c
--- 7025aaba9c2ab8b4bba2e33b64db3824810bb5df
+++ 384b949c991b8ab324ce67c9d9ba761fd07672ff
@@ -1,15 +1,15 @@
COMMAND LINE
-/usr/bin/avocado run sleeptest.py
+/usr/bin/avocado run passtest.py
TOTAL TIME
-1.00 s
+0.00 s
TEST RESULTS
-1-sleeptest.py:SleepTest.test: PASS
+1-passtest.py:PassTest.test: PASS
...
Extensible by plugins
Avocado has a plugin system that can be used to extend it in a clean way. The
avocado
command line tool has a builtin plugins
command that lets you
list available plugins. The usage is pretty simple:
$ avocado plugins
Plugins that add new commands (avocado.plugins.cli.cmd):
exec-path Returns path to Avocado bash libraries and exits.
run Run one or more tests (native test, test alias, binary or script)
sysinfo Collect system information
...
Plugins that add new options to commands (avocado.plugins.cli):
remote Remote machine options for 'run' subcommand
journal Journal options for the 'run' subcommand
...
For more information about plugins, please visit the Plugin System section on the Avocado User’s Guide.
Utility libraries
When writing tests, developers often need to perform basic tasks on OS and end up having to implement these routines just to run they tests.
Avocado has more than 40 utility modules that helps you to perform basic operations.
Below a small subset of our utility modules:
utils.vmimage: This utility provides a API to download/cache VM images (QCOW) from the official distributions repositories.
utils.memory: Provides information about memory usage.
utils.cpu: Get information from the current’s machine CPU.
utils.software_manager: Software package management library.
utils.download: Methods to download URLs and regular files.
utils.archive: Module to help extract and create compressed archives.
Avocado Python API
If the command-line is limiting you, then you can use our new API and create custom jobs and test suites:
import sys
from avocado.core.job import Job
with Job.from_config({'resolver.references': ['/bin/true']}) as job:
sys.exit(job.run())
How to install
It is super easy, just run the follow command:
$ pip3 install --user avocado-framework
This will install the avocado command in your home directory.
Note
For more details and alternative methods, please visit the Installing section on Avocado User’s Guide
Documentation
Please use the following links for full documentation, including installation methods, tutorials and API or browse this site for more content.
Bugs/Requests
Please use the GitHub issue tracker to submit bugs or request features.
Changelog
Please consult the Avocado Releases for fixes and enhancements of each version.
License
Except where otherwise indicated in a given source file, all original contributions to Avocado are licensed under the GNU General Public License version 2 (GPLv2) or any later version.
By contributing you agree that these contributions are your own (or approved by your employer) and you grant a full, complete, irrevocable copyright license to all users and developers of the Avocado project, present and future, pursuant to the license of the project.
Build and Quality Status
Contents:
- Avocado User’s Guide
- Avocado Test Writer’s Guide
- Writing an Executable Test
- Writing Avocado Tests with Python
- Basic example
- Multiple tests and naming conventions
- Convenience Attributes
- Test statuses
- Test methods
- Turning errors into failures
- Turning errors into cancels
- Saving test generated (custom) data
- Accessing test data files
- Accessing test parameters
- Running multiple variants of tests
unittest.TestCase
heritage- Running tests under other
unittest
runners - Setup and cleanup methods
- Running third party test suites
- Fetching asset files
- Test log, stdout and stderr in native Avocado modules
- Setting a Test Timeout
- Skipping Tests
- Canceling Tests
- Docstring Directives
- Declaring test as not being avocado-instrumented
- Declaring test as being avocado-instrumented
- (Deprecated) enabling recursive discovery
- Categorizing tests
- Python
unittest
Compatibility Limitations And Caveats - Environment Variables for Tests
- Docstring Directives Rules
- Signal Handlers
- Wrap Up
- Defining what to run using recipe files
- Advanced logging capabilities
- Test parameters
- Utility Libraries
- Subclassing Avocado
- Integrating Avocado
- Avocado Contributor’s Guide
- Avocado Refefence Guide
- Avocado Releases
- How we release Avocado
- Long Term Stability Releases
- Regular Releases
- 108.0 - Oppenheimer
- 107.0 - The Godfather
- 106.0 - Taking Off
- 105.0 - Poor Things
- 104.0 - Restore Point
- 103.0 - Sound of Freedom
- 102.0 - 102 Dalmatians
- 101.0 One Hundred and One Dalmatians
- 100.1 The 100
- 100.0 The 100
- 99.0 The Chosen
- 98.0 The Thing
- 97.0 Dopesick
- 96.0 Nueve reinas
- 95.0 PAW Patrol: The Movie
- 94.0 Gran Torino
- 93.0 The Book of Eli
- 92.0 Monsters, Inc.
- 91.0 Thelma & Louise
- 90.0 Bladerunner
- 89.0 Shrek
- 88.1 The Serpent
- 88.0 The Serpent
- 87.0 Braveheart
- 86.0 The Dig
- 85.0 Bacurau
- 84.0 The Intouchables
- 83.0 Crime and Punishment
- 82.0 Avengers: Endgame
- 81.0 Avengers: Infinity War
- 80.0 Parasite
- 79.0 La vita è bella
- 78.0 Outbreak
- 77.0 The Hangover
- 76.0 Hotel Mumbai
- 75.1 Voyage to the Prehistoric Planet (minor release)
- 75.0 Voyage to the Prehistoric Planet
- 74.0 Home Alone
- 73.0 Pulp Fiction
- 72.0 Once upon a time in Holywood
- 71.0 Downton Abbey
- 70.0 The Man with the Golden Gun
- 69.0 The King’s Choice
- 68.0 The Marvelous Mrs. Maisel
- 67.0 A Beautiful Mind
- 66.0 Les Misérables
- 65.0 Back to the Future
- 64.0 The man who would be king
- 63.0 Greed in the Sun
- 62.0 Farewell
- 61.0 Seven Pounds
- 60.0 Better Call Saul
- 59.0 The Lobster
- 58.0 Journey to the Christmas Star
- 57.0 Star Trek: Discovery
- 56.0 The Second Mother
- 55.0 Never Let Me Go
- 54.1 House of Cards (minor release)
- 54.0 House of Cards
- 53.0 Rear Window
- 52.0 Pat & Mat
- 51.0 The White Mountains
- 50.0 A Dog’s Will
- 49.0 The Physician
- 48.0 Lost Boundaries
- 47.0 The Lost Wife
- 46.0 Burning Bush
- 45.0 Anthropoid
- 44.0 The Shadow Self
- 43.0 The Emperor and the Golem
- 42.0 Stranger Things
- 41.0 Outlander
- 40.0 Dr Who
- 39.0 The Hateful Eight
- 38.0 Love, Ken
- 37.0 Trabant vs. South America
- 36.0 LTS
- 35.0 Mr. Robot
- 0.34.0 The Hour of the Star
- 0.33.0 Lemonade Joe or Horse Opera
- 0.32.0 Road Runner
- 0.31.0 Lucky Luke
- 0.30.0 Jimmy’s Hall
- 0.29.0 Steven Universe
- 0.28.0 Jára Cimrman, The Investigation of the Missing Class Register
- 0.27.1
- 0.27.0 Terminator: Genisys
- 0.26.0 The Office
- 0.25.0 Blade
- 109.0 - TBD
- Avocado’s Configuration Reference
- assets.fetch.ignore_errors
- assets.fetch.references
- assets.fetch.timeout
- assets.list.days
- assets.list.overall_limit
- assets.list.size_filter
- assets.purge.days
- assets.purge.overall_limit
- assets.purge.size_filter
- assets.register.name
- assets.register.sha1_hash
- assets.register.url
- cache.clear
- cache.list
- config.datadir
- core.paginator
- core.show
- core.verbose
- datadir.paths.base_dir
- datadir.paths.cache_dirs
- datadir.paths.data_dir
- datadir.paths.logs_dir
- datadir.paths.test_dir
- diff.create_reports
- diff.filter
- diff.html
- diff.jobids
- diff.open_browser
- diff.strip_id
- distro.distro_def_arch
- distro.distro_def_create
- distro.distro_def_name
- distro.distro_def_path
- distro.distro_def_release
- distro.distro_def_type
- distro.distro_def_version
- filter.by_tags.include_empty
- filter.by_tags.include_empty_key
- filter.by_tags.tags
- human_ui.omit.statuses
- job.output.loglevel
- job.output.testlogs.logfiles
- job.output.testlogs.statuses
- job.output.testlogs.summary_statuses
- job.replay.source_job_id
- job.run.dependency
- job.run.logging_buffer_size
- job.run.result.json.enabled
- job.run.result.json.output
- job.run.result.tap.enabled
- job.run.result.tap.include_logs
- job.run.result.tap.output
- job.run.result.xunit.enabled
- job.run.result.xunit.job_name
- job.run.result.xunit.max_test_log_chars
- job.run.result.xunit.output
- job.run.store_logging_stream
- job.run.timeout
- jobs.show.job_id
- json.variants.load
- list.recipes.write_to_directory
- list.write_to_json_file
- plugins.cache.order
- plugins.cli.cmd.order
- plugins.cli.order
- plugins.disable
- plugins.init.order
- plugins.job.prepost.order
- plugins.jobscripts.post
- plugins.jobscripts.pre
- plugins.jobscripts.warn_non_existing_dir
- plugins.jobscripts.warn_non_zero_status
- plugins.ordered_list
- plugins.resolver.order
- plugins.result.order
- plugins.result_events.order
- plugins.runnable.runner.order
- plugins.skip_broken_plugin_notification
- plugins.spawner.order
- plugins.suite.runner.order
- plugins.test.post.order
- plugins.test.pre.order
- plugins.varianter.order
- resolver.exec_runnables_recipe.arguments
- resolver.references
- resolver.run_executables
- run.dict_variants
- run.dict_variants.variant_id_keys
- run.dry_run.enabled
- run.dry_run.no_cleanup
- run.execution_order
- run.failfast
- run.ignore_missing_references
- run.job_category
- run.journal.enabled
- run.keep_tmp
- run.log_test_data_directories
- run.max_parallel_tasks
- run.results.archive
- run.results_dir
- run.shuffle
- run.spawner
- run.status_server_auto
- run.status_server_buffer_size
- run.status_server_listen
- run.status_server_uri
- run.suite_runner
- run.test_parameters
- run.unique_job_id
- runner.exectest.clear_env
- runner.exectest.exitcodes.skip
- runner.identifier_format
- runner.output.color
- runner.output.colored
- runner.task.interval.from_hard_termination_to_verification
- runner.task.interval.from_soft_to_hard_termination
- spawner.lxc.arch
- spawner.lxc.create_hook
- spawner.lxc.dist
- spawner.lxc.release
- spawner.lxc.slots
- spawner.podman.avocado_spawner_egg
- spawner.podman.bin
- spawner.podman.image
- spawner.podman.image_tag_prefix
- sysinfo.collect.commands_timeout
- sysinfo.collect.enabled
- sysinfo.collect.installed_packages
- sysinfo.collect.locale
- sysinfo.collect.optimize
- sysinfo.collect.per_test
- sysinfo.collect.profiler
- sysinfo.collect.sysinfodir
- sysinfo.collectibles.commands
- sysinfo.collectibles.fail_commands
- sysinfo.collectibles.fail_files
- sysinfo.collectibles.files
- sysinfo.collectibles.profilers
- task.timeout.running
- variants.contents
- variants.debug
- variants.inherit
- variants.json_variants_dump
- variants.summary
- variants.tree
- variants.variants
- vmimage.get.arch
- vmimage.get.distro
- vmimage.get.version
Test API
- Test APIs
- Internal (Core) APIs
- Subpackages
- Submodules
- avocado.core.app module
- avocado.core.data_dir module
- avocado.core.decorators module
- avocado.core.dispatcher module
- avocado.core.enabled_extension_manager module
- avocado.core.exceptions module
- avocado.core.exit_codes module
- avocado.core.extension_manager module
- avocado.core.job module
- avocado.core.job_id module
- avocado.core.jobdata module
- avocado.core.main module
- avocado.core.messages module
- avocado.core.output module
FilterInfoAndLess
FilterTestMessage
FilterTestMessageOnly
FilterWarnAndMore
LOG_JOB
LOG_ROOT
LOG_UI
MemStreamHandler
Paginator
ProgressStreamHandler
STD_OUTPUT
StdOutput
TERM_SUPPORT
TEST_STATUS_DECORATOR_MAPPING
TEST_STATUS_MAPPING
TermSupport
Throbber
add_log_handler()
del_last_configuration()
disable_log_handler()
early_start()
log_plugin_failures()
reconfigure()
split_loggers_and_levels()
- avocado.core.parameters module
- avocado.core.parser module
- avocado.core.parser_common_args module
- avocado.core.plugin_interfaces module
- avocado.core.references module
- avocado.core.resolver module
- avocado.core.result module
- avocado.core.settings module
- avocado.core.settings_dispatcher module
- avocado.core.streams module
- avocado.core.suite module
- avocado.core.sysinfo module
- avocado.core.tags module
- avocado.core.tapparser module
- avocado.core.test module
- avocado.core.test_id module
- avocado.core.teststatus module
- avocado.core.tree module
- avocado.core.varianter module
- avocado.core.version module
- Module contents
- Utilities APIs
- Subpackages
- Submodules
- avocado.utils.ar module
- avocado.utils.archive module
- avocado.utils.asset module
- avocado.utils.astring module
- avocado.utils.aurl module
- avocado.utils.build module
- avocado.utils.cloudinit module
- avocado.utils.cpu module
FamilyException
VENDORS_MAP
cpu_has_flags()
cpu_online_list()
get_arch()
get_cpu_arch()
get_cpu_vendor_name()
get_cpufreq_governor()
get_cpuidle_state()
get_family()
get_freq_governor()
get_idle_state()
get_model()
get_numa_node_has_cpus()
get_pid_cpus()
get_revision()
get_va_bits()
get_vendor()
get_version()
get_x86_amd_zen()
is_hotpluggable()
lscpu()
numa_nodes_with_assigned_cpus()
offline()
online()
online_count()
online_cpus_count()
online_list()
set_cpufreq_governor()
set_cpuidle_state()
set_freq_governor()
set_idle_state()
total_count()
total_cpus_count()
- avocado.utils.crypto module
- avocado.utils.data_factory module
- avocado.utils.data_structures module
- avocado.utils.datadrainer module
- avocado.utils.debug module
- avocado.utils.diff_validator module
- avocado.utils.disk module
DiskError
clean_disk()
create_linux_raw_partition()
create_loop_device()
delete_loop_device()
delete_partition()
freespace()
fs_exists()
get_absolute_disk_path()
get_all_disk_paths()
get_available_filesystems()
get_dir_mountpoint()
get_disk_blocksize()
get_disk_mountpoint()
get_disk_partitions()
get_disks()
get_filesystem_type()
get_io_scheduler()
get_io_scheduler_list()
get_size_of_disk()
is_dir_mounted()
is_disk_mounted()
is_root_device()
rescan_disk()
set_io_scheduler()
- avocado.utils.distro module
- avocado.utils.dmesg module
- avocado.utils.download module
- avocado.utils.exit_codes module
- avocado.utils.file_utils module
- avocado.utils.filelock module
- avocado.utils.gdb module
- avocado.utils.genio module
- avocado.utils.git module
- avocado.utils.iso9660 module
- avocado.utils.kernel module
- avocado.utils.linux module
- avocado.utils.linux_modules module
- avocado.utils.lv_utils module
LVException
get_device_total_space()
get_devices_total_space()
get_diskspace()
lv_check()
lv_create()
lv_list()
lv_mount()
lv_reactivate()
lv_remove()
lv_revert()
lv_revert_with_snapshot()
lv_take_snapshot()
lv_umount()
vg_check()
vg_create()
vg_list()
vg_ramdisk()
vg_ramdisk_cleanup()
vg_reactivate()
vg_remove()
- avocado.utils.memory module
MemError
MemInfo
check_hotplug()
drop_caches()
freememtotal()
get_blk_string()
get_buddy_info()
get_huge_page_size()
get_num_huge_pages()
get_page_size()
get_supported_huge_pages_size()
get_thp_value()
hotplug()
hotunplug()
is_hot_pluggable()
memtotal()
memtotal_sys()
node_size()
numa_nodes()
numa_nodes_with_memory()
read_from_meminfo()
read_from_numa_maps()
read_from_smaps()
read_from_vmstat()
rounded_memtotal()
set_num_huge_pages()
set_thp_value()
- avocado.utils.multipath module
MPException
add_mpath()
add_path()
device_exists()
fail_path()
flush_path()
form_conf_mpath_file()
get_mpath_from_dm()
get_mpath_name()
get_mpath_paths_status()
get_mpath_status()
get_multipath_details()
get_multipath_wwid()
get_multipath_wwids()
get_path_status()
get_paths()
get_policy()
get_size()
get_svc_name()
is_mpath_dev()
is_path_a_multipath()
reinstate_path()
remove_mpath()
remove_path()
resume_mpath()
suspend_mpath()
- avocado.utils.nvme module
NvmeException
attach_ns()
create_full_capacity_ns()
create_max_ns()
create_namespaces()
create_one_ns()
delete_all_ns()
delete_ns()
detach_ns()
get_block_size()
get_controller_id()
get_controller_name()
get_current_ns_ids()
get_current_ns_list()
get_equal_ns_size()
get_free_space()
get_lba()
get_max_ns_supported()
get_ns_status()
get_nslist_with_pci()
get_total_capacity()
is_ns_exists()
ns_rescan()
- avocado.utils.output module
- avocado.utils.partition module
- avocado.utils.path module
- avocado.utils.pci module
bind()
change_domain()
change_domain_check()
get_cfg()
get_disks_in_pci_address()
get_domains()
get_driver()
get_interfaces_in_pci_address()
get_iommu_group()
get_mask()
get_memory_address()
get_nics_in_pci_address()
get_num_interfaces_in_pci()
get_pci_addresses()
get_pci_class_name()
get_pci_fun_list()
get_pci_id()
get_pci_id_from_sysfs()
get_pci_info()
get_pci_prop()
get_slot_from_sysfs()
get_slot_list()
get_vendor_id()
get_vpd()
rescan()
rescan_check()
reset()
reset_check()
unbind()
- avocado.utils.pmem module
- avocado.utils.podman module
- avocado.utils.process module
CmdError
CmdInputError
CmdResult
FDDrainer
SubProcess
binary_from_shell_cmd()
can_sudo()
cmd_split()
get_capabilities()
get_children_pids()
get_command_output_matching()
get_owner_id()
get_parent_pid()
getoutput()
getstatusoutput()
has_capability()
kill_process_by_pattern()
kill_process_tree()
pid_exists()
process_in_ptree_is_defunct()
run()
safe_kill()
system()
system_output()
- avocado.utils.script module
- avocado.utils.service module
- avocado.utils.softwareraid module
- avocado.utils.ssh module
- avocado.utils.stacktrace module
- avocado.utils.sysinfo module
- avocado.utils.vmimage module
CentOSImageProvider
CirrOSImageProvider
DebianImageProvider
FedoraImageProvider
FedoraImageProviderBase
FedoraSecondaryImageProvider
FreeBSDImageProvider
IMAGE_PROVIDERS
Image
ImageProviderBase
ImageProviderError
JeosImageProvider
OpenSUSEImageProvider
QEMU_IMG
UbuntuImageProvider
VMImageHtmlParser
get()
get_best_provider()
list_providers()
- avocado.utils.wait module
- Module contents
- Extension (plugin) APIs
- Subpackages
- Submodules
- avocado.plugins.archive module
- avocado.plugins.assets module
- avocado.plugins.beaker_result module
- avocado.plugins.bystatus module
- avocado.plugins.cache module
- avocado.plugins.config module
- avocado.plugins.dependency module
- avocado.plugins.dict_variants module
- avocado.plugins.diff module
- avocado.plugins.distro module
- avocado.plugins.exec_path module
- avocado.plugins.human module
- avocado.plugins.jobs module
- avocado.plugins.jobscripts module
- avocado.plugins.journal module
- avocado.plugins.json_variants module
- avocado.plugins.jsonresult module
- avocado.plugins.list module
- avocado.plugins.plugins module
- avocado.plugins.replay module
- avocado.plugins.requirement_cache module
- avocado.plugins.resolvers module
- avocado.plugins.run module
- avocado.plugins.runner_nrunner module
- avocado.plugins.sysinfo module
- avocado.plugins.tap module
- avocado.plugins.testlogs module
- avocado.plugins.teststmpdir module
- avocado.plugins.variants module
- avocado.plugins.vmimage module
- avocado.plugins.xunit module
- Module contents
- Optional Plugins API