Verify Script

The verify_odin.sh script is designed for generating regression test results.

./verify_odin.sh [args]

Arguments

*The tool requires a task to run hence -t <task directory> must be passed in

arg	equivalent form	Following argument	Description
-t	path to directory containing a test	The test types are a task.conf,task_list.conf or a predefined test
-j	-- nb_of_process	# of processors requested	This allows the user to choose how many processor are used in the computer, the default is 1
-d	--output_dir	path to directory	This allows the user to change the directory of the run output
-C	--config	path to configuration file	This directs the compiler to a configuration file to append the configuration file of the test
-t	--test	path to directory containing a test	The test types are a task.conf,task_list.conf or a predefined test
-h	--help	N/A	Prints all possible commands and directives
-g	--generate_bench	N/A	Generates input and output vectors for test
-o	--generate_output	N/A	Generates output vector for test given its input vector
-b	--build_config	path to directory	Generates a configuration file for a given directory
-c	--clean	N/A	Clean temporary directory
-f	--force_simulate	N/A	Force the simulation to be executed regardless of the configuration file
N/A	--override	N/A	if a configuration file is passed in, override arguments rather than append
N/A	--dry_run	N/A	performs a dry run to check the validity of the task and flow
N/A	--randomize	N/A	performs a dry run randomly to check the validity of the task and flow
N/A	--regenerate_expectation	N/A	regenerates expectation and overrides the expected value only if there's a mismatch
N/A	--generate_expectation	N/A	generate the expectation and overrides the expectation file

Examples

The following examples are being performed in the odin_ii directory:

Generating Results for a New Task

To generate new results, synthesis_parse_file and simulation_parse_file must be specified in task.conf file.

The following commands will generate the results of a new regression test using N processors:

make sanitize

./verify_odin.sh --generate_expectation -j N -t <regression_test/benchmark/task/<task_name>

A synthesis_result.json and a simulation_result.json will be generated in the task’s folder. The simulation results for each benchmark are only generated if they synthesize correctly (no exit error), thus if none of the benchmarks synthesize there will be no simulation_result.json generated.

Regenerating Results for a Changed Test

The following commands will only generate the results of the changes. If there are new benchmarks it will add to the results. If there are deleted benchmarks or modified benchmarks the results will be updated accordingly.

make sanitize

./verify_odin.sh --regenerate_expectation -t <regression_test/benchmark/task/<task_name>

Generating Results for a Suite

The following commands generate the results for all the tasks called upon in a suite.

make sanitize

NOTE

If the suite calls upon the large test DO NOT run make sanitize. Instead run make build.

./verify_odin.sh --generate_expectation -t <regression_test/benchmark/suite/<suite_name>

Checking the configuration file

The following commands will check if a configuration file is being read properly.

make build

./verify_odin.sh --dry_run -t <regression_test/benchmark/<path/to/config_file/difrectory>

Running a subset of tests in a suite

The following commands will run only the tests matching <test regex>:

./verify_odin.sh -t <regression_test/benchmark/suite/<suite_name> <test regex>

You may specify as many test regular expressions as desired and the script will run any test that matches at least one regex

NOTE

This uses grep’s extended regular expression syntax for matching test names.
Test names matched are of the form <suite_name>/<test_name>/