1 Magic Architectural Verification Test Suite
The purpose of the test suite is to validate both the software simulation and the physical implementation of the Magic-1 computer. Some validation features, such as microcode coverage analysis, are applicable only when running the tests under the simulator, m1sim. Because Magic-1 is not pipelined, instructions will be tested as individual units. In practical terms, this greatly simplifies the verification process as it allows us to ignore sequences of instructions and the associated nasty combinatorial problems. The tests are structured in a series of groups based on increasingly complicated functionality. They should be run in order, as later tests rely on the correct behavior of instructions validated in earlier tests.
The tests are written in Magic-1 assembly language, with the caveat that the test files must first be run through a C preprocessor before assembling with qas, the Magic-1 assembler. The primary purpose of the preprocessing is to insert in each test some boiler-plate code for startup and pass/fail reporting. The test files are of the form <letter>_<3-digit number>.s. For example, “d_002.s refers to the second test in test group d. The output of the preprocessor will yield the same file name, but with an “sss” suffix - d_002.sss in the previous example.
In general, the tests will follow a pass/fail convention based on leaving special values in fixed registers and then executing a halt instruction. When running the simulator in test mode (-t flag), the simulator will print either PASS or FAIL to standard output, along with the test name. The exception here is the first group of tests - those which validate the basic instructions required to execute the compare/pass/fail portions of subsequent tests. Because they cannot assume that any portion of the implementation is correct, special constraints (number of instructions executed and values registers should contain) are passed to the simulator to check for correctness. Also, when running in test made the simulator will enforce a “deadman switch” to detect , abort and fail a test which appears to be looping.
The root of the testing structure is the "tests" directory. In it there are working directories for each of the test groups (though I'll probably delete these sometime soon). To really run tests, go to the "all" directory. There you'll find a perl script, "run.pl". To run, do:
Where [tests] is a match string for the test names and [options] are compiler options. There is also a special value for [tests] -> "all", which will run all tests. Here are some examples:
There is also a directory called "stresstest" which contains a single test file which includes all of the individual tests into a single test. Run using the "run" batch file you'll find in the directory.
Here are the tests:
1.3 Test Conventions
Each test should (with the exception of group a) should use the macros defined in tmacros.h. The following macros should be used:
TEST_INIT(group,num) -> group is the single char group name. num is the test number. This macro also provides definitions for some useful memory values (listed below).
SUBTEST(number) -> Causes reinitialization of registers and copies number to a special memory location and register C. Useful for quickly detecting which subtest failed.
SYM(label) -> should be used to predefine all branch targets and other labels. The macro must be invoked before any use of the label. It will redfine the name with a unique prefix based on the group letter and test number. The purpose of this silliness is to allow simple concatenation of any combination of test files without fear of label duplication.
END_TEST -> Typically expands to a branch to pass. Should be the last statement in the test (with control falling into it if no failures have been detected).