Home

Awesome

tests Build Status

Common tests for all clients to test against.

Test Formats

See descriptions of the different test formats in the official documentation at http://ethereum-tests.readthedocs.io/.

Note:
The format of BlockchainTests recently changed with the introduction of a new field sealEngine (values: NoProof | Ethash), see related JSON Schema change or BlockchainTest format docs for reference.

This means that you can skip PoW validation for NoProof tests but also has the consequence that it is not possible to rely on/check PoW related block parameters for these tests any more.

Clients using the library

The following clients make use of the tests from this library. You can use these implementations for inspiration on how to integrate. If your client is missing, please submit a PR (requirement: at least some minimal test documentation)!

Using the Tests

We do versioned tag releases for tests and you should aim to run your client libraries against the latest repository snapshot tagged.

Generally the develop branch in ethereum/tests is always meant to be stable and you should be able to run your test against.

Contribute to the Test Suite

See the dedicated section in the docs on how to write new tests.

If you want to follow up with current tasks and what is currently in the works, have a look at the issues and PRs on this repo, there is also a Google spreadsheet with a list of current TODOs regarding test coverage.

Currently the C++ Aleth client is the reference client for generating tests. Have a look at issues and PRs tagged with testeth to get an idea what is currently being worked on regarding the test generation process.

Contents of this repository

Do not change test files in folders:

It is being created by the testFillers which could be found at src folder. The filler specification and wiki are in development so please ask on gitter channel for more details.

If you want to modify a test filler or add a new test please contact @winsvega at https://gitter.im/ethereum/aleth

All files should be of the form:

{
	"test1name":
	{
		"test1property1": ...,
		"test1property2": ...,
		...
	},
	"test2name":
	{
		"test2property1": ...,
		"test2property2": ...,
		...
	}
}

Arrays are allowed, but don't use them for sets of properties - only use them for data that is clearly a continuous contiguous sequence of values.

Test Set Sanitation

Setup

(Requires virtualenv)

#> virtualenv -p python3 .env3
#> . .env3/bin/activate
#> python3 -m pip install -r requirements.txt

Checkers

Several basic checks against the test-set are performed to ensure that they have been filled and are formatted correctly. Currently, there are three types of checks that we can perform:

The constant TEST_PREFIX is a path prefix to the test-set you're interested in performing the checks on. For instance:

These checks are all performed by the file ./test.py, which can be invoked on individual files as well. Run ./test.py with no arguments for help.

Sanitizers

The above checkers are packaged together into sanitizers for each test-suite, marking which testsuites are passing which testers. See the TODOs in the Makefile to see which checkers are enabled for which test-suites.

Runners/Fillers

The tests can also be run/filled with the same Makefile.