Skip to content

Commit

Permalink
✨ feat: Migrate to state test
Browse files Browse the repository at this point in the history
  • Loading branch information
raxhvl committed Nov 7, 2024
2 parents 1fabe17 + d43c28a commit 611d5d1
Show file tree
Hide file tree
Showing 5 changed files with 34 additions and 17 deletions.
27 changes: 25 additions & 2 deletions docs/writing_tests/types_of_tests.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,28 @@ There are currently two types of tests that can be produced by a test spec:

## State Tests

State tests span a single block and, ideally, a single transaction. For example:
### Purpose

Tests the effects of individual transactions (ideally a single one) that span a single block in a controlled environment.

### Use cases

- Test a single opcode behavior.
- Verify opcode gas costs.
- Test interactions between multiple smart contracts.
- Test creation of smart contracts.

!!! info

The fill function will automatically generate a `blockchain_test` fixture from `state_tests`, consisting of one block and one transaction.

## Blockchain Tests

Blockchain tests span multiple blocks which may or may not contain transactions and mainly focus on the block to block effects to the Ethereum state. For example:
### Purpose

Blockchain tests span multiple blocks which may or may not contain transactions and mainly focus on the block to block effects to the Ethereum state.

### Use cases

- Verify system-level operations such as coinbase balance updates or withdrawals.
- Verify fork transitions.
Expand All @@ -38,3 +50,14 @@ def test_blob_type_tx_pre_fork(
Reject blocks with blobs before blobs fork
"""
```

## Deciding on a test type

### Prefer `state_test` for single transactions

Whenever possible, use `state_test` to examine individual transactions. This method is more straightforward and less prone to external
influences that can occur during block building.

This provides more targeted testing since it does not invoke the client's block-building machinery. This reduces the risk of
encountering false positives, particularly in exception scenarios (e.g., see issue
[#343: "Zero max_fee_per_blob_gas test is ineffective"](https://github.com/ethereum/execution-spec-tests/issues/343)).
3 changes: 2 additions & 1 deletion docs/writing_tests/writing_a_new_test.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@

## Test Functions

Every test case is defined as a python function that defines a single `StateTest` or `BlockchainTest` by using one of the `state_test` or `blockchain_test` objects made available by the framework. Test cases, respectively test modules, must fulfill the following requirements:
Every test case is defined as a Python function that implements a single `StateTest` or `BlockchainTest` using the `state_test` or `blockchain_test` objects made available by the framework ([learn how to decide on a test type](./types_of_tests.md#deciding-on-a-test-type)). Test
cases, and the respective test modules, must fulfill the following requirements:

| Requirement | When |
| -----------------------------------------------------------------------|---------------------------------------------|
Expand Down
4 changes: 2 additions & 2 deletions src/cli/gentest/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
from ethereum_test_base_types import Hash

from .source_code_generator import get_test_source
from .test_context_providers import BlockchainTestContextProvider
from .test_context_providers import StateTestProvider


@click.command()
Expand All @@ -27,7 +27,7 @@ def generate(transaction_hash: str, output_file: TextIO):
OUTPUT_FILE is the path to the output python script.
"""
provider = BlockchainTestContextProvider(transaction_hash=Hash(transaction_hash))
provider = StateTestProvider(transaction_hash=Hash(transaction_hash))

source = get_test_source(provider=provider, template_path="blockchain_test/transaction.py.j2")
output_file.write(source)
Expand Down
13 changes: 3 additions & 10 deletions src/cli/gentest/templates/blockchain_test/transaction.py.j2
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,7 @@ from typing import Dict

import pytest

from ethereum_test_tools import (
Account,
Block,
BlockchainTestFiller,
Environment,
Storage,
Transaction,
)
from ethereum_test_tools import Account, Environment, StateTestFiller, Storage, Transaction

REFERENCE_SPEC_GIT_PATH = "N/A"
REFERENCE_SPEC_VERSION = "N/A"
Expand All @@ -28,7 +21,7 @@ def env(): # noqa: D103
@pytest.mark.valid_from("Paris")
def test_transaction_{{ tx_hash }}( # noqa: SC200, E501
env: Environment,
blockchain_test: BlockchainTestFiller,
state_test: StateTestFiller,
):
"""
Gentest autogenerated test for tx.hash:
Expand All @@ -40,4 +33,4 @@ def test_transaction_{{ tx_hash }}( # noqa: SC200, E501

tx = {{ transaction | stringify }}

blockchain_test(genesis_environment=env, pre=pre, post=post, blocks=[Block(txs=[tx])])
state_test(env=env, pre=pre, post=post, tx=tx)
4 changes: 2 additions & 2 deletions src/cli/gentest/test_context_providers.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,9 +37,9 @@ def get_context(self) -> Dict:
pass


class BlockchainTestContextProvider(Provider):
class StateTestProvider(Provider):
"""
Provides context required to generate a `blockchain_test` using pytest.
Provides context required to generate a `state_test` using pytest.
"""

transaction_hash: Hash
Expand Down

0 comments on commit 611d5d1

Please sign in to comment.