Experiment with functional testing #575
Conversation
| """ | ||
|
|
||
|
|
||
| # dateadd |
There was a problem hiding this comment.
This file contains the "project" file contents, all defined as Python strings. These can also live right alongside the test cases, but it can be nice to keep the concerns separated.
| import os | ||
| import pytest | ||
| from dbt.tests.util import run_dbt | ||
| from tests.functional.cross_db_utils.fixtures import ( |
There was a problem hiding this comment.
Here's where we import those file contents
| ) | ||
|
|
||
|
|
||
| class BaseCrossDbMacro: |
There was a problem hiding this comment.
This class sets up the basic scaffolding for all test cases below. Child methods will inherit methods that don't change (e.g. packages, macros, test_build_assert_equal), and can reimplement others that do (seeds + models)
| # actual test sequence | ||
| def test_build_assert_equal(self, project): | ||
| run_dbt(['deps']) | ||
| run_dbt(['build']) # seed, model, test -- all handled by dbt |
There was a problem hiding this comment.
This is actually running dbt, and making sure the thing passes. We could also:
- add
assertstatements - inspect results / logs
- ensure that dbt fails if given bad input, via
expect_pass=False
| class BaseDateAdd(BaseCrossDbMacro): | ||
| @pytest.fixture(scope="class") | ||
| def seeds(self): | ||
| return {"data_dateadd.csv": seeds__data_dateadd_csv} | ||
|
|
||
| @pytest.fixture(scope="class") | ||
| def models(self): | ||
| return { | ||
| "test_dateadd.yml": models__test_dateadd_yml, | ||
| "test_dateadd.sql": models__test_dateadd_sql, | ||
| } |
There was a problem hiding this comment.
This is it, our three ingredients:
- seed with inputs
- model that calls the macro being tested
- test (
assert_equal) to ensure thatactual = expectedin the model
| class TestDateDiff(BaseDateAdd): | ||
| pass |
There was a problem hiding this comment.
By convention, pytest "discovers" tests as:
- methods whose name starts with
test_ - defined on classes starting with
Test - in files starting with
test_
Everything else is just setup, helpers, etc
| fi | ||
|
|
||
| . $VENV | ||
| python3 -m pytest tests/functional --profile $1 |
There was a problem hiding this comment.
Once all the setup is sorted, this is really it:
python3 -m pytest tests/functional --profile postgres
| @@ -0,0 +1,3 @@ | |||
| pytest | |||
| pytest-dotenv | |||
| dbt-tests-adapter | |||
There was a problem hiding this comment.
Realized this isn't needed! Just installed out of habit
* Skip BigQuery dateadd for now * Split into one fixture and test per cross-database macro * Move the skip decorator * Scaffolding for fixtures and tests * Functional tests for any_value macro * Remove TODO * Functional tests for bool_or macro * Functional tests for concat macro * Functional tests for date_trunc macro * Functional tests for hash macro * Functional tests for last_day macro * Functional tests for length macro * Functional tests for listagg macro * Functional tests for position macro * Functional tests for replace macro * Functional tests for right macro * Functional tests for safe_cast macro * Functional tests for split_part macro * Missing newline * Conform bool_or macro to actual/expected test pattern * Conform any_value macro to actual/expected test pattern * Functional tests for current_timestamp macro * Remove extraneous newline * Functional tests for current_timestamp_in_utc macro * Functional tests for cast_bool_to_text macro * Functional tests for string_literal macro * Functional tests for escape_single_quotes macro * Functional tests for except macro * Functional tests for intersect macro * Fix scaffolded SQL so that tests can pass * Postgres does not treat integers as booleans without casting * Refactor test case override into the base class * Refactor the two variants for escaping single quotes into two different test cases * Fix functional tests for dateadd macro for BigQuery
|
Work continued within #577 |
This is a:
Description & motivation
Set up functional testing within the repo, and experiment with how to factor these test cases. A lot of the changes in this PR are boilerplate pytest / CircleCI stuff; I'll leave comments to call out the cool parts.
Each test case is:
actualandexpectedactual(macro result) andexpectedactual = expectedAll the heavy lifting is being handled within dbt, just like with current integration tests.Should we have 1:1 mapping between macros and test cases? Or one big test case to run with a single method that defines which macros are supported? The latter is faster to run, and would require less code to define—but it's trickier to reason about, and more difficult for databases that can only support a subset of these macros.
Checklist
timestampvs.datetimebehavior!I followed guidelines to ensure that my changes will work on "non-core" adapters by:We're changing it up, baby!star()source)limit_zero()macro in place of the literal string:limit 0dbt_utils.type_*macros instead of explicit datatypes (e.g.dbt_utils.type_timestamp()instead ofTIMESTAMP