Testing Data Pipelines ====================== Thinking through testing best practices for Data Pipelines Date: November 16, 2020 # Lint/Format/Doc - black - flake8 - interrogate - mypy # Pipeline Assertions - pipeline constructs - pipeline as expected nodes - pipeline has minimum nodes - test minimum tags - test alternate tags # Catalog Assertions - test catalog follows naming structure - # Node Tests - test function does the correct operations on test data # Great Expectations