(hamilton) branch main updated: Allow bypassing data quality checks via a config key (#1598)

[sk...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

skrawcz pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hamilton.git


The following commit(s) were added to refs/heads/main by this push:
     new f39b2105 Allow bypassing data quality checks via a config key (#1598)
f39b2105 is described below

commit f39b2105d0bf949adf2b7660f0712509f63b7197
Author: Dev-iL <65...@users.noreply.github.com>
AuthorDate: Mon Jun 1 02:12:07 2026 -0400

    Allow bypassing data quality checks via a config key (#1598)
    
    * Fix preëxisting issues in test_sklearn_plot_extensions
    
    * Add `disable_data_quality_checks` config key
    
    ## Context
    
    `@check_output` and `@check_output_custom` decorators let users attach data validators to function outputs. At graph-construction time, each decorated node is expanded into a subgraph:
    
    - `{name}_raw` — runs the actual function
    - `{name}_{validator}` — one node per validator (tagged `hamilton.data_quality.contains_dq_results`)
    - `{name}` — aggregates validation results and returns the raw output
    
    This flag is useful for two main reasons:
    1. Validation may be crucial during development, but carries a real runtime cost in production where the pipeline is already trusted. Previously there was no way to turn it off short of removing the decorators from the source code.
    2. When generating graph visualizations and we don't want to crowd them with validation nodes, passing  `disable_data_quality_checks=True` at construction time causes the `_raw` and validator nodes not to be added to the graph - so any visualization of that driver's graph will only show the "business logic" nodes.
    
    ## What was changed
    
    ### `hamilton/function_modifiers/validation.py`
    
    `BaseDataValidationDecorator` (parent of both `check_output` and `check_output_custom`) now:
    
    1. Overrides `optional_config()` to declare `{"disable_data_quality_checks": False}`. This registers the key with Hamilton's config-filtering pipeline so it is threaded through to `transform_node` automatically — no driver plumbing needed.
    
    3. Adds an early-return guard at the top of `transform_node`:
    
       ```python
       if config.get("disable_data_quality_checks", False):
           return [node_]
       ```
    
       When the flag is set, the node is returned unchanged — no `_raw` node, no validator nodes, and no expansion happens. The cost is literally zero: the extra nodes are never created.
    
    ### `hamilton/driver.py`
    
    `Builder` gains a convenience method:
    
    ```python
    def with_data_quality_disabled(self) -> "Builder":
        return self.with_config({"disable_data_quality_checks": True})
    ```
    
    This is a thin wrapper over `with_config` — discoverable by IDE autocomplete and explicit about intent. Because `with_config` does a dict `.update()`, a later call like `.with_config({"disable_data_quality_checks": False})` will re-enable validation, which is the expected last-write-wins behavior.
    
    ## Usage
    
    **Via the `Builder` convenience method (recommended):**
    
    ```python
    dr = (
        hamilton.driver.Builder()
        .with_modules(my_pipeline)
        .with_data_quality_disabled()
        .build()
    )
    ```
    
    **Via `with_config` directly (equivalent, useful when config is assembled dynamically):**
    
    ```python
    dr = (
        hamilton.driver.Builder()
        .with_modules(my_pipeline)
        .with_config({"disable_data_quality_checks": True})
        .build()
    )
    ```
    
    **Legacy `Driver` constructor:**
    
    ```python
    dr = hamilton.driver.Driver(
        {"disable_data_quality_checks": True},
        my_pipeline,
        adapter=DefaultAdapter(),
    )
    ```
    
    ## Tests added
    
    | File | Test | What it covers |
    |------|------|----------------|
    | `tests/function_modifiers/test_validation.py` | `test_check_output_disabled_via_config_returns_original_node` | `check_output_custom` returns the original node unchanged when flag is set |
    | `tests/function_modifiers/test_validation.py` | `test_check_output_builtin_disabled_via_config_returns_original_node` | `check_output` (built-in validators) also respects the flag |
    | `tests/test_end_to_end.py` | `test_builder_with_data_quality_disabled_removes_validator_nodes` | No DQ-tagged nodes appear in `list_available_variables()` when disabled |
    | `tests/test_end_to_end.py` | `test_builder_with_data_quality_disabled_still_executes_correctly` | Driver executes correctly and returns the function's real output when disabled |
    | `tests/test_end_to_end.py` | `test_disable_data_quality_checks_config_key_works_directly` | Raw `with_config` path (no convenience method) also suppresses validator nodes |
    
    All 13 pre-existing validation tests continue to pass.
    
    ## Design notes
    
    - **Graph-construction time, not execution time.** Disabling at construction eliminates the extra nodes entirely. An execution-time approach (e.g., a lifecycle adapter) would still pay graph construction and scheduling overhead, and would be harder to reason about.
    
    - **Config key, not a subclass.** A `NoValidationBuilder` subclass was considered. It was rejected: this is a single boolean flag, not a fundamentally different execution model. The existing `Builder` method pattern (`with_config`, `with_adapters`, …) is the right weight here. The `Builder.with_data_quality_disabled()` convenience method is offered as a discoverable alias — if maintainers prefer, the method alone (without the raw config key) or the raw key alone (without the method) i [...]
    
    - **`optional_config()` is the correct hook.** Hamilton's `resolve_config` / `filter_config` machinery passes only declared config keys to each decorator. Registering the key via `optional_config()` means the flag is silently ignored (defaulting to `False`) by all existing drivers that never set it — no breaking change.
    
    ## Checklist
    
    - [x] PR has an informative and human-readable title (this will be pulled into the release notes)
    - [x] Changes are limited to a single goal (no scope creep)
    - [x] Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
    - [x] Any _change_ in functionality is tested
    - [x] New functions are documented (with a description, list of inputs, and expected output)
    - [x] Placeholder code is flagged / future TODOs are captured in comments
    - [x] Project documentation has been updated if adding/changing functionality.
---
 docs/how-tos/run-data-quality-checks.rst      | 43 +++++++++++++++++++++++++++
 docs/reference/decorators/check_output.rst    | 19 ++++++++++++
 hamilton/driver.py                            | 14 +++++++++
 hamilton/function_modifiers/validation.py     |  6 ++++
 tests/function_modifiers/test_validation.py   | 38 +++++++++++++++++++++++
 tests/plugins/test_sklearn_plot_extensions.py | 11 ++++---
 tests/test_end_to_end.py                      | 42 ++++++++++++++++++++++++++
 7 files changed, 167 insertions(+), 6 deletions(-)

diff --git a/docs/how-tos/run-data-quality-checks.rst b/docs/how-tos/run-data-quality-checks.rst
index 361476ea..d258b547 100644
--- a/docs/how-tos/run-data-quality-checks.rst
+++ b/docs/how-tos/run-data-quality-checks.rst
@@ -16,4 +16,47 @@ Async validators
 
 For validation logic that requires async operations (e.g., async database queries or API calls), use ``AsyncDataValidator`` or ``AsyncBaseDefaultValidator`` from ``hamilton.data_quality.base``. These define ``async def validate()`` and work with ``AsyncDriver``. You can mix sync and async validators in a single ``@check_output_custom`` call.
 
+Disabling validators at runtime
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Validators are useful during development but may be unnecessary overhead in a trusted production pipeline. You can disable all ``@check_output`` and ``@check_output_custom`` validators at graph-construction time, so no extra nodes are ever created:
+
+.. code-block:: python
+
+    dr = (
+        hamilton.driver.Builder()
+        .with_modules(my_pipeline)
+        .with_data_quality_disabled()
+        .build()
+    )
+
+This is equivalent to passing ``{"hamilton.data_quality.disable_checks": True}`` via ``.with_config()``, which is useful when the flag is controlled dynamically (e.g., from an environment variable):
+
+.. code-block:: python
+
+    import os
+
+    dr = (
+        hamilton.driver.Builder()
+        .with_modules(my_pipeline)
+        .with_config({"hamilton.data_quality.disable_checks": os.getenv("DISABLE_DQ", "false") == "true"})
+        .build()
+    )
+
+Because the flag is resolved at graph-construction time, disabled drivers carry zero runtime overhead from validation — no validator nodes are created at all.
+
+A second use case is graph visualization. Each decorated function normally expands into several nodes (``{name}_raw``, one per validator, and the final ``{name}`` node), which can clutter a visualization when you want to communicate pipeline structure rather than validation wiring. Building a driver with ``with_data_quality_disabled()`` gives a clean visualization with only the business-logic nodes:
+
+.. code-block:: python
+
+    dr_viz = (
+        hamilton.driver.Builder()
+        .with_modules(my_pipeline)
+        .with_data_quality_disabled()
+        .build()
+    )
+    dr_viz.display_all_functions("pipeline.png")
+
+Note that this requires a separate driver instance from the one used for execution if you still want validations to run.
+
 See the :doc:`check_output reference <../reference/decorators/check_output>` and `data quality writeup <https://github.com/apache/hamilton/blob/main/writeups/data_quality.md>`_ for details and examples.
diff --git a/docs/reference/decorators/check_output.rst b/docs/reference/decorators/check_output.rst
index 1a27c3fd..47cad979 100644
--- a/docs/reference/decorators/check_output.rst
+++ b/docs/reference/decorators/check_output.rst
@@ -32,6 +32,25 @@ from ``hamilton.data_quality.base`` as your base class instead of the sync varia
 See `data_quality <https://github.com/apache/hamilton/blob/main/data\_quality.md>`_ for more information on
 available validators and how to build custom ones.
 
+Disabling validators
+~~~~~~~~~~~~~~~~~~~~
+
+All ``@check_output`` and ``@check_output_custom`` validators can be disabled at graph-construction
+time using ``Builder.with_data_quality_disabled()``:
+
+.. code-block:: python
+
+    dr = (
+        hamilton.driver.Builder()
+        .with_modules(my_pipeline)
+        .with_data_quality_disabled()
+        .build()
+    )
+
+This eliminates all validator nodes from the graph — no ``_raw`` or validator nodes are created, so
+there is zero runtime cost. It is equivalent to ``.with_config({"hamilton.data_quality.disable_checks": True})``.
+See :doc:`../drivers/Driver` for full ``Builder`` documentation.
+
 Note we also have a plugins that allow for validation with the pandera and pydantic libraries. There are two ways to access these:
 
 1. ``@check_output(schema=pandera_schema)`` or ``@check_output(model=pydantic_model)``
diff --git a/hamilton/driver.py b/hamilton/driver.py
index 541b1da0..9ca646ae 100644
--- a/hamilton/driver.py
+++ b/hamilton/driver.py
@@ -45,6 +45,7 @@ from hamilton.caching.adapter import HamiltonCacheAdapter
 from hamilton.caching.stores.base import MetadataStore, ResultStore
 from hamilton.dev_utils import deprecation
 from hamilton.execution import executors, graph_functions, grouping, state
+from hamilton.function_modifiers.validation import DISABLE_DATA_QUALITY_CHECKS_CONFIG_KEY
 from hamilton.graph_types import HamiltonNode
 from hamilton.io import materialization
 from hamilton.io.materialization import ExtractorFactory, MaterializerFactory
@@ -1843,6 +1844,19 @@ class Builder:
         self.adapters.extend(adapters)
         return self
 
+    def with_data_quality_disabled(self) -> Self:
+        """Disables all ``@check_output`` / ``@check_output_custom`` validators at graph-construction
+        time. No validator nodes are created, so there is zero runtime cost.
+
+        This is equivalent to ``.with_config({DISABLE_DATA_QUALITY_CHECKS_CONFIG_KEY: True})``.
+        Note that a subsequent ``.with_config({DISABLE_DATA_QUALITY_CHECKS_CONFIG_KEY: False})``
+        will re-enable validation, since ``with_config`` always wins on the last write.
+
+        :return: self
+        """
+
+        return self.with_config({DISABLE_DATA_QUALITY_CHECKS_CONFIG_KEY: True})
+
     def with_materializers(self, *materializers: ExtractorFactory | MaterializerFactory) -> Self:
         """Add materializer nodes to the `Driver`
         The generated nodes can be referenced by name in `.execute()`
diff --git a/hamilton/function_modifiers/validation.py b/hamilton/function_modifiers/validation.py
index fd9f7094..e54490b9 100644
--- a/hamilton/function_modifiers/validation.py
+++ b/hamilton/function_modifiers/validation.py
@@ -30,6 +30,7 @@ from hamilton.function_modifiers import base
 
 IS_DATA_VALIDATOR_TAG = "hamilton.data_quality.contains_dq_results"
 DATA_VALIDATOR_ORIGINAL_OUTPUT_TAG = "hamilton.data_quality.source_node"
+DISABLE_DATA_QUALITY_CHECKS_CONFIG_KEY = "hamilton.data_quality.disable_checks"
 
 
 class BaseDataValidationDecorator(base.NodeTransformer):
@@ -42,9 +43,14 @@ class BaseDataValidationDecorator(base.NodeTransformer):
         """
         pass
 
+    def optional_config(self) -> dict[str, Any]:
+        return {DISABLE_DATA_QUALITY_CHECKS_CONFIG_KEY: False}
+
     def transform_node(
         self, node_: node.Node, config: dict[str, Any], fn: Callable
     ) -> Collection[node.Node]:
+        if config.get(DISABLE_DATA_QUALITY_CHECKS_CONFIG_KEY, False):
+            return [node_]
         raw_node = node.Node(
             name=node_.name
             + "_raw",  # TODO -- make this unique -- this will break with multiple validation decorators, which we *don't* want
diff --git a/tests/function_modifiers/test_validation.py b/tests/function_modifiers/test_validation.py
index d37f1865..18675016 100644
--- a/tests/function_modifiers/test_validation.py
+++ b/tests/function_modifiers/test_validation.py
@@ -313,6 +313,44 @@ async def test_async_validator_wrapper_returns_validation_result():
     assert result_fail.passes is False
 
 
+def test_check_output_disabled_via_config_returns_original_node():
+    """With hamilton.data_quality.disable_checks=True, transform_node returns the original node unchanged."""
+    decorator = check_output_custom(
+        SampleDataValidator2(dataset_length=1, importance="warn"),
+        SampleDataValidator3(dtype=np.int64, importance="warn"),
+    )
+
+    def fn(input: pd.Series) -> pd.Series:
+        return input
+
+    node_ = node.Node.from_fn(fn)
+    subdag = decorator.transform_node(
+        node_, config={"hamilton.data_quality.disable_checks": True}, fn=fn
+    )
+    assert len(subdag) == 1
+    assert subdag[0] is node_
+
+
+def test_check_output_builtin_disabled_via_config_returns_original_node():
+    """check_output (not custom) also respects hamilton.data_quality.disable_checks."""
+    decorator = check_output(
+        importance="warn",
+        default_validator_candidates=DUMMY_VALIDATORS_FOR_TESTING,
+        dataset_length=1,
+        dtype=np.int64,
+    )
+
+    def fn(input: pd.Series) -> pd.Series:
+        return input
+
+    node_ = node.Node.from_fn(fn)
+    subdag = decorator.transform_node(
+        node_, config={"hamilton.data_quality.disable_checks": True}, fn=fn
+    )
+    assert len(subdag) == 1
+    assert subdag[0] is node_
+
+
 def test_sync_wrapper_guards_against_unawaited_coroutine():
     """Sync wrapper should raise TypeError if validator accidentally returns a coroutine."""
 
diff --git a/tests/plugins/test_sklearn_plot_extensions.py b/tests/plugins/test_sklearn_plot_extensions.py
index 5de722b0..e14009d9 100644
--- a/tests/plugins/test_sklearn_plot_extensions.py
+++ b/tests/plugins/test_sklearn_plot_extensions.py
@@ -15,6 +15,7 @@
 # specific language governing permissions and limitations
 # under the License.
 
+import inspect
 import pathlib
 
 import numpy as np
@@ -141,12 +142,10 @@ def decision_boundary_display() -> DecisionBoundaryDisplay:
     grid = np.vstack([feature_1.ravel(), feature_2.ravel()]).T
     tree = DecisionTreeClassifier().fit(iris.data[:, :2], iris.target)
     y_pred = np.reshape(tree.predict(grid), feature_1.shape)
-    kwargs = dict(xx0=feature_1, xx1=feature_2, response=y_pred)
-    # sklearn 1.8+ requires n_classes
-    sig = inspection.DecisionBoundaryDisplay.__init__
-    if "n_classes" in sig.__code__.co_varnames:
-        kwargs["n_classes"] = 3
-    decision_curve = inspection.DecisionBoundaryDisplay(**kwargs)
+    dbd_kwargs = dict(xx0=feature_1, xx1=feature_2, response=y_pred)
+    if "n_classes" in inspect.signature(inspection.DecisionBoundaryDisplay.__init__).parameters:
+        dbd_kwargs["n_classes"] = len(np.unique(iris.target))
+    decision_curve = inspection.DecisionBoundaryDisplay(**dbd_kwargs)
     return decision_curve
 
 
diff --git a/tests/test_end_to_end.py b/tests/test_end_to_end.py
index bd401a69..8a28a7e4 100644
--- a/tests/test_end_to_end.py
+++ b/tests/test_end_to_end.py
@@ -600,3 +600,45 @@ def test_driver_v2_inputs_can_be_none():
     with pytest.raises(ValueError):
         # validate that None doesn't cause issues
         dr.execute(["e"], inputs=None)
+
+
+def test_builder_with_data_quality_disabled_removes_validator_nodes():
+    """with_data_quality_disabled() eliminates validator nodes from the graph entirely."""
+    dr = (
+        driver.Builder()
+        .with_modules(tests.resources.data_quality)
+        .with_data_quality_disabled()
+        .build()
+    )
+    all_vars = dr.list_available_variables()
+    dq_nodes = [
+        var for var in all_vars if var.tags.get("hamilton.data_quality.contains_dq_results", False)
+    ]
+    assert len(dq_nodes) == 0
+
+
+def test_builder_with_data_quality_disabled_still_executes_correctly():
+    """Driver built with data quality disabled returns correct output without raising."""
+    dr = (
+        driver.Builder()
+        .with_modules(tests.resources.data_quality)
+        .with_data_quality_disabled()
+        .build()
+    )
+    result = dr.execute(["data_might_be_in_range"], inputs={"data_quality_should_fail": True})
+    assert list(result["data_might_be_in_range"]) == [10.0]
+
+
+def test_disable_data_quality_checks_config_key_works_directly():
+    """hamilton.data_quality.disable_checks can also be passed via with_config directly."""
+    dr = (
+        driver.Builder()
+        .with_modules(tests.resources.data_quality)
+        .with_config({"hamilton.data_quality.disable_checks": True})
+        .build()
+    )
+    all_vars = dr.list_available_variables()
+    dq_nodes = [
+        var for var in all_vars if var.tags.get("hamilton.data_quality.contains_dq_results", False)
+    ]
+    assert len(dq_nodes) == 0