causal_testing.testing.metamorphic_relation

This module contains the ShouldCause and ShouldNotCause metamorphic relations as defined in our ICST paper [https://eprints.whiterose.ac.uk/195317/].

Module Contents

Classes

MetamorphicRelation

Class representing a metamorphic relation.

ShouldCause

Class representing a should cause metamorphic relation.

ShouldNotCause

Class representing a should cause metamorphic relation.

Functions

generate_metamorphic_relation(→ MetamorphicRelation)

Construct a metamorphic relation for a given node pair implied by the Causal DAG, or None if no such relation can

generate_metamorphic_relations(→ list[MetamorphicRelation])

Construct a list of metamorphic relations implied by the Causal DAG.

generate_causal_tests(dag_path, output_path[, ...])

Generate and output causal tests for a given DAG.

Attributes

logger

causal_testing.testing.metamorphic_relation.logger
class causal_testing.testing.metamorphic_relation.MetamorphicRelation

Class representing a metamorphic relation.

base_test_case: causal_testing.testing.base_test_case.BaseTestCase
adjustment_vars: Iterable[str]
__eq__(other)

Return self==value.

to_json_stub(skip: bool = False, estimate_type: str = 'coefficient', effect_type: str = 'direct', estimator: str = 'LinearRegressionEstimator', alpha: float = 0.05) → dict

Convert to a JSON frontend stub string for user customisation. :param skip: Whether to skip the test (default False). :param effect_type: The type of causal effect to consider (total or direct) :param estimate_type: The estimate type to use when evaluating tests :param estimator: The name of the estimator class to use when evaluating the test :param alpha: The significance level to use when calculating the confidence intervals

class causal_testing.testing.metamorphic_relation.ShouldCause

Bases: MetamorphicRelation

Class representing a should cause metamorphic relation.

to_json_stub(skip: bool = False, estimate_type: str = 'coefficient', effect_type: str = 'direct', estimator: str = 'LinearRegressionEstimator', alpha: float = 0.05) → dict

Convert to a JSON frontend stub string for user customisation. :param skip: Whether to skip the test (default False). :param effect_type: The type of causal effect to consider (total or direct) :param estimate_type: The estimate type to use when evaluating tests :param estimator: The name of the estimator class to use when evaluating the test :param alpha: The significance level to use when calculating the confidence intervals

__str__()

Return str(self).

class causal_testing.testing.metamorphic_relation.ShouldNotCause

Bases: MetamorphicRelation

Class representing a should cause metamorphic relation.

to_json_stub(skip: bool = False, estimate_type: str = 'coefficient', effect_type: str = 'direct', estimator: str = 'LinearRegressionEstimator', alpha: float = 0.05) → dict

Convert to a JSON frontend stub string for user customisation. :param skip: Whether to skip the test (default False). :param effect_type: The type of causal effect to consider (total or direct) :param estimate_type: The estimate type to use when evaluating tests :param estimator: The name of the estimator class to use when evaluating the test :param alpha: The significance level to use when calculating the confidence intervals

__str__()

Return str(self).

causal_testing.testing.metamorphic_relation.generate_metamorphic_relation(node_pair: tuple[str, str], dag: causal_testing.specification.causal_dag.CausalDAG, nodes_to_ignore: set = None) → MetamorphicRelation

Construct a metamorphic relation for a given node pair implied by the Causal DAG, or None if no such relation can be constructed (e.g. because every valid adjustment set contains a node to ignore).

Parameters:

  • node_pair – The pair of nodes to consider.

  • dag – Causal DAG from which the metamorphic relations will be generated.

  • nodes_to_ignore – Set of nodes which will be excluded from causal tests.

Returns:

A list containing ShouldCause and ShouldNotCause metamorphic relations.

causal_testing.testing.metamorphic_relation.generate_metamorphic_relations(dag: causal_testing.specification.causal_dag.CausalDAG, nodes_to_ignore: set = None, threads: int = 0, nodes_to_test: set = None) → list[MetamorphicRelation]

Construct a list of metamorphic relations implied by the Causal DAG. This list of metamorphic relations contains a ShouldCause relation for every edge, and a ShouldNotCause relation for every (minimal) conditional independence relation implied by the structure of the DAG.

Parameters:

  • dag – Causal DAG from which the metamorphic relations will be generated.

  • nodes_to_ignore – Set of nodes which will be excluded from causal tests.

  • threads – Number of threads to use (if generating in parallel).

  • nodes_to_test – Set of nodes to test the relationships between (defaults to all nodes).

Returns:

A list containing ShouldCause and ShouldNotCause metamorphic relations.

causal_testing.testing.metamorphic_relation.generate_causal_tests(dag_path: str, output_path: str, ignore_cycles: bool = False, threads: int = 0, test_inputs: bool = False, **json_stub_kargs)

Generate and output causal tests for a given DAG.

Parameters:

  • dag_path – Path to the DOT file that specifies the causal DAG.

  • output_path – Path to save the JSON output.

  • ignore_cycles – Whether to bypass the check that the DAG is actually acyclic. If set to true, tests that include variables that are part of a cycle as either treatment, outcome, or adjustment will be omitted from the test set.

  • threads – The number of threads to use to generate tests in parallel. If unspecified, tests are generated in serial. This is tylically fine unless the number of tests to be generated is >10000.

  • test_inputs – Whether to test independences between inputs (i.e. root nodes in the DAG). Defaults to False as they will typically be independent by construction.

  • json_stub_kargs – Kwargs to pass into to_json_stub (see docstring for details.)