EMMAA Model Test Framework (emmaa.model_tests
)¶
This module implements the object model for EMMAA model testing.
-
class
emmaa.model_tests.
ModelManager
(model, mode='local')[source]¶ Bases:
object
Manager to generate and store properties of a model and relevant tests.
Parameters: - model (emmaa.model.EmmaaModel) – EMMAA model
- mode (str) – If ‘local’ (default), does not save any exports/images to S3. It is only set to ‘s3’ mode in update_model_manager.py script.
-
mc_mapping
¶ A dictionary mapping a ModelChecker type to a corresponding method for assembling the model and a ModelChecker class.
Type: dict
-
mc_types
¶ A dictionary in which each key is a type of a ModelChecker and value is a dictionary containing an instance of a model, an instance of a ModelChecker and a list of test results.
Type: dict
-
applicable_tests
¶ A list of EMMAA tests applicable for given EMMAA model.
Type: list[emmaa.model_tests.EmmaaTest]
-
answer_intervention_query
(query, bucket='emmaa')[source]¶ Answer user intervention query by simulating a PySB model.
-
answer_queries
(queries, **kwargs)[source]¶ Answer all queries registered for this model.
Parameters: queries (list[emmaa.queries.Query]) – A list of queries to run. Returns: responses – A list of tuples each containing a query, mc_type and result json. Return type: list[tuple(json, json)]
-
get_updated_mc
(mc_type, stmts, add_ns=False, edge_filter_func=None)[source]¶ Update the ModelChecker and graph with stmts for tests/queries.
-
hash_response_list
(response)[source]¶ Return a dictionary mapping a hash with a response in a response list.
-
process_response
(mc_type, result)[source]¶ Return a dictionary in which every key is a hash and value is a list of tuples. Each tuple contains a sentence describing either a step in a path (if it was found) or result code (if a path was not found) and a link leading to a webpage with more information about corresponding sentence.
-
run_all_tests
(filter_func=None, edge_filter_func=None)[source]¶ Run all applicable tests with all available ModelCheckers.
-
class
emmaa.model_tests.
RefinementTestConnector
[source]¶ Bases:
emmaa.model_tests.TestConnector
Determines applicability of a test to a model by checking if test entities or their refinements are in the model.
-
class
emmaa.model_tests.
ScopeTestConnector
[source]¶ Bases:
emmaa.model_tests.TestConnector
Determines applicability of a test to a model by overlap in scope.
-
class
emmaa.model_tests.
StatementCheckingTest
(stmt, configs=None)[source]¶ Bases:
emmaa.model_tests.EmmaaTest
Represent an EMMAA test condition that checks a PySB-assembled model against an INDRA Statement.
-
class
emmaa.model_tests.
TestConnector
[source]¶ Bases:
object
Determines if a given test is applicable to a given model.
-
class
emmaa.model_tests.
TestManager
(model_managers, tests)[source]¶ Bases:
object
Manager to generate and run a set of tests on a set of models.
Parameters: - model_managers (list[emmaa.model_tests.ModelManager]) – A list of ModelManager objects
- tests (list[emmaa.model_tests.EmmaaTest]) – A list of EMMAA tests
-
make_tests
(test_connector)[source]¶ Generate a list of applicable tests for each model with a given test connector.
Parameters: test_connector (emmaa.model_tests.TestConnector) – A TestConnector object to use for connecting models to tests.
-
emmaa.model_tests.
load_tests_from_s3
(test_name, bucket='emmaa')[source]¶ Load Emmaa Tests with the given name from S3.
Parameters: test_name (str) – Looks for a test file in the emmaa bucket on S3 with key ‘tests/{test_name}’. Returns: List of EmmaaTest objects loaded from S3. Return type: list of EmmaaTest
-
emmaa.model_tests.
model_to_tests
(model_name, upload=True, bucket='emmaa')[source]¶ Create StatementCheckingTests from model statements.
-
emmaa.model_tests.
run_model_tests_from_s3
(model_name, test_corpus='large_corpus_tests', upload_results=True, bucket='emmaa')[source]¶ Run a given set of tests on a given model, both loaded from S3.
After loading both the model and the set of tests, model/test overlap is determined using a ScopeTestConnector and tests are run.
Parameters: Returns: Instance of ModelManager containing the model data, list of applied tests and the test results.
Return type: