Quality Evaluation

When data is brought in, it will be subjected to a number of quality evaluation steps at different parts of the pipeline.

1. CSV text import: This ensures that the text of the submitted CSVs can be loaded into the database. This will be as permissive as possible. All errors at this stage are critical errors and failing inputs will not be used.
2. OMOP constraint application: This step will convert data types, apply OMOP database constraints, and apply the transfer format deletion / row update logic. All errors at this stage are critical errors and failing input data will not be used.
3. Site level quality evaluation: The combined individual dataset for each contributing site will undergo quality evaluation. This will look at the data more closely and use sites and controls for each other. The quality evaluation from this phase will need to be responded to on a case by case basis, as appropriate.
4. Combined data quality evaluation: Quality checks eventually will be run over the merged dataset. This will attempt to detect issues that occurred in merging, and if the merged data has lower quality than the individual site data.