A Model for Quality Defects Detection in Record Sequences and its Application for Obstetric Data in Electronic Medical Records

摘要

Scenarios, in which real-world state transitions are documented by a sequence of data records, introduce unique data quality (DQ) challenges. Due to time and workload constraints, one might choose to update the values only for a subset of the required attributes, while replicating previous values of others. As a result, the record sequence might reflect the current state incorrectly and fail to capture critical transitions. Our model addresses such scenarios by evaluating record sequences and alerting on high likelihood of erroneous replication. The metrics consider attribute characteristics, the distances between consecutive values, and the likelihood of value transition. The potential contribution of the model is demonstrated with a preliminary evaluation of 200 real-world records collected in an Obstetrics unit in a large hospital. A model trained with 100 records reached accuracy level of 85% with detecting data acquisition flaws in the other 100. The manuscript introduces the model development, describes the preliminary evaluation with real-world data, and highlights directions for future research progress.