Reply To: Need to build an interface for XML to SQL Datadata

Reply To: Need to build an interface for XML to SQL Datadata

Home Forums NAACCR XML Standard Need to build an interface for XML to SQL Datadata Reply To: Need to build an interface for XML to SQL Datadata

#7475
AnonymousJeff Reed
Spectator

Thank you Rich,

The data generation tool Fabian pointed me to in the github community looks like it did a good job of generating test files, (haven’t used the files I generated yet). The tool populates 40-50 field variables so it is a good source to test basic load functionality but shy of being able to use for timing on a fully populated record load test.

Though my needs are only for processing ‘I’ records, the tool will be able to handle all record types. I am currently working on incorporating a user customized XML reference file to define the fields to extract into the load file.

I am handling the data as one flat file with each row representing a case/tumor. Patient/header data will be repeated on each row. Much of the validation and breakdown of the data will be done in the DB. I still see the database needed to check for duplicates and generate aggregate totals.

Building multiple files based on a relational data model is not envisioned at this point but I could see where that would be useful if there is a global data model of the data stores to match the XML. Generic SQL to generate schemas could be a component of that effort. Relationally we consider a unique case/tumor key on the facility_id, Accession_nbr and sequence_nbr. Identification of the patient is a different story which I am sure there is a healthy thread out there somewhere…

A correlated side project will be to use the EDIT50.DLL from the CDC to apply edit checks/scoring for each tumor record. This will be incorporated into the NAACCR XML file parser I am building (looking for a name for this beast). This process would also create KB delimited file(s) for use in bulk loading.

Thank you for all your feedback

J

Copyright © 2018 NAACCR, Inc. All Rights Reserved | naaccr-swoosh-only See NAACCR Partners and Sponsors