Keck Hospital Test

[NickKramer87]

As a database generator, I want to validate the accuracy of the synthetic database by comparing the summary statistics of a database of patients that would go to Keck Hospital to the actual summary statistics from Keck Hospital to determine if significant changes are needed to the database generation program.

Requirements:

1. Task "Automatic Summary Generation" and "Database Summary Validation" must be completed prior to start.

Potential subtasks:

1. A radius or list of zip codes for patients who are likely to go to Keck Hospital.
2. A synthetic database of patients within the area from subtask 1.
3. A passing grade according to the success criterium from subtask 36.

Acceptance Criteria:

1. A tool that will compare the synthetic and real summaries and give a percent similarity or possibly a correlation coefficient if that is easier.
2. A brief report specifying the threshold for deeming a dataset similar and the reasons behond that threshold.
3. A report where at least five different databases are tested using the tool from task 3 wherein all five (or 95% if a large number is possible) of the datasets pass the similarity threshold.

[TravisHaussler]

Its possible to override the hospitals list to only include Keck and then set the location for the run to be somewhat close. We did a test run with Pasadena and it produced results.

[TravisHaussler]

Sub task notes:

• Test out of location population to see what generates
• Add location as a possible command line
• Compare running with a local hospital vs a far hospital

[rileeki]

Medicare did an analysis comparing Synthea's data to real Medicare claims data. This might be something of a model for the analysis. See pages 13-27 of this document.

[TravisHaussler]

https://github.com/orchid-initiative/synthetic-database-project/blob/main/csv_formatted_data_09-11-2023_134827.csv

Here is a 500 male and 500 female run of synthea in los angeles area with only keck as a possible hospital

[rileeki]

Thanks, @TravisHaussler! lol They all still live in Massachusetts somehow.

[rileeki]

Hm, the layout of this doesn't seem quite right and it looks like the diagnosis codes are still SNOMED. Could you upload the log and fixed-width output too? @TravisHaussler

[TravisHaussler]

I’ll take a look tomorrow morning. I am surprised the patients themselves didn’t generate in the right location at least, that seems strange to me. I’ll check the diagnosis too, that’s odd (we expect the procedure ones to be still though)

…

On Thu, Nov 9, 2023 at 4:19 PM rileeki ***@***.***> wrote: Hm, the layout of this doesn't seem quite right and it looks like the diagnosis codes are still SNOMED. Could you upload the log and fixed-width output too? @TravisHaussler <https://github.com/TravisHaussler> — Reply to this email directly, view it on GitHub <#64 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZL3CWKUWLQUBG3Q5WXD3C3YDVXIRAVCNFSM6AAAAAA6H67RDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMBUHA3TSMJUGY> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>

[TravisHaussler]

Here is what I see for the first bunch of rows from that file (taking out the mass of extra blank diagnosis and procedure code fields):
The diagnosis codes are ICD, while the Procedure codes are still SNOMED and I see all the addresses in CA

<style type="text/css"></style>

Type of Care	Facility Identification Number	Facility Name	Date of Birth	Sex	Ethnicity	Race	Not in Use 1	Admission Date	Point of Origin	Route of Admission	Type of Admission	Discharge Date	Principal Diagnosis	Present on Admission for Principal Diagnosis	Diagnosis 2	Present on Admission 2	Diagnosis 3	Present on Admission 3	Diagnosis Codes	Present on Admission	Principal Procedure Code	Principal Procedure Date	Procedure Code 2	Procedure Date 2	Procedure Codes	Procedure Dates	External Causes of Morbidity and Present on Admission	Patient SSN	Disposition of Patient	Total Charges	Abstract Record Number (Optional)	Prehospital Care & Resuscitation - DNR Order	Payer Category	Type of Coverage	Plan Code Number	Preferred Spoken Language	Patient Address - Address Number and Street Name	Patient Address - City	Patient Address - State	Patient Address - Zip Code	Patient Address - Country Code	Patient Address - Homeless Indicator
1	10735	KECK MEDICAL CENTER OF USC	19621021	F	E2	R5		19980816	6	3	2	19980817	Z9851	N					('Z9851',)	('N',)								999713129	85	12088	1	Y	6				140 Flatley Arcade Apt 56	Los Angeles	California	90036	US
1	10735	KECK MEDICAL CENTER OF USC	19621021	F	E2	R5		19980816	1	3	3	19980817	Z9851	N					('Z9851',)	('N',)								999713129	84	12088	2	N	6				140 Flatley Arcade Apt 56	Los Angeles	California	90036	US
1	10735	KECK MEDICAL CENTER OF USC	19621021	F	E2	R5		19980816	6	3	4	19980817	Z9851	N					('Z9851',)	('N',)								999713129	85	12088	3	N	6				140 Flatley Arcade Apt 56	Los Angeles	California	90036	US
1	10735	KECK MEDICAL CENTER OF USC	19621021	F	E2	R5		19980816	8	3	5	19980817	Z9851	N					('Z9851',)	('N',)								999713129	50	12088	4	Y	6				140 Flatley Arcade Apt 56	Los Angeles	California	90036	US
1	10735	KECK MEDICAL CENTER OF USC	19621021	F	E2	R5		20180807	1	1	1	20180810	S83519	U	S83519	Y			('S83519', 'S83519')	('U', 'Y')	699253003	20180807	133899007	20180807	('699253003', '133899007')	('20180807', '20180807')		999713129	86	29949	5	Y	6				140 Flatley Arcade Apt 56	Los Angeles	California	90036	US
1	10735	KECK MEDICAL CENTER OF USC	19621021	F	E2	R5		20180807	2	3	9	20180810	S83519	U	S83519	Y			('S83519', 'S83519')	('U', 'Y')	699253003	20180807	133899007	20180807	('699253003', '133899007')	('20180807', '20180807')		999713129	21	29949	6	N	6				140 Flatley Arcade Apt 56	Los Angeles	California	90036	US
1	10735	KECK MEDICAL CENTER OF USC	19621021	F	E2	R5		20180807	E	1	1	20180810	S83519	U	S83519	Y			('S83519', 'S83519')	('U', 'Y')	699253003	20180807	133899007	20180807	('699253003', '133899007')	('20180807', '20180807')		999713129	81	29949	7	Y	6				140 Flatley Arcade Apt 56	Los Angeles	California	90036	US
1	10735	KECK MEDICAL CENTER OF USC	19621021	F	E2	R5		20180807	5	3	3	20180810	S83519	U	S83519	Y			('S83519', 'S83519')	('U', 'Y')	699253003	20180807	133899007	20180807	('699253003', '133899007')	('20180807', '20180807')		999713129	62	29949	8	Y	6				140 Flatley Arcade Apt 56	Los Angeles	California	90036	US
1	10735	KECK MEDICAL CENTER OF USC	19971128	F	E2	R5		20230308	F	3	2	20230309	Z9851	W	Z302	W	Z9851	N	('Z9851', 'Z302', 'Z9851')	('W', 'W', 'N')	287664005	20230308	133899007	20230308	('287664005', '133899007')	('20230308', '20230308')		999904365	91	16155	9	N	6				707 Considine Way Apt 91	Los Angeles	California	90061	US
1	10735	KECK MEDICAL CENTER OF USC	19971128	F	E2	R5		20230308	4	3	9	20230309	Z9851	W	Z302	W	Z9851	N	('Z9851', 'Z302', 'Z9851')	('W', 'W', 'N')	287664005	20230308	133899007	20230308	('287664005', '133899007')	('20230308', '20230308')		999904365	0	16155	10	Y	6				707 Considine Way Apt 91	Los Angeles	California	90061	US
1	10735	KECK MEDICAL CENTER OF USC	19971128	F	E2	R5		20230308	1	3	9	20230309	Z9851	W	Z302	W	Z9851	N	('Z9851', 'Z302', 'Z9851')	('W', 'W', 'N')	287664005	20230308	133899007	20230308	('287664005', '133899007')	('20230308', '20230308')		999904365	87	16155	11	Y	6				707 Considine Way Apt 91	Los Angeles	California	90061	US
1	10735	KECK MEDICAL CENTER OF USC	19971128	F	E2	R5		20230308	1	3	5	20230309	Z9851	W	Z302	W	Z9851	N	('Z9851', 'Z302', 'Z9851')	('W', 'W', 'N')	287664005	20230308	133899007	20230308	('287664005', '133899007')	('20230308', '20230308')		999904365	84	16155	12	Y	6				707 Considine Way Apt 91	Los Angeles	California	90061	US
1	10735	KECK MEDICAL CENTER OF USC	19690208	F	E2	R5		20040820	8	3	5	20040821	Z9851	Y					('Z9851',)	('Y',)								999925265	83	9185	13	N	8				320 Bayer Crossing Suite 37	Los Angeles	California	90291	US
1	10735	KECK MEDICAL CENTER OF USC	19690208	F	E2	R5		20040820	6	3	4	20040821	Z9851	Y					('

[rileeki]

@TravisHaussler You are totally right. I'm not even sure what I was looking at... I'm sorry about that!

[rileeki]

@TravisHaussler I'll pick this up for the next two weeks. I plan to slice and dice the data you provided and provide a report at our next check-in comparing this dataset to the publicly available summary statistics.

[TravisHaussler]

Ok, let me know if you want it run with larger numbers too. I hit programmatic errors at first reading in the huge synthea csvs but that’s a little better now that we only load specific desired columns. I can also run it a handful of separate times and concatenate the results I guess, but that’s maybe slightly statistically different

…

On Fri, Nov 17, 2023 at 1:29 PM rileeki ***@***.***> wrote: @TravisHaussler <https://github.com/TravisHaussler> I'll pick this up for the next two weeks. I plan to slice and dice the data you provided and provide a report at our next check-in comparing this dataset to the publicly available summary statistics. — Reply to this email directly, view it on GitHub <#64 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AZL3CWJJAADJ4DCP4OJBWULYE7JJZAVCNFSM6AAAAAA6H67RDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMJXGEZTANBXHE> . You are receiving this because you were mentioned.Message ID: ***@***.*** com>