Creating shareable "albums" from locally stored DICOM images

[dipak101505]

Hi @pradeeban !

I'd like to discuss the project's problem statement to ensure we're on the same page.

My understanding:

The project aims to create albums of DICOM files based on query parameters (modality, gender, etc.). These DICOM files are stored on a research server.

Implementation Proposal:

I propose building an application that can:
-Schedule scans on the server to identify new DICOM files.
-Create and manage a file system on the server to store the DICOM files.
-Extract and store metadata (DICOM file path, instance UID, modality, patient information, etc.) from the DICOM files into a database.
-Support a query feature that allows users to search for DICOM files based on specific criteria and displays the results in a table.
-Send the queried DICOM files to Kheops to create albums ( Api Documentation).

User Input:

The user will be able to:
-Enter query parameters in the application.
-Review the results table to verify the retrieved DICOM files.
-Enter a name for the album to add queried dicoms.

Is my understanding of the project goals correct? Does my proposed approach seem feasible? If so, may I proceed with working on it?

I've also attached a diagram for your reference, providing a high-level overview of the proposed solution.

[anbhimi]

Hi @dipak101505,

Thank you for you interest in this project. Your understanding of the project is correct.

While the first two points of the implementation details seems thoughtful - metadata extraction and DICOM querying were already existing features in Niffler repository. You can adjust the implementation details accordingly to avoid code repetition.

Attaching the above mentioned functionality for reference -

1. DICOM Metadata Extraction - https://github.com/Emory-HITI/Niffler/tree/master/modules/meta-extraction
2. DICOM Querying - The results of metadata extraction allows to play with data to filter, slice, and dice in an effective manner.

[dipak101505]

I went through the code of extractor.py. It uses dicom4che to extract Metadata. I found the thread execution implementation a bit complicated.
Personally, I would have used spring framework and configured thread pool to take care of multithreading

[pradeeban]

Sure!
We are using Niffler to get the images. But once you have the images, creating albums does not need Niffler.

So, I think you don't need to understand the complexities of Niffler source code for this project (unless you are doing that for your own curiosity).

An attempt refactor (rather than reusing as needed) Niffler or any other dependencies will divert your time away from your own implementation of creating shareable albums from local DICOM files.

[EraxDhruv]

Hi @anbhimi and @pradeeban,

I'm Dhruv, a B.Tech Computer Science student at the Vellor Institute of Technology. I'm excited to share my progress on this GSoC project.

I've built a functional demo showcasing the following features:

-Secure Login and Registration (JWT Authentication)
-DICOM Metadata Extraction and Storage (DICOM file path, modality, patient info, etc.) using Dicom4che to a MySQL database.
-Searchable DICOM Files: Users can search for DICOM files based on specific criteria with results displayed in a table (currently under development).
-Patient List: Users can view a list of patients parsed from DICOM files.
-Sorting, Filtering, and Pagination: Implemented for both Hospital and Patient lists.
-Album Functionality (In Progress): Users can select and add patients to an Album within the Patient List.

Next Steps:

A scheduler will be implemented to automatically:
-Scan a designated folder for DICOM files.
-Parse the metadata.
-Move the processed files to a file system.
Integration with the Kheops application.
Further improvements based on your valuable feedback.

Screenshots for your reference .

I'd be grateful if you could review my repository and provide guidance for the next steps. I'm highly motivated and eager to learn from your expertise. With your support, I'm confident in completing this project comprehensively.

Thank you for your time and consideration.

[EraxDhruv]

Readme file added.
On a side note, I am solely applying for this project and I truly appreciate your time in helping me make my application stand out.

[pradeeban]

Hi @EraxDhruv, aha, thanks for clarifying. The README makes it better, highlighting Apache Maven as a requirement, making it easier to understand your approach.

Yes, it is great to use a build management framework such as Apache Maven. MEDIator uses Apache Maven as well. So you will be able to easily integrate with that if needed.

You can go ahead and make your proposal draft, including a link to the repository. That way, we can give feedback to the entire draft.

One major comment I like to clarify in your approach is, you mention
“Hospital Management:
• Create, Update, and Delete Hospitals: Add new hospitals to the system, maintain their details, and remove them if necessary.
• Hospital List Retrieval: View a comprehensive list of all registered hospitals.”

I understand why you consider managing hospitals as a use case is a good idea. But in practice, that is unlikely to be a use case for us (as this tool targets a single hospital network, rather than a case where we want to manage data from multiple hospitals).
You should rather have a DICOM Source Management instead. A DICOM Source is anything that can send us radiographic images. This includes scanners (Xray, CT, MR, …) and PACS (Picture archiving and communication system). That is a realistic and common use case as we choose to add and remove endpoints as needed. This approach will also scale to cover multiple hospitals (for instance, hospital 1 has 10 MR scanners and hospital 2 has 20 MR scanners. If you don’t want scanners from hospital 2, you can remove all their scanners). In many cases, a hospital also has a single PACS, thus removing the PACS can remove the hospital (unless individual scanners are added).

If you must add “Hospital Management,” indirectly, you will still be adding/removing all the associated DICOM sources when you add/remove hospitals. (Since, we deal with DICOM files, and DICOM endpoints – source and targets, are the one we see as sources, rather than the site/building names of the hospitals.)

Please let me know if that makes sense. Thanks for the great effort in coding and explaining.

[EraxDhruv]

Hi @pradeeban,

Thanks so much for the green light on the proposal! I'm excited to jump in and get started right away.

Your feedback was incredibly insightful, and I completely understand the logic behind it. Moving forward, instead of grouping images from multiple sources under a single hospital, I'll be adding each source individually. This ensures all patients within the list will have the same modality for a specific particular source.

Looking ahead, here's what I'll be working on:

1. Streamlining Source Management: The hospital list will be removed, simplifying the structure.
2. Refined Patient List: The modality filter within the patient list will be eliminated as all studies from the same source will inherently have the same modality.
3. Enhanced DICOM Integration: A dedicated DICOM source list will be implemented, allowing for advanced filtering options like modality and name. This shift enables adding DICOMs directly to individual sources instead of hospitals.
   I believe this revised approach will effectively meet the project requirements.

For an even more scalable solution:

We could explore the possibility of introducing subcategorical source rows for each hospital. This would provide a way to group sources within their respective hospitals.

Please let me know if you have any further questions or require additional clarification.

Thanks again for your support!

[pradeeban]

Great.
DICOM standard structures the images neatly. A modality sees many Patients (PatientID). A patient may have several studies (StudyInstanceUID). A study will have several series (SeriesInstanceUID). Then a Series has several DICOM images. Imagine this to be like a tree structure, or a folder hierarchy. You should use these pre-existing DICOM entries (PatientID, StudyInstanceUID, and SeriesInstanceUID) to further categorize your images (rather than introducing new custom fields).
Check https://www.dicomstandard.org/concepts and https://dicom.innolitics.com/ciods/ct-image/general-series/0020000e

[EraxDhruv]

Yes @pradeeban.

Thanks for sharing the links and your feedback! I'll definitely focus on extracting all the relevant metadata from the data to improve the user experience. Additionally, I can handle situations where a study has multiple series with different Series Instance UIDs.

Let me know about any other thoughts you have on improving the application.

[kar-hik]

Subject: Progress Update and Request for Guidance on GSoC Project

Dear @anbhimi and @pradeeban,

I hope this email finds you both well. My name is Karthik, and I am currently pursuing a B.Tech in Computer Science at Karunya University. I am reaching out to provide an update on the progress of our GSoC project focused on creating a utility for generating shareable "albums" from locally stored DICOM images.

I am thrilled to share that I have developed a functional demo showcasing several key features. These include:

• Secure Login and Registration using JWT Authentication.
• DICOM Metadata Extraction and Storage into a MySQL database, facilitated by Dicom4che.
• Implementation of a searchable DICOM file system, with ongoing development for search functionality.
• Patient List functionality, allowing users to view parsed patient data from DICOM files.
• Sorting, Filtering, and Pagination capabilities for improved user experience.
• In-progress development of Album Functionality, enabling users to create and manage albums within the application.

As we move forward, I would greatly appreciate your guidance on the next steps for the project. Specifically, I am seeking advice on:

1. Implementing a Scheduler to automate tasks such as file scanning and metadata parsing.
2. Integrating the utility with the Kheops application for enhanced functionality.
3. Incorporating any feedback or suggestions for further improvements.

Your expertise and support are instrumental in ensuring the success of this project, and I am eager to learn from your insights.

Thank you for your time and consideration. I look forward to your valuable feedback.

Warm regards,
Karthik

[pradeeban]

As we reach the deadline for the application (April 2nd), I would recommend sharing the proposal draft at this point. Please also make sure to submit the final proposal on time through the GSoC site!

[yashitz07]

Hey @pradeeban and @anbhimi I am looking forward to deep dive in this project idea of creating shareable albums, Is there any amount of work done on this?
For now, I am looking to grind on problem statement to get a complete understanding.

[anbhimi]

Hi @yashitz07,

I appreciate your interest in the problem. It was proposed for GSoC 2024 but has not been worked on at any technical level.

[yashitz07]

Oh okay, So is the organization is planning to go with this statement this time? Or is there any another set of problem statements on which I can look into.

[anbhimi]

The Schedule for GSoC 2025 have been released yet. Irrespective of that, the problem statement does not change. If you are interested in any other problem, please continue to work on it - which could involve identifying the tech stack to solve the problem, and solving existing issues.

[nilaygit-10721]

Hey @pradeeban and @anbhimi, I'm excited to explore the concept of shareable albums further. Has any work been done on this so far? Right now, I want to focus on refining the problem statement to gain a thorough understanding.

[anbhimi]

Hi, The above problem was proposed for GSoC 2024. Unfortunately it has not been worked in GSoC 2024. The problem statement did not change nor there has been any work done on this since then.

[nilaygit-10721]

I’m really excited about the concept of shareable albums and would love to contribute to this project. Since the problem statement hasn’t been updated or worked on yet, I see this as a great opportunity to dive in and start building. Looking forward to contributing!

[Amine2000s]

hey @anbhimi i already proposed an approach on solving this project problem, I would like to know your opinion about my approach

[Amine2000s]

Hello,

There is an Arabic quote that says, "Understanding the question is half of the answer." Inspired by this, I’ve been reading the project description and familiarizing myself with the DICOM ecosystem. I’ve also reviewed last year’s proposals, but I found their solutions somewhat unclear in addressing the actual problem.

To ensure that I fully understand the scope of the project, I’d like to outline my thoughts and ask a few clarifying questions

Researchers require a utility tool to automate the workflow of creating subsets (or "albums") from locally stored DICOM data. This involves the following steps:

Researchers retrieve DICOM datasets using C-MOVE to store them on a local server or cluster.
Locally stored DICOM images need to be divided into subsets by querying their metadata. These subsets effectively represent "albums."
Currently, this workflow is manual and lacks a utility tool (which is the one where are going to build )

Available Tools:
From my research, I understand the roles of existing tools in this ecosystem:

Kheops: Provides an interface for creating and viewing albums, with integration into OHIF Viewer for image visualization.
MEDIator: Focuses on subset creation and shareability via unique URLs.
NIFFLER: Handles metadata extraction and querying for locally stored DICOM datasets.
To avoid reinventing the wheel, we should aim to integrate these tools where appropriate while adding value to the overall workflow.

I intend to build a proof-of-concept desktop application using JavaFX as a utility for:

Querying metadata from locally stored DICOM datasets.
Creating and managing subsets or "albums."
Leveraging the existing features of tools like Kheops, MEDIator, and NIFFLER, while adding desktop-based enhancements for usability.
This application will provide researchers with a more streamlined way to interact with their datasets, query metadata, and create albums in a user-friendly environment.

My Questions are :

-Is my understanding of the problem and its steps accurate?
-Does the approach of building a small desktop utility using JavaFX seem valid for this proof of concept?
-Are there specific aspects of the problem or tools that I might have overlooked?

I would greatly appreciate it if one of the mentors could provide guidance on these points to help refine my understanding and approach.
with all respect and regards
@anbhimi and @pradeeban

[anbhimi]

Hi,

• Your understanding of the problem is stop-on. Niffler's metadata extraction module would help to query metadata from DICOM datasets.
• The problem does not require for a dekstop application, but if that is the best way to approach the problem - please go ahead. And there are no boundaries on tech stack. Our idea was to be able to create, and delete albums as well as add and remove information from existing albums through Kheops.

Thank You

[Amine2000s]

hey @anbhimi since there is an emphasize on integrating albums managment through kheops , i am thinking of building a small offline albums mangement feature that is synchronized with kheops ? (are we aiming to build a 3rd party kheops utility ? ) also any other solution (other platforms that you suggest ?) , i feel that there something missing and i don't what is it ( probably lack of clear user needs), i would be gratful for any guidance

Thank you

[EbrahemMordy]

Hello,

I want to explain what I understand to see if it's correct or if there is any mistake with it.

After reading the problem I see it's like there is some manual work that needs to be done so they can share some albums from DICOM images.

Like in the last photo, they need to download images and group them then make them any type of folder in Google Drive or zip file, and send it, etc.

But this is manual and takes time.

So we need to make it easy to make an album and share it so we need a middle point that I send to it which images I want to include in the album (ID or anything unique for each image) and it creates the album and makes it a shareable and gives me a URL for it so I can just share the link with others and no need to download the images and make it in my PC

Does this is correct or is there anything I understand wrong?

I would appreciate it if you check this and give me feedback
@anbhimi @pradeeban

[anbhimi]

There have been a few discussions above regarding "Understanding the problem", please refer to them.

We are trying to solve the problem of creating/deleting/managing albums (similar to photo albums) for DICOM Images. The framework to download the images and extract metadata from images is already in place (Niffler - DICOM Metadata Extraction).

[Amine2000s]

Hello ,

since the current focus is on solving the problem of "Albums management" i began to design a few interfaces for this utility i will drop down the interfaces design and the functionalities each interface contains , for the next 2-3 days i ll try to implment the majority of them ,
i hope i get feedback soon from @anbhimi and @pradeeban to know if i am in the right path
(this is just POC a lot of things can get changed à

this is the main interface , user either can go and filter the metadata query , or go to manage albums ( i ll explain each one with it's own workflow ) or settings

this one is where the the metadata query operation happens , and the selected files can either be added to an album (existing one or create a new one directly ) , i will add the option to choose the path that leads to imageslocation

this is where perfom albums management operations like seeing the content of the albums , performing CRUD on them and theability to export them to the desired types

@anbhimi @pradeeban

Thank You

[42bitstogo]

DICOM Album Manager - Initial Implementation Discussion

Hi @anbhimi @pradeeban,

I'm Aditya Marada, a computer science student at the University of Amsterdam. I've gone through the project description and also the past discussions and implemented something basic as a starting point.

Current Understanding

• The core problem is making locally stored DICOM images more accessible and shareable between researchers
• Traditional PACS environments and manual sharing methods are cumbersome and time-consuming
• We want to build a system that can organise DICOM data hierarchically (Patient → Study → Series → Image) and create shareable subsets
• Kheops integration would be a more pragmatic approach as we wouldn't have to reinvent the wheel for native OHIF Viewer support and sharing capabilities

Initial Implementation

I went ahead and implemented a small proof of concept / demo (https://github.com/42bitstogo/dicom_album_manager). Built a Python-based system that can:

• process and organise local DICOM files maintaining the standard hierarchy
• query DICOM metadata effectively
• Create and manage "albums" (collections of DICOM images)
• Store album metadata persistently using JSON
• Provide a CLI interface for all operations

You can run the project with the instructions in the README. I would recommend using the CLI.

Core Components:

• DicomDataManager: handles the DICOM file processing and hierarchy management
• DicomQuery: query interface for searching through the DICOM hierarchy
• AlbumManager: manages album creation and persistence

Using JSON for storage to keep things simple initially, extensive logging for debugging, and test suite with support for real DICOM data (for now I downloaded one sample dicom dataset and have added it to the DICOM folder)

Next Steps

1. Integration Phase

• Integrate with Kheops for image viewing
• Implement URL-based sharing mechanism
• Add OHIF Viewer integration

2. User Interface

• Develop web-based interface for album management

3. Security

• Implement proper access controls
• Add authentication system

If you have time, I'd appreciate some initial feedback. Do you think I'm on the right track and is this approach aligned with the project's goals? Any recommended changes to the current design, code or documentation?

Regards
Aditya