ePADlite Updates
We are introducing ePADlite, a reimplementation of ePad with state of art technologies: React.js and Node.js.
The new infrastructure leverages docker containers and docker-compose to facilitate modularization and easy installation in addition to improving security using keycloak.
ePADlite is the lightweight mode of ePAD that is focused on collecting annotations in a rapid manner to facilitate populating annotated datasets for training and testing machine learning models. Thus; it only has annotation, worklist and progress tracking functionalities.
New Infrastructure
The new infrastructure of ePAD is illustrated in Fig. 1. All of the components are docker containers and they are bound together with docker compose. The infrastructure has six main containers.
- keycloak: An authentication service that supports openid.
- mariadb: (EPADDB) SQL database for holding data on projects, worklists, user access rights, etc.
- couchdb: NOSQL database for holding JSON data: AIM, AIM Template and DICOM JSON model objects
- DICOMweb Server: lightweight DICOMweb server
- epadlite: the REST API for managing ePAD functionalities and user access rights
- epadjs: the REACT user interface
Fig.1 New ePAD Infrastructure
The authentication service we are using is keycloak, it is opensource and free and it supports openid. By using keycloak, we have achieved a more secure authentication system. The ePAD interface forwards the user to keycloak interface to authenticate and uses the JSON Web Token (JWT) that is generated by keycloak with an Authorization header when making requests to the ePAD REST API services. ePAD REST API services connects to keycloak to validate the JWT token.
New User Interface
The new user interface is implemented with React.js which is a Javascript framework for building user interfaces. We are using Cornerstone, which is an open source JavaScript library, for displaying medical images in browsers. It also provides common tools in an extensible framework manner. This enabled us to add more annotation tools to our application like freehand drawing tool, interpolation and smart brush.
Fig.2 Freehand Tool
Freehand tool (Fig. 2), which can be used in combination with interpolation tool, is a tool that enables the annotators draw the region of interest (ROI) more precisely than the previous spline tool. The slower the annotator moves the mouse, the tool adds more points to the ROI making the shape more precise. Sculpt tool can be used to edit the freehand shape once the drawing is completed.
If the interpolation is enabled and the annotator draws an ROI on a slice, then moves to another and draws one more; ePAD automatically interpolates the annotation to the slices in between and adjusts the size of the shape for each slice. For example, if the annotator draws an ROI on 10th slice, then scrolls to 13th and draws a slightly smaller one, marking the lesion on that slice; the interpolation tool will draw two new ROIs, one on the 11th slice, another on 12th, each one getting smaller. An illustration of this can be seen in Fig.3.
Fig.3 Interpolation illustration
The smart brush tool can be used to improve the time required to create segmentations. It has eight preset Hounsfield Unit gates defined in the system and the annotator can use any other range by clicking the “Custom” radio box and defining the lower and upper bounds. Fig. 4 illustrates lungs segmented using the smart brush tool with Lung preset.
\
Fig.4 Lungs segmented with smart brush annotation tool
We have also added shortcuts to facilitate reading and annotating faster. The template designer or study designer can define shortcuts for the answers in an AIM Template and the annotator can use those shortcuts to select the answers. The annotator can also move to the next series by using shortcut keys.
New backend with NODE.JS, COUCDB and lightweight DICOMweb server
The new backend is implemented with state of art javascript framework Fastify.js, moved all our data to JSON objects and save them in CouchDB, a NOSQL database.
Using JSON objects and saving them in NOSQL database instead of XML files or SQL tables enables us to improve our performance. JSON is a much lightweight data format then XML which makes parsing and processing much easier and faster; and you can achieve the same functionalities like schema specification and validation. We have converted AIM and AIM Template models which were defined as XMLs and validated by XML Schemas defined by Annotation and Image project supported by NCI to JSON objects and converted the XML Schemas to JSON Schemas. We have implemented a command line software aimconvert to convert legacy AIM and AIM Template files to JSON format.
ePAD backend consists of REST API endpoints for controlling ePAD functionalities like project, worklist and reporting. It talks to CouchDB to save and query AIM JSON objects and AIM Template JSON objects and talks to DICOMweb server to get list of available images. In ePADlite, it manages the worklists and progress tracking using a MariaDB database
During these reimplementation efforts, we decided to implement a lightweight DICOM server that will support DICOMweb standard and host the DICOM files as JSON objects. We are using dcmjs to generate the DICOM JSON Model representations of the DICOM P10 files and save original DICOM files as attachments to the JSON object. The DICOMweb server enables the search and filtering of DICOM files as JSON objects which improves the performance. Also, as we designed the new ePAD to use DICOMweb standard to talk to the server it can be plugged out and replaced by any PACS that is supporting DICOMweb.
New Worklist Assigning and Progress Tracking Interface:
Within early releases of ePADlite, we are going to introduce a new worklist assignment and progress tracking functionality to support creating worklists with specific requirements and assigning those worklists to one or many annotators. The requirements specified by the worklist creator define what is needed to mark a study or series as finished. And the progress view automatically calculates the completeness percentages for each study and cumulates by patient and annotator.
Fig. 5 illustrates the worklist management interface which displays the name, due date and description of the worklist in addition to a list of assigned users and a summary of requirements. The requirement for a worklist specifies how many annotations of a specific template are required for patient, study or series to be marked as completed. For example, the Clinical trial 1 worklist specifies that if there are two annotation with RECIST template in a study, then the study is completed; whereas for a study to be marked as completed in Clinical trial 2 worklist, each series under that study should have one annotation with ROI template. The requirements can be changed after defining the worklist. ePAD will recalculate the progress for each study that is added to the worklist for each assignee if the requirements are changed.
Fig.5 Worklist Management Interface
When the user logs in to ePAD, they can use Worklist view to see the list of studies they need to annotate and click on the eye icon to open that study (Fig.6).
Fig.6 Worklist view to display the studies in a worklist
Annotators and project managers can use progress view to track the progress of the worklists (Fig. 7). Progress view opens with a menu on the left which lists both the worklists created by the user and worklists assigned to the user with cumulated completeness percentages. On the right side, progress view supports two modes: user based and patient based view. User based view which is displayed in Fig. 8 groups the users whereas the patient based view groups the patients. Both views cumulate the percentages after grouping using erithmetic mean. User can see the completeness percentages of each user by clicking the triangle in each row. For example, Fig. 8 shows that admin user completed 50% of the required annotations for patient 7; whereas testuser has completed 100% of them and ePAD gets the arithmetic mean of both to show a completion percentage of 75%.
Fig.7 Progress view