Deep Learning Analysis of COVID-19 lung X-Rays using MATLAB: Part 5

*** DISCLAIMER ***

I have no medical training. Nothing presented here should be considered in any way as informative from a medical point-of-view. This is simply an exercise in image analysis via Deep Learning using MATLAB, with lung X-rays as a topical example in these times of COVID-19. 

INTRODUCTION

In this Part 5 in my series of blog articles on exploring Deep Learning of lung X-rays using MATLAB, the observations for Part 4 -- whereby the grad-CAM technique was used to identify which regions of the X-ray images were being activated for all 19 network architectures under consideration -- serve as the basis for a new network to discriminate between those models which are utilising the lung regions (as desired) rather than outside the lung regions, for a given image-under-test. The resulting network can then be used as a discriminating filter applied to the outputs of the main X-ray classifiers in order to choose between those classifiers which focus (correctly) on the lung regions rather than elsewhere.

DATASET

The image dataset for the grad-CAM Discriminating Filter comprised a set of grad-CAM images as presented in Part 4.  Specifically, the dataset was composed by generating a total of 14,333 grad-CAM image files across all 19 network types and X-ray sample images. For training of the Deep Neural Network, these were split into three classes: INSIDE_LUNGS (whereby the grad-CAM images contain activation regions which are focused on the interior of one or both lungs -- the desirable scenario); OUTSIDE_LUNGS (whereby the grad-CAM images contain activation regions which are focused outside of the lungs or even outside of the body -- undesirable scenario); and RIBCAGE_CENTRAL (whereby the grad-CAM images contain activation regions which are focused in the central part of the ribcage rather then explicitly within either lung -- an intermediate scenario which happened to occur commonly so was considered necessary to be included). Sample images of each of these are shown below.

A sample image generated from the grad-CAM technique presented in Part 4 applied to a lung X-ray analysed via a (Transfer Learning) Deep Neural Network from Part 3. This example has been assigned the label INSIDE_LUNGS for the sake of creating a test dataset for the training of the Deep Neural Network Discriminating Filter, the central focus of this current article. Ideally, all the grad-CAM images generated from all the classifiers applied to all the lung X-rays would fall within this INSIDE_LUNGS class. But the results of Part 4 show this not to be the case (and hence the motivation for devising the Discriminating Filter to sort the relevant classifications from the less relevant) .

A sample grad-CAM image which falls within the OUTSIDE_LUNGS category. The purpose of the Discriminating Filter described in this current article is to identify such cases where the X-ray analysis classifier has wrongly focused on regions outside of the lungs (or indeed the body).

A sample grad-CAM image which falls within the RIBCAGE_CENTRAL category. This occurs quite often with the networks from Part 3. The idea is that such cases can be considered less positively definitive than INSIDE_LUNGS, but better than OUTSIDE_LUNGS, when it comes to determining the validity for lung X-ray classification. 

GROUND TRUTH DATA LABELLING VIA AMAZON SAGEMAKER

In order to assign each of the 14,333 grad-CAM images into the appropriate class (INSIDE_LUNGS, OUTSIDE_LUNGS, or RIBCAGE_CENTRAL) in preparation for training the Deep Neural Network to be used as the Discriminating Filter, the Amazon Mechanical Turk service (part of the Amazon SageMaker Ground Truth for Data Labelling product suite) was utilised. This unique service leverages an on-demand, scalable, human workforce to perform the image labelling. The service employs thousands of human workers willing to do piecemeal work at their convenience, and is a far more attractive solution than attempting to manually label all the images oneself (!)

TRAINING THE DEEP NEURAL NETWORKS VIA TRANSFER LEARNING

Once the grad-CAM images had been sorted (via AWS Mechanical Turk) into the three classes  (INSIDE_LUNGS, OUTSIDE_LUNGS, and RIBCAGE_CENTRAL), all 19 pre-trained networks available in MATLAB were used for Transfer Learning on these grad-CAM images, directly analogous to the approach presented in Part 3 for the underlying X-ray image classifier training. 

RESULTS

The results from the (Transfer Learning) training of all the networks is summarised as follows. From consideration of the classification accuracies on the validation dataset, the "best" performing networks were found to be (where the name refers to the base pre-trained network used in the Transfer Learning): googlenet for determining if INSIDE_LUNGS (75% accuracy);  darknet19 for determining if OUTSIDE_LUNGS (85% accuracy); and mobilenetv2 for determining if RIBCAGE_CENTRAL (86% accuracy). The validation Confusion Matrix for each of these is included below.

Confusion Matrix (on  the validation dataset) for a network trained on grad-CAM images via Transfer Learning starting with the pre-trained googlenet. Of all the networks that were tried, this one had the highest accuracy (75%) for the INSIDE_LUNGS class.

Confusion Matrix (on  the validation dataset) for a network trained on grad-CAM images via Transfer Learning starting with the pre-trained darknet19. Of all the networks that were tried, this one had the highest accuracy (85%) for the OUTSIDE_LUNGS class.

Confusion Matrix (on  the validation dataset) for a network trained on grad-CAM images via Transfer Learning starting with the pre-trained mobilenetv2. Of all the networks that were tried, this one had the highest accuracy (86%) for the RIBCAGE_CENTRAL class.

DISCUSSION & NEXT STEPS

The results demonstrate that the technique of Transfer Learning can be used to devise Deep Neural Networks which can successfully distinguish (with reasonable accuracy) the validity of a given lung X-ray classifier network applied to a given X-ray image by determining whether the corresponding grad-CAM image focuses on regions INSIDE the lungs (suggesting that the X-ray lung classification is valid), OUSTIDE the lungs (suggesting that the the X-ray lung classification is not valid), or in the RIBCAGE CENTRAL region (suggesting that the lung X-ray classification may be of some validity: i.e., more relevant than OUTSIDE the lungs though not as relevant as INSIDE the lungs). The Deep Neural Networks presented here can therefore serve as a Discrimination Filter to assist in choosing between all the various networks (presented in Part 3) for X-ray lung image classification.

The next step will  be to combine the results of this article with the results from Part 3 to determine the "best" network (or  combination of networks) for lung X-ray image classification.