Methodologic Assessment of a Deep Learning Approach Measuring Lung Opacification in COVID-19 at Chest CT
Editor:
We read with great interest the article by Dr Huang and colleagues, published in March 2020 in Radiology: Cardiothoracic Imaging, in which they report a deep learning approach to quantify coronavirus disease 2019 (COVID-19) in chest CT 1. We are firm believers that research on artificial intelligence (AI) applied to quantitative radiology will help improve patient care in the near future. Nevertheless, we think some methodologic comments are appropriate.
First, the reported model training is lacking important information essential to the reproducibility of the method; the recently published CLAIM checklist 2 addresses important points for authors presenting their research on AI applied to medical imaging. Training data annotation was done every five slices, but it is unclear whether the remaining slices were simply excluded or if some technique such as label interpolation was utilized. There was also no mention of the level at which the data were partitioned; to prevent data leakage, data should be partitioned at the patient level. The manufacturer and parameters of the scanner from which training data were obtained are also unclear. No information on the model’s loss function was reported.
Second, the authors state that 8.7% of the cases were excluded due to poor segmentation, but it is unclear whether this refers only to segmentation of opacities or to the lung segmentation as well. Juxtapleural opacities are frequently reported as problematic for lung segmentation algorithms and could be a relevant issue 2. It would be interesting to know if prior structural lung damage, atelectasis, or pleural effusion affects performance (if any cases were included in the study). Also, it would be interesting to know if the model is expected to perform equally well on both lung and body kernels.
Finally, there is little information on how the percentage of lung opacification (quantitative CT percentage of lung opacification [QCT-PLO] score) is measured. It is not explicit whether the algorithm is able to differentiate ground-glass opacities from consolidations or other findings, and if they all have the same weight in the final score.
Despite the comments above, we believe the work done by Dr Lu Huang and colleagues is a phenomenal use case of AI to aid in prognosis and, ultimately, in patient care.
Notas
Institution from which the work originated (with address): Diagnóstico da América, R. Gilberto Sabino, 215, Pinheiros - 05425-020 - São Paulo - SP, Brazil. Funding: This Letter received no funding. Disclosures: M.R.F.d.M. machine learning engineer at DASA. F.C.K. consultant for MD.ai; employed by DASA. I.P. consultant for MD.ai. M.S.T. employed by DASA; payment for lectures by DASA.
