© FUJIFILM Corporation

Reinforcement learning for automatically generation of radiology reports using imbalanced dataset


Radiologists use a lot of time to write medical reports manually from medical images. Automated generation of medical reports will help them to reduce their workload. However, many datasets have imbalanced distribution of finding labels (Figure 1), and it is difficult to train models accurately using those datasets. Authors introduced the clinical correctness of generated reports as a reward for reinforcement learning (Figure 2). In addition, they proposed a data augmentation to additionally train the model on infrequent findings. Experimental results showed that the reports generated by the proposed model describe the findings in the input image more correctly. Although the experiment used chest images, the method will be expected to reduce the cost of constructing learning datasets and generate reports for other body parts as well.

Figure 1. Imbalanced distribution of the part of the finding labels in a dataset. [1]

Figure 2. Proposed training architecture. [1] (The clinical correctness of generated reports is used as reward for a reinforcement learning.)
[1] Toru Nishino, Ryota Ozaki, Yohei Momoki, Tomoki Taniguchi, Ryuji Kano, Norihisa Nakano, Yuki Tagawa, Motoki Taniguchi, Tomoko Ohkuma, and Keigo Nakamura. 2020. Reinforcement Learning with Imbalanced Dataset for Data-to-Text Medical Report Generation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2223–2236, Online. Association for Computational Linguistics. 

DOI: 10.18653/v1/2020.findings-emnlp.202

CAUTION:This is Fujifilm Global Website. Fujifilm makes no representation that products on this website are commercially available in all countries. Approved uses of products vary by country and region. Specifications and appearance of products are subject to change without notice.