Jiashu Xu

A Review of Self-supervised Learning Methods in the Field of Medical Image Analysis

Full Text (PDF, 1148KB), PP.33-46

Views: 0 Downloads: 0

Author(s)

Jiashu Xu ^1,*

1. National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, 03056, Ukraine

* Corresponding author.

DOI: https://doi.org/10.5815/ijigsp.2021.04.03

Received: 6 Apr. 2021 / Revised: 2 May 2021 / Accepted: 20 May 2021 / Published: 8 Aug. 2021

Index Terms

Medical image analysis, Self-Supervised learning, Unsupervised learning, Visual feature learning, Contrastive Learning.

Abstract

In the field of medical image analysis, supervised deep learning strategies have achieved significant development, while these methods rely on large labeled datasets. Self-Supervised learning (SSL) provides a new strategy to pre-train a neural network with unlabeled data. This is a new unsupervised learning paradigm that has achieved significant breakthroughs in recent years. So, more and more researchers are trying to utilize SSL methods for medical image analysis, to meet the challenge of assembling large medical datasets. To our knowledge, so far there still a shortage of reviews of self-supervised learning methods in the field of medical image analysis, our work of this article aims to fill this gap and comprehensively review the application of self-supervised learning in the medical field. This article provides the latest and most detailed overview of self-supervised learning in the medical field and promotes the development of unsupervised learning in the field of medical imaging. These methods are divided into three categories: context-based, generation-based, and contrast-based, and then show the pros and cons of each category and evaluates their performance in downstream tasks. Finally, we conclude with the limitations of the current methods and discussed the future direction.

Cite This Paper

Jiashu Xu, " A Review of Self-supervised Learning Methods in the Field of Medical Image Analysis", International Journal of Image, Graphics and Signal Processing(IJIGSP), Vol.13, No.4, pp. 33-46, 2021. DOI: 10.5815/ijigsp.2021.04.03

Reference

[1]S. Gidaris, P. Singh, N. Komodakis.” Unsupervised Representation Learning by Predicting Image Rotations,” Proceedings of the International Conference on Learning Representations, 2018.

[2]M. Noroozi, P. Favaro. “Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles,” In European conference on computer vision, 2016, pp: 69-84.

[3]T. Nima, Y. Hu, J. Cao, X. Yan, Y. Xiao, Y. Lu, J. Liang, et al. “Surrogate supervision for medical image analysis: Effective deep learning from limited quantities of labeled data.” In 2019 IEEE 16th International Symposium on Biomedical Imaging,2019, pp. 1251-1255.

[4]C. Szegedy, S. Ioffe, V. Vanhoucke, "Inception-v4, inception-resnet and the impact of residual connections on learning." In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31, no. 1. 2017.

[5]A. Hatamizadeh, P. Shilpa, X. Ding, D. Terzopoulos, and N. Tajbakhsh. "Automatic segmentation of pulmonary lobes using a progressive dense V-network." In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 2018, pp. 282-290.

[6]Imran, A. A. Z., Huang, C., Tang, H., Fan, W., Xiao, Y., Hao, D., et al. “Partly Supervised Multitask Learning.” arXiv preprint arXiv:2005.02523,2020.

[7]A. R. Venkatakrishnan, S. T. Kim, R. Eisawy, F. Pfister, & Navab, N. “Self-Supervised Out-of-Distribution Detection in Brain CT Scans.” arXiv e-prints, arXiv-2011,2020.

[8]Y. Li, J. Chen, X. Xie, K. Ma, & Y. Zheng. “Self-Loop Uncertainty: A Novel Pseudo-Label for Semi-supervised Medical Image Segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention.” pp. 614-623, Oct. 2020

[9]A. Taleb,W. Loetzsch, N. Danz, J. Severin, T. Gaertner, B. Bergner, & C. Lippert. “3d self-supervised methods for medical imaging.” arXiv preprint arXiv:2006.03829, 2020.

[10]X. Zhuang, Y. Li, Y. Hu, K. Ma, Y. Yang, and Y. Zheng, “Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik’s Cube.” arXiv preprint arXiv:1910.02241, 2019.

[11]X. Tao, Y. Li, W. Zhou, K. Ma, & Y. Zheng, Revisiting Rubik’s cube: self-supervised learning with volume-wise transformation for 3D medical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2020, October. pp. 238-248.

[12]J. Zhu, Y. Li, Y. Hu, K. Ma, S. K. Zhou, & Y. Zheng. Rubik’s Cube+: A self-supervised feature learning framework for 3D medical image analysis. Medical Image Analysis, 2020, 64, 101746.

[13]S. Li, K. Yu, and K. Batmanghelich. "Context Matters: Graph-based Self-supervised Representation Learning for Medical Images." arXiv preprint arXiv:2012.06457 ,2020.

[14]Bai, W., Chen, C., Tarroni, G., Duan, J., Guitton, F., Petersen, S. E., ... & Rueckert, D. "Self-supervised learning for cardiac mr image segmentation by anatomical position prediction." In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 541-549. 2019.

[15]Zhou, Z., Sodha, V., Pang, J., Gotway, M. B., & Liang, J. and Jianming Liang. "Model’s genesis." Medical image analysis 67: 101840, 2021.

[16]Haghighi, F., Taher, M. R. H., Zhou, Z., Gotway, M. B., & Liang, J. "Learning Semantics-enriched Representation via Self-discovery, Self-classification, and Self-restoration." In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 137-147. 2020.

[17]Zhang, X., Zhang, Y., Zhang, X., & Wang, Y. Universal Model for 3D Medical Image Analysis. arXiv preprint arXiv:2010.06107, 2020.

[18]Zhou, Z., Sodha, V., Siddiquee, M. M. R., Feng, R., Tajbakhsh, N., Gotway, M. B., & Liang, J. "Model’s genesis: Generic autodidactic models for 3d medical image analysis." In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 384-393.2019.

[19]Chen, L., Bentley, P., Mori, K., Misawa, K., Fujiwara, M., & Rueckert, D. Self-supervised learning for medical image analysis using image context restoration. Medical image analysis, 58, 101539. 2019.

[20]Yu, L., Zhang, W., Wang, J., & Yu, Y. Seqgan: Sequence generative adversarial nets with policy gradient. In Proceedings of the AAAI conference on artificial intelligence. 2017, February,Vol. 31, No. 1.

[21]Ross, T., Zimmerer, D., Vemuri, A., Isensee, F., Wiesenfarth, M., Bodenstedt, S., ... & Maier-Hein, L. Exploiting the potential of unlabeled endoscopic video data with self-supervised learning. International journal of computer assisted radiology and surgery, 13(6), 925-933. 2018.

[22]M. Arjovsky, S. Chintala, L. Bottou, “Wasserstein gan,” arXiv:1701.07875,2017.

[23]Larsson, G., Maire, M., & Shakhnarovich, G. Colorization as a proxy task for visual understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 6874-6883. 2017.

[24]Liu, H., Liu, J., Hou, S., Tao, T., & Han, J. Perception consistency ultrasound image super-resolution via self-supervised CycleGAN. Neural Computing and Applications, 1-11. 2021.

[25]Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. A simple framework for contrastive learning of visual representations. In International conference on machine learning. 2020, November, pp. 1597-1607.

[26]He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 9729-9738. 2020.

[27]Grill, J. B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., ... & Valko, M. Bootstrap your own latent: A new approach to self-supervised learning. arXiv preprint arXiv:2006.07733. 2020.

[28]Chen, X., & He, K. Exploring Simple Siamese Representation Learning. arXiv preprint arXiv:2011.10566.2020.

[29]Zbontar, J., Jing, L., Misra, I., LeCun, Y., & Deny, S. Barlow Twins: Self-Supervised Learning via Redundancy Reduction. arXiv preprint arXiv:2103.03230. 2021.

[30]Zhang, P., Wang, F., & Zheng, Y. Self-supervised deep representation learning for fine-grained body part recognition. In 2017 IEEE 14th International Symposium on Biomedical Imaging. April ,2017, pp. 578-582. 2017.

[31]Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. vol. 2, pp. 1735–1742. 2006.

[32]Henaff, O.J. Data-efficient image recognition with contrastive predictive coding. In IEEE Conference on Computer Vision and Pattern Recognition,pp. 4182-4192, 2019.

[33]Chaitanya, K., Erdil, E., Karani, N., & Konukoglu, E. Contrastive learning of global and local features for medical image segmentation with limited annotations. arXiv preprint arXiv:2006.10511. 2020.

[34]Oord, A. V. D., Li, Y., & Vinyals, O. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. 2018.

[35]Yan, K., Cai, J., Jin, D., Miao, S., Harrison, A. P., Guo, D., et al. Self-supervised Learning of Pixel-wise Anatomical Embeddings in Radiological Images. arXiv preprint arXiv:2012.02383.2020.

[36]Zhu, J., Li, Y., Hu, Y., & Zhou, S. K. Embedding Task Knowledge into 3D Neural Networks via Self-supervised Learning. arXiv preprint arXiv:2006.05798.2020

[37]Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. vol. 34, pp. 2274–2282,2012.

[38]Sowrirajan, H., Yang, J., Ng, A. Y., & Rajpurkar, P. MoCo-CXR: MoCo Pretraining Improves Representation and Transferability of Chest X-ray Models. E-print arXiv:2010.05352,2020

[39]Sriram, A., Muckley, M., Sinha, K., Shamout, F., Pineau, J., Geras, K. J., ... & Moore, W. (2021). COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction. E-print arXiv:2101.04909,2021.

[40]Vu, Y. N. T., Wang, R., Balachandar, N., Liu, C., Ng, A. Y., & Rajpurkar, P. MedAug: Contrastive learning leveraging patient metadata improves representations for chest X-ray interpretation. arXiv preprint arXiv:2102.10663.2021.

[41]Ciga, O., Martel, A. L., & Xu, T. Self-supervised contrastive learning for digital histopathology. arXiv preprint arXiv:2011.13971. 2020.

[42]Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., et al. Big Self-Supervised Models Advance Medical Image Classification. arXiv preprint arXiv:2101.05224. 2021.

[43]Xie, Y., Zhang, J., Liao, Z., Xia, Y., & Shen, C. (2020). PGL: Prior-Guided Local Self-Supervised Learning for 3D Medical Image Segmentation. arXiv preprint arXiv:2011.12640. 2020.

[44]Yang, Y., & Xu, Z. Rethinking the value of labels for improving class-imbalanced learning. arXiv preprint arXiv:2006.07529. 2020.

[45]Chen, S., Ma, K., & Zheng, Y. Med3d: Transfer learning for 3d medical image analysis. arXiv preprint arXiv:1904.00625. 2019.

[46]Setio, A. A. A., Traverso, A., De Bel, T., Berens, M. S., van den Bogaard, C., Cerello, P., ... & Jacobs, C. Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge. Medical image analysis, 42, 1-13. 2017.

[47]Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv preprint arXiv:1811.02629. 2018.

[48]Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 2536-2544. 2016.

[49]Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P. A., & Bottou, L. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. Journal of machine learning research, 11(12). 2010.

[50]Caron, M., Bojanowski, P., Joulin, A., & Douze, M. Deep clustering for unsupervised learning of visual features. In Proceedings of the European Conference on Computer Vision. pp.132-149. 2018.

[51]Heller, N., Isensee, F., Maier-Hein, K. H., Hou, X., Xie, C., Li, F., et al. The state of the art in kidney and kidney tumor segmentation in contrast-enhanced ct imaging: Results of the kits19 challenge. Medical Image Analysis, 67, 101821. 2021.

[52]Simpson, A. L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., Van Ginneken, B., et al. A large annotated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXiv:1902.09063. 2019.

[53]Gordienko, Y., Gang, P., Hui, J., Zeng, W., Kochura, Y., Alienin, O., ... & Stirenko, S. Deep learning with lung segmentation and bone shadow exclusion techniques for chest X-ray analysis of lung cancer. In International Conference on Computer Science, Engineering and Education Applications. Springer, Cham, pp. 638-647, 2018.

[54]Yang, Y. H., Xu, J. S., Gordienko, Y., & Stirenko, S. Abnormal Interference Recognition Based on Rolling Prediction Average Algorithm. In International Conference on Computer Science, Engineering and Education Applications. Springer, Cham, pp. 306-316. 2020.

[55]Sulema, Y., Kerre, E. and Shkurat, O. Vector Image Retrieval Methods Based on Fuzzy Patterns. International Journal of Modern Education and Computer Science, 12(3), 2020.

[56]Md. Rahat Khan, A. S. M. Shafi, " Statistical Texture Features Based Automatic Detection and Classification of Diabetic Retinopathy", International Journal of Image, Graphics and Signal Processing, Vol.13, No.2, pp. 53-61, 2021.

[57]Farzaneh Nikroorezaei, Somayeh Saraf Esmaili, " Application of Models based on Human Vision in Medical Image Processing: A Review Article", International Journal of Image, Graphics and Signal Processing, Vol.11, No.12, pp. 23-28, 2019.

[58]Kama, R., Chinegaram, K., Tummala, R.B. and Ganta, R.R. Segmentation of Soft Tissues and Tumors from Biomedical Images using Optimized K-Means Clustering via Level Set formulation. International Journal of Intelligent Systems and Applications, 11(9), p.18, 2019.

International Journal of Image, Graphics and Signal Processing (IJIGSP)