open access publication

Article, 2024

SuperCUT, an unsupervised multimodal image registration with deep learning for biomedical microscopy

Briefings in Bioinformatics, ISSN 1477-4054, 1467-5463, Volume 25, 2, Page bbae029, 10.1093/bib/bbae029

Contributors

Grexa, Istvan [1] [2] Iván, Zsanett Zsófia [1] [2] Migh, Ede 0000-0002-8312-6463 [2] Kovács, Ferenc [2] [3] Bolck, Hella Anna 0000-0001-5157-0490 [4] Zheng, Xiang 0000-0001-7588-825X [5] Mund, Andreas 0000-0002-7843-5341 [5] Moshkov, Nikita 0000-0002-5823-4884 [2] Miczán, Vivien 0000-0003-0457-4899 [2] Koos, Krisztian [2] Horvath, Peter (Corresponding author) [2] [3] [6] [7]

Affiliations

  1. [1] University of Szeged
    2. [NORA names: Hungary; Europe, EU; OECD];
  2. [2] Biological Research Centre
    3. [NORA names: Hungary; Europe, EU; OECD];
  3. [3] Single-Cell Technologies Ltd, Temesvári körút 62, Szeged 6726, Hungary.
    4. [NORA names: Hungary; Europe, EU; OECD];
  4. [4] University Hospital of Zurich
    5. [NORA names: Switzerland; Europe, Non-EU; OECD];
  5. [5] University of Copenhagen
    6. [NORA names: KU University of Copenhagen; University; Denmark; Europe, EU; Nordic; OECD];

Abstract

Numerous imaging techniques are available for observing and interrogating biological samples, and several of them can be used consecutively to enable correlative analysis of different image modalities with varying resolutions and the inclusion of structural or molecular information. Achieving accurate registration of multimodal images is essential for the correlative analysis process, but it remains a challenging computer vision task with no widely accepted solution. Moreover, supervised registration methods require annotated data produced by experts, which is limited. To address this challenge, we propose a general unsupervised pipeline for multimodal image registration using deep learning. We provide a comprehensive evaluation of the proposed pipeline versus the current state-of-the-art image registration and style transfer methods on four types of biological problems utilizing different microscopy modalities. We found that style transfer of modality domains paired with fully unsupervised training leads to comparable image registration accuracy to supervised methods and, most importantly, does not require human intervention.

Keywords

SuperCut, acceptable solution, accuracy, accurate registration, analysis, analysis process, annotated data, biological problems, biological samples, biomedical microscopy, comprehensive evaluation, computer, computer vision tasks, correlation analysis, correlation analysis process, data, deep learning, domain, evaluation, experts, human intervention, image registration, image registration accuracy, images, imaging modalities, imaging techniques, inclusion, information, intervention, learning, method, microscopy, microscopy modalities, modal domain, modalities, molecular information, multimodal image registration, multimodal imaging, pipeline, problem, process, registration, registration accuracy, registration method, registration of multimodal images, resolution, samples, solution, style, style transfer, style transfer method, supervised methods, task, technique, training, transfer method, unsupervised pipeline, unsupervised training, vision tasks

Funders

  • Hungarian Scientific Research Fund
  • National Research, Development and Innovation Office
  • Cancer Society of Finland
  • Hungarian Academy of Sciences
  • Novo Nordisk Foundation
  • Novo Nordisk (Denmark)
  • Sigrid Jusélius Foundation

Data Provider: Digital Science

LINKS
-

Matching Records in NORA

SUBJECTS
+

METRICS
+