1 LIVIA, ILLS, Dept. of Systems Engineering, ÉTS Montréal · 2 Goodman Cancer Institute, McGill University · 3 Dept. of Biochemistry, McGill University · 4 Gerald Bronfman Dept. of Oncology, McGill University
Confocal fluorescence microscopy is one of biology's most powerful windows into the living cell, but every photon comes at a price. Crank up the light for a crisp image and you bleach the dye and damage the specimen. SR-CACO-2 asks whether super-resolution AI can buy back that quality, and gives the field a real benchmark to measure it on.
01 · MotivationThe microscopist's dilemma
Scanning confocal microscopy captures beautiful images from thick, three-dimensional samples, but it relies on intense illumination. That light causes photobleaching (the fluorescent markers fade) and phototoxicity (the cells themselves are harmed), which is a serious problem for live-cell imaging. You can dial the light down to protect the sample, but you pay for it directly in image quality.
Single-image super-resolution (SISR) offers a tempting escape: capture a fast, low-light, low-resolution image and let a model upscale it to a high-resolution one. SISR has been spectacularly successful on natural photos, largely because there are enormous public datasets to train on. Microscopy has had no such luxury, and that scarcity has quietly capped progress.
The few existing SISR microscopy datasets are mostly private, so the methods that could most help biologists are the ones hardest to fairly evaluate.
02 · The datasetWhat SR-CACO-2 provides
SR-CACO-2 is a large scanning-confocal dataset of low- and high-resolution image pairs, captured for three different fluorescent markers on the human epithelial cell line Caco-2 (ATCC HTB-37). It is built to evaluate SISR at three upscaling levels, ×2, ×4 and ×8, using real low-resolution images, not synthetic ones.
From 2,200 unique fields of view captured at four resolutions, the team derived 9,937 patches ready for SISR experiments, and benchmarked 16 state-of-the-art methods spanning the main families of the field. Everything, data, code and pretrained weights, is released under a permissive CC BY-NC-SA 4.0 license.
03 · CaptureBuilt for real diversity
Acquisition was automated for both scale and rigour. Sets of 100 images were stitched into 10 × 10 tiles, so each of the 22 tiles represents 100 unique fields of view, and every image was captured at four resolutions for three markers. Collecting a single tile across all four resolutions takes 12 to 16 hours. The 22 wells were drawn from four independent experiments, giving the dataset genuine biological diversity rather than 22 views of the same thing.
An object-based analysis of the high-resolution fields reveals roughly 16,800 multi-cellular objects. Grown in a 3D protein matrix, the cells form tissue-like cyst structures whose size spans two orders of magnitude, single-layered, multi-layered and solid phenotypes alike, a far richer range than the flat 2D cultures used in many other microscopy datasets.
04 · The key insightReal low-res is not bicubic low-res
Most SISR research fakes its low-resolution inputs by bicubically downscaling the high-resolution image. SR-CACO-2 shows why that shortcut breaks down in microscopy. Comparing real (microscope) low-resolution images against bicubic ones, the differences are stark: real LR is noticeably noisier, it comes from a single scan, while the HR target is averaged over nine, and the two can differ wildly in pixel intensity, especially over the cells themselves. Bicubic LR, by contrast, is unnaturally smooth and even preserves structure it shouldn't.
The takeaway is direct: bicubic LR cannot stand in for real LR. To build SISR models that survive deployment, you have to train and test on the real thing, which is exactly what SR-CACO-2 makes possible.
05 · StructurePatches & distribution
High-resolution fields are tiled into patches, balanced so that training and evaluation see a representative spread of cellular content rather than mostly background.
06 · BenchmarksHow well does SISR work here?
Sixteen leading SISR methods were trained and evaluated across the three scales. The verdict: limited success. Current models struggle to reproduce the high-resolution textures that matter biologically, confirming that SR-CACO-2 is a genuinely challenging problem, not a solved one dressed up as new.
07 · Beyond pixelsDoes it help the biology?
Image quality is only a proxy, what biologists care about is whether they can detect and segment cells reliably. So the benchmark goes further, evaluating super-resolved images on downstream cell-detection and segmentation tasks. Several methods produced promising results, hinting that SISR could meaningfully assist real analysis pipelines even where the raw textures aren't perfect.
08 · TakeawaysWhy it matters
SR-CACO-2 closes a real gap: a public, reproducible benchmark of real LR/HR microscopy pairs across three scales and three markers. Its difficulty is a feature, it gives the community honest ground to improve on. And the payoff reaches beyond fixed imaging: SISR models trained here could one day let scientists film living cells faster and at lower light, opening the door to tracking instantaneous inter-cellular events with far less damage to the cells.
09 · AccessGet the data, poster & slides
SR-CACO-2 is freely available under CC BY-NC-SA 4.0. Grab the data via the download instructions; code and pretrained weights are on GitHub and Hugging Face.
10 · CiteCitation
@inproceedings{belharbi24-sr-caco-2,
title={SR-CACO-2: A Dataset for Confocal Fluorescence
Microscopy Image Super-Resolution},
author={Belharbi, S. and Whitford, M.K.M. and Hoang, P. and
Murtaza, S. and McCaffrey, L. and Granger, E.},
booktitle={NeurIPS},
year={2024}
}Acknowledgments, supported in part by the Canadian Institutes of Health Research, NSERC, and the Digital Research Alliance of Canada.