Content-aware image restoration - pushing the limits of fluorescence microscopy

Authors: Martin Weigert, Uwe Schmidt, Tobias Boothe, Andreas Müller, Alexandr Dibrov, Akanksha Jain, Benjamin Wilhelm, Deborah Schmidt, Coleman Broaddus, Siân Culley, Mauricio Rocha-Martins, Fabián Segovia-Miranda, Caren Norden, Ricardo Henriques, Marino Zerial, Michele Solimena, Jochen Rink, Pavel Tomancak, Loic Royer, Florian Jug, Eugene W. Myers
Technologies: CARE, NanoJ-SQUIRREL and NanoJ-SRRF
Paper published in Nature Methods, November 2018
Publisher: Nature Publishing Group US New York

The manuscript "Content-aware image restoration: pushing the limits of fluorescence microscopy" by Weigert et al. introduces CARE (content-aware image restoration) networks, a new approach for restoring fluorescence microscopy data. CARE networks enable the investment of saved photon budget into improvements of acquisition parameters, such as speed, photo-toxicity, isotropy, and resolution. The researchers developed strategies to generate synthetic training data and applied common convolutional neural network architectures to various image restoration tasks, including denoising, surface projection, and the restoration of sub-diffraction structures. The results demonstrate that CARE networks produce high-quality restorations, even with very low SNR images, and significantly improve restoration error compared to several classical denoising methods. The manuscript also discusses the limitations of CARE networks and their potential impact on downstream image analysis, such as segmentation and tracking of cells in Drosophila and Tribolium. The manuscript provides examples of image restoration experiments using deep learning techniques, including the use of zebrafish embryos, mouse liver imaging, and rat insulin-secreting beta cells. The methods used include tensorflow, keras, and Fiji/ImageJ, among others, and the software and training data are made available for reproduction.