Content-aware frame interpolation (CAFI): Deep Learning-based temporal super-resolution for fast bioimaging

Session

Poster Session

Authors

David Gaboriau (2), Martin Priessner (2), Arlo Sheridan (1), Tchern Lenn (5), Carlos Garzon-Coral (3), Alexander Dunn (4), Aiden Tousley (4), Robbie Majzner (4), Jonathan Chubb (5), Uri Manor (6), Ramon Vilar (2), Romain Laine (5)

Affiliations

1. E11 Bio
2. Imperial College London
3. Roche
4. Stanford
5. UCL
6. UCSD

Keywords

Image Segmentation, Object Tracking, AI & Machine/Deep Learning, New Tools, Open Science, Web/Cloud Interfaces

Abstract text

The development of high-resolution microscopes has made it possible to investigate cellular processes in 4D (3D over time). However, observing fast cellular dynamics remains challenging as a consequence of photobleaching and phototoxicity. These issues become increasingly problematic with the depth of the volume acquired and the speed of the biological events of interest. Here, we report the implementation of two content-aware frame interpolation (CAFI) deep learning networks, Zooming SlowMo (ZS) and Depth-Aware Video Frame Interpolation (DAIN), that are highly suited for accurately predicting images in between image pairs, therefore improving the temporal resolution of image series (tCAFI) as a post-acquisition analysis step. CAFI also works in the axial dimension (zCAFI), and in combination of temporal and axial dimensions (tzCAFI). We show that CAFI predictions are capable of understanding the motion context of biological structures to perform better than standard interpolation methods. We benchmark CAFI’s performance on twelve different datasets, obtained from four different light microscopy modalities (point-scanning confocal, spinning disk confocal, lattice light sheet and confocal brightfield microscopy) and on an electron microscopy dataset. We demonstrate its capabilities for single-particle tracking methods applied to the study of lysosome trafficking and on segmentation experiments of labelled nuclei. CAFI therefore allows for reduced light exposure and phototoxicity on the sample and extends the possibility of long-term live-cell imaging. Both DAIN and ZS as well as the training and testing data are made available for use by the wider community via the ZeroCostDL4Mic platform.