CalibFPA: a focal plane array imaging system based on online deep learning calibration

Abstract

Compressive focal plane arrays (FPA) enable cost-effective high-resolution (HR) imaging by acquisition of several multiplexed measurements on a low-resolution (LR) sensor. Multiplexed encoding of the visual scene is often attained via electronically controllable spatial light modulators (SLM). To capture system non-idealities such as optical aberrations, a system matrix is measured via additional offline scans, where the system response is recorded for a point source at each spatial location on the imaging grid. An HR image can then be reconstructed by solving an inverse problem that involves encoded measurements and the calibration matrix. However, this offline calibration framework faces limitations due to challenges in encoding single HR grid locations with a fixed coded aperture, lengthy calibration scans repeated to account for system drifts, and computational burden of reconstructions based on dense system matrices. Here, we propose a novel compressive FPA system based on online deep-learning calibration of multiplexed LR measurements (CalibFPA). To acquire multiplexed measurements, we devise an optical setup where a piezo-stage locomotes a pre-printed fixed coded aperture. We introduce a physics-driven deep-learning method to correct for the influences of optical aberrations in multiplexed measurements without the need for offline calibration scans. The corrected measurement matrix is of block-diagonal form, so it can be processed efficiently to recover HR images with a user-preferred reconstruction algorithm including least-squares, plug-and-play, or unrolled techniques. On simulated and experimental datasets, we demonstrate that CalibFPA outperforms state-of-the-art compressive FPA methods. We also report analyses to validate the design elements in CalibFPA and assess computational complexity.