Open access
Author
Date
2024Type
- Doctoral Thesis
ETH Bibliography
yes
Altmetrics
Abstract
Hyperspectral images (HSIs) are useful in numerous domains. The key obstacle to fully unleashing the potential of HSIs is the challenge of data acquisition -- acquiring HSIs of high spatial and high spectral resolution at a high frame rate is a grand challenge. This thesis addresses the challenge from two perspectives: 1) developing data-efficient HSI super-resolution (SR) methods and 2) developing a single-shot hyperspectral imaging method by jointly learning color filter array and spectral recovery. In particular, the thesis has made three contributions.
Firstly, we have observed that HSI SR and RGB image SR are correlated and have developed a novel multi-tasking network to train them jointly so that the auxiliary task RGB image SR can provide additional supervision. With this contribution, our method is able to learn from heterogeneous datasets and lift the requirement of having a large amount of high-resolution HSI training samples. Extensive experiments on four standard datasets show that our HSI SR method outperforms existing methods significantly.
Secondly, we have developed a novel test-time domain adaptation method for HSI SR. In this setting, a model trained on rather limited training data can further adapt itself to generalize to new and different data during testing. We propose to adapt by test-time model tuning with synthesized pseudo samples. The novel sample synthesis method is designed based on the teacher-student learning framework and self-training paradigm, which can generate authentic LR-to-HR relationship and accurate pseudo ground-truth. Our test-time training method can boost the performance of HSI SR by a large margin.
Lastly, we have developed a new method that simultaneously addresses the problem of spectral band selection, color filter design, image demosaicing, and spectral image recovery in a joint learning framework for single-shot HS imaging. We have proposed a reinforcement learning based method for spectral band selection and a novel neural network for CFA generation, image demosaicing, and HS image recovery. Our final method delivers a simple setup -- as simple as an RGB camera -- for HS imaging. Experimental results show the potential of our method for fast and low-cost HS imaging.
Overall, this thesis has made three contributions on efficient hyperspectral imaging -- the first two on methods to super-resolve low-resolution HSI images and the last one on learning a new color filter array for efficient hyperspectral imaging. They have been extensively evaluated on multiple datasets and have shown good results. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000671284Publication status
publishedExternal links
Search print copy at ETH Library
Contributors
Examiner: Gool, Luc Van
Examiner: Süsstrunk, Sabine
Examiner: T. Tan, Robby
Examiner: Li, Zhen
Publisher
ETH ZurichSubject
Hyperspectral Imaging, Deep Learning, Hyperspectral Image Super-resolutionOrganisational unit
03514 - Van Gool, Luc / Van Gool, Luc
More
Show all metadata
ETH Bibliography
yes
Altmetrics