Skip to main content
Log in

Hyperspectral image super-resolution using recursive densely convolutional neural network with spatial constraint strategy

  • Extreme Learning Machine and Deep Learning Networks
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Hyperspectral images (HSIs) have been widely applied in real life, such as remote sensing, geological exploration, and so on. Many deep networks have been proposed to raise the resolution of HSIs for their better applications. But training their huge number of model parameters (weights and biases) needs more memory for storage and computation, which may bring some difficulties when they are applied in mobile terminal devices. In order to condense the deep networks and still keep the reconstruction effect, this paper proposes a compact deep network for HSI super-resolution (SR) by fusing the idea of recursion, dense connection, and spatial constraint (SCT) strategy. We name this method as recursive densely convolutional neural network with a spatial constraint strategy (SCT-RDCNN). The proposed method uses a novel designed recursive densely convolutional neural network (RDCNN) to learn the mapping relation between the low-resolution (LR) HSI and the high-resolution (HR) HSI and then adopts the SCT to improve the determined HR HSI. Compared with some existing deep-network-based HSI SR methods, the proposed method can use much less parameters (weight and bias) to attain or exceed the performance of methods with similar convolution layers because of the recursive structure and dense connection. It is significant and meaningful for the practical applications of the network in HSI SR due to the limitations of hardware devices. Some experiments on three HSI databases illustrate that our proposed SCT-RDCNN method outperforms several state-of-the-art HSI SR methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Asadzadeh S, de Souza CR (2016) A review on spectral processing methods for geological remote sensing. Int J Appl Earth Obs Geoinform 47:69–90

    Article  Google Scholar 

  2. Uzair M, Mahmood A, Mian A (2013) Hyperspectral face recognition using 3D-DCT and partial least squares. In: 2013 Electronic proceedings of the British machine vision conference (BMVC), vol 57, pp 1–10

  3. Tarabalka Y, Chanussot J, Benediktsson JA (2010) Segmentation and classification of hyperspectral images using minimum spanning forest grown from automatically selected markers. IEEE Trans Syst Man Cybern Part B Cybern 40(5):1267–1279

    Article  Google Scholar 

  4. Mei SH, Yuan X, Ji JY, Zhang YF, Wan S, Du Q (2017) Hyperspectral image spatial super-resolution via 3D full convolutional neural network. Remote Sens 9(11):1–22

    Article  Google Scholar 

  5. Li YS, Hu J, Zhao X, Xie WY, Li JJ (2017) Hyperspectral image super-resolution using deep convolutional neural network. Neurocomputing 266:29–41

    Article  Google Scholar 

  6. Carper WJ, Lillesand TM, Kiefer RW (1990) The use of intensity-hue-saturation transformations for merging SPOT panchromatic and multispectral image data. Photogram Eng Remote Sens 56(4):459–467

    Google Scholar 

  7. Shettigara VK (1992) A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set. Photogram Eng Remote Sens 58(5):561–567

    Google Scholar 

  8. Nunez J, Otazu X, Fors O, Prades A, Pala V, Arbiol R (1999) Multiresolution-based image fusion with additive wavelet decomposition. IEEE Trans Geosci Remote Sens 37(3):1204–1211

    Article  Google Scholar 

  9. Aiazzi B, Baronti S, Lotti F, Selva M (2009) A comparison between global and context-adaptive pansharpening of multispectral images. IEEE Geosci Remote Sens Lett 6(2):302–306

    Article  Google Scholar 

  10. Zhukov B, Oertel D, Lanzl F, Reinhackel G (1999) Unmixing-based multisensor multiresolution image fusion. IEEE Trans Geosci Remote Sens 37(3):1212–1226

    Article  Google Scholar 

  11. Minghelli-Roman A, Polidori L, Mathieu-Blanc S, Loubersac L, Cauneau F (2006) Spatial resolution improvement by merging MERIS-ETM images for coastal water monitoring. IEEE Geosci Remote Sens Lett 3(2):227–231

    Article  Google Scholar 

  12. Zurita-Milla R, Clevers JGPW, Schaepman ME (2008) Unmixing based landsat TM and MERIS FR data fusion. IEEE Geosci Remote Sens Lett 5(3):453–457

    Article  Google Scholar 

  13. Kopf J, Cohen MF, Lischinski D, Uyttendaele M (2007) Joint bilateral upsampling. ACM Trans Gr 26(3):1–8

    Article  Google Scholar 

  14. Candès E J (2006) Compressive sampling. In: Proceedings of the international congress of mathematicians (ICM 2006). vol 3, pp 1433–1452

  15. Zhao YQ, Yang JX, Zhang QY, Song L, Cheng YM, Pan Q (2011) Hyperspectral imagery super-resolution by sparse representation and spectral regularization. EURASIP J Adv Signal Process 1(87):1–10

    Google Scholar 

  16. Simoes M, Bioucas-Dias J, Almeida LB, Chanussot J (2015) A convex formulation for hyperspectral image super resolution via subspace-based regularization. IEEE Trans Geosci Remote Sens 53(6):3373–3388

    Article  Google Scholar 

  17. Dong WS, Fu FZ, Shi GM, Cao X, Wu JJ, Li GY, Li X (2016) Hyperspectral image super-resolution via non-negative structured sparse representation. IEEE Trans Image Process 25(5):2337–2352

    Article  MathSciNet  Google Scholar 

  18. Sun Y, Chen YH, Wang XG, Tang XO (2014) Deep learning face representation by joint identification verification. In: Advances in neural information processing systems (NIPS). pp 2337–2352

  19. Donahue J, Hendricks LA, Rohrbach M, Venugopalan S, Guadarrama S, Saenko K, Darrell T (2017) Long-term recurrent convolutional networks for visual recognition and description. IEEE Trans Pattern Anal Mach Intell 39(4):677–691

    Article  Google Scholar 

  20. Yang CY, Ma C, Yang MH, Yang MH (2014) Single-image super-resolution: a benchmark. In: Proceedings of the 13th European conference on computer vision (ECCV 2014). Springer, Berlin, pp 372–386

  21. Yuan Y, Zheng XT, Lu XQ (2017) Hyperspectral image superresolution by transfer learning. IEEE J Sel Top Appl Earth Obs Remote Sens 10(5):1963–1974

    Article  Google Scholar 

  22. Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  23. He Z, Liu L (2018) Hyperspectral image super-resolution inspired by deep Laplacian pyramid network. Remote Sens 10(1939):1–24

    Google Scholar 

  24. Srivastava R K, Greff K, Schmidhuber J (2015) Training very deep networks. In: Advances in neural information processing systems (NIPS 2015). vol 28, pp 2377–2385

  25. Huang G, Liu Z, Van DML, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the 30th IEEE conference on computer vision and pattern recognition (CVPR 2017). pp 2264–2269

  26. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision (ECCV 2016). pp 630–645

  27. Kim J, Lee JW, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the 29th IEEE conference on computer vision and pattern recognition (CVPR 2016). pp 1637–1645

  28. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  29. Yasuma F, Mitsunaga T, Iso D, Nayar SK (2010) Generalized assorted pixel camera: postcapture control of resolution, fynamic range, and spectrum. IEEE Trans Image Process 19(9):2241–2253

    Article  MathSciNet  Google Scholar 

  30. Foster DH, Amano K, Nascimento SMC, Foster MJ (2006) Frequency of metamerism in natural scenes. J Opt Soc Am A 23(10):2359–2372

    Article  Google Scholar 

  31. Chakrabarti A, Zickler T (2011) Statistics of real-world hyperspectral images. In: Proceedings of the 24th IEEE conference on computer vision and pattern recognition (CVPR 2011). IEEE, pp 193–200

  32. Wycoff E, Chan TH, Jia K, Ma WK, Ma Y (2013) A non-negative sparse promoting algorithm for high resolution hyperspectral imaging. In: 2013 IEEE international conference on acoustics, speech, and signal processing (ICASSP). Institute of Electrical and Electronics Engineers Inc, Piscataway, pp 1409–1413

  33. Yokoya N, Yairi T, Iwasaki A (2012) Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion. IEEE Trans Geosci Remote Sens 50(2):528–537

    Article  Google Scholar 

Download references

Acknowledgements

This work was funded by the National Natural Science Foundation of China (61571410) and the Zhejiang Provincial Nature Science Foundation of China (LY18F020018 and LSY19F020001).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jianwei Zhao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Research involving human participants and/or animals

This study did not involve human participants and animals.

Informed consent

The all authors of this paper have consented to the submission.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Huang, T. & Zhou, Z. Hyperspectral image super-resolution using recursive densely convolutional neural network with spatial constraint strategy. Neural Comput & Applic 32, 14471–14481 (2020). https://doi.org/10.1007/s00521-019-04484-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04484-3

Keywords

Navigation