Skip to main content
Log in

Multimodal image feature detection with ROI-based optimization for image registration

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

Image registration plays an imperative part of multimodal video analysis system. In video surveillance applications, change in the environmental conditions makes the registration process hard. Use of multiple sensors makes the system more robust to environmental changes as compared to single sensor imaging system. Using multiple modalities such as infrared(IR)/thermal sensors and CMOS image sensors augment the sturdiness of the surveillance system. Here we propose hardware implementation of feature detection on Genesys 2 Kintex-7 FPGA for a multimodal surveillance system, which is robust in poor lighting conditions and affine changes. To reduce the processing time, a region of interest (ROI) is identified and feature extraction is performed in this region. Design optimization in hardware architecture resulted in achieving the real-time performance of image registration on HD 720p video.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Dıaz, S., Soto, J.E., Inostroza, F., Godoy, S.E., Figueroa, M.: An embedded system for image segmentation and multimodal registration in noninvasive skin cancer screening. In: 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju Island, South Korea, South Korea (2017)

  2. Davis, J.W., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 106(2–3), 162–182 (2007)

    Article  Google Scholar 

  3. Inostroza, F., Cárdenas, J., Godoy, S.E.: Embedded multimodal registration of visible images on long-wave infrared video in real time. In: Euromicro Conference on Digital System Design (DSD), Limassol, Cyprus (2016)

  4. Jie, Z.: A novel image registration algorithm using SIFT feature descriptors. In: International Conference on Smart City and Systems Engineering, Hunan, China (2016)

  5. Yang, Z., Cohen, F.S.: Image registration and object recognition using affine invariants and convex hulls. IEEE Trans. Image Process. 8(7), 934–946 (1999)

    Article  MathSciNet  Google Scholar 

  6. Zitová, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)

    Article  Google Scholar 

  7. Zhu, Y., Cheng, S., Stankovic´, V., Stankovic, L.: Image registration using BP-SIFT. J. Vis. Commun. Image Represent. 24(4), 448–457 (2013)

    Article  Google Scholar 

  8. Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24(4), 325–376 (1992)

    Article  Google Scholar 

  9. Claus, C., Huitl, R., Rausch, J., Stechele, W.: Optimizing the SUSAN corner detection algorithm for a high speed FPGA implementation. In: FPL 2009. International Conference on Field Programmable Logic and Applications, Prague, Czech Republic (2009)

  10. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. 60(2), 91–110 (2004)

  11. David, J.W.: Dataset 03: OSU Color-Thermal Database, OTCBVS Benchmark Dataset Collection. (2004). http://vcipl-okstate.org/pbvs/bench/. Accessed 1 Sep 2017

  12. Vourvoulakis, J., Kalomiros, J., Lygouras, J.: Fully pipelined FPGA-based architecture for real-time SIFT extraction. Microprocess. Microsyst. 40, 53–73 (2015)

    Article  Google Scholar 

  13. Mizuno, K., Kamino, T., Ariki, Y.: FPGA based accelerated orientation calculation in SIFT using LUTs. In: A Low-Power Real-Time SIFT Descriptor Generation Engine for Full-HDTV Video Recognition, vol. E94, no. C, pp. 448–457 (2013)

  14. Wang, J., Zhong, S., Yan, L., Cao, Z.: An embedded system-on-chip architecture for real-time visual detection and matching. In: IEEE Transactions on Circuits and Systems for Video Technology (2013)

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Rajesh Nandalike.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nandalike, R., Sarojadevi, H. Multimodal image feature detection with ROI-based optimization for image registration. J Real-Time Image Proc 17, 1007–1013 (2020). https://doi.org/10.1007/s11554-018-0847-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-018-0847-z

Keywords

Navigation