Abstract
Image registration plays an imperative part of multimodal video analysis system. In video surveillance applications, change in the environmental conditions makes the registration process hard. Use of multiple sensors makes the system more robust to environmental changes as compared to single sensor imaging system. Using multiple modalities such as infrared(IR)/thermal sensors and CMOS image sensors augment the sturdiness of the surveillance system. Here we propose hardware implementation of feature detection on Genesys 2 Kintex-7 FPGA for a multimodal surveillance system, which is robust in poor lighting conditions and affine changes. To reduce the processing time, a region of interest (ROI) is identified and feature extraction is performed in this region. Design optimization in hardware architecture resulted in achieving the real-time performance of image registration on HD 720p video.
Similar content being viewed by others
References
Dıaz, S., Soto, J.E., Inostroza, F., Godoy, S.E., Figueroa, M.: An embedded system for image segmentation and multimodal registration in noninvasive skin cancer screening. In: 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju Island, South Korea, South Korea (2017)
Davis, J.W., Sharma, V.: Background-subtraction using contour-based fusion of thermal and visible imagery. Comput. Vis. Image Underst. 106(2–3), 162–182 (2007)
Inostroza, F., Cárdenas, J., Godoy, S.E.: Embedded multimodal registration of visible images on long-wave infrared video in real time. In: Euromicro Conference on Digital System Design (DSD), Limassol, Cyprus (2016)
Jie, Z.: A novel image registration algorithm using SIFT feature descriptors. In: International Conference on Smart City and Systems Engineering, Hunan, China (2016)
Yang, Z., Cohen, F.S.: Image registration and object recognition using affine invariants and convex hulls. IEEE Trans. Image Process. 8(7), 934–946 (1999)
Zitová, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)
Zhu, Y., Cheng, S., Stankovic´, V., Stankovic, L.: Image registration using BP-SIFT. J. Vis. Commun. Image Represent. 24(4), 448–457 (2013)
Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24(4), 325–376 (1992)
Claus, C., Huitl, R., Rausch, J., Stechele, W.: Optimizing the SUSAN corner detection algorithm for a high speed FPGA implementation. In: FPL 2009. International Conference on Field Programmable Logic and Applications, Prague, Czech Republic (2009)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. 60(2), 91–110 (2004)
David, J.W.: Dataset 03: OSU Color-Thermal Database, OTCBVS Benchmark Dataset Collection. (2004). http://vcipl-okstate.org/pbvs/bench/. Accessed 1 Sep 2017
Vourvoulakis, J., Kalomiros, J., Lygouras, J.: Fully pipelined FPGA-based architecture for real-time SIFT extraction. Microprocess. Microsyst. 40, 53–73 (2015)
Mizuno, K., Kamino, T., Ariki, Y.: FPGA based accelerated orientation calculation in SIFT using LUTs. In: A Low-Power Real-Time SIFT Descriptor Generation Engine for Full-HDTV Video Recognition, vol. E94, no. C, pp. 448–457 (2013)
Wang, J., Zhong, S., Yan, L., Cao, Z.: An embedded system-on-chip architecture for real-time visual detection and matching. In: IEEE Transactions on Circuits and Systems for Video Technology (2013)
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Nandalike, R., Sarojadevi, H. Multimodal image feature detection with ROI-based optimization for image registration. J Real-Time Image Proc 17, 1007–1013 (2020). https://doi.org/10.1007/s11554-018-0847-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-018-0847-z