Skip to main content
SpringerLink
Log in

Image de-fencing framework with hybrid inpainting algorithm

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Detection and removal of fences from digital images become essential when an important part of the scene turns to be occluded by such unwanted structures. Image de-fencing is challenging because manually marking fence boundaries is tedious and time-consuming. In this paper, a novel image de-fencing algorithm that effectively detects and removes fences with minimal user input is presented. The user is only requested to mark few fence pixels; then, color models are estimated and used to train Bayes classifier to segment the fence and the background. Finally, the fence mask is refined exploiting connected component analysis and morphological operators. To restore the occluded region, a hybrid inpainting algorithm is proposed that integrates exemplar-based technique with a pyramid-based interpolation approach. In contrast to previous solutions which work only for regular pattern fences, the proposed technique is able to remove both regular and irregular fences. A large number of experiments are carried out on a wide variety of images containing different types of fences demonstrating the effectiveness of the proposed approach. The proposed approach is also compared with state-of-the-art image de-fencing and inpainting techniques and showed convincing results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. http://vision.cse.psu.edu/data/data.shtml.

References

  1. Arias, P., Facciolo, G., Caselles, V., Sapiro, G.: A variational framework for exemplar-based image inpainting. Int. J. Comput. Vis. 93, 319–347 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  2. Ballester, C., Bertalmio, M., Caselles, V., Sapiro, G., Verdera, J.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  3. Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: SIGGRAPH, pp. 417–424 (2000)

  4. Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. IEEE Trans. Image Process. 12(8), 882–889 (2003)

    Article  Google Scholar 

  5. Bornemann, F., März, T.: Fast image inpainting based on coherence transport. J. Math. Imaging Vis. 28, 259–278 (2007)

    Article  MathSciNet  Google Scholar 

  6. Bruni, V., et al.: Semi-transparent blotches removal from sepia images exploiting visibility laws. Signal Image Video Process. 7(1), 11–26 (2013)

    Article  Google Scholar 

  7. Cheng, W.H., et al.: Robust algorithm for exemplar-based image inpainting. In: Proceeding of International Conference on Computer Graphics, Imaging and Visualization, pp. 64–69 (2005)

  8. Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 721–728 (2003)

  9. Criminisi, A., Perez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

    Article  Google Scholar 

  10. Farid, M., Khan, H.: Image inpainting using dynamic weighted kernels. In: Proceeding of the IEEE International Conference on Computer Science and Information Technology (ICCSIT), vol. 8, pp. 252–255 (2010)

  11. Farid, M., Khan, H., Mahmood, A.: Image inpainting based on pyramids. In: Proceeding of the IEEE International Conference on Signal Processing (ICSP), pp. 711–715 (2010)

  12. Freeman, W., Jones, T., Pasztor, E.: Example-based super-resolution. IEEE Comput. Graph. Appl. 22(2), 56–65 (2002)

    Article  Google Scholar 

  13. Glasner, D., Bagon, S., Irani, M.: Super-resolution from a single image. In: Proceedings of the IEEE ICCV, pp. 349–356 (2009)

  14. Hays, J., Leordeanu, M., Efros, A.A., Liu, Y.: Discovering texture regularity as a higher-order correspondence problem. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 522–535 (2006)

  15. Leordeanu, M., Hebert, M.: A spectral technique for correspondence problems using pairwise constraints. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1482–1489 (2005)

  16. Leung, T.K., Malik, J.: Detecting, localizing and grouping repeated scene elements from an image. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 546–555. Springer (1996)

  17. Liu, Y., Belkina, T., Hays, J., Lublinerman, R.: Image de-fencing. In: Proceedings of the IEEE CVPR, pp. 1–8 (2008)

  18. Liu, Y., Collins, R., Tsin, Y.: A computational model for periodic pattern perception based on frieze and wallpaper groups. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 354–371 (2004)

    Article  Google Scholar 

  19. Lobay, A., Forsyth, D.A.: Shape from texture without boundaries. Int. J. Comput. Vis. 67, 71–91 (2006)

    Article  Google Scholar 

  20. Mirkamali, S., Nagabhushan, P.: Object removal by depth-wise image inpainting. Signal Image Video Process. 9(8), 1785–1794 (2015)

    Article  Google Scholar 

  21. Oliveira, M.M., Bowen, B., Mckenna, R., sung Chang, Y.: Fast digital image inpainting. In: Proceedings of the International Conference Visualization, Imaging and Image Processing (VIIP), pp. 106–107 (2001)

  22. Papafitsoros, K., Schnlieb, C.: A combined first and second order variational approach for image reconstruction. J. Math. Imaging Vis. 48(2), 308–338 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  23. Park, M., et al.: Deformed lattice detection in real-world images using mean-shift belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1804–1816 (2009)

    Article  Google Scholar 

  24. Park, M., Brocklehurst, K., Collins, R.T., Liu, Y.: Image de-fencing revisited. In: Proceedings of the Asian Conference on Computer Vision (ACCV), pp. 422–434 (2011)

  25. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990)

    Article  Google Scholar 

  26. Serra, J.: Image Analysis and Mathematical Morphology. Academic Press Inc., Orlando (1983)

    Google Scholar 

  27. Shen, J., Chan, T.F.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  28. Tian, J., Ma, K.K.: A survey on super-resolution imaging. Signal Image Video Process. 5(3), 329–342 (2011)

    Article  MathSciNet  Google Scholar 

  29. Tuytelaars, T., Turina, A., Van Gool, L.: Noncombinatorial detection of regular repetitions under perspective skew. IEEE Trans. Pattern Anal. Mach. Intell. 25, 418–432 (2003)

    Article  Google Scholar 

  30. Wu, J., Ruan, Q.: Object removal by cross isophotes exemplar-based inpainting. Proc. Int. Conf. Pattern Recognit. (ICPR) 3, 810–813 (2006)

    Google Scholar 

  31. Xu, Z., Sun, J.: Image inpainting by patch propagation using patch sparsity. IEEE Trans. Image Process. 19, 1153–1165 (2010)

    Article  MathSciNet  Google Scholar 

  32. Yang, C.K., Yeh, Y.C.: Stain removal in 2D images with globally varying textures. Signal Image Video Process. 8(7), 1373–1382 (2014)

    Article  Google Scholar 

  33. Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19(11), 2861–2873 (2010)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dipartimento di Informatica, Università degli Studi di Torino, Torino, Italy

    Muhammad Shahid Farid & Marco Grangetto

  2. Punjab University College of Information Technology, University of the Punjab, Lahore, Pakistan

    Muhammad Shahid Farid

  3. Department of Computer Science and Engineering, College of Engineering, Qatar University, Doha, Qatar

    Arif Mahmood

Authors
  1. Muhammad Shahid Farid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arif Mahmood
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco Grangetto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Muhammad Shahid Farid.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Farid, M.S., Mahmood, A. & Grangetto, M. Image de-fencing framework with hybrid inpainting algorithm. SIViP 10, 1193–1201 (2016). https://doi.org/10.1007/s11760-016-0876-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-016-0876-7

Keywords

Search

Navigation