Skip to main content
SpringerLink
Log in

MOONLIT: momentum-contrast and large-kernel for multi-fine-grained deraining

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Deep learning-based methods have achieved excellent performance in image-deraining tasks. Unfortunately, most existing deraining methods incorrectly assume a uniform rain streak distribution and a fixed fine-grained level. And this uncertainty of rain streaks will result in the model not being competent at repairing all fine-grained rain streaks. In addition, some existing convolution-based methods extend the receptive field mainly by stacking convolution kernels, which frequently results in inaccurate feature extraction. In this work, we propose momentum-contrast and large-kernel for multi-fine-grained deraining network (MOONLIT). To address the problem that the model is not competent at all fine-grained levels, we use the unsupervised dictionary contrastive learning method to treat different fine-grained rainy images as different degradation tasks. Then, to address the problem of inaccurate feature extraction, we carefully constructed a restoration network based on large-kernel convolution with a larger and more accurate receptive field. In addition, we designed a data enhancement method to weaken features other than rain streaks in order to be better classified for different degradation tasks. Extensive experiments on synthetic and real-world deraining datasets show that the proposed method MOONLIT achieves the state-of-the-art performance on some datasets. Code is available at https://github.com/awhitewhale/moonlit.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Availability of data and materials

The code generated during and analysed during the current study are available in the github repository, https://github.com/awhitewhale/moonlit.

References

  1. Pal SK, Pramanik A, Maiti J, Mitra P (2021) Deep learning in multi-object detection and tracking: state of the art. App Intell 51(9):6400–6429

    Article  Google Scholar 

  2. Lin G, Milan A, Shen C, Reid I (2017) Refinenet: Multi-path refinement networks for high-resolution semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1925–1934

  3. Chen X, Pan J, Jiang K, Li Y, Huang Y, Kong C, Dai L, Fan Z (2022) Unpaired deep image deraining using dual contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2017–2026

  4. Li M, Xie Q, Zhao Q, Wei W, Gu S, Tao J, Meng D (2018) Video rain streak removal by multiscale convolutional sparse coding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 6644–6653

  5. Jiang K, Wang Z, Yi P, Chen C, Huang B, Luo Y, Ma J, Jiang J (2020) Multi-scale progressive fusion network for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8346–8355

  6. Du Y, Xu J, Zhen X, Cheng M-M, Shao L (2020) Conditional variational image deraining. IEEE Tran Image Process 29:6288–6301

    Article  MATH  Google Scholar 

  7. Rai SN, Saluja R, Arora C, Balasubramanian VN, Subramanian A, Jawahar C (2022) Fluid: Few-shot self-supervised image deraining. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp 3077–3086

  8. Deng S, Wei M, Wang J, Feng Y, Liang L, Xie H, Wang FL, Wang M (2020) Detail-recovery image deraining via context aggregation networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 14560–14569

  9. Wang P, Zhu H (2021) Single-image de-raining using joint filter and multi-scale deep alternate-connection dense network. Neurocomputing 457:306–321

    Article  Google Scholar 

  10. Yasarla R, Sindagi VA, Patel VM (2020) Syn2real transfer learning for image deraining using gaussian processes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2726–2736

  11. Tan F, Qian Y, Kong Y, Zhang H, Zhou D, Fan Y, Chen L (2021) Dbswin: transformer based dual branch network for single image deraining. J Intell Fuzzy Syst (Preprint), 1–15

  12. Wang S, Liu Y, Qing Y, Wang C, Lan T, Yao R (2020) Detection of insulator defects with improved resnest and region proposal network. IEEE Access 8:184841–184850. https://doi.org/10.1109/ACCESS.2020.3029857

    Article  Google Scholar 

  13. Liang H, Ji W, Wang R, Ma Y, Chen J, Chen M (2022) A scene-dependent sound event detection approach using multi-task learning. IEEE Sens J 22(18):17483–17489. https://doi.org/10.1109/JSEN.2021.3098325

    Article  Google Scholar 

  14. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  15. Yang W, Wang S, Xu D, Wang X, Liu J (2020) Towards scale-free rain streak removal via self-supervised fractal band learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 12629–12636

  16. Wang C, Xing X, Wu Y, Su Z, Chen J (2020) Dcsfn: deep cross-scale fusion network for single image rain removal. In: Proceedings of the 28th ACM International Conference on Multimedia, pp 1643–1651

  17. Chen C, Li H (2021) Robust representation learning with feedback for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 7742–7751

  18. Lin X, Huang Q, Huang W, Tan X, Fang M, Ma L (2021) Single image deraining via detail-guided efficient channel attention network. Comput Graph 97:117–125

    Article  Google Scholar 

  19. Zhang J, Pan J, Ren J, Song Y, Bao L, Lau RW, Yang M-H (2018) Dynamic scene deblurring using spatially variant recurrent neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 2521–2529

  20. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2020) Residual dense network for image restoration. IEEE Trans Pattern Anal Machine Intell 43(7):2480–2495

    Article  Google Scholar 

  21. Yuntong Y, Changfeng Y, Yi C, Lin Z, Xile Z, Luxin Y, Yonghong T (2022) Unsupervised deraining: where contrastive learning meets self-similarity. arXiv preprint arXiv:2203.11509

  22. Liu Y, Yue Z, Pan J, Su Z (2021) Unpaired learning for deep image deraining with rain direction regularizer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 4753–4761

  23. Zou W, Wang Y, Fu X, Cao Y (2022) Dreaming to prune image deraining networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 6023–6032

  24. Yi Q, Li J, Dai Q, Fang F, Zhang G, Zeng T (2021) Structure-preserving deraining with residue channel prior guidance. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 4238–4247

  25. Xiao J, Zhou M, Fu X, Liu A, Zha Z-J (2021) Improving de-raining generalization via neural reorganization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 4987–4996

  26. Zhang H, Sindagi V, Patel VM (2019) Image de-raining using a conditional generative adversarial network. IEEE Trans Circuits Syst Video Technol 30(11):3943–3956

    Article  Google Scholar 

  27. Mishra S, Shah A, Bansal A, Choi J, Shrivastava A, Sharma A, Jacobs D (2020) Learning visual representations for transfer learning by suppressing texture. arXiv preprint arXiv:2011.01901

  28. Ding X, Zhang X, Han J, Ding G (2022) Scaling up your kernels to 31x31: revisiting large kernel design in CNNS. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 11963–11975

  29. Zhang H, Patel VM (2018) Density-aware single image de-raining using a multi-stream dense network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 695–704

  30. Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J (2017) Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3855–3863

  31. Wang T, Yang X, Xu K, Chen S, Zhang Q, Lau RW (2019) Spatial attentive single-image deraining with a high quality real rain dataset. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 12270–12279

  32. Yang H, Zhou D, Cao J, Zhao Q, Li M (2022) Rainformer: a pyramid transformer for single image deraining. J Supercomput. https://doi.org/10.1007/s11227-022-04895-5

    Article  Google Scholar 

  33. Shen H, Zhao Z-Q, Liao W, Tian W, Huang D-S (2022) Joint operation and attention block search for lightweight image restoration. Pattern Recognit 132:108909

    Article  Google Scholar 

  34. Gao F, Mu X, Ouyang C, Yang K, Ji S, Guo J, Wei H, Wang N, Ma L, Yang B (2022) Mltdnet: an efficient multi-level transformer network for single image deraining. Neural Comput Appl 34:14013–14027

    Article  Google Scholar 

  35. Yasarla R, Patel VM (2020) Confidence measure guided single image de-raining. IEEE Trans Image Process 29:4544–4555

    Article  MATH  Google Scholar 

  36. Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp 1597–1607. PMLR

  37. He K, Fan H, Wu Y, Xie S, Girshick R (2020) Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 9729–9738

  38. Wang C, Shen Q, Wang X, Jiang G (2022) Momentum feature comparison network based on generative adversarial network for single image super-resolution. Signal Proces Image Commun 106:116726

    Article  Google Scholar 

  39. Li B, Liu X, Hu P, Wu Z, Lv J, Peng X (2022) All-in-one image restoration for unknown corruption. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 17452–17462

  40. Peng C, Zhang X, Yu G, Luo G, Sun J (2017) Large kernel matters–improve semantic segmentation by global convolutional network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4353–4361

  41. Feng H, Wang L, Li Y, Du A (2022) Lkasr: large kernel attention for lightweight image super-resolution. Knowl Based Syst 252:109376

    Article  Google Scholar 

  42. Liu X, Shen F, Zhao J, Nie C (2022) Randommix: a mixed sample data augmentation method with multiple mixed modes. arXiv preprint arXiv:2205.08728

  43. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European Conference on Computer Vision, pp 630–645. Springer

  44. Liu Z, Lin Y, Cao Y, Hu H, Wei Y, Zhang Z, Lin S, Guo B (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 10012–10022

  45. Agarap AF (2018) Deep learning using rectified linear units (relu). arXiv preprint arXiv:1803.08375

  46. Hadsell R, Chopra S, LeCun Y (2006) Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), vol 2, pp 1735–1742. IEEE

  47. Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp 764–773

  48. Wang X, Yu K, Dong C, Loy CC (2018) Recovering realistic texture in image super-resolution by deep spatial feature transform. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 606–615

  49. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  50. Contributors M (2018) MMCV: openMMLab computer vision foundation. https://github.com/open-mmlab/mmcv

  51. Wang Z, Cun X, Bao J, Zhou W, Liu J, Li H (2022) Uformer: a general u-shaped transformer for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 17683–17693

  52. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  53. Mittal A, Moorthy AK, Bovik AC (2012) No-reference image quality assessment in the spatial domain. IEEE Trans Image Process 21(12):4695–4708

    Article  MathSciNet  MATH  Google Scholar 

  54. OpenAI: GPT-4 technical report (2023)

  55. Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? the kitti vision benchmark suite. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp 3354–3361. https://doi.org/10.1109/CVPR.2012.6248074

  56. Menze M, Geiger A (2015) Object scene flow for autonomous vehicles. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 3061–3070

  57. Zhang K, Li D, Luo W, Ren W (2021) Dual attention-in-attention model for joint rain streak and raindrop removal. IEEE Trans Image Process 30:7608–7619

    Article  Google Scholar 

  58. Zhang K, Li D, Luo W, Ren W, Liu W (2022) Enhanced spatio-temporal interaction learning for video deraining: faster and better. IEEE Trans Pattern Anal Machine Intell 45(1):1287–1293

    Article  Google Scholar 

Download references

Funding

This work was supported in part by the project of National Natural Science Foundation of China under Grant No. 62272178.

Author information

Authors and Affiliations

  1. Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, 430074, People’s Republic of China

    Yifan Liu, Jincai Chen, Ping Lu, Chuanbo Zhu, Yugen Jian, Chao Sun & Han Liang

  2. School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, People’s Republic of China

    Jincai Chen & Ping Lu

  3. Key Laboratory of Information Storage System, Engineering Research Center of Data Storage Systems and Technology, Ministry of Education of China, Wuhan, 430074, People’s Republic of China

    Yifan Liu, Jincai Chen, Ping Lu, Chuanbo Zhu, Yugen Jian, Chao Sun & Han Liang

Authors
  1. Yifan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jincai Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ping Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chuanbo Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yugen Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Han Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yifan Liu, Chuanbo Zhu, and Yugen Jian wrote the main manuscript text. Yifan Liu, Chao Sun, and Han Liang designed the deraining method. Yifan Liu, Jincai Chen, and Ping Lu improved the performance of the model and designed the ablation experiments. Yifan Liu performed the experiments. All authors reviewed the manuscript.

Corresponding author

Correspondence to Jincai Chen.

Ethics declarations

Ethical approval

Not applicable.

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Chen, J., Lu, P. et al. MOONLIT: momentum-contrast and large-kernel for multi-fine-grained deraining. J Supercomput 79, 15729–15759 (2023). https://doi.org/10.1007/s11227-023-05286-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05286-0

Keywords

Search

Navigation