Mobile QR Code QR CODE

  1. (Department of Electronics and Communication Engineering, VIgnan’s Foundation for Science Technology and Research Deemed to be University / Guntur, Andhra Pradesh 522213, India {drml_ece, drvvr_ece}@vignan.ac.in)



Convolutional neural network, Deep learning, Gaussian noise, Image denoising

1. Introduction

Image denoising is a significant task for computer vision applications since it affects computer vision algorithms in the recognition of images. The goal of denoising is to remove the noise by conserving all characteristics of images. In general, denoising methods are available in spatial and transform domains [1,2]. Spatial domain methods use the spatial connection of pixels to eliminate noise. Spatial domain methods are further classified into linear filters and non-linear filters.

Linear approaches [3,4] are frequently used to remove additive noise in images by blurring the original signal. Gaussian filtered detection of forgery images was implemented by extracting feature vectors from the Gaussian filter residual in the spatial domain and performs excellently [5]. The SVD which is a non-linear spatial filter also eliminate noise in images, another spatial non-linear filter but it is lacking in distinguishing significant and non-significant singular values [6,7]. A bilateral filter is also based on a nonlinear technique that considers both the intensity and geometric closeness of a pixel to perform image denoising. The transform domain-based methods differentiate the data signal and noise signal by sparse depiction and obtain good denoising performance [8].

In general, conventional image processing techniques depend on prior knowledge for the removal of noise, and the computational complexity is higher. Moreover, in a state-of-art denoising algorithm, a missing part is the consideration of a residual image. A residual image can be defined as the dissimilarity between a noise-affected image and a denoised image. The residual image has several properties that have to be explored. The residual-image statistics like the mean square error and structural similarity index support denoising [9]. By considering the statistical moments and correlation of patches in a residual image, noise reduction by preservation of texture and contrast can be obtained [10].

In recent technology development, a convolutional neural network (CNN) using deep learning gives good results in not only the fields of object detection, classification, but also in image noise reduction. We presents a Gaussian filtered residual using a deep learning CNN for the noise reduction in an image. The denoising CNN architecture is based on residual denoising and has some small structure details that have not been evaluated. This problem is addressed in our work using a Gaussian residual filter for a residual image. Finally, the approach was evaluated using the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).

The paper stream chart is, in section 2 background ground theory of the proposed work is described briefly. Section 3 discusses the scheme presented, and Section 4 covers the results and comparisons. The final section discusses the conclusion.

2. Background

Neural networks have already been explored in signal denoising. A CNN for image denoising [11] gives good denoising performance for different variance levels. In recent days, deep CNNs have created a revolution in image classification, identification, restoration, etc. CNNs can be used in image de-speckling of satellite aperture radar images using Euclidean loss and total variation loss [12]. The restoration of an image from noise [13] removes the noise by identifying the difference between noisy image and denoised image. The result is a clean image, and this is called residual learning. A residual-based CNN has more layers, more parameters, and more computational cost. Residual image denoising using a dilated convolutional layer reduces the computational cost and receptive field size using mathematical calculation [14].

To prevents the problem of vanishing gradients, a CNN with 17 layers and residual learning was presented [15]. The residual network converges quickly due to a gradient skipping scheme at the learning stage to remove different noise at a different level [16]. Multi-scale residual learning influences the network depth as well as the number of models required for a learning process by using a dropout layer that slows the training process [17]. The training speed of the network is increased by using pre-processing and post-processing layers after the non-linear mapping improves the denoising performance, which is called FFDNet [18].

Algorithm complexity is reduced by recognizing the pixels at lower scale. To enrich the spatial information of the image, the structural detail of the image can be found by using a guided filter-based CNN. The network has two sub-networks. The networks extract the informative features, and these features are given as input to another network, where the non-consistent features are suppressed. This architecture is applicable for image denoising, up-sampling, and texture separation [19]. In proposed method we use the gaussian filter residual image which is different from residual image methodologies presented in other papers are used to evaluate the impact on image denoising.

3. The Proposed Scheme

The deep architecture has three basic layers with one skip layer. The first layer and last layer of the architecture have only a convolutional layer and rectified linear unit (ReLU). In the second level of operation, a convolutional layer plus a batch normalization layer and ReLu layer find the feature map. The same batch of operations is performed up to 28 levels.

The skip layer is used for a residual image. The convolutional layer has 30 filters with a kernel size of 3x3, and hence, 3x3x3 is used to generate 30 feature maps. The feature map gives the presence or absence of features in terms of intensity values. The convolution layer multiplies the input array with a weight matrix called the kernel. The operation of the convolutional layer is shown in Fig. 1.

The feature map of the convolutional layer is given to the ReLU layer to introduce non-linearity, so that the output will not be a sum. Otherwise, the network will not obtain its objective. The ReLU maintains the positive pixels and makes the negative pixels zero, as shown in Fig. 2. The activation map is the output of the ReLU layer. The next one is the batch normalization layer used to normalize the outputs of the previous activation layer. It normalizes them by subtracting the batch mean from the output and dividing them by the standard deviation. The last layer is once again a convolutional and ReLU layer. In this architecture, stochastic gradient descent is used as an optimization technique. Stochastic gradient descent de-normalizes the output by changing the mean and standard deviation.

The architecture shows that the filter structure plays a major role in extracting the feature map in the convolutional layer. The value of the filters is learned by the CNN by itself during a training process. The feature map size is fixed by three parameters like the depth, stride, and zero padding. We has used a depth of three, stride of one, and padding of one. The filters used in each convolutional layer is different, so simple specific characteristics of images can be obtained.

We have handled a color image, and the image is divided into three planes, where each plane is fed to the network separately and finally concatenated to find the original denoised color image. The presented method’s denoising performance is better than that of a denoising CNN. This can be seen in Fig. 3, which shows the residual image from using a denoising CNN in Fig. 3(a) and the proposed method in Fig. 3(b). Fig. 3(a) shows that the image has very small features with more noise compared to our result. Fig. 3(a) also shows that even when the noisy image is passing through various layers for prediction of noise, small structured details are present in the residual image, which violates the assumption of independent identically distributed noise.

In order to overcome the above problem effectively, by capturing the left-out structure in a residual image, Gaussian convolution is done on the residual image. The image filtered by Gaussian convolution is represented by:

(1)
$ GC\left(I_{p}\right)=\sum _{q\in s}G_{\sigma }(p-qI_{q} $
(2)
$ G_{\sigma }\left(x\right)=\frac{1}{2\pi \sigma ^{2}}~ \exp \left(-\frac{x^{2}}{2\sigma ^{2}}\right) $

I denotes the image, p denotes the position of a pixel, s denotes the spatial location of image, $\left\| p-q\right\| $ denotes the Euclidian distance between the pixels p and q, which defines the size of neighborhood, and finally, $G_{\sigma }$(x) is the Gaussian kernel.

The spatial distance between the pixels plays a role in Gaussian convolution but not in their values. In Gaussian convolution, a bright pixel has control over a dark pixel that is adjacent to it, even when the values of the two pixels are different. In the result, edges are blurred, and the structures are captured since discontinuities are averaged together.

The proposed method uses Gaussian convolution, which is linear and more effective at smoothing images. Smoothing has a strong effect on the contours of the image objects as the contrast is not preserved at the edges. Therefore, the last convolution layer feature map is Gaussian filtered (GF), and this filtered feature is subtracted from the last layers feature map to obtain the Gaussian filter residual (GFR). Hence, GFR has much less structure of the original image and more noise. Finally, a denoised image is obtained by subtracting the GFR from the original image. This concept is applied for all three planes of the input noisy image to denoise it. The presented scheme is shown in Fig. 4.

Fig. 1. Operation of convolutional layer.
../../Resources/ieie/IEIESPC.2021.10.2.096/fig1.png
Fig. 2. Activation map.
../../Resources/ieie/IEIESPC.2021.10.2.096/fig2.png
Fig. 3. Residual image (a) Denoising CNN method, (b) Proposed method.
../../Resources/ieie/IEIESPC.2021.10.2.096/fig3.png
Fig. 4. Presented network architecture for image denoising.
../../Resources/ieie/IEIESPC.2021.10.2.096/fig4.png

4. Results and Discussion

The efficacy of the presented scheme is assessed in MATLAB. The presented scheme was verified on standard test images. The test images are ``Lena,'' ``Pepper,'' and ``Baboon.'' To test the technique, ``imnoise'' is used, and adaptive white Gaussian noise is added to the input image at four different noise levels of 0.01, 0.02, 0.03, and 0.04. A learning rate of 0.0001 and minimum batch size of 128 were adopted for the stochastic descent gradient optimization technique.

In the presented scheme, PSNR is used to quantify the signal strength of the denoised images. The presented scheme is compared with a denoising CNN in terms of the PSNR and SSIM. Tables 1 and 2 show the performance comparisons of the proposed method for three test images. Table 1 shows that the presented process performs better than the denoising CNN with an average result of 0.3 dB. Another metric, SSIM, displays the error measure of the perceptual image in Table 2.

A visual comparison between our method and the denoising CNN for a noise level of 0.04 is shown in Fig. 5 for the Lena image and Fig. 6 for the Monarch image. Figs. 5(c) and 6(c) indicate that our approach restores the image better than the denoising CNN. Hence, the combination of a CNN and Gaussian filter residual is effective for image denoising.

Fig. 5. Lena image (a) Noise-affected image, (b) Denoised image of denoising CNN method, (c) Denoised image of proposed method.
../../Resources/ieie/IEIESPC.2021.10.2.096/fig5.png
Fig. 6. Monarch image (a) Noise-affected image, (b) Denoised image of denoising CNN method, (c) Denoised image of proposed method.
../../Resources/ieie/IEIESPC.2021.10.2.096/fig6.png
Table 1. Noisy PSNR and denoised PSNR values of images of Lena, Pepper, and Baboon.

Image

Noise variance (σ)

Denoising CNN method

Proposed method

Noisy PSNR

Denoised PSNR

Noisy PSNR

Denoised PSNR

Lena

0.01

20.21

30.56

20.21

30.59

0.02

17.37

28.79

17.37

28.89

0.03

15.76

27.64

15.76

27.76

0.04

14.67

26.68

14.67

26.86

Pepper

0.01

20.32

29.92

20.32

29.95

0.02

17.51

28.23

17.51

28.32

0.03

15.96

27.01

15.96

27.18

0.04

14.88

25.98

14.88

26.26

Baboon

0.01

20.17

24.22

20.17

24.62

0.02

17.30

22.80

17.30

23.09

0.03

15.70

22.01

15.70

22.27

0.04

14.60

21.45

14.60

21.71

Table 2. Noisy SSIM and denoised SSIM values of images of Lena, Pepper, and Baboon.

Image

Noise variance (σ)

Denoising CNN method

Proposed method

Noisy SSIM

Denoised SSIM

Noisy SSIM

Denoised SSIM

Lena

0.01

0.8035

0.9703

0.8035

0.9786

0.02

0.6856

0.9621

0.6856

0.9678

0.03

0.6025

0.9579

0.6025

0.9601

0.04

0.5419

0.9491

0.5419

0.9515

Pepper

0.01

0.8200

0.9701

0.8200

0.9713

0.02

0.7140

0.9529

0.7140

0.9598

0.03

0.6416

0.9487

0.6416

0.9506

0.04

0.5862

0.9381

0.5862

0.9418

Baboon

0.01

0.7584

0.8477

0.7584

0.8596

0.02

0.6376

0.7932

0.6376

0.8307

0.03

0.5529

0.7590

0.5529

0.7670

0.04

0.5033

0.7304

0.5033

0.7413

5. Conclusion

The work that we presented is a Gaussian residual learning CNN to remove noise from an image affected by Gaussian noise. The network is built by stacking a CNN along with a Gaussian filter to identify a Gaussian filtered residual. The Gaussian filtered residual helps in obtaining a clean image. Moreover, we showed that the proposed scheme removes noise in a better way than a denoising CNN in terms of the performance metrics PSNR and SSIM. The experiential results of visual comparison also exhibit that our proposed method performs better.

REFERENCES

1 
Talebi H., Zhu X., Milanfar P., 2013, How to SAIF-ly boost denoising performance, IEEE Trans. Image Process., Vol. 22, No. 4, pp. 1470-1485DOI
2 
Varghese G., Wang Z., 2010, Video denoising based on a spatiotemporal Gaussian scale mixture model, IEEE Trans. Circuits Syst. Video Technol., Vol. 20, No. 7, pp. 1032-1040DOI
3 
Ozkan M. K., Erdem A. T., Sezan M. I., Tekalp A. M., 1992, Efficient multiframe Wiener restoration of blurred and noisy image sequences, IEEE Trans. Image Process., Vol. 1, pp. 453-476DOI
4 
Rosenfeld A., Kak A. C., 1982, Digital picture processing, Second edition, Academic, New York, USADOI
5 
Hwang J. J., Rhee K. H., Research, Gaussian filtering detection based on features of residuals in image forensics, in Proc. of IEEE Int. Conf. Computing & Communication Technologies, Research, Innovation, and Vision for the Future, Hanoi, Vietnam, pp. 153-157DOI
6 
Patterson H. C., Andrews C. L., Speech, Singular value decompositions and digital image processing, IEEE Trans. Acoust., Speech, Signal Process., Vol. 24, pp. 26-53DOI
7 
Lee H-C., Lee H-J., Kwon H., Liang J., 1991, Digital image noise suppression method using SVD block transform, U.S. Patent 5 010 504Google Search
8 
Chang S. G., Yu B., Vetterli M., 2000, Adaptive wavelet thresholding for image denoising and compression, IEEE Trans. Image Process., Vol. 9, No. 9, pp. 1532-1546DOI
9 
Brunet D., Vrscay E. R., Wang Z., July 2009, The use of residuals in image denoising, in Proc. of 6th Int. Conf. Image Analysis and Recognition, Halifax, Canada, pp. 1-12DOI
10 
Riot P., Almansa A., Gousseau Y., Tupin F., Sept. 2016, Penalizing local correlations in the residual improves image denoising performance, in Proc. of 24th European Conf. Signal Processing, Budapest, Hungary, pp. 1867-1871DOI
11 
Koziarski M., Cyganek B. L., 2016, Deep neural image denoising, in Proc. of Int. Conf. Computer Vision and Graphics, pp. 163-173DOI
12 
Wang P., Zhang H., Patel V. M., 2017, SAR image despeckling using a convolutional neural network, IEEE Signal Process., Letters, Vol. 24, No. 2, pp. 1763-1767DOI
13 
Zhang K., Zuo W., Chen Y., Meng D., Zhang L., 2017, Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising, IEEE Trans. Image Process., Vol. 26, No. 7, pp. 3142-3155DOI
14 
Wang T., Sun M., Hu K., Boston, Dilated deep residual network for image denoising, in Proc. of IEEE 29th Int. Conf. Tools with Artificial Intelligence, Boston, MA, USA, pp. 1272-1279DOI
15 
Tian C., Xu Y., Fei L., Wang J., Luo J. Wen and N., 2019, Enhanced CNN for image denoising, IET-CAAI Trans. Intelligence Technology, Vol. 4, No. 1, pp. 17-23DOI
16 
Zhang F., Cai N., Wu J., G , Cen , Wang H., Chen X., 2018, Image denoising method based on a deep convolution neural network, IET Image Process., Vol. 12, No. 4, pp. 485-493DOI
17 
Chen C., Xu Z., 2018, Aerial-image denoising based on convolutional neural network with multi-scale residual learning approach, Information Journal, Vol. 9, No. 7, pp. 169-186DOI
18 
Tassano M., Delon J., Veit T., 2019, An analysis and implementation of the FFDNet image denoising method, Image Processing On Line Journal, Vol. 9, pp. 1-25DOI
19 
Li Y., Huang J-B., Ahuja N., Yang M. H., 2019, Joint image filtering with deep convolutional networks, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 41, No. 8, pp. 1-14DOI

Author

M. Laavanya
../../Resources/ieie/IEIESPC.2021.10.2.096/au1.png

M. Laavanya currently serves as Associate Professor in Department of Electronics and Communication Engineering, Vignan’s Foundation for Science, Technology and Research University, Guntur, Andhra Pradesh. She received her B. E., degree in Electronics and Communication from Madurai Kamaraj University, Madurai in the year 2003. She received her M. E., degree in Applied Electronics from Anna University, Chennai in the year 2005. She did her Ph. D., research work in the domain of Image Denoising and awarded Ph. D., in the year 2019 by Anna University, Chennai. Her areas of research include Signal, Image, Video Processing and Deep Learning. She is the life member of Indian Society for Technical Education and Overseas member of The Institute of Electronics, Information and Communication Engineers.

V. Vijayaraghavan
../../Resources/ieie/IEIESPC.2021.10.2.096/au2.png

V. Vijayaraghavan currently serves as Associate Professor in Department of Electronics and Communication Engineering, Vignan’s Foundation for Science, Technology and Research University, Guntur, Andhra Pradesh. He received his B. E., degree in Electronics and Communication from Madurai Kamaraj University, Madurai in the year 2003. He received his M. E., degree in Computer Science and Engineering from Anna University, Tiruchirappalli in the year 2010. He did his Ph. D., research work in the domain of Image Denoising and awarded Ph. D., in the year 2019 by Anna University, Chennai. His areas of research include Image Processing, Deep Learning, Embedded Systems, Wireless Networks, and Network Security. He is a life member of Indian Society for Technical Education.