1. Introduction

The number of vehicles in cities is growing, resulting in an increase in traffic congestion as a result of society's rapid development and the growing variety of people's modes of transportation ^[1]. Manual management methods cannot effectively alleviate the traffic management pressure. With the development of computer technology, intelligent traffic systems using intelligent algorithms are gradually becoming popular.

One of the key uses of computer vision in intelligent transportation systems is license plate identification, and precise license plate recognition helps with vehicle management. Intelligent license plate recognition is mainly achieved by image processing and pattern recognition technology, which automatically detects, segments, recognizes, and compares license plates from images ^[2]. License plate recognition algorithms typically require manual feature extraction and then use the extracted features to recognize license plate characters. Although this method is relatively simple in principle, the selection of features directly affects the recognition accuracy.

Convolutional neural networks (CNNs) are deep learning algorithms that can automatically extract image features using convolutional kernels ^[3], which makes them suitable for license plate image recognition. Wang et al. ^[4] proposed an automatic license plate recognition approach and verified its effectiveness through experiments. Vetriselvi et al. ^[5] proposed an effective vehicle license plate number recognition (DLVLPNR) model based on deep learning for recognizing alphanumeric characters in license plates, and the results validated its effectiveness. In order to recognize and locate regions of interest, Nurhaida et al. ^[6] developed an enhanced horizontal-vertical edge projection and found an improvement over previous research.

All of the above studies have proposed algorithms related to license plate recognition, but the feature extraction of license plate images in these studies was not detailed enough. The purpose of this study is to use LeNet-5 to improve the performance of license plate recognition. The features extracted by a basic CNN are not deep enough, and there are not enough detail features. Therefore, this study introduces the inception-SE convolution module to process the convolutional features in parallel to improve the depth of features and increase the detail features. This study provides a simple introduction to the LeNet-5 model used for license plate image recognition and an improved LeNet-5 model with the inception-SE convolution module. The optimized LeNet-5 model was compared with the original one and a back-propagation neural network (BPNN) through simulation experiments.

2. License Plate Recognition based on Optimized LeNet-5

To extract contour information from license plate images, rule-based image processing methods are applied ^[7], such as binarization, filtering, and edge detection. Character information is then extracted from the contour. These methods require manual rule design and rely heavily on human experience. Intelligent algorithms based on feature extraction recognize license plates by using features such as the scale invariant feature transform (SIFT) and histogram of oriented gradients (HOG) ^[8] combined with classifiers using various combinations ^[9]. But the selection of features is crucial and directly affects the accuracy of the algorithm. Intelligent algorithms based on deep learning train recognition models with large datasets, enabling the algorithms to autonomously extract features from license plate images and accurately recognize the features.

2.1 LeNet-5 Model

The CNN algorithm is often used to recognize license plate images. Compared with other deep learning algorithms, the CNN algorithm can use the sliding convolution operation of the convolution kernel on the image ^[10] to directly extract the features from the image without additional feature extraction processing. Fig. 1 shows the basic structure of LeNet-5 in the CNN algorithm. When applying LeNet-5 to license plate recognition, a license plate image with a size of $32\times 32$ is input into the input layer, followed by a convolution operation in convolution layer 1 using a convolution kernel with a size of $5\times 5$. The convolution formula is:

(1)

$y_{j}^{l}=f\left(\sum _{i=1}^{N_{j}^{l-1}}w_{i,j}\otimes x_{i}^{l-1}+b_{j}^{l}\right),j=1,2,\cdots ,m$,

where $l$ denotes the current layer, $w$ represents the convolution kernel weight matrix, $x_{i}^{l-1}$ represents the output feature map matrix, $f$ denotes the activation function, $\otimes $ denotes the convolution operation, and $b_{j}^{l}$ denotes the bias of the $j$-th feature map of the $l$-th layer ^[11].

The convolutional feature map is then compressed in pooling layer 1, which uses a pooling frame with a size of $2\times 2$. After the pooling and compression, the convolutional feature map is sent to convolutional layer 2, which also uses a convolutional kernel with a size of $5\times 5$. The pooling and compression is performed again in pooling layer 2, which has the same parameters as pooling layer 1. The compressed convolutional feature maps are computed in two fully connected layers, and the results are finally output in the output layer ^[12].

Fig. 1. Basic structure of conventional LeNet-5.

2.2 Improved LeNet-5 Model for License Plate Recognition

The LeNet-5 model is mainly used for recognizing handwritten digits. However, during license plate recognition, license plate images are easily affected by external factors, such as the environment. Moreover, license plate recognition in China involves not only recognizing digits, but also identifying Chinese characters and letters that represent the region to which the license plate belongs. To improve the recognition performance of LeNet-5 for license plate images, this study introduces the inception-SE convolution module ^[13].

The inception-SE convolution module is a fusion of the inception module and the SE module. The aim is to increase the accuracy and efficiency of the model while keeping relatively small model parameters. The module can be applied to various computer vision tasks. Fig. 2 shows the basic structure of the inception-SE convolution module ^[14].

The module consists of the inception module and the SE model. The inception model has four convolution branches that process images from the previous layer in parallel, and the features extracted from different convolution branches are ultimately concatenated into a deeper feature map. By using the multiple convolution branches in the inception module, more target features can be acquired, increasing the depth and breadth of the network structure ^[15]. The SE module is a gating mechanism that adaptively adjusts the relationships between channels to enhance the important information within the feature map and thus improve the performance of the model ^[16]. First, global pooling is performed on the channel according to the feature map at each position, and then the activation functions of multiple hidden layers are combined in an interleaved structure. Finally, sigmoid compression and multiplication operations are performed on the results to obtain feature maps with enhanced effective features and suppressed ineffective features.

The basic structure of the LeNet-5 model after the introduction of the inception-SE convolution module is shown in Fig. 3. It was improved by using the inception-SE module to replace convolution layer 2 and the first fully connected layer in the original model. The recognition steps are listed below.

① The license plate image is pre-processed, including grayscale conversion, filtering, noise reduction, binarization, edge detection ^[17], and so on. Finally, the characters in the image are segmented ^[18]. Since the license plate has uniform specifications, the character segmentation can be realized after designing the corresponding matching template according to the specifications of the license plate. The formulas for grayscale conversion and noise reduction filtering are:

(2)

$\left\{\begin{array}{l} gray=\frac{R+G+B}{3}\\ g(x,y)=A\cdot e^{-\frac{(x-x_{0})^{2}}{2\sigma _{x}^{2}}-\frac{(y-y_{0})^{2}}{2\sigma _{y}^{2}}} \end{array}\right.$,

where $gray$ stands for the gray level of a pixel, $R$, $G$, and $B$ are the color components of the pixel, $g(x,y)$ is the Gaussian filter function, $A$ is the function amplitude, $(x_{0},y_{0})$ is the central coordinates, and $\sigma _{x}$ and $\sigma _{y}$ are the width in the directions of $x$ and $y$. The image binarization is performed using the maximum between-cluster variance method, and its expression is:

(3)

$\left\{\begin{array}{l} \mu =\omega _{1}\mu _{1}+\omega _{2}\mu _{2}\\ g(t)=\omega _{1}(\mu -\mu _{1})^{2}+\omega _{2}(\mu -\mu _{2})^{2}\\ T=argmax(g(t)) \end{array}\right.$,

where $\mu $ is the overall average gray level of the image, $\mu _{1}$ and $\mu _{2}$ are the average gray levels after dividing by gray threshold $T$, $\omega _{1}$ and $\omega _{2}$ are the percentages of pixels after dividing by gray threshold $T$, and $g(t)$ is the variance between the two parts.

② The image is passed to the input layer.

③ Convolutional feature extraction and compression are performed in the convolutional and pooling layers.

④ The compressed convolutional feature map is convolved in the inception-SE convolution module to expand the depth and breadth of the convolutional feature map and highlight the effective features. Compression is then performed again.

⑤ The convolution operation is performed in the second inception-SE convolution module ^[19] and then sent to the fully connected layer for forward computation. The recognition result is output in the output layer.

In the training phase, the recognition results of the model need to be compared with the labeling results of the corresponding training samples to obtain the error between the them. The cross entropy was used as the training error in this study ^[20].

Fig. 2. Basic structure of the inception-SE convolution module.

Fig. 3. Basic structure of the optimized LeNet-5 model.

3. Simulation Experiments

The previous section described the algorithm used for license plate image recognition, which improves recognition performance by incorporating the inception-SE convolutional module into the traditional LeNet-5 model. To evaluate the improved LeNet-5 model's recognition performance on license plate images, this section conducts performance tests and compares it with both the traditional LeNet-5 model and BPNN model by experiments.

3.1 Experimental Data

The license plate images used in the simulation experiments were collected from different intersections. Cameras were placed at different locations of the intersections, and then the license plate images of the vehicles in the path were collected by the cameras. The collected images had different light and angles. Some of the images are shown in Fig. 4. A total of 1,500 license plate images were collected, of which 1,000 were randomly selected as the training set, and the remaining 500 were used as the test set.

Fig. 4. Vehicle images.

4. Experimental Results

Before recognizing the license plate images, they were pre-processed, and the characters to be recognized were segmented. Fig. 5 shows some license plate images and their pre-processed segmentation contact character images. The color performance for different license plates was different due to the influence of external environmental factors such as lighting, especially the base color between conventional-energy vehicle license plates and new-energy vehicle license plates. After pre-processing and character segmentation, the characters in the license plate image were clearer, and the color difference between different license plates was eliminated, which is beneficial for recognition.

Fig. 6 shows the convergence curves of the three license plate recognition model algorithms when training was performed. The training bias of all three recognition algorithms decreased as the number of training sessions increased and eventually stabilized. The improved LeNet-5 decreased the fastest and converged earlier with stability, and the error with stability was also the smallest.

Table 2 presents the recognition results of the three recognition algorithms for some license plate images. The character recognition results of the improved LeNet-5 model matched the characters in the images. The character recognition results of the original LeNet-5 model differed from the characters in the images, and sometimes, "0" was recognized as "O". The character recognition results of the BPNN model differed the most from the characters in the images, recognizing not only "0" as "O", but also "B" and "5" as "8" and "S", respectively.

Fig. 7 shows the recognition performance of the three recognition algorithms for characters such as Chinese characters, letters, numbers, and the whole license plate image. The specific recognition accuracy can be obtained from the data labels in Fig. 7. It was seen that the recognition accuracy of the improved LeNet-5 model was the highest for recognizing Chinese characters, letters, numbers, and the whole license plate image, followed by the original LeNet-5 model, and the BPNN had the lowest recognition accuracy.

Fig. 5. License plate images and their pre-processed character images.

Fig. 6. Convergence curves of the three recognition algorithms when training was performed.

Fig. 7. Recognition accuracy of three recognition algorithms.

Table 2. Partial Recognition Results of License Plate by Three Algorithms.

License plate	Recognition results of the BPNN model	Recognition results of the original LeNet-5 model	Recognition results of the optimized LeNet-5 model
	苏M·MAOOO	苏M·MA00O	苏M·MA000
	粤8·FI234S	粤B·FI2345	粤B·F12345
	鲁F·BL62O	鲁F·8L62O	鲁F·8L620
	苏B·FOII11	苏B·FO1111	苏B·F01111

5. Discussion

Intelligent recognition of license plate images, as one of the important applications in the field of computer vision and pattern recognition, plays a key role in traffic safety and intelligent transportation. By combining image preprocessing, feature extraction, and machine learning or deep learning methods, an efficient and accurate license plate recognition system can be realized. This is of great significance for traffic violation monitoring, parking management, and traffic flow statistics. This study used a LeNet-5 CNN as the classifier. As a CNN, it does not need to extract image features separately when recognizing images. The convolutional kernel in the convolutional layer can automatically extract image features, so it has an advantage in recognition efficiency.

In order to improve the recognition accuracy of LeNet-5, this study introduced the inception-SE convolutional module into it, and then the simulation experiments compared the BPNN model, the original LeNet-5 model, and the improved LeNet-5 model. The improved LeNet-5 model converged faster, stabilized with a smaller error during the training process, and had the highest accuracy in recognizing license plates. The BPNN model needs to extract image features separately when recognizing license plate images, and in this process, image features are lost, resulting in lower recognition accuracy. As CNNs, the original LeNet-5 and improved LeNet-5 do not need to extract additional image features, and the local features extracted by the convolutional kernel can be combined into the global features. This can maintain the integrity of the image features as much as possible.

Therefore, these methods perform better than the BPNN model in terms of training efficiency and recognition accuracy. The inception-SE convolutional module introduced into the improved LeNet-5 model obtains more detailed features in the image through parallel convolution and then enhances the effective features through the gating mechanism. Thus, it is better than the original LeNet-5 model in terms of recognition performance.

One limitation of this study is that only Chinese license plate images were used in the training and testing of the recognition algorithm. This makes the algorithm less generalizable. Therefore, a future research direction is to add license plate images from different countries to improve the generalizability of the algorithm.

6. Conclusion

This study improved the LeNet-5 model by introducing the inception-SE convolution module. In simulation experiments, the optimized LeNet-5 model was compared with the original model and BPNN. After pre-processing and character segmentation, the characters in the license plate images were clearer than before, and the color differences between different license plates were almost eliminated. As the training time increased, the errors of the three recognition algorithms all decreased, but the optimized LeNet-5 model decreased the fastest and had earlier stable convergence. In the character recognition of license plate images, the improved LeNet-5 model had the highest accuracy, followed by the original LeNet-5 model, and the BPNN model had the lowest accuracy.