3.1. CNN-based Image Aesthetic Assessment for Advertising Design

Aesthetic evaluation of graphic advertising design images can further improve the aesthetic quality of advertising design. Therefore, this paper proposes an aesthetic evaluation model of advertising design images based on CNN and data aggregation. In this paper, a convolutional network based binary path image aesthetic evaluation network is proposed. Visual Geometry Group and RESNET-50 are selected as the feature extraction network, and multi-scale convolution layer and ResNet are selected as the multi-scale information fusion network. The aesthetics of advertising design were evaluated through these two networks. After the model is established, a model training optimization method based on data aggregation is proposed to further determine the model optimization direction and improve the aesthetic evaluation accuracy. CNN is a deep learning model that is particularly suitable for processing image data. It extracts features from images through multilayer convolution and pooling operations and performs classification or regression tasks through fully connected layers ^{[19, 20]}. Therefore, the study proposes a CNN-based IAA method for AD as shown in Fig. 1.

The IAA method for AD of CNN employs two path sub-networks aimed at extracting features of the image and modeling them for aesthetic assessment of the image for AD. The first sub-network is the region of interest sub-network. In order to extract the features of these regions more flexibly, the study determines the degree of interest of a local region based on the information density of the region, and a region with higher information density may be more appealing. The study designed a network to model multiple local regions with high information density by extracting them. This network can be trained end-to-end without manual labeling, thus avoiding the interference of subjective noise. The second path sub-network is a multi-scale information sub-network designed to provide rich and diverse global descriptive features to further enhance the performance of the model. The study combines shallow and deep features through a multi-layer information fusion network structure to support the decision-making process. Finally, the decision results from the two networks are fused to produce a final decision judgment. This method can effectively assess the aesthetic quality of AD images with high accuracy and reliability. The prediction function for AD image quality assessment in the region of interest sub-network is shown in Eq. (1).

In Eq. (1), the prediction function is $\phi$, and the output conditional probability distribution is $P(\hat{y}^{(i)} | Z^{(i)})$. The prediction result is $\hat{y}^{(i)}$, the output region of interest variable feature is $Z^{(i)}$, and the deep learning feature vector of the image object region is $F^{(i)}$. The region of interest subnetwork structure is shown in Fig. 2.

In the design of the region of interest sub-network, the study chooses Visual Geometry Group (VGG) and Residual Network-50 (ResNet-50) as the feature extraction network, which have excellent performance in the field of image recognition. The VGG network can capture the fine features of the image through its deep convolutional layer, while the ResNet-50 solves the problem of gradient disappearance in deep network training through residual connections, making the network deeper and thus extracting richer features. The local features and global features of images are extracted through these two networks respectively. Meanwhile, the study employs multilayer perceptron as a predictive classifier for image aesthetics in order to achieve high-precision evaluation of image aesthetics. The center point in the image object is calculated as shown in Eq. (2).

In Eq. (2), the centroid of the object region is $\{c_j\}$, the centroid detected by the target is $\{c_{BBi}\}$, the centroid of the connected subgraph computed by the binarized mapping map is $\{c_{SALi}\}$, and the sum of the number of region points is $n_1+n_2$, $n_1$ and $n_2$ are the number of region points entered by the two convolutional networks, respectively. The final set of subject regions obtained after the introduction of the anchor point mechanism is shown in Eq. (3).

In Eq. (3), the set of subject regions is $s\{c_j\}$, the number of anchor boundaries is $k$, and the number of sets of subject regions is $k \times (n_1 + n_2)$. the cost function for semantic evaluation is shown in Eq. (4).

In Eq. (4), the number of regions contained in the subject region set is $M$, one of the regions is $p_i$, and the corresponding length and width of the region are $h_i$ and $\omega_i$. The significant sum of all the pixel points in the region is $S(p_i)$, the evaluation function is $H$, and the weight coefficients of the three functions are $\alpha$, $\beta$, and $\gamma$, respectively. The feature expression of the original image extracted by the first four layers of convolution is shown in Eq. (5).

In Eq. (5), the feature extraction function is $Ext$, the original image input is $I^{(i)}$, and the corresponding deep convolutional features of the region are $P^{(i)}$. The transformation relationship between a given set of feature vectors and the feature time of the region of interest is shown in Eq. (6).

In Eq. (6), the given set of feature vectors is $F$, the feature vector weights are $W$, the region of interest features are $Z$, the softmax function is $\sigma$, the activation function is $\psi$, and the multiplication operation is $\otimes$. The structure of the multi-scale information sub-network is shown in Fig. 3.

In the multiscale information subnetwork structure, different scale features are adjusted by small convolutional layers, and the transverse connection output is shown in Eq. (7).

In Eq. (7), the output features of the small convolutional block are $f_{b2}$, $f_{b3}$, and $f_{b4}$, whose feature weights are $W_{b2}$, $W_{b3}$, and $W_{b4}$, respectively. the laterally connected outputs are $f'_{b2}$, $f'_{b3}$, and $f'_{b4}$, whose feature weights are $W'_{b2}$, $W'_{b3}$, and $W'_{b4}$, respectively. the shallow features are generated as shown in Eq. (8).

In Eq. (8), the shallow feature is $f_{shallow}$. The deep feature output from the last layer of the network is shown in Eq. (9).

In Eq. (9), the output of the last layer of the network is $f_{b5}$. During the feature encoding process, the study uses two stacked $3 \times 3$ convolutional layers to extract the middle layer features of the image, and ensures the output feature size and the number of channels by adjusting the convolutional layer step size and adding $1 \times 1$ convolutional layers to realize the shallow feature noise reduction and the image down sampling. In the feature fusion stage, the three low-level features are merged into shallow features by global average pooling of feature layers, as well as dimensional compression and splicing, while the original ResNet model is used to output deep features. The overall loss of model training is shown in Eq. (10).

In Eq. (10), the overall loss is $L_M$, the shallow and deep feature losses are $L_s$ and $L_d$, respectively, and the weights of the two parts of the loss are $\lambda_s$ and $\lambda_d$, respectively.

3.2. Training Strategies Based on Data Aggregation

After the aesthetic evaluation model based on CNN is established, a training strategy based on data aggregation is proposed. Through this training strategy, the model performance is further optimized and the aesthetic evaluation accuracy is improved. In order to better mine the sparsely distributed samples in the dataset, the study utilizes feature similarity as the classification basis and proposes a DA-based dataset partitioning method. Subsequently, a training method that combines sparse data that are correctly dispersed with compact samples is devised to enhance the model's generalization ability and provide the model an optimization direction. The DA-based training strategy is shown in Fig. 4.

In the DA-based training strategy, a CNN model is first trained using advertisement image aesthetic data to generate high-level deep features. This approach captures the semantic and abstract information in the images and is therefore more suitable for assessing the semantic similarity of the images. Next, a density clustering-based segmentation method is used to classify the dataset for semantic similarity. This method calculates the local densities and distances of the samples in a high-dimensional space in order to achieve an aggregated representation of the dataset and thus a better understanding of the semantic associations between the samples. On this basis, the study further adopts the Compact-to-Sparse training strategy to divide the learning process into a start-up phase and a retraining phase. In the start-up phase, an initialization model is first trained using the entire dataset, and then the model is used to extract features and divide the dataset into three subsets. Subsequently, learning starts from the compact subset to obtain the startup model. Upon entering the retraining phase, the features are reextracted and the three subsets are re-divided. In this phase, sparse subsets are added for learning to achieve further improvement of the model performance. Such a training approach makes it possible to use DA to enhance the model's functionality and increase its accuracy and dependability when handling jobs pertaining to advertising aesthetics. The Euclidean distance between features in the dataset division is calculated as shown in Eq. (11).

In Eq. (11), the Euclidean distance between features $f(P_i)$ and $f(P_j)$ is $D_{ij}$. the local density of each map is calculated as shown in Eq. (12).

In Eq. (12), the local density of the image is $\rho_i$, the number of points in the image is $j$, and the constant associated with the sequence of alignments is $d_c$. the density taking function is $X(d)$, as shown in Eq. (13).

In Eq. (13), the density function takes the value of 1 when the unit distance $d$ is less than 1, and in other cases it takes the value of 0. The distance $\theta_i$ for each image is defined as shown in Eq. (14).

In Eq. (14), the clustering center is selected based on the distance between images and local density, and the principle of selection is to find the image that has the highest local density and the largest distance from other images. These photos make good clustering centers because they show points with high local density encircled by points with low local density. At the same time, points with large distances and high local densities can be considered as anomalies. The Compact-to-Sparse training strategy is shown in Fig. 5.

The goal of the training process for aesthetic assessment of advertising images is to discover potential image features and aesthetic rules. Due to the huge parameter search space of the CNN model, the initial learning direction is crucial for the model to converge to a better local optimum. The study designed a Compact-to-Sparse training strategy to learn the segmented dataset in stages. The aggregation of samples in the dataset is partitioned using a density clustering-based algorithm before training begins. A CNN model is first trained on the full training data, then high-level features are extracted and the dataset is divided. In the startup phase, a compactly distributed subset from the delineated dataset is taken and a new model is trained to learn the regular aesthetic features and rules. After the model has converged, the degree of aggregation of the samples in the training set is re-evaluated and delineated using the new model and fine-tuned based on it. The study also added sparsely distributed images to the dataset but gave them less weight to allow the model to learn more unique and complex aesthetic rules. Compared to standard neural network training, the Compact-to-Sparse training strategy learns relatively simple but effective decision boundaries. The overall loss function during data training is shown in Eq. (15).

In Eq. (15), the overall loss is $L$, and the losses for each stage of data are $L_0$, $L_1$, and $L_2$, whose corresponding weights are $\omega_0$, $\omega_1$, and $\omega_2$, respectively.

4.1. Effects of Aesthetic Feature Extraction

In order to establish the connection between different features, the study extracts feature from low convolutional layers and coordinates the different scale features. In order to test the extraction effect of the feature extraction method designed by the study, the experiment compares the feature extraction effect of CNN-based multi-scale feature extraction with that of scale invariant feature transform (SIFT), speeded-up robust features (SURF), and histogram of orientation gradients (HOG). Firstly, using different data volume images, feature extraction methods are used to extract features and compare the differentiation, computational efficiency and relevance of the features extracted by the four algorithms. The specific results are shown in Fig. 6.

As can be seen in Fig. 6(a), the discriminability of the features extracted by CNN is 0.68, while that of SIFT is 0.44, that of SURF is 0.40, and that of HOG is 0.50. The discriminability of the features extracted by the four algorithms is in a relatively good interval. Among them, the discriminability of CNN is significantly higher than that of the other three algorithms. Fig. 6(b) shows that the correlation coefficient of CNN is 0.004, and the variables are linearly independent of each other. Compared to the other three algorithms, CNN extracts better features. This is due to its parallel computing nature and the optimization of the deep learning framework. Although CNN requires more computing resources in feature extraction, its efficient parallel processing capability keeps the overall computing efficiency at a high level. Fig 6(c) compares the correlation of features extracted by the four algorithms. The correlation of CNN is 0.82, SIFT is 0.70, SURF is 0.68, and HOG is 0.75. The feature correlation of CNN is high, which indicates that the features extracted by CNN have good consistency among different images, which is helpful for the subsequent image classification and recognition tasks.

Utilizing the features derived by the four algorithms, the experiment is trained to further assess the efficacy of the feature extraction techniques developed in the study. The training results are shown in Fig. 7.

In Fig. 7, the convergence of the training model using the features extracted by CNN is better. In Fig. 7(a), the CNN trained model is stabilized by training up to 12 times to get the target accuracy. While the models trained under the other three features all need to be trained more than 16 times. In Fig. 7(b), the recall of the CNN training model reaches convergence in 14 iterations, which is better compared to the other three algorithms. In Fig. 7(c), the loss value of the CNN training model starts to converge at 10 iterations, which is more than 5 iterations less compared to the other three algorithms. This demonstrates how CNN-extracted features can enhance the model's training convergence.

4.2. Comparative Analysis of Model Evaluation Results

Most of the current image aesthetic quality assessment divides images into two tiers, high and low, and there are fewer methods for image aesthetic quality laddering metrics. In order to verify the advantages of the model constructed by the research (Model 1), the AVA dataset was chosen for the experiment. The aesthetic assessment was performed on four categories, namely, natural scenery, urban architecture, portraits, and animals. And the results obtained from the scores were compared and analyzed with the AVA dataset scores. The results are shown in Fig. 8.

According to Figs. 8(a)-8(d), it can be seen that the evaluation results of aesthetic evaluation on the four categories of natural scenery, urban architecture, portrait, and animal images are very close to the rating results of the AVA dataset, with an average fitting degree of 0.907. The fitting degree for urban architecture is the highest, reaching 0.921, while the fitting degree for animal categories is 0.893. This is because images of urban architecture and portrait categories often have obvious aesthetic features and regularities, making it easier for models to capture these features and make accurate evaluations. In contrast, images of natural landscapes and animal categories are relatively difficult to evaluate due to their complex and varied natural elements and dynamic features, resulting in slightly lower fitting accuracy. To further examine the model performance, the study utilized it to compare with the IAA model based on the lightweight network model (Model 2) and the IAA model based on the context-aware attention mechanism (Model 3) on the AADB and AVAI databases. The comparison results are shown in Fig. 9.

In Fig. 9(a), the classification accuracy of Model 1 reaches 85.79%, while that of Model 2 is 80.43%. The classification accuracy of the other two models is below 80%. This is due to the advantages of Model 1 in feature extraction and model structure design. Model 1 adopts the residual network structure in deep learning, which enables the network to learn the feature representation of images more deeply while alleviating the problem of gradient vanishing. In Fig. 9(b), the mean value of scoring error of Model 1 is 0.521, which is 0.485 lower compared to Model 2, its mean value of error is 0.600 lower compared to Model 3, and its mean value of error is 0.584 lower compared to Model 4. It can be seen that Model 1 has a high accuracy in IAA.

4.3. Data Aggregation-based Assessment Reasonableness Analysis

The study divides the aggregation of samples in the dataset based on the density clustering algorithm and adds sparsely distributed images to the dataset, giving smaller weights to allow the model to learn more and complex aesthetic rules. To test the reasonableness of using DA in the model, the experiment divides the dataset into compact (C), sparse (S) and highly sparse (HS) according to a certain ratio. Different training data were used for training. Pearson's sorting relevance (SPRC) is quoted, and classification accuracy is used as the evaluation index, and the final aesthetic evaluation results are shown in Table 1.

According to Table 1, when training with a compact dataset, the binary classification accuracy of the model is 79.14%, the multi classification accuracy is 61.52%, and the SPRC is 0.628. After adding sparse datasets, the binary classification accuracy increased to 80.74%, the multi classification accuracy increased to 66.78%, and the SPRC further improved to 0.704. This indicates that adding sparsely distributed images to the dataset and giving them smaller weights can help the model learn more and more complex aesthetic rules, thereby improving the accuracy of evaluation. In addition, when further incorporating highly sparse datasets, the classification accuracy and SPRC metrics slightly decrease. This may be because the highly sparse dataset introduces too much noise, which has a negative impact on the training of the model. In summary, researching evaluation methods based on data aggregation can effectively improve the rationality of image aesthetic evaluation. Sparse and compact datasets are used to compare the impact of various weights on the model in order to better investigate the performance of this dataset. by tracking the model's classification accuracy and adjusting the weights in the loss function. Fig. 10 presents the findings.

In Figs. 10(a) and 10(b), when the weight of the loss function is [1, 0.5], the classification accuracy of the model reaches 80.4%. This indicates that appropriately reducing the weight of sparse datasets when evaluating them can help improve the classification accuracy of the model. However, when the weight of the loss function is further adjusted to [1, 0.3] or [1, 0.7], the classification accuracy decreases. This indicates that when evaluating highly sparse datasets, excessively reducing or increasing their weights may result in the model being unable to fully learn the aesthetic rules in these datasets, thereby affecting the accuracy of the evaluation.

4.4. Effects of Model Application

The study utilized the model in actual AD, and 20 AD practitioners were selected to use the model for one month in their regular design. After using the model for one month, the practitioners' experience was analyzed through a questionnaire. The results are shown in Table 2.

Table 2 shows that after one month of use, more than 80% of the trialists chose satisfied. In the survey on the assessment accuracy of the model, 75% of the trialists chose the option of very accurate. This shows that the institute constructed model can help practitioners to design more professionally in practical applications.

To further enrich the depth and breadth of the research, qualitative samples and results are added to evaluate the applicability and effectiveness of the research model. The study first collected qualitative samples from different advertising design fields, including graphic design, video advertising, online advertising, etc. The samples cover a variety of styles and topics to ensure that the models can handle diverse aesthetic needs. In collaboration with advertising design practitioners, the research team analyzed and labeled these samples in detail to ensure that they were representative of design needs in real work. After collecting enough qualitative samples, the research team fed these into the built model for testing. The model's assessment of these samples was compared with the professional judgment of advertising design practitioners. The comparison results are shown in Table 3.

As shown in Table 3, the evaluation results of the model are highly consistent with the professional judgments of advertising design practitioners. Specifically, 78.5% of model evaluations were considered "very consistent" with practitioners' judgments, while 12.3% of model evaluations were considered "fairly consistent with industry standards," 6.2% of model evaluations were considered "general," and only 3.0% were considered "very inconsistent." These results show that the model can accurately evaluate the aesthetic quality of advertising design images and meet the professional standards of the industry. The results show that the model can accurately identify the aesthetic elements in the design in most cases, and give reasonable evaluation results. This further verifies the validity and applicability of the model in practical applications.