3.1. Federated Learning Image Classification Algorithm Based on Adaptive Gradient Descent

Image classification is a processing method that distinguishes images of different categories based on their semantic information. Traditional methods mainly rely on Convolutional Neural Network (CNN), which is an ANN specifically designed for image processing and recognition ^[14]. In Fig. 1, CNN is mainly composed of a 5-layer structure.

As an important image classifying tool, CNN has the ability to extract hierarchical features from original images. CNN can gradually extract high-level semantic features from low-level pixel features by processing multiple convolutional layers layer-by-layer. However, training CNN requires a large amount of data, which may lead to a decrease in the model's generalization ability. To address this issue, FL is proposed. Its core idea is to train models in a distributed manner across multiple devices, rather than concentrating all data on one device ^[15]. In FL, the updating and aggregation of model parameters are represented by Eq. (1).

In Eq. (1), $G_{t} +1$ represents the model parameters for the $t+1$ round. $G_{t} $ refers to the model parameters for round $t$. ${\eta }$ is the learning rate. ${n}$ means the number of samples. ${m}$ represents the number of devices participating in FL. $L{i}_{{t}} +1$ refers to the model parameters of the $i$th device in round $t+1$ ^[16]. FL can be classified based on the characteristics of the data and model of the participants. According to the distribution of the data sources of the participants, FL can be divided into horizontal FL, vertical FL, and federated transfer learning ^[17]. Fig. 2 is a schematic diagram of FL in both horizontal and vertical directions.

Horizontal FL is suitable for scenarios where the dataset of participants has the same features but different users. Vertical FL is suitable for situations where the participants in the dataset have the same user but different characteristics. Federated transfer learning is suitable for situations where the participating dataset users and features are different ^[18]. Fig. 3 shows the framework of FL.

In Fig. 3, FL mainly contains two parts, namely FL server and multiple client devices. The FL server is the core of the entire FL system, responsible for coordinating and managing communication and data exchange between various client devices ^[19]. The client device is an important component of FL, responsible for local model training and data preprocessing. The objective function of the central server is represented by Eq. (2).

In Eq. (2), $f_{i} (w)$ represents the loss function of parameter $w$ in the sample. ${m}$ refers to the total client devices when training. ${i}$ means the number of clients, then the objective function of the ${i}$th client is represented by Eq. (3).

In Eq. (3), $d_{k} $ represents the local dataset of the $k$th client. Adaptive gradient descent algorithm is an optimization method mainly used for parameter optimization in machine learning and deep learning models. Common adaptive gradient descent algorithms include Adagrad, Adadelta, Adam, etc. Adagrad can automatically adjust the learning rate based on the historical gradient size of each parameter, for sparse or non-stationary data. The historical sum of parameter gradient squared is represented by Eq. (4).

In Eq. (4), $cache$ represents the accumulator. At each time step $t$, the gradient on the $i$th dimension of the parameter vector $f$ is $gradient$. Eq. (5) is the update of the parameter vector.

In Eq. (5), $learning\_ rate$ represents the learning rate. $epsilon$ refers to a smaller constant used to prevent errors in dividing by zero. Adadelta utilizes the Root Mean Square Prop (RMSprop), which scales the learning rate by exponentially moving the average of the gradient squared. In the Adadelta algorithm, the exponential decay moving average of the square of the parameter update difference $\Delta W$ is represented by Eq. (6).

In Eq. (6), $\Delta X2t-2$ represents the $\Delta X2t$ value of the previous step. $\Delta Wt-1$ refers to the difference in parameter updates for the current step. $\odot $ means element wise multiplication, which is the Hadamard product. The parameter update of Adadelta is represented by Eq. (7).

In Eq. (7), $ht$ represents the exponential decay moving average of the square of the gradient $\partial L\partial W$ for each iteration. $\varepsilon $ refers to a smaller constant ^[20]. Adam is a stochastic optimization method with adaptive momentum. The first-order moment estimation calculation of gradient information is represented by Eq. (8).

In Eq. (8), $g$ represents the current gradient. $t$ refers to the quantity of iterations. $\beta _{1} $ is the first-order moment's exponential decay rate of the gradient. Adam combines the characteristics of Adagrad and RMSprop, while considering the exponential moving average of historical gradients and gradient squares of parameters. Therefore, the Adam algorithm is integrated into the FL image classification algorithm to improve the training efficiency and performance of the model. The study process does not screen the demographics of specific medical image data or the details of ethical considerations based on the subjects' medical data.

3.2. Construction of Federated Learning Privacy Protection Model Based on Fedavg

Privacy protection occupies a core position in FL systems, mainly because such systems often involve a large amount of user data. In traditional model training, data are usually stored centrally and trained uniformly. However, this method carries the risk of data privacy leakage. Due to all data being stored in the same location, user privacy information may be stolen in the event of data leakage. To address this issue, homomorphic encryption technology is introduced into FL systems. Homomorphic encryption is based on the computational complexity theory of mathematical problems, allowing for the processing of encrypted data and obtaining an output ^[21]. After decrypting this output, the result is consistent with the output obtained after the same processing as the unencrypted original data. This technology provides strong privacy protection for data processing in FL systems, ensuring the security of user data while maintaining the accuracy of data processing and analysis. Fig. 4 shows homomorphic encryption technology.

Assuming that $E(m)$ represents the result of encrypting message $m$ and $F(m')$ represents the result of decrypting encrypted result $m'$, Eq. (9) needs to be satisfied in encryption.

In Eq. (9), $m$ represents a plaintext message. $c$ refers to ciphertext. In decryption, Eq. (10) needs to be satisfied.

The decrypted plaintext is equal to the original plaintext. For any function $f(x)$, if there are valid algorithms A and B, the homomorphic expression of the encryption method is represented by Eq. (11).

In Eq. (11), $F$ represents the decryption function. $E$ refers to the encryption function. $f$ is a mapping function from plaintext messages to ciphertext. FedAvg is the first algorithm proposed by Google that fully defines the federated optimization process, allowing multiple users to train a machine learning model simultaneously. There is no need to upload any private data to the server during training. Local users are responsible for training local data to obtain local models. The central server is responsible for weighted aggregation of local models to obtain global models. After multiple iterations, a model that tends towards centralized machine learning results is finally obtained. This effectively reduces many privacy risks associated with traditional machine learning source data aggregation. Fig. 5 shows the FL structure based on FedAvg.

In FedAvg's FL, the optimization objective function for each medical institution is represented by Eq. (12).

In Eq. (12), $L(\hat{y},y)$ refers to the loss function at each medical institution. The loss function at each medical institution is represented by Eq. (13).

In Eq. (13), $\theta $ refers to the model parameters. $h(x\wedge (i); \theta )$ represents the predicted output of the model for the $i$th sample. $y\wedge (i)$ refers to the true label of the $i$th sample. $\Sigma $ is the sum of all samples ^[22]. The study will scale local parameters for updates to replace the original parameter update amount to enhance the protection of specific participating clients. The scaled parameter update amount is represented by Eq. (14).

In Eq. (14), $m_{k} $ represents the local model updated by each client. In FL, a method of adding noise in each iteration update is studied to blur the update amount of the local model. After the $i$th iteration, the update of the global model is represented by Eq. (15).

In Eq. (15), $\tau ^{2} S^{2} $ represents the average noise. Fig. 6 shows the construction of MIC based on FL.

In Fig. 6, firstly, based on hybrid reality technology, a secure Transport Layer Security (TLS) channel is established between various medical institutions and central servers ^[23]. To ensure the privacy and security of data in the entire FL system, a homomorphic encryption module is adopted. After starting the model training process, the central server and various medical institutions follow the FL process for model training. During this process, the transmission of data and model parameters is protected through homomorphic encryption modules. To ensure the privacy of data from various medical institutions, the classification model is evaluated locally using a test set. After the training is completed, each medical institution can deploy the obtained optimal model to their own platform for practical application. The accurate analysis of medical image data can be realized on the premise of ensuring data privacy through the above processes.

The data set is divided into training set, verification set, and test set according to the ratio of 8:1:1 in the data preprocessing stage to verify the performance of this algorithm. Among them, 80% of the data are used as a training set, namely 152 images, for the algorithm training. Of the remaining 20%, half of the data are used as a validation set, a total of 19 images, to adjust the hyperparameters of the model and optimize the model structure during training. The other half of the data are used as a test set, a total of 19 images, to objectively evaluate the model's performance on previously unseen data. After the model training is completed, the test set data are input into the trained model. The prediction results of the model are compared with the real labels of the test set to calculate the performance indicators of the model. Specificity and sensitivity are two important indexes to evaluate the performance of the model. Specificity can measure the model's ability to correctly identify negative cases. Specificity is calculated by counting the samples in the test set correctly judged negative by the model and the negative samples incorrectly judged positive by the model. Sensitivity is a measuring index of the model's ability to correctly identify positive examples. The sensitivity is calculated by counting the samples in the test set correctly judged positive by the model and the positive samples incorrectly judged negative by the model.

4.1. Performance Analysis of Federated Learning Image Classification Algorithm Based on Adaptive Gradient Descent

To verify the performance of the research algorithm, in the data preprocessing stage, the research adopted a fixed training method and divided the data set into training set, verification set, and test set according to the ratio of 8:1:1. Among them, 80% of the data were used as a training set, namely 152 images, for the algorithm model training. Of the remaining 20%, half of the data were used as a validation set, a total of 19 images, to adjust the hyperparameters of the model and optimize the model structure during training. The other half of the data were used as a test set, a total of 19 images, to objectively evaluate the model's performance on previously unseen data. The traditional MIC algorithms SVM and KNN were selected as reference algorithms to carry out the accuracy comparison experiment. The programming environment used in the experiment was Python 3.7, the hardware facility used Intel(R) Core (TM) i7-10510U, the CPU frequency was 1.8GHz, and the memory was 16GB, which provided sufficient computing resources for the experiment to run complex algorithms and process a large amount of data. In terms of operating system, the research chose Windows 11 64-bit operating system. Fig. 7 shows the accuracy of different algorithms.

In Fig. 7, when the iteration was 3, the research algorithm's accuracy curve tended to stabilize, fluctuating around 95%. The traditional model KNN's accuracy curve tended to stabilize at an iteration of 25, fluctuating around 90%. The accuracy curve of SVM tended to stabilize at iteration 30, fluctuating around 85%. In summary, the research algorithm was significantly superior to these two traditional models in terms of iteration efficiency and accuracy. During training, the experiment delved into the impact of different numbers of client participation on FL training and compared the performance of FL with KNN and SVM with different client participation. The quantity of clients set ranged from 3 to 12 to comprehensively evaluate the algorithm performance in different scenarios. Fig. 8 shows the algorithm performance curves for different numbers of clients.

Fig. 8(a) shows a comparison of algorithm performance curves with 8-12 clients. In the case of 8-12 clients, as the clients increased, the Area Under the Curve (AUC) of these three algorithms showed a decreasing trend. However, compared to KNN and SVM, this research algorithm had shown superior performance. When the clients were 8, its AUC was the highest, reaching 92.23%. Fig. 8(b) shows a comparison graph of algorithm performance curves with clients of 3-7. When the clients were 3-7, the algorithm in this study showed better advantages compared to KNN and SVM. When the clients were 3, its AUC value was the highest, at 97.58%. To evaluate the performance trends of the research algorithm and its various local models during the increasing communication rounds, experimental verification was conducted on FL, KNN, and SVM, respectively, in Fig. 9.

Fig. 9(a) shows the comparison of algorithm performance curves for different communication rounds when the local clients are 8. As the communication rounds increased, the accuracy curves of the three algorithms showed an upward trend. When the communication rounds were 20, the accuracy of this research algorithm tended to stabilize at 88.54%. KNN tended to stabilize with an accuracy of 86.95% when the communication rounds were 25. When the communication rounds were 130, the accuracy of KNN reached 86.43%. Fig. 9(b) shows the comparison of algorithm performance curves for different communication rounds when the local clients are 12. As the communication rounds increased, the accuracy curves of these three algorithms also showed an upward trend. This research algorithm tended to stabilize when the communication rounds were 10, with an accuracy of 88.67%. KNN tended to stabilize with an accuracy of 86.25% when the communication rounds were 20. When the communication rounds were 60, KNN's accuracy reached 85.73%. According to Figs. 9(a) and 9(b), the performance of the research algorithm in the number of clients 3-7 was higher than that in the number of clients 8-12, which might be because the data distribution might become more diversified and complex with the increase of the number of clients, which increased the difficulty of model learning. To verify the performance of different deep learning algorithms in MIC, the research was based on adaptive gradient descent FL image classification algorithm and deep learning method. Convolutional Neural Networks (CNN), residual networks, ResNet, Transfer Learning, Generative Adversarial Networks (GANs) and Capsule Networks (CapsNet) were compared. The performance comparison table of different deep learning methods is shown in Table 1.

From Table 1, compared with other deep learning methods, the research algorithm had the highest accuracy of 95.4%, which indicated that it had high recognition accuracy in MIC tasks. In terms of overfitting value, the research algorithm had the lowest overfitting value, which was 0.015, which indicated that it effectively prevented the occurrence of overfitting in the training process. The performance of the research algorithm was better than other deep learning methods, and it effectively improved the accuracy of MIC.

4.2. Performance Analysis of Differential Privacy Medical Image Classification Model Based on Federated Learning

The learning rate directly affects the efficiency and final performance of model training. To comprehensively evaluate the research algorithm's performance under different learning rates, three representative learning rates of 0.1, 0.01, and 0.001 were selected for experiments. Meanwhile, robustness comparison experiments were conducted between the models under these learning rates and traditional KNN. Fig. 10 shows the impact of initial learning rate on model performance.

Fig. 10(a) presents KNN's accuracy line graph under different learning rates. When the learning rate was 0.1 and the iteration was 9, the accuracy of KNN reached its highest level, at 98%. Fig. 10(b) presents KNN's loss curve under different learning rates. When the iteration was 8 and the learning rate was 0.1, the loss curve of KNN tended to stabilize, with a loss value of 0.07, which was the lowest. Fig. 10(c) presents the research algorithm's accuracy line graph under different learning rates. When the learning rate was 0.1 and the iteration was 10, the accuracy of the research algorithm was the highest, at 99.82%. Fig. 10(d) presents the research algorithm's loss curve under different learning rates. As iterations increased, the loss curve of the research algorithm became smoother compared to KNN. In summary, under the same learning rate, the research algorithm had higher accuracy and smoother loss curve compared to KNN. To verify the impact of different subsets on the research algorithm, Task 1 was set to divide the dataset into 10 subsets, and Task 2 was set to divide the dataset into 5 subsets. Table 2 shows the performance comparison of different models under different conditions.

In Table 2, the research algorithm in Task 1 outperformed KNN and SVM in F1, AUC, accuracy, and specificity indicators. The F1 of this research algorithm was 78.93%, AUC was 90.38%, accuracy was 90.11%, specificity was 89.37%, sensitivity was 77.25%, higher than 76.49% of KNN and 75.31% of SVM. In Task 2, the research algorithm also performed well. Its F1 reached 89.96%, AUC was 98.05%, accuracy was 95.27%, specificity was 95.89%, and sensitivity was 89.35%. In summary, this research algorithm had excellent performance and applicability in FL. These data indicated that the research algorithm had good generalization ability and stability and could adapt to the needs of different datasets and task scenarios. Fig. 11 shows the loss convergence curves of the study model for server-side training and local training under different subsets.

Fig. 11(a) shows the training loss curve of the server for Task 1. When the iterations reached 50, the loss convergence curve of the research algorithm tended to stabilize, and its loss value was 0.18. Fig. 11(b) shows the training loss curve of the server for Task 2. When the iterations reached 45, the loss convergence curve of the research algorithm tended to stabilize. Fig. 11(c) shows a comparison of the loss curves between the local training model and the research algorithm for Task 1. As the iterations increased, its loss curve's convergence trend was consistent with that of the local training model. Fig. 11(d) presents the comparison of the loss curves between the locally trained model and the research algorithm in Task 2. Their loss curves showed a consistent trend. The final loss value of the research algorithm was 0.38, indicating that it had good learning performance.