3.1 Optimization design of full-angle AEA based on EKF

In sports action recognition, accurately tracking and estimating an athlete's dynamic posture is a challenging task, especially when fast and complex motion sequences are involved. The key of action recognition lies in how to extract useful dynamic information accurately and timely from real-time data. Traditional motion recognition technology, such as video analysis or simple sensor data processing, is often limited by data processing delay and noise interference, and it is difficult to meet the requirements of high precision and real-time. Therefore, the extended Kalman filter is used to design a new motion recognition algorithm. The extended Kalman filter can update state estimates at each time step based on new observed data, thus providing continuous and accurate prediction of dynamic system state. This feature makes it particularly suitable for processing real-time multi-dimensional dynamic data collected by MEMS sensors during sports movements. By continuously predicting and updating the state of the system, EKF can not only effectively suppress the observation noise, but also adapt to the rapid changes in the process of motion ^[12]. In the application scenario of this study, due to the existence of numerous nonlinear functions, EKF was used to process these nonlinear discrete systems. The attitude estimation framework structure under the EKF algorithm is Fig. 1.

In Fig. 1, EKF serves as a nonlinear form of Kalman filtering, with the attitude quaternion of the vehicle as the state variable, the gyroscope reading as the prediction step, and the attitude angle output from the accelerometer and magnetometer as the observation update value. The various updated values obtained are processed using EKF, and then the attitude estimation results are output.

In the EKF algorithm, assuming the state of a nonlinear discrete system is $X$, the state history at time $k$ is obtained as shown in Eq. (1).

In Eq. (1), $X(k)$ represents the state variable at time $k$. $U$ represents the output variable. $W$ represents the process noise sampled. $f\left(\cdot \right)$ represents the state function.

Assuming that the output variable of a nonlinear discrete system is $Z$, the output variable equation at time $k$ is obtained as shown in Eq. (2).

In Eq. (2), $Z(k)$ represents the output variable at time $k$. $V$ represents the sampled measurement noise. $h\left(\cdot \right)$ represents the output function.

The statistical characteristics of process noise and measurement noise can be expressed using Eq. (3).

In Eq. (3), $W(k)$ and $V(k)H$ represent the process noise and measurement noise values at time $k$, respectively. $W(k)\sim N\left(0,Q\right)$ represents that $W(k)$ is a multivariate normal distribution with a mean of 0 and a covariance matrix of $Q$. $V(k)\sim N\left(0,R\right)$ represents that $V(k)$ is also a multivariate normal distribution with a mean of 0 but a covariance matrix of $R$.

Considering that both $W$ and $V$ belong to zero mean white noise sequences and are not correlated with each other, first-order linearization can be applied to Eqs. (1) and (2) to obtain the processed first-order state equation and output equation as shown in Eq. (4).

In Eq. (4), $A$ and $\Gamma $ are the partial derivatives (PD) of the state function with respect to $X$ and $W$. and $\wedge $ represent the PD of the output function over $X$ and $V$, respectively. $\tilde{X}(k)$ and $\tilde{Z}(k)$ are the approximate values of $X(k)$ and $Z(k)$ at time $k$, respectively. $\hat{X}$ represents a posterior estimate of $X$.

The Kalman filtering algorithm is applied to the first-order state equation and first-order output equation in Eq. (4), and the formula for calculating the filter gain is Eq. (5) ^[13].

In Eq. (5), $K_{g} (k)$ represents the filter gain at time $k$. $\bar{P}(k)$ is the covariance matrix of the estimation error at time $k$. $H^{T} $ and $\wedge ^{T} $ represent the transposes of $H$ and $\wedge $, respectively. $R(k)$ represents the covariance matrix of measurement noise at time $k$. $H(k)$ and $\wedge (k)$ represent the PD of the output function with respect to $X$ and $V$ at time $k$.

According to Eqs. (1) to (5), the time and state update equation of the nonlinear dynamic model under the EKF algorithm can be obtained. The update equation for time is Eq. (6).

In Eq. (6), $\bar{X}(k)$ represents the estimated state at $k$. $A(k)$ represents the PD of the state function over $X$ at $k$. $\hat{P}$ represents a posterior estimate of $P$. $A^{T} $ and $\Gamma ^{T} $ represent the transposes of $A$ and $\Gamma $, respectively. The update equation for the state is Eq. (7).

According to Eqs. (6) and (7), different real-time update time values and update status values can be obtained. The obtained values are filtered and processed, and then the athlete's pose estimation results are output.

In the accurate estimation of motion attitude, MEMS sensors are widely used in dynamic data acquisition because of their miniaturization and portability. However, these sensors are susceptible to various factors during fast or complex motion capture processes, such as mechanical vibration and temperature changes, which can lead to increased errors and noise in data acquisition. Especially in the acquisition of angular velocity and acceleration data, the output data of MEMS sensors often contain large errors and deviations due to the nonlinear characteristics and limited measurement range, which will affect the accuracy and reliability of attitude estimation. Therefore, this study introduces a full angle attitude calculation method (FAACM) to optimize the EKF algorithm. By considering all possible attitude Angle variations, this method enables more comprehensive utilization of the output data of MEMS sensors. In addition, by introducing this method for optimization, nonlinear and high dynamic changes in MEMS sensor data can be handled more effectively, ensuring the robustness and high-precision performance of the algorithm in practical applications.

In order to overcome the problem of excessive pitch angle leading to increased error in roll angle calculation, this study first performs a rotation operation on the coordinate system, and then calculates quaternions in that rotation coordinate system. Next, the coordinate system is rotated back to its initial state, and after this process, the attitude quaternion is obtained. This FAACM can effectively calculate the attitude angle in a wide range of pitch angles and avoid increasing the error in roll angle calculation.

Before introducing the rotational coordinate system, the roll angle measured by the MEMS accelerometer is Eq. (8).

In Eq. (8), $\theta $ represents the initial measured roll angle. $g$ represents gravitational acceleration. $y_{g} $ represents the projection of gravitational acceleration on the Y axis. At this point, the formula for calculating the attitude angle is Eq. (9).

In Eq. (9), $\phi $ represents the initial measured attitude angle. $x_{g} $ and $z_{g} $ represent the projection of gravitational acceleration on the X and Z axes, respectively. Introducing a new rotating coordinate system, the roll angle and attitude angle under the FAACM are obtained as shown in Eq. (10).

In Eq. (10), $\theta^{ '}$ and $\phi^{ '}$ represent the roll angle and attitude angle in the rotating coordinate system, respectively. $y_{g} {}^{{'} } $, $x_{g} {}^{{'} } $ and $z_{g} {}^{{'} } $ represent the projections of gravitational acceleration on the Y-axis, X-axis, and Z-axis of the rotating coordinate system, respectively. According to Eq. (10), further to calculate the value of azimuth, as shown in Eq. (11).

In Eq. (11), $\Psi $ represents the azimuth in the rotating coordinate system. $y^{'}_{2T}$ and $x^{'}_{2T}$ represent the projections of the geomagnetic field vector on the $Y$-axis and $X$-axis of the rotating coordinate system, respectively. The quaternion rotating in a rotating coordinate system is denoted as D, and its expression is Eq. (12).

Combining Eqs. (8) to (12), the formula for calculating the final attitude angle is Eq. (13).

By combining the FAACM based on rotational coordinate system with EKF, the optimized algorithm is recorded as Extended Kalman Filter Full Angle Attention Estimation Algorithm (EKF-FAAEA). The operational flowchart of EKF-FAAEA is Fig. 2.

Fig. 2 shows the operation flow of the EKF-FAAEA algorithm, where $\alpha $ and $\alpha _{0} $ represent the calculated pitch Angle and pitch Angle threshold respectively. In Fig. 2, the data of the accelerometer and magnetometer in the MEMS sensor system were first obtained, and then the pitch angle was calculated using the EKF algorithm. To compare the pitch angle calculation value with the threshold value. If the pitch angle calculation value ${\leqslant}$ the threshold value, directly to use EKF to calculate the roll angle and azimuth angle, then to obtain the quaternion of the attitude angle, and finally output the result. If the calculated elevation angle is greater than the threshold, the attitude angle is calculated based on the rotation coordinate system and the quaternion in the rotation coordinate system is output.

3.2 Construction of a sports action recognition model integrating FAAEA and MEMS

After completing the optimization design of the EKF-FAAEA algorithm, this study further analyzed the performance indicators of various MEMS sensors and built a sports action recognition model combining AEA and various MEMS sensors. The aim of this study is to collect athlete movement information and perform AE-MR using this model. The action recognition model built is referred to as EKF-FAAEA-MEMS, and the framework structure of EKF-FAAEA-MEMS is Fig. 3.

In Fig. 3, the entire EKF-FAAEA-MEMS framework mainly consists of three parts: attitude measurement unit (AMU), AEA, and data processing and communication (DPC). AMU is composed of various types of MEMS sensors, which athletes wear on various parts of their bodies to obtain initial motion data ^[14]. AEA calculates the posture information of athletes based on the data collected by AMU. The values calculated by AEA can be stored in the DPC unit, which mainly collects, stores, and sends real-time data information to the computer to assist operators in completing final action recognition based on attitude estimation results. The hardware structure design of EKF-FAAEA-MEMS is Fig. 4.

In Fig. 4, the hardware design of the entire model is mainly focused on the AMU and DPC units ^[15]. AMU mainly consists of MEMS sensors such as accelerometers, gyroscopes, magnetometers, and microprocessors. Considering the different requirements for sensor size, wearing method, and accuracy in practical applications, this study designed two types of AMUs, one is a small AMU and the other is a multifunctional AMU. The DPC unit includes attitude data storage unit (ADSU) and Bluetooth communication unit (BCU). ADSU can store various motion information collected by AMU in a storage card when the data volume is too large or real-time calculation is not required, and offline data processing can be performed at this time. BCU is mainly responsible for real-time sending various collected motion information to data relay nodes, achieving real-time collection, calculation, and analysis functions of human motion information. The specific composition structure of AMU is Fig. 5.

Fig. 5 shows the structural design of miniature AMUs and multifunctional AMUs, where SPI represents the Serial Peripheral Interface. When designing miniature AMUs, due to strict requirements for volume and weight, an integrated three-axis accelerometer, gyroscope, and magnetometer nine axis sensor MPU9250 was chosen. Considering the insufficient accuracy of the built-in magnetometer in MPU9250, a higher precision three-axis MEMS magnetometer HMC5983 was used as a replacement, and STM32F401CEU6 was selected as the microprocessor chip. For multifunctional AMUs, due to lower volume requirements, higher precision MEMS sensors ADXL355, ADXRS453, and AK09970N, as well as the more powerful STM32F407VGT6 microprocessor, were selected to meet higher measurement accuracy and expansion requirements. To enhance the stability of the multifunctional AMU, this study chose to use the 9-axis motion sensor ICM20948 as a backup for the ADXL355 accelerometer, ADXRS453 gyroscope, and AK09970N magnetometer to achieve redundant functionality. The extended functional structure diagram of the multifunctional AMU is Fig. 6.

In Fig. 6, the expanded multifunctional AMU can add a global positioning system or ultra wideband chip according to actual needs, thereby achieving indoor and outdoor motion positioning. Due to its low volume requirements, this unit can expand more human motion information collection functions and is suitable for installation on the waist, back, or sports equipment of the human body. In addition to global positioning systems and ultra wideband chips, pressure sensors and airspeed sensors can also be integrated to measure the height and speed of movement. The expanded multifunctional AMU is more conducive to posture measurement during winter sports.

4.1 Performance testing of FAAWA

The publicly available SportsPose serves as the dataset for the experimental process. SportsPose is a 3D human pose dataset designed specifically for sports action recognition, which contains over 176000 3D poses. These postures were collected from 24 different subjects while engaging in 5 different sports activities. The five sports activities are running, basketball, badminton, swimming, and high jump. 176000 3D poses were divided into training and testing sets in an 8:2 ratio. To avoid experimental errors, this study conducted various experiments using the same equipment. Table 1 shows the specific parameter settings.

Table 1 provides the parameter settings for this experimental environment. EKF, Mask-Region-Based Convolutional Neural Network (Mask R-CNN), and Pose Network (PoseNet) are used as comparison algorithms. The iterative fitness values of EKF, Mask R-CNN, PoseNet, and EKF-FAAEA under the same experimental environment were obtained, as shown in Fig. 7.

Fig. 7 shows the iteration of fitness values for EKF, Mask R-CNN, PoseNet, and EKF-FAAEA. As the number of iterations increases, the optimal fitness values of the four pose estimation algorithms are continuously decreasing. When the number of iterations is 58, 56, 45, and 37, EKF, Mask R-CNN, PoseNet, and EKF-FAAEA all begin to reach a stable state, and the optimal fitness values for each algorithm are 0.23, 0.20, 0.26, and 0.18, respectively. By comparing the fitness value iteration of the four algorithms, it can be found that the decline speed and final value of EKF-FAAEA are superior to the other three algorithms, indicating that EKF-FAAEA algorithm has better performance when dealing with non-linear and dynamic sports action data. The reason for this result is that the full Angle attitude calculation method and nonlinear processing method are introduced into the EKF-FAAEA algorithm, so that EKF-FAAEA can track and predict the dynamic attitude of athletes more accurately.

Figs. 8(a) and 8(b) show the fluctuation results of pose estimation errors for EKF, Mask R-CNN, PoseNet, and EKF-FAAEA in the training and testing datasets, respectively. In Fig. 8(a), the fluctuation ranges of pose estimation for EKF, Mask R-CNN, PoseNet, and EKF-FAAEA in the training set are $-0.2\sim0.3$, $-0.2 \sim 0.2$, $-0.1 \sim 0.2$, and $-0.1 \sim 0.1$, respectively. In Fig. 8(b), the fluctuation ranges of pose estimation for EKF, Mask R-CNN, PoseNet, and EKF-FAAEA in the test set are $-0.2 \sim 0.3$, $-0.2 \sim 0.3$, $-0.1 \sim 0.2$, and $-0.1 \sim 0.1$, respectively. Overall, the error fluctuation range of EKF-FAAEA is the smallest, indicating that this algorithm has the most accurate estimation of posture.

Table 2 shows the measured values of each roll angle under different algorithms. When the actual roll angles are $-30^\circ$, 0$^\circ$, 30$^\circ$, and 60$^\circ$, the measured values of EKF are $-31.24^\circ$, 1.05$^\circ$, 31.08$^\circ$, and 61.32$^\circ$, respectively. The measurement values of Mask R-CNN are $-30.38^\circ$, 0.52$^\circ$, 30.47$^\circ$, and 60.63$^\circ$, respectively. The measurement values of PoseNet are $-30.18^\circ$, 0.09$^\circ$, 30.14$^\circ$, and 60.31$^\circ$, respectively. The measured values of EKF-FAAEA are $-30.01^\circ$, 0.02$^\circ$, 30.03$^\circ$, and 60.01$^\circ$, respectively. EKF-FAAEA has a very small error in the measurement of each roll Angle, which is almost close to the actual value, showing a high accuracy. In contrast, other algorithms such as Mask R-CNN and PoseNet, while also showing high accuracy, still have slight deviations in the measurement of some angles. The high precision of EKF-FAAEA algorithm is due to its application of MEMS sensor optimization processing and data fusion technology in the algorithm, which effectively reduces the impact of sensor error and environmental noise, and improves the algorithm's adaptability to dynamic changes and measurement accuracy. Overall, the measured values of EKF-FAAEA are closest to the actual values, so this algorithm can more accurately estimate the specific pose angle.

4.2 Analysis of the application effect of sports action recognition model

To verify the good practicality of the designed EKF-FAAEA-MEMS model, this study selected object keypoint similarity (OKS), recognition time, and recognition accuracy as evaluation indicators. The experiment compared the recognition performance of three models: EKF-FAAEA-MEMS, 3D CNNs, and long short term memory networks (LSTM) in actual badminton sports. Fig. 9 shows the variation of OKS values for EKF-FAAEA-MEMS, 3D CNNs, and LSTM in different datasets.

Figs. 9(a) and 9(b) show the OKS values of EKF-FAAEA-MEMS, 3D CNNs, and LSTM in the training and testing datasets, respectively. In Fig. 9(a), as the number of training samples increases from 0 to 100, the OKS values of all three models show a trend of first increasing and then stabilizing. Finally, the optimal OKS values for EKF-FAAEA-MEMS, 3D CNNs, and LSTM were 0.98, 0.88, and 0.75. In Fig. 9(b), the optimal OKS values for EKF-FAAEA-MEMS, 3D CNNs, and LSTM in the test dataset are 0.97, 0.86, and 0.77. The OKS value is usually within 0 to 1. The closer the value is to 1, the closer the predicted key points are to the actual key points, indicating better performance in motion recognition. In summary, the EKF-FAAEA-MEMS model has better action recognition performance.

Figs. 10(a)-10(c) show the accuracy and time consumption of EKF-FAAEA-MEMS, 3D CNNs, and LSTM in identifying three different badminton movements, respectively. In Fig. 10, numbers 1, 2, and 3 correspond to three types of badminton movements: serve, drop, and spike. EKF-FAAEA-MEMS has the highest accuracy in identifying serve, drop, and spike movements, with values of 0.99, 0.98, and 0.98, respectively. It takes the shortest time, with values of 2.01 seconds, 1.88 seconds, and 1.96 seconds, respectively. The accuracy of 3D CNNs in identifying serve, drop, and spike movements is 0.87, 0.92, and 0.85, respectively, with a time consumption of 6.86 seconds, 4.82 seconds, and 4.98 seconds. The accuracy of LSTM in identifying serve, drop, and spike movements is 0.76, 0.84, and 0.78, respectively, with a time consumption of 7.21 seconds, 10.96 seconds, and 8.64 seconds.

In Fig. 11, three movements of serving, dribbling, and spiking are selected for recognition. Each set of images, from left to right, displays the recognition results of the original image, EKF-FAAEA-MEMS, 3D CNNs, and LSTM. Figs. 11(a)-11(c) indicate that the recognition box of EKF-FAAEA-MEMS can select all postures of athletes, thus enabling better recognition of their movements. On the contrary, the recognition boxes of 3D CNNs and LSTM cannot accurately frame all postures, so they cannot recognize various actions well.