2.1. Bidirectional Encoder Representations from Transformers

A feedforward neural network and self-attention mechanism construct the Transformer. The self-attention mechanism focuses on the current vocabulary and considers the context and context, significantly enhancing the ability to comprehend the context ^{[10, 11]}. The Transformer adopts an Encoder-Decoder structure, each composed of 6 repeating modules, as shown in BERT-Base. The encoder consists of 6 layers: the first layer is Multihead-Attention, the second layer is feedforwarding neural network, residual connection, and layer standardization are applied between layers, and the output dimension is 512. The decoder structure is similar; the first layer is the mask Multihead-Attention, and the last two layers correspond to the encoder, which also adopts residual and standardized operations ^[12].

BERT is a pre-trained model based on Transformer architecture. It has vibrant language knowledge representation through self-supervised learning on massive unlabeled text, especially for context-sensitive vocabulary understanding and complex sentence meaning analysis.

BERT applies the Attention mechanism to modern deep learning to improve the performance and alignment effect of machine translation ^[13]. Imagine the constituent elements in Source as composed of a series of < Keyi, Valuei > data pairs. Given a specific query item in the target, calculate the similarity between the Query and different Key, perform weighted summation, and calculate the corresponding weight coefficient of Value. The formula is shown in Eq. (1).

The Similarity table measures the correlation between the Query vector (Query) and the Key vector (Key). Value represents the information content associated with each Key, a vector that stores specific information, where L is the length of the input sentence, Query is the word sequence in Target, and (i = 1, 2, ..., n) is the specific sequence coding. Source points to the semantic coding corresponding to each word in the input sentence, so the result obtained is more accurate ^[14]. The calculation steps of the Attention mechanism are roughly as follows: first, the weight coefficient is calculated according to Query and Key, and the process can be subdivided into first calculating the similarity or correlation between Query and Key and then performing normalization processing Finally, the Value is weighted and summed according to the weight coefficient.

In the first stage, a two-vector dot product, two-vector Cosine similarity, or neural network is used to calculate the similarity between Query and Key. The formulas are shown in Eqs. (2)-(4), where MLP represents a multi-layer perceptron.

Query represents the characteristics of content found or retrieved from the information set. Key matches the Query and determines which parts of the information set the Query is related to. In the second stage, a function is introduced to numerically normalize the calculation results of the previous stage, using the following formula (5): SoftMax is the activation function, and e is the input vector.

A represents a specific variable, and L is the input sentence length. In the third stage, the calculation result Ai of the weight coefficient corresponding to Valuei is weighted and summed, as shown in formula (6):

The Valuei represents the corresponding weight coefficient. The attention mechanism is simple to calculate, with few parameters and fast speed, which solves the problem of RNN being unable to be calculated in parallel ^{[15, 16]}. Due to distance limitation, NLP effectively reduces the dependence complexity between the source sequence and target sequence, and the empirical results show that the model has a superior effect. The Key part of BERT is the self-attention mechanism. First, the self-attention mechanism will calculate three new vectors: Query (v), Key (w), and Value (x). The vectors have the same dimension. The self-attention mechanism mainly obtains the representation of words by adjusting the association coefficient matrix between words and words in sentences, and the formula is shown in Eq. (7).

Among them, Q, K, and V are the input vector matrices of characters, the dimensions of Q and K are both dk and the vector dimension of V is dv. The multi-head attention mechanism of BERT projects Q, K, and V through linear transformation, apply Scaled Dot-Product Attention and splices different self-attention results to extract sentence semantics. The formulas are shown in Eqs. (8)-(9).

Among them, $W_i^Q \in$ Rdmodelxdk, $W_i^K \in$ Rdmodelxdk, $W_i^V \in$ Rdmodelxdk, WO $\in$ Rdmodelxdk, head is the number of attention heads, i is the head index, and WO is the weight parameter. Regarding calculating probability, specifically, the probability of the next word appearing is calculated from left to right ^[17]. It is mainly a training set composed of sentences w1, w2, ..., wm, and then a language model with occurrence probability is obtained by neural network training, as shown in formula (10).

p indicates the probability that the next word will appear. S denotes the input sequence, and w refers to the number of sequences. Traditional language models are static and cannot represent word meaning and grammar dynamically. Pre-trained models such as ELMo, GPT, and BERT are pre-loaded with semantic information, and large-scale corpus is used to enhance word meaning expression, improve model robustness, and avoid repeated training from zero ^{[18, 19]}. BERT is based on Transformer’s Encoder, which has a deeper model structure, efficiently captures context, and significantly enhances feature extraction capabilities ^[20]. BERT training is divided into two stages: pre-training and fine-tuning, and its structure is shown in Fig. 1. Pre-training includes a mask language model and next-sentence prediction, and the fine-tuning stage is adjusted for specific tasks.

Fig. 1. BERT structure.

The pre-training of BERT includes a mask language model and next sentence prediction (NSP) ^[21]. In the masking task, 10% of words are randomly masked, of which 80% are replaced with [Mask], 10% remain unchanged, and 10% are replaced with random words. The Chinese dataset uses full-word masking. The NSP task predicts whether two sentences are connected and uses [CLS] and [SEP] to mark the sentences. If there is a connection, [CLS] outputs IsNest; otherwise, it outputs NotNest. The BERT input contains [CLS] and [SEP] tags to identify the beginning and end of the sentence. The input comprises words, words, and position vectors; sentence vectors assist NSP tasks; position vectors supplement time series information, and the model learns sequence features through absolute coding.

2.2. Bidirectional Long Short-term Memory

BiLSTM is a variant that solves the gradient problem in recurrent neural networks. It efficiently uses context, deeply mines semantics, reduces workload, improves entity recognition accuracy, and is widely used for text semantic information extraction ^{[22, 23]}. BiLSTM contains three stages internally: the forgetting stage selects to forget old data through the forget gate; The memory stage stores new data through the input gate and updates the state; The output phase determines the output and activates the status information. One-way LSTM handles one-way information; two-way LSTM is more suitable for text context processing. The structure of BiLSTM can be represented by the following formulas (11)-(12):

where ft represents a forgetting gate, it represents an input gate, $\sigma$ is the sigmoid activation function, Wf and Wi represent the weight value and the output gate respectively, and ht−1 represents the hidden layer state at t − 1 time, xt represents the current input; bf, bi is the deviation term. LSTM stores key information through a gating mechanism and forgets unimportant parts. Compared with single-layer storage RNN, LSTM can dynamically capture information and have stronger memory. BiLSTM deeply mines long-distance text information through memory units and control gates, solves gradient disappearance, and has significant application value in information retrieval, automatic question answering, and knowledge graph construction.

2.3. BERT-BiLSTM Fusion Mechanism

The challenges we face include the increased difficulty of training and inference due to the high complexity of the model, the limited scale of training data that may limit the improvement of the generalization ability of the system, and the huge consumption of computing resources in the training and inference process. In response to these limitations, we explore building more efficient model architectures, leveraging larger and more diverse training datasets, and optimizing the use of computing resources.

The technical route of BERT-BiLSTM integration shows its unique charm and strong potential. It integrates the advantages of two deep learning models, dramatically improving the translation quality and speeding up the translation process, making it an ideal choice to meet the needs of modern communication ^{[24, 25]}.

In English phrase translation, BERT can keenly capture the specific contextual meaning behind each word, thus avoiding ambiguity and misunderstanding caused by literal translation and ensuring the accuracy of the translation. Meanwhile, BiLSTM, with its unique bidirectional memory unit, can simultaneously retain past and future contextual information in sequence data processing, which is crucial for correctly identifying and transforming grammatical structures in phrases ^{[26, 27]}. In the scenario of English phrase translation, BiLSTM can effectively track the relationship chain between words and help the model understand complex structures, such as attributive clauses and non-predicate verbs, so that the final translation is more in line with grammatical norms and reads naturally and fluently.

Integrating BERT and BiLSTM perfectly balances semantic deep mining and grammatical detail control ^[28]. In a specific implementation, the powerful pre-training function of BERT is usually used to obtain the high-dimensional semantic features of source language phrases. Then, these features are input into the BiLSTM network, and the latter can reorganize these features based on its excellent sequence processing skills to generate corresponding expressions in the target language. This process fully uses BERT’s advantages in word-level and sentence-level understanding. It gives full play to BiLSTM’s expertise in grammatical reconstruction, contributing to a significant leap in translation effect.

The BERT-BiLSTM convergence solution also brings additional efficiency gains. Since many common language patterns have been accumulated in the pre-training phase of BERT, only a small amount of fine-tuning often needs to be used in specific translation tasks, significantly reducing the data and computing resources required for model training. Coupled with the rapid response characteristics of BiLSTM when processing fixed-length inputs, it can effectively improve the operating efficiency of the overall system.

3.1. Genetic Algorithm Optimization

The BERT-BiLSTM model combines the bidirectional Transformer of BERT and the bidirectional memory of BiLSTM to construct a powerful language processing architecture. BERT accurately understands the source language and captures the context of the words; BiLSTM is further processed to capture sequence dependencies and improve the ability to translate complex syntax and long-distance dependencies. In order to optimize this model, we introduce a genetic algorithm to simulate natural selection and heredity and efficiently explore the optimal solution in parameter space, including initialization, selection, crossover, and mutation to termination criteria. The fitness function is designed to evaluate the performance of model parameters and guide the optimization process. Charts and flowcharts are provided to visually demonstrate the integration and interaction of BERT, BiLSTM, and genetic algorithms. At the same time, the implementation details, such as libraries and frameworks, hardware specifications, data preprocessing, training, and testing steps, are detailed, providing comprehensive guidance for system deployment and performance evaluation.

We selected individuals based on fitness, balancing exploration and utilization to avoid precocious convergence. In crossover, we combined genetic information using various techniques to expand the search space. Mutation introduced random changes to maintain diversity. We also addressed initialization, elite selection, and termination conditions, impacting algorithm performance. Our genetic algorithm optimized translation system parameters like learning rate, batch size, and hidden nodes. Despite challenges like high computational complexity and local optima, adaptive mutation, elite strategies, and parallel computing helped us find the optimal parameter combination, significantly improving translation accuracy, speed, and resource efficiency. Genetic algorithms outperformed traditional methods like grid and random search.

Genetic algorithm simulates “genetic inheritance” and “natural selection” to achieve optimization. Its core idea is “survival of the fittest.” When resources are limited, excellent individuals survive, inferior individuals are eliminated, and mutations may occur in reproduction to form new individuals ^[29]. As a global search tool, the genetic algorithm encodes feasible solutions into chromosomes, calculates and selects chromosomes with high fitness, generates new chromosomes through crossover and mutation, iteratively eliminates individuals with low fitness, and retains those with high fitness until the stop condition is met, and the optimal solution is output. Fig. 2 shows the genetic algorithm architecture. The main steps involve coding, population initialization, fitness function design, selection, crossover, and mutation operations.

Fig. 2. Genetic algorithm architecture.

Chromosome coding is the first step of the genetic algorithm, which affects crossover and mutation and is related to the feasibility of solution and algorithm performance. Commonly used codes include binary, natural numbers, floating-point numbers, etc. Although binary coding is efficient, improper length affects the search, and there may be no similarity between adjacent solutions, so it is not suitable for vehicle routing problems. Accurate coding is direct and clear, suitable for extensive space search, and accelerates the convergence to the optimum. Therefore, this paper uses accurate coding to solve the problem. Initializing the population affects the algorithm’s efficiency and the solution’s quality. Standard methods include random, clustering, and heuristic initialization. Population size is significant. If it is too small, it reduces the searchability, and if it is too large, it increases the amount of calculation. The fitness function reflects the advantages and disadvantages of individuals. The larger the value, the stronger the ability to adapt to the environment and the greater the probability of being retained. Its design is often related to the objective function of the problem, which directly affects the algorithm’s convergence. Select operation, that is, survival of the fittest, and individuals with high fitness are more likely to enter the next generation. Fitness guarantees excellent individuals in the later period.

Variable neighborhood search algorithm belongs to local search. Starting from any initial solution, it searches in a small range, expands the range when the solution quality is no longer improved, and retracts after the quality improvement. It is good at optimizing solutions, but its efficiency is low and suitable for TSP, image coloring, etc ^[30]. In variable neighborhood search, max-ordered neighborhoods k1 to max are combined, and the local optimum is searched from the minimum neighborhood k1. If it is better than the current optimum, the minimum neighborhood is replaced and re-searched; Otherwise, search for max step by step.

Individuals with high fitness have many breeding opportunities, so the selection operation screens them accordingly. There are various selection strategies of genetic algorithms, and the probability of the roulette strategy is proportional to fitness. However, the survival of the fittest is weak in the later stage, and the outstanding individuals may be eliminated by mistake. The elite retains and prevents the best, but the local optimum affects the global search. This paper adopts dynamic, elite retention combined with roulette optimization: less retention or no retention in the initial stage, full roulette selection. With the iteration, the number of outstanding individuals gradually increases, the right number of elites are retained, and the rest are roulette selected. At the beginning of the algorithm iteration, the pure roulette method is used to select individuals, and the elite is not retained. As the iteration deepens, the adaptability improves, and the number of elite retentions gradually increases to be constant in the middle of the iteration. Combining dynamic, elite strategy with roulette can not only prevent a single outstanding individual from dominating evolution and enhance population diversity but also ensure the retention of high-quality individuals, accelerate convergence and improve solution efficiency.

In genetic algorithm applications, crossover and mutation settings crucially affect searchability and solution quality. Too low probability leads to less gene exchange, reducing offspring diversity and risking premature convergence or local optima. Too high probability expands the search space but may disrupt existing excellent solutions, reducing efficiency. Mutation aims to broaden the search range; too low probability decreases population diversity, risking loss of high-quality solutions, while too high makes search nearly random, drastically increasing time cost. Hence, parameter settings must balance search efficiency and solution quality based on the specific problem.

The computational resource consumption of model training and inference is critical. In the training stage, the deep structure and bidirectional LSTM layer of BERT-BiLSTM lead to long-term consumption, and the iterative genetic algorithm increases the time cost. Large-scale datasets and model parameters significantly increase memory usage, and GPU-accelerated training comes with increased energy consumption and cost. In the inference stage, the time consumption is less, but the text needs to be processed and the results are generated, the memory needs to be sufficient, and the GPU utilization can reduce latency, but when the real-time performance requirements are high, more efficient algorithms or hardware acceleration are required.

3.2. Construction of College English Phrase Translation System

Data preprocessing includes three parts: data cleaning, word segmentation, and annotation. During the data cleansing process, we remove extraneous information, correct typos, and standardize text formatting to improve dataset quality and reduce noise from degraded model performance. In the word segmentation stage, we precisely divide the text into meaningful units, such as words or phrases, a step that is critical to the translation system because it is directly related to whether the model can correctly understand and translate the text. The annotation process involves labeling the data with the correct translation, either manually annotated by experts or aided by automated tools to ensure that the model learns the correct translated phrases. In particular, we discuss the combined impact of these preprocessing steps on model performance and present experimental results that demonstrate that data cleaning, word segmentation, and annotation significantly improve the accuracy and efficiency of translation systems.

The English phrase recognition system covers five steps: preprocessing, positioning, correction, segmentation, and recognition, each of which must be efficient and accurate. The system integrates machine vision, image processing, and feature recognition technologies to achieve automatic recognition. It is composed of hardware (such as mobile phones) and software (image processing, character recognition algorithms), with hardware supporting operation and software processing data. Text positioning is the foundation that directly affects the recognition effect and system performance, and the key is to extracting image text information. In this study, a genetic algorithm was introduced to optimize localization and improve the accuracy of low-quality image recognition. Character segmentation technology separates individual characters. During the operation, the system first grayscales the color image and then processes it into binarity so as to analyze and process it in depth and obtain accurate recognition results. At the same time, this study also discusses the optimization of the English phrase translation system based on BERT-BiLSTM and genetic algorithm, focusing on the generalization ability of the model to improve the practicability and adaptability of the system.