Preeti
The SLKIR model consists of three main components: a backbone network, a prompt classification layer, and a key information boundary word discrimination layer. The backbone network uses Transformer-XL encoding blocks with pre-trained parameters from the Ernie3.0 language model. Prompts are crucial for guiding the model’s generative direction. Multi-task learning strategies are employed to enhance model generalization. A prompt classification layer is added to boost the semantic expressive capacity. The key information boundary word discrimination layer uses two linear layers to identify start and end boundaries efficiently.
The backbone network transforms input sequences into high-dimensional word vectors containing contextual information. A MHLA mechanism captures latent representations associated with boundary words, integrated into the backbone network outputs through an adaptive fusion module. Exponentially Weighted BCE Loss and Weighted BCE Loss optimize the model training process due to class imbalance in positive and negative samples.
The MHLA mechanism analyzes the association of key information boundary words with nearby characters at varying distances. An adaptive fusion module fuses features outputted by MHLA with those from the backbone network. The loss function includes EBCE and WBCE loss for binary classification problems with imbalanced samples. The EBCE loss applies exponential weights to sparse sample categories, while the WBCE loss allocates weights based on class imbalance ratios. The final loss function combines start and end EBCE loss with WBCE loss to optimize model training.
Source link
Source link: https://www.nature.com/articles/s41598-024-60675-6
GIPHY App Key not set. Please check settings