ALBERT

Description: The medical literature contains valuable knowledge, such as the clinical symptoms, diagnosis, and treatments of a particular disease. Named Entity Recognition (NER) is the initial step in extracting this knowledge from unstructured text and presenting it as a Knowledge Graph (KG). However, the previous approaches of NER have often suffered from small-scale human-labelled training data. Furthermore, extracting knowledge from Chinese medical literature is a more complex task because there is no segmentation between Chinese characters. Recently, the pretraining models, which obtain representations with the prior semantic knowledge on large-scale unlabelled corpora, have achieved state-of-the-art results for a wide variety of Natural Language Processing (NLP) tasks. However, the capabilities of pretraining models have not been fully exploited, and applications of other pretraining models except BERT in specific domains, such as NER in Chinese medical literature, are also of interest. In this paper, we enhance the performance of NER in Chinese medical literature using pretraining models. First, we propose a method of data augmentation by replacing the words in the training set with synonyms through the Mask Language Model (MLM), which is a pretraining task. Then, we consider NER as the downstream task of the pretraining model and transfer the prior semantic knowledge obtained during pretraining to it. Finally, we conduct experiments to compare the performances of six pretraining models (BERT, BERT-WWM, BERT-WWM-EXT, ERNIE, ERNIE-tiny, and RoBERTa) in recognizing named entities from Chinese medical literature. The effects of feature extraction and fine-tuning, as well as different downstream model structures, are also explored. Experimental results demonstrate that the method of data augmentation we proposed can obtain meaningful improvements in the performance of recognition. Besides, RoBERTa-CRF achieves the highest F1-score compared with the previous methods and other pretraining models.

Description: Using absolute intensity methods (metabolic equivalent of energy (METs), etc.) to determine exercise intensity in exercise prescriptions is straightforward and convenient. Using relative intensity methods (heart rate reserve (%HRR), maximal heart rate (%HRmax), etc.) is more recommended because it is more personalized. Taking target heart rate (THR) given by the relative method as an example, compared with just presenting the THR value, intuitively providing the setting parameters for achieving the THR with specific sport equipment is more user-friendly. The objective of this study was to find a method which combines the advantages (convenient and personalized) of the absolute and relative methods and relatively avoids their disadvantages, helping individuals to meet the target intensity by simply setting equipment parameters. For this purpose, we recruited 32 males and 29 females to undergo incremental cardiopulmonary exercise testing with cycling equipment. The linear regression model of heart rate and exercise wattage (the setting parameter of the equipment) was constructed for each one (R2 = 0.933, p 〈 0.001), and the slopes of the graph of these models were obtained. Next, we used an iterative algorithm to obtain a multiple regression model (adjusted R2 = 0.8336, p 〈 0.001) of selected static body data and the slopes of participants. The regression model can accurately predict the slope of the general population through their static body data. Moreover, other populations can guarantee comparable accuracy by using questionnaire data for calibration. Then, the predicted slope can be utilized to calculate the equipment’s settings for achieving a personalized THR through our equation. All of these steps can be assigned to the intelligent system.

Description: Named Entity Recognition (NER) is the fundamental task for Natural Language Processing (NLP) and the initial step in building a Knowledge Graph (KG). Recently, BERT (Bidirectional Encoder Representations from Transformers), which is a pre-training model, has achieved state-of-the-art (SOTA) results in various NLP tasks, including the NER. However, Chinese NER is still a more challenging task for BERT because there are no physical separations between Chinese words, and BERT can only obtain the representations of Chinese characters. Nevertheless, the Chinese NER cannot be well handled with character-level representations, because the meaning of a Chinese word is quite different from that of the characters, which make up the word. ERNIE (Enhanced Representation through kNowledge IntEgration), which is an improved pre-training model of BERT, is more suitable for Chinese NER because it is designed to learn language representations enhanced by the knowledge masking strategy. However, the potential of ERNIE has not been fully explored. ERNIE only utilizes the token-level features and ignores the sentence-level feature when performing the NER task. In this paper, we propose the ERNIE-Joint, which is a joint model based on ERNIE. The ERNIE-Joint can utilize both the sentence-level and token-level features by joint training the NER and text classification tasks. In order to use the raw NER datasets for joint training and avoid additional annotations, we perform the text classification task according to the number of entities in the sentences. The experiments are conducted on two datasets: MSRA-NER and Weibo. These datasets contain Chinese news data and Chinese social media data, respectively. The results demonstrate that the ERNIE-Joint not only outperforms BERT and ERNIE but also achieves the SOTA results on both datasets.

Description: Background and objectives To explore the relationship between dietary patterns, physical activity and lipid-related indices in Chinese Population. Methods and study design This study included 21,472 (72.3% men) participants aged 16 to 78 years. Data of anthropometric measurements, biochemical tests and questionnaires were collected through a physical examination. Diet patterns were identified through factor analysis and five patterns were retained (“meat,” “high-energy,” “high-protein,” “healthy” and “traditional Chinese”). Physical activity was classified into low, moderate, or high. Abnormalities in lipid indices were assessed using the Adult Treatment Panel III criterion. Results Higher factor scores of “high-protein” pattern and “healthy” pattern were found to be related to favorable lipid indices. Quartiles 3 and 4 of “meat” pattern showed increased risks of having elevates total cholesterol and low-density lipoprotein cholesterol concentrations. Participants with higher levels of physical activity showed lowest risk of abnormal lipid profiles. All the associations were equally established among men, while most were no longer significant among women. Conclusions Higher physical activity level and a dietary pattern consists of high-quality protein foods, vegetables and fruits were associated with favorable lipid profiles, and these lifestyle factors were related to the risk of dyslipidemia in a sex-specific way.

Description: Objective: To evaluate the bone response to an 8 month aerobic gymnastics training program in young opioid-addicted women. Design: Randomized controlled trial (parallel design). Setting: Women's Specific Drug Rehabilitation Center in China. Patients: One hundred and two young women with low bone quality and previous opioid addiction were divided into two groups: (a) the low bone quality intervention experimental group (n = 55; age: 30.3 ± 6.1) and (b) the low bone quality observed control group (observation group; n = 47; age: 29.0 ± 5.3). Interventions: The intervention group took aerobic gymnastics regularly for 80 min/d and 5 d/wk for 8 months and completed follow-up testing. Main Outcome Measures: Substance use history and other life habits affecting bone quality were assessed by questionnaire-based interviews. Bone quality (stiffness-index, T-score, Z-score) was examined with quantitative ultrasound. Anthropometric characteristics (body weight, fat-free mass, fat mass) were obtained by bioelectrical impedance analysis. Results: After the 8 month intervention, the stiffness index of bone quality increased significantly (before: 82 ± 6, after: 108 ± 14, p 〈 0.05) in the experimental group. However, the bone quality did not change significantly in the controls (before: 79 ± 10, after: 77 ± 13, p 〉 0.05). The bone change in the difference group was significant (experimental group: 31.7% vs. observation group: −0.03%). Fat mass decreased in the experimental group (experimental group: before: 19.6 ± 3.7 kg, after: 18.8 ± 4.0 kg, p 〈 0.05). Meanwhile, the change in fat-free mass was the determination of the change in bone quality in the experimental group. Conclusions: Our results suggested that aerobic gymnastics intervention can be an effective strategy for the prevention and treatment of drug-induced osteoporosis in detoxification addicts.