A new theoretical approach to language has emerged in the past 10-15 years that allows linguistic observations about form-meaning pairings constructions — to be stated directly. Constructionist approaches aim to acco...A new theoretical approach to language has emerged in the past 10-15 years that allows linguistic observations about form-meaning pairings constructions — to be stated directly. Constructionist approaches aim to account for the full range of facts about language, without assuming that a particular subset of the data is part of a privileged 揷ore? Researchers argue that unusual constructions shed light on more general issues, and serve to illuminate what is required for a complete account of language.展开更多
Recently, the emergence of pre-trained models(PTMs) has brought natural language processing(NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP. We first briefly introduce language rep...Recently, the emergence of pre-trained models(PTMs) has brought natural language processing(NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP. We first briefly introduce language representation learning and its research progress. Then we systematically categorize existing PTMs based on a taxonomy from four different perspectives. Next,we describe how to adapt the knowledge of PTMs to downstream tasks. Finally, we outline some potential directions of PTMs for future research. This survey is purposed to be a hands-on guide for understanding, using, and developing PTMs for various NLP tasks.展开更多
2009年，微博客受到了人们的极大关注。简单快捷的操作方式、随时随地发布信息的互动形式，成为互联网的一个亮点。令球语苦监测机构（Global Language Monitor）公布的数据显示，微博客“Twitter”成为2009年最热门的英语单词。Twitte...2009年，微博客受到了人们的极大关注。简单快捷的操作方式、随时随地发布信息的互动形式，成为互联网的一个亮点。令球语苦监测机构（Global Language Monitor）公布的数据显示，微博客“Twitter”成为2009年最热门的英语单词。Twitter热浪也重新燃起了国内微博客的激情，展开更多
ChatG PT,an artificial intelligence generated content (AIGC) model developed by OpenAI,has attracted worldwide attention for its capability of dealing with challenging language understanding and generation tasks in th...ChatG PT,an artificial intelligence generated content (AIGC) model developed by OpenAI,has attracted worldwide attention for its capability of dealing with challenging language understanding and generation tasks in the form of conversations.This paper briefly provides an overview on the history,status quo and potential future development of ChatGPT,helping to provide an entry point to think about ChatGPT.Specifically,from the limited open-accessed resources,we conclude the core techniques of ChatGPT,mainly including large-scale language models,in-context learning,reinforcement learning from human feedback and the key technical steps for developing ChatGPT.We further analyze the pros and cons of ChatGPT and we rethink the duality of ChatGPT in various fields.Although it has been widely acknowledged that ChatGPT brings plenty of opportunities for various fields,mankind should still treat and use ChatG PT properly to avoid the potential threat,e.g.,academic integrity and safety challenge.Finally,we discuss several open problems as the potential development of ChatGPT.展开更多
文摘A new theoretical approach to language has emerged in the past 10-15 years that allows linguistic observations about form-meaning pairings constructions — to be stated directly. Constructionist approaches aim to account for the full range of facts about language, without assuming that a particular subset of the data is part of a privileged 揷ore? Researchers argue that unusual constructions shed light on more general issues, and serve to illuminate what is required for a complete account of language.
基金the National Natural Science Foundation of China(Grant Nos.61751201 and 61672162)the Shanghai Municipal Science and Technology Major Project(Grant No.2018SHZDZX01)and ZJLab。
文摘Recently, the emergence of pre-trained models(PTMs) has brought natural language processing(NLP) to a new era. In this survey, we provide a comprehensive review of PTMs for NLP. We first briefly introduce language representation learning and its research progress. Then we systematically categorize existing PTMs based on a taxonomy from four different perspectives. Next,we describe how to adapt the knowledge of PTMs to downstream tasks. Finally, we outline some potential directions of PTMs for future research. This survey is purposed to be a hands-on guide for understanding, using, and developing PTMs for various NLP tasks.
基金supported by National Key Research and Development Program of China (2021YFB1714300)National Natural Science Foundation of China (62293502, 61831022, 61976211)Youth Innovation Promotion Association CAS。
文摘ChatG PT,an artificial intelligence generated content (AIGC) model developed by OpenAI,has attracted worldwide attention for its capability of dealing with challenging language understanding and generation tasks in the form of conversations.This paper briefly provides an overview on the history,status quo and potential future development of ChatGPT,helping to provide an entry point to think about ChatGPT.Specifically,from the limited open-accessed resources,we conclude the core techniques of ChatGPT,mainly including large-scale language models,in-context learning,reinforcement learning from human feedback and the key technical steps for developing ChatGPT.We further analyze the pros and cons of ChatGPT and we rethink the duality of ChatGPT in various fields.Although it has been widely acknowledged that ChatGPT brings plenty of opportunities for various fields,mankind should still treat and use ChatG PT properly to avoid the potential threat,e.g.,academic integrity and safety challenge.Finally,we discuss several open problems as the potential development of ChatGPT.
