AI models from Tencent and Alibaba demonstrate better understanding of Chinese language than humans on CLUE benchmark

Tencent Holdings and Alibaba have AI models that have passed a CLUE benchmark test. These test results indicate that these AI models can understand the Chinese language better than humans. However, the researchers say that AI still has to go through many advances before it really understands languages.

Until recently, no AI models had achieved significant scores in the Chinese Language Understanding Assessment (CLUE) benchmark, which gives a set of tasks to assess how well models or machines can understand and respond to the Chinese language. But Tencent and Alibaba’s AI models have recently hit all-time high scores in CLUE.

The assessment of Chinese language comprehension is called CLUE. It is composed of many NLU datasets. This is a community-based project that combines nine challenges on the original Chinese text, including machine reading comprehension and many well-known single-sentence categorization tasks. It’s the first time these two AI models have outperformed humans on CLUE since academics set the benchmark three years ago.

According to 29^e November 2022 ranking on the CLUE website, the two models of the tech titans, namely Tencent’s “Hunyuan AI model”, ranked first with a score of 86.918, and the second place was won by Alibaba’s “AliceMind” with a score of 86.685 respectively. Both scored higher than humans, who received a score of 86.678 points, followed by Oppo (smartphone manufacturing company) and Meituan (food delivery company).

In a recent article, Alibaba Cloud, the cloud computing arm of the e-commerce giant, said, “Although rankings often fluctuate, high scores have never surpassed humans until today.” According to the latest findings, “Chinese language understanding of the AI ​​model has reached a new level.”

There are also other ongoing advances and research beyond these topics. On China’s notoriously tough national university admissions tests, known locally as gaokao, Du Xiaoxiao produced an essay that outscored the majority of students, according to a statement earlier this year by Baidu, a Chinese Internet research company. Chinese Big Tech giants have improved NLP technology that powers voice assistants, machine translation and spam detection like Alibaba’s AliGenie and Tencent’s Xiaowei.

Some US-based researchers claimed that in the past year, many AIs outperformed humans in some comprehensive tasks, but still could not determine whether words and letters in a sentence had been arbitrarily rearranged. This is one of the advantages that humans still have in understanding language. According to research conducted by the University of Cambridge, the human brain can read words with mixed letters if the first and last letters are the same. This capacity of the human brain is not yet acquired by these human models.

There are many elements in a language that these AI models have yet to conquer. The context and emotion of a sentence also matters, which these models struggle with. Although there is still a long way to go to achieve a full understanding of these languages, AI models have covered a huge gap between them and humans. Emotions in a language are a huge challenge for AI models, but affirmative and ongoing research gives hope to address all of these challenges.

Check Reference article. All credit for this research goes to the researchers on this project. Also don’t forget to register. our Reddit page and discord channelwhere we share the latest AI research news, cool AI projects, and more.