Medlinker, a leading digital healthcare company in China, undertook a real world comparison of AI doctors accessible through its MedGPT service and their human equivalents in Chengdu on June 30 2023.
The evaluation, streamed live, involved 120 real patients and ten attending physicians from West China Hospital, Sichuan University. Seven eminent physicians from leading Chinese hospitals reviewed and scored the results based on criteria including consultation accuracy, diagnosis, treatment recommendations, suggested additional examinations, data analysis, explanation provided to patients, and the use of natural language in patient interactions. Human doctors scored an average of 7.5 out of 10, while MedGPT scored 7.2. MedGPT’s recommendations were found to be 96% as good as those from top-tier physicians.
Launched by Medlinker on May 25 2023, MedGPT is China’s first homegrown medical large-scale language model founded on the Transformer architecture. Unlike general-purpose large-scale models, MedGPT was designed to provide practical diagnostic value in real medical scenarios, offering intelligent diagnosis and treatment capabilities throughout the entire process, from disease prevention to diagnosis, treatment, and rehabilitation. Medlinker’s ability to release a large-scale medical model ahead of its peers is the result of its accumulated expertise in artificial intelligence, big data, and internet-based healthcare. The world’s first real patient evaluation confirms Medlinker’s leadership in the research, development and application of medical large-scale language models.
The consultation session between doctor and patient was planned to fully ensure the impartiality and fact-based reliability of the evaluation, following a carefully established process:
- The patient explained their medical situation, including symptoms and previous treatments, to the medical assistant;
- The medical assistant relayed the collected information simultaneously to the human doctor and to MedGPT through online text input;
- The medical assistant acted as the intermediary as doctor and patient completed multiple rounds of communication until sufficient information had been collected for a diagnosis;
- Based on the information, the human doctor and MedGPT ordered specific follow-up examinations or, if the information provided was sufficient, provided a diagnosis to the patient;
- If further examinations were required, the patient underwent examinations at the hospital;
- After obtaining the examination results, the patient returned for a final consultation where MedGPT and the human doctor independently provided a clinical diagnosis and treatment.
Throughout the entire process, the human doctor and MedGPT worked independently with no sharing of information, to ensure that both were operating under the same conditions.
During the evaluation session, the medical experts on the panel were impressed by MedGPT’s performance. They generally agreed that MedGPT ensured medical accuracy by collecting sufficient information through multiple rounds of questioning, resulting in a relatively low probability of misdiagnosis or missed diagnosis. What was particularly surprising was that, unlike doctors specialized in one field, MedGPT could diagnose diseases outside its field of expertise and suggest a treatment program based on other clinical observations and factors, something that can be quite challenging to achieve in routine consultations undertaken by specialists. The panel concluded that MedGPT’s knowledge coverage surpassed that of human doctors with limited experience.
“The integration of AI doctors and telemedicine services is certain to disrupt the entire medical field,” said Wang Shirui, founder and CEO of Medlinker. “Looking into the future, we firmly believe that through Medlinker’s MedGPT, both the efficiency and the effectiveness of doctors will be vastly enhanced. At the same time, it will become possible to rapidly replicate high-quality medical resources and make them universally available. Furthermore, Medlinker’s telemedicine service network will be able to leverage MedGPT to provide more convenient, efficient and accessible medical services to a significantly larger pool of patients, vastly expanding medical inclusion, and taking a huge step towards solving the biggest dilemma facing the medical community today, providing medical services that are efficient and deliver meaningful treatment to the patient while remaining affordable.”
