NEJM publishes groundbreaking study on AI in tumor knowledge #AIinMedicine

This study focuses on comparing the performance of large language models (LLMs) in different medical topics, particularly in clinical oncology. The research aims to evaluate the effectiveness of LLMs in addressing oncology problems, comparing various models such as GPT-3.5, GPT-4, PaLM 2, Claude-v1, and LLaMA 1. The study found that while LLMs provide valuable suggestions, they do not yet reach the performance level of human experts. The research design included testing five LLMs on a set of 2044 questions from various medical fields, assessing their consistency and accuracy. The study also explored strategies to enhance model confidence in answering questions, such as model selection and prompt repetition. Overall, the study highlights the differences in performance among LLMs in oncology problem-solving and suggests the need for fine-tuning models for specific medical domains and providing users with access to original sources for verification. Additionally, the study emphasizes the importance of addressing the issue of hallucinations in AI-generated content and the need for effective UI/UX design to assist users in reviewing and utilizing the information generated by LLMs.

Source link

Source link: https://medium.com/@michael_han/nejm-ai%E5%88%8A%E7%99%BB%E9%87%8D%E7%A3%85%E7%A0%94%E7%A9%B6-%E5%A4%A7%E5%9E%8B%E8%AA%9E%E8%A8%80%E6%A8%A1%E5%9E%8B%E5%9C%A8%E8%85%AB%E7%98%A4%E7%9F%A5%E8%AD%98%E4%B8%8A%E7%9A%84%E6%95%88%E8%83%BD%E9%A9%97%E8%AD%89-49daf709a86b?source=rss——llm-5