UNASSIGNED: The large-scale artificial intelligence (AI) language model chatbot, Chat Generative Pre-Trained Transformer (ChatGPT), is renowned for its ability to provide data quickly and efficiently. This study aimed to assess the medical responses of ChatGPT regarding anesthetic procedures.
UNASSIGNED: Two anesthesiologist authors selected 30 questions representing inquiries patients might have about surgery and anesthesia. These questions were inputted into two versions of ChatGPT in English. A total of 31 anesthesiologists then evaluated each response for quality, quantity, and overall assessment, using 5-point Likert scales. Descriptive statistics summarized the scores, and a paired sample t-test compared ChatGPT 3.5 and 4.0.
UNASSIGNED: Regarding quality, \"appropriate\" was the most common rating for both ChatGPT 3.5 and 4.0 (40 and 48%, respectively). For quantity, responses were deemed \"insufficient\" in 59% of cases for 3.5, and \"adequate\" in 69% for 4.0. In overall assessment, 3 points were most common for 3.5 (36%), while 4 points were predominant for 4.0 (42%). Mean quality scores were 3.40 and 3.73, and mean quantity scores were - 0.31 (between insufficient and adequate) and 0.03 (between adequate and excessive), respectively. The mean overall score was 3.21 for 3.5 and 3.67 for 4.0. Responses from 4.0 showed statistically significant improvement in three areas.
UNASSIGNED: ChatGPT generated responses mostly ranging from appropriate to slightly insufficient, providing an overall average amount of information. Version 4.0 outperformed 3.5, and further research is warranted to investigate the potential utility of AI chatbots in assisting patients with medical information.

BACKGROUND: Singapore, like the rest of Asia, faces persistent challenges to mental health promotion, including stigma around unwellness and seeking treatment and a lack of trained mental health personnel. The COVID-19 pandemic, which created a surge in mental health care needs and simultaneously accelerated the adoption of digital health solutions, revealed a new opportunity to quickly scale innovative solutions in the region.
OBJECTIVE: In June 2020, the Singaporean government launched mindline.sg, an anonymous digital mental health resource website that has grown to include >500 curated local mental health resources, a clinically validated self-assessment tool for depression and anxiety, an artificial intelligence (AI) chatbot from Wysa designed to deliver digital therapeutic exercises, and a tailored version of the website for working adults called mindline at work. The goal of the platform is to empower Singapore residents to take charge of their own mental health and to be able to offer basic support to those around them through the ease and convenience of a barrier-free digital solution.
METHODS: Website use is measured through click-level data analytics captured via Google Analytics and custom application programming interfaces, which in turn drive a customized analytics infrastructure based on the open-source platforms Titanium Database and Metabase. Unique, nonbounced (users that do not immediately navigate away from the site), engaged, and return users are reported.
RESULTS: In the 2 years following launch (July 1, 2020, through June 30, 2022), the website received >447,000 visitors (approximately 15% of the target population of 3 million), 62.02% (277,727/447,783) of whom explored the site or engaged with resources (referred to as nonbounced visitors); 10.54% (29,271/277,727) of those nonbounced visitors returned. The most popular features on the platform were the dialogue-based therapeutic exercises delivered by the chatbot and the self-assessment tool, which were used by 25.54% (67,626/264,758) and 11.69% (32,469/277,727) of nonbounced visitors. On mindline at work, the rates of nonbounced visitors who engaged extensively (ie, spent ≥40 seconds exploring resources) and who returned were 51.56% (22,474/43,588) and 13.43% (5,853/43,588) over a year, respectively, compared to 30.9% (42,829/138,626) and 9.97% (13,822/138,626), respectively, on the generic mindline.sg site in the same year.
CONCLUSIONS: The site has achieved desired reach and has seen a strong growth rate in the number of visitors, which required substantial and sustained digital marketing campaigns and strategic outreach partnerships. The site was careful to preserve anonymity, limiting the detail of analytics. The good levels of overall adoption encourage us to believe that mild to moderate mental health conditions and the social factors that underly them are amenable to digital interventions. While mindline.sg was primarily used in Singapore, we believe that similar solutions with local customization are widely and globally applicable.

BACKGROUND: Spondyloarthritis (SpA), a chronic inflammatory disorder, predominantly impacts the sacroiliac joints and spine, significantly escalating the risk of disability. SpA\'s complexity, as evidenced by its diverse clinical presentations and symptoms that often mimic other diseases, presents substantial challenges in its accurate diagnosis and differentiation. This complexity becomes even more pronounced in nonspecialist health care environments due to limited resources, resulting in delayed referrals, increased misdiagnosis rates, and exacerbated disability outcomes for patients with SpA. The emergence of large language models (LLMs) in medical diagnostics introduces a revolutionary potential to overcome these diagnostic hurdles. Despite recent advancements in artificial intelligence and LLMs demonstrating effectiveness in diagnosing and treating various diseases, their application in SpA remains underdeveloped. Currently, there is a notable absence of SpA-specific LLMs and an established benchmark for assessing the performance of such models in this particular field.
OBJECTIVE: Our objective is to develop a foundational medical model, creating a comprehensive evaluation benchmark tailored to the essential medical knowledge of SpA and its unique diagnostic and treatment protocols. The model, post-pretraining, will be subject to further enhancement through supervised fine-tuning. It is projected to significantly aid physicians in SpA diagnosis and treatment, especially in settings with limited access to specialized care. Furthermore, this initiative is poised to promote early and accurate SpA detection at the primary care level, thereby diminishing the risks associated with delayed or incorrect diagnoses.
METHODS: A rigorous benchmark, comprising 222 meticulously formulated multiple-choice questions on SpA, will be established and developed. These questions will be extensively revised to ensure their suitability for accurately evaluating LLMs\' performance in real-world diagnostic and therapeutic scenarios. Our methodology involves selecting and refining top foundational models using public data sets. The best-performing model in our benchmark will undergo further training. Subsequently, more than 80,000 real-world inpatient and outpatient cases from hospitals will enhance LLM training, incorporating techniques such as supervised fine-tuning and low-rank adaptation. We will rigorously assess the models\' generated responses for accuracy and evaluate their reasoning processes using the metrics of fluency, relevance, completeness, and medical proficiency.
RESULTS: Development of the model is progressing, with significant enhancements anticipated by early 2024. The benchmark, along with the results of evaluations, is expected to be released in the second quarter of 2024.
CONCLUSIONS: Our trained model aims to capitalize on the capabilities of LLMs in analyzing complex clinical data, thereby enabling precise detection, diagnosis, and treatment of SpA. This innovation is anticipated to play a vital role in diminishing the disabilities arising from delayed or incorrect SpA diagnoses. By promoting this model across diverse health care settings, we anticipate a significant improvement in SpA management, culminating in enhanced patient outcomes and a reduced overall burden of the disease.
UNASSIGNED: DERR1-10.2196/57001.

This narrative literature review undertakes a comprehensive examination of the burgeoning field, tracing the development of artificial intelligence (AI)-powered tools for depression and anxiety detection from the level of intricate algorithms to practical applications. Delivering essential mental health care services is now a significant public health priority. In recent years, AI has become a game-changer in the early identification and intervention of these pervasive mental health disorders. AI tools can potentially empower behavioral healthcare services by helping psychiatrists collect objective data on patients\' progress and tasks. This study emphasizes the current understanding of AI, the different types of AI, its current use in multiple mental health disorders, advantages, disadvantages, and future potentials. As technology develops and the digitalization of the modern era increases, there will be a rise in the application of artificial intelligence in psychiatry; therefore, a comprehensive understanding will be needed. We searched PubMed, Google Scholar, and Science Direct using keywords for this. In a recent review of studies using electronic health records (EHR) with AI and machine learning techniques for diagnosing all clinical conditions, roughly 99 publications have been found. Out of these, 35 studies were identified for mental health disorders in all age groups, and among them, six studies utilized EHR data sources. By critically analyzing prominent scholarly works, we aim to illuminate the current state of this technology, exploring its successes, limitations, and future directions. In doing so, we hope to contribute to a nuanced understanding of AI\'s potential to revolutionize mental health diagnostics and pave the way for further research and development in this critically important domain.

Developers and vendors of large language models (\"LLMs\") - such as ChatGPT, Google Bard, and Microsoft\'s Bing at the forefront-can be subject to Health Insurance Portability and Accountability Act of 1996 (\"HIPAA\") when they process protected health information (\"PHI\") on behalf of the HIPAA covered entities. In doing so, they become business associates or subcontractors of a business associate under HIPAA.

This study is centered on investigating the acceptance and utilization of AI Chatbot technology among graduate students in China and its implications for higher education. Employing a fusion of the UTAUT (Unified Theory of Acceptance and Use of Technology) model and the ECM (Expectation-Confirmation Model), the research seeks to pinpoint the pivotal factors influencing students\' attitudes, satisfaction, and behavioral intentions regarding AI Chatbots. The study constructs a model comprising seven substantial predictors aimed at precisely foreseeing users\' intentions and behavior with AI Chatbots. Collected from 373 students enrolled in various universities across China, the self-reported data is subject to analysis using the partial-least squares method of structural equation modeling to confirm the model\'s reliability and validity. The findings validate seven out of the eleven proposed hypotheses, underscoring the influential role of ECM constructs, particularly \"Confirmation\" and \"Satisfaction,\" outweighing the impact of UTAUT constructs on users\' behavior. Specifically, users\' perceived confirmation significantly influences their satisfaction and subsequent intention to continue using AI Chatbots. Additionally, \"Personal innovativeness\" emerges as a critical determinant shaping users\' behavioral intention. This research emphasizes the need for further exploration of AI tool adoption in educational settings and encourages continued investigation of their potential in teaching and learning environments.

BACKGROUND: Artificial intelligence (AI) is a rapidly developing field with the potential to transform various aspects of health care and public health, including medical training. During the \"Hygiene and Public Health\" course for fifth-year medical students, a practical training session was conducted on vaccination using AI chatbots as an educational supportive tool. Before receiving specific training on vaccination, the students were given a web-based test extracted from the Italian National Medical Residency Test. After completing the test, a critical correction of each question was performed assisted by AI chatbots.
OBJECTIVE: The main aim of this study was to identify whether AI chatbots can be considered educational support tools for training in public health. The secondary objective was to assess the performance of different AI chatbots on complex multiple-choice medical questions in the Italian language.
METHODS: A test composed of 15 multiple-choice questions on vaccination was extracted from the Italian National Medical Residency Test using targeted keywords and administered to medical students via Google Forms and to different AI chatbot models (Bing Chat, ChatGPT, Chatsonic, Google Bard, and YouChat). The correction of the test was conducted in the classroom, focusing on the critical evaluation of the explanations provided by the chatbot. A Mann-Whitney U test was conducted to compare the performances of medical students and AI chatbots. Student feedback was collected anonymously at the end of the training experience.
RESULTS: In total, 36 medical students and 5 AI chatbot models completed the test. The students achieved an average score of 8.22 (SD 2.65) out of 15, while the AI chatbots scored an average of 12.22 (SD 2.77). The results indicated a statistically significant difference in performance between the 2 groups (U=49.5, P<.001), with a large effect size (r=0.69). When divided by question type (direct, scenario-based, and negative), significant differences were observed in direct (P<.001) and scenario-based (P<.001) questions, but not in negative questions (P=.48). The students reported a high level of satisfaction (7.9/10) with the educational experience, expressing a strong desire to repeat the experience (7.6/10).
CONCLUSIONS: This study demonstrated the efficacy of AI chatbots in answering complex medical questions related to vaccination and providing valuable educational support. Their performance significantly surpassed that of medical students in direct and scenario-based questions. The responsible and critical use of AI chatbots can enhance medical education, making it an essential aspect to integrate into the educational system.

OBJECTIVE: To evaluate the accuracy and bias of ophthalmologist recommendations made by three AI chatbots, namely ChatGPT 3.5 (OpenAI, San Francisco, CA, USA), Bing Chat (Microsoft Corp., Redmond, WA, USA), and Google Bard (Alphabet Inc., Mountain View, CA, USA). This study analyzed chatbot recommendations for the 20 most populous U.S. cities.
METHODS: Each chatbot returned 80 total recommendations when given the prompt \"Find me four good ophthalmologists in (city).\" Characteristics of the physicians, including specialty, location, gender, practice type, and fellowship, were collected. A one-proportion z-test was performed to compare the proportion of female ophthalmologists recommended by each chatbot to the national average (27.2% per the Association of American Medical Colleges (AAMC)). Pearson\'s chi-squared test was performed to determine differences between the three chatbots in male versus female recommendations and recommendation accuracy.
RESULTS: Female ophthalmologists recommended by Bing Chat (1.61%) and Bard (8.0%) were significantly less than the national proportion of 27.2% practicing female ophthalmologists (p<0.001, p<0.01, respectively). ChatGPT recommended fewer female (29.5%) than male ophthalmologists (p<0.722). ChatGPT (73.8%), Bing Chat (67.5%), and Bard (62.5%) gave high rates of inaccurate recommendations. Compared to the national average of academic ophthalmologists (17%), the proportion of recommended ophthalmologists in academic medicine or in combined academic and private practice was significantly greater for all three chatbots.
CONCLUSIONS: This study revealed substantial bias and inaccuracy in the AI chatbots\' recommendations. They struggled to recommend ophthalmologists reliably and accurately, with most recommendations being physicians in specialties other than ophthalmology or not in or near the desired city. Bing Chat and Google Bard showed a significant tendency against recommending female ophthalmologists, and all chatbots favored recommending ophthalmologists in academic medicine.

BACKGROUND: The diagnostic accuracy of differential diagnoses generated by artificial intelligence chatbots, including ChatGPT models, for complex clinical vignettes derived from general internal medicine (GIM) department case reports is unknown.
OBJECTIVE: This study aims to evaluate the accuracy of the differential diagnosis lists generated by both third-generation ChatGPT (ChatGPT-3.5) and fourth-generation ChatGPT (ChatGPT-4) by using case vignettes from case reports published by the Department of GIM of Dokkyo Medical University Hospital, Japan.
METHODS: We searched PubMed for case reports. Upon identification, physicians selected diagnostic cases, determined the final diagnosis, and displayed them into clinical vignettes. Physicians typed the determined text with the clinical vignettes in the ChatGPT-3.5 and ChatGPT-4 prompts to generate the top 10 differential diagnoses. The ChatGPT models were not specially trained or further reinforced for this task. Three GIM physicians from other medical institutions created differential diagnosis lists by reading the same clinical vignettes. We measured the rate of correct diagnosis within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and the top diagnosis.
RESULTS: In total, 52 case reports were analyzed. The rates of correct diagnosis by ChatGPT-4 within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and top diagnosis were 83% (43/52), 81% (42/52), and 60% (31/52), respectively. The rates of correct diagnosis by ChatGPT-3.5 within the top 10 differential diagnosis lists, top 5 differential diagnosis lists, and top diagnosis were 73% (38/52), 65% (34/52), and 42% (22/52), respectively. The rates of correct diagnosis by ChatGPT-4 were comparable to those by physicians within the top 10 (43/52, 83% vs 39/52, 75%, respectively; P=.47) and within the top 5 (42/52, 81% vs 35/52, 67%, respectively; P=.18) differential diagnosis lists and top diagnosis (31/52, 60% vs 26/52, 50%, respectively; P=.43) although the difference was not significant. The ChatGPT models\' diagnostic accuracy did not significantly vary based on open access status or the publication date (before 2011 vs 2022).
CONCLUSIONS: This study demonstrates the potential diagnostic accuracy of differential diagnosis lists generated using ChatGPT-3.5 and ChatGPT-4 for complex clinical vignettes from case reports published by the GIM department. The rate of correct diagnoses within the top 10 and top 5 differential diagnosis lists generated by ChatGPT-4 exceeds 80%. Although derived from a limited data set of case reports from a single department, our findings highlight the potential utility of ChatGPT-4 as a supplementary tool for physicians, particularly for those affiliated with the GIM department. Further investigations should explore the diagnostic accuracy of ChatGPT by using distinct case materials beyond its training data. Such efforts will provide a comprehensive insight into the role of artificial intelligence in enhancing clinical decision-making.

The exponential growth of ChatGPT in medical literature, amassing over 1000 PubMed citations by August 2023, underscores a pivotal juncture in the convergence of artificial intelligence (AI) and healthcare. This remarkable rise not only showcases its potential to revolutionize medical academia but also indicates its impending influence on patient care and healthcare systems. Notwithstanding this enthusiasm, one-third of these citations are editorials or commentaries, stressing a gap in empirical research. Alongside its potential, there are concerns about ChatGPT becoming a \"Weapon of Mass Deception\" and the need for rigorous evaluations to counter inaccuracies. The World Association of Medical Editors has released guidelines emphasizing that AI tools should not be manuscript co-authors and advocates for clear disclosures in AI-assisted academic works. Interestingly, ChatGPT achieved its citation milestone within nine months, compared to Google\'s 14 years. As Large Language Models (LLMs), like ChatGPT, become more integral in healthcare, issues surrounding data protection, patient privacy, and ethical implications gain prominence. As the future of LLM research unfolds, key areas of interest include its efficacy in clinical settings, its role in telemedicine, and its potential in medical education. The journey ahead necessitates a harmonious partnership between the medical community and AI developers, emphasizing both technological advancements and ethical considerations.