UNASSIGNED: Teaching medical students the skills required to acquire, interpret, apply, and communicate clinical information is an integral part of medical education. A crucial aspect of this process involves providing students with feedback regarding the quality of their free-text clinical notes.
UNASSIGNED: The goal of this study was to assess the ability of ChatGPT 3.5, a large language model, to score medical students\' free-text history and physical notes.
UNASSIGNED: This is a single-institution, retrospective study. Standardized patients learned a prespecified clinical case and, acting as the patient, interacted with medical students. Each student wrote a free-text history and physical note of their interaction. The students\' notes were scored independently by the standardized patients and ChatGPT using a prespecified scoring rubric that consisted of 85 case elements. The measure of accuracy was percent correct.
UNASSIGNED: The study population consisted of 168 first-year medical students. There was a total of 14,280 scores. The ChatGPT incorrect scoring rate was 1.0%, and the standardized patient incorrect scoring rate was 7.2%. The ChatGPT error rate was 86%, lower than the standardized patient error rate. The ChatGPT mean incorrect scoring rate of 12 (SD 11) was significantly lower than the standardized patient mean incorrect scoring rate of 85 (SD 74; P=.002).
UNASSIGNED: ChatGPT demonstrated a significantly lower error rate compared to standardized patients. This is the first study to assess the ability of a generative pretrained transformer (GPT) program to score medical students\' standardized patient-based free-text clinical notes. It is expected that, in the near future, large language models will provide real-time feedback to practicing physicians regarding their free-text notes. GPT artificial intelligence programs represent an important advance in medical education and medical practice.

UNASSIGNED: In many countries, health care professionals are legally obliged to share information from electronic health records with patients. However, concerns have been raised regarding the sharing of notes with adolescents in mental health care, and health care professionals have called for recommendations to guide this practice.
UNASSIGNED: The aim was to reach a consensus among authors of scientific papers on recommendations for health care professionals\' digital sharing of notes with adolescents in mental health care and to investigate whether staff at child and adolescent specialist mental health care clinics agreed with the recommendations.
UNASSIGNED: A Delphi study was conducted with authors of scientific papers to reach a consensus on recommendations. The process of making the recommendations involved three steps. First, scientific papers meeting the eligibility criteria were identified through a PubMed search where the references were screened. Second, the results from the included papers were coded and transformed into recommendations in an iterative process. Third, the authors of the included papers were asked to provide feedback and consider their agreement with each of the suggested recommendations in two rounds. After the Delphi process, a cross-sectional study was conducted among staff at specialist child and adolescent mental health care clinics to assess whether they agreed with the recommendations that reached a consensus.
UNASSIGNED: Of the 84 invited authors, 27 responded. A consensus was reached on 17 recommendations on areas related to digital sharing of notes with adolescents in mental health care. The recommendations considered how to introduce digital access to notes, write notes, and support health care professionals, and when to withhold notes. Of the 41 staff members at child and adolescent specialist mental health care clinics, 60% or more agreed with the 17 recommendations. No consensus was reached regarding the age at which adolescents should receive digital access to their notes and the timing of digitally sharing notes with parents.
UNASSIGNED: A total of 17 recommendations related to key aspects of health care professionals\' digital sharing of notes with adolescents in mental health care achieved consensus. Health care professionals can use these recommendations to guide their practice of sharing notes with adolescents in mental health care. However, the effects and experiences of following these recommendations should be tested in clinical practice.

BACKGROUND: Real-time surveillance of emerging infectious diseases necessitates a dynamically evolving, computable case definition, which frequently incorporates symptom-related criteria. For symptom detection, both population health monitoring platforms and research initiatives primarily depend on structured data extracted from electronic health records.
OBJECTIVE: This study sought to validate and test an artificial intelligence (AI)-based natural language processing (NLP) pipeline for detecting COVID-19 symptoms from physician notes in pediatric patients. We specifically study patients presenting to the emergency department (ED) who can be sentinel cases in an outbreak.
METHODS: Subjects in this retrospective cohort study are patients who are 21 years of age and younger, who presented to a pediatric ED at a large academic children\'s hospital between March 1, 2020, and May 31, 2022. The ED notes for all patients were processed with an NLP pipeline tuned to detect the mention of 11 COVID-19 symptoms based on Centers for Disease Control and Prevention (CDC) criteria. For a gold standard, 3 subject matter experts labeled 226 ED notes and had strong agreement (F1-score=0.986; positive predictive value [PPV]=0.972; and sensitivity=1.0). F1-score, PPV, and sensitivity were used to compare the performance of both NLP and the International Classification of Diseases, 10th Revision (ICD-10) coding to the gold standard chart review. As a formative use case, variations in symptom patterns were measured across SARS-CoV-2 variant eras.
RESULTS: There were 85,678 ED encounters during the study period, including 4% (n=3420) with patients with COVID-19. NLP was more accurate at identifying encounters with patients that had any of the COVID-19 symptoms (F1-score=0.796) than ICD-10 codes (F1-score =0.451). NLP accuracy was higher for positive symptoms (sensitivity=0.930) than ICD-10 (sensitivity=0.300). However, ICD-10 accuracy was higher for negative symptoms (specificity=0.994) than NLP (specificity=0.917). Congestion or runny nose showed the highest accuracy difference (NLP: F1-score=0.828 and ICD-10: F1-score=0.042). For encounters with patients with COVID-19, prevalence estimates of each NLP symptom differed across variant eras. Patients with COVID-19 were more likely to have each NLP symptom detected than patients without this disease. Effect sizes (odds ratios) varied across pandemic eras.
CONCLUSIONS: This study establishes the value of AI-based NLP as a highly effective tool for real-time COVID-19 symptom detection in pediatric patients, outperforming traditional ICD-10 methods. It also reveals the evolving nature of symptom prevalence across different virus variants, underscoring the need for dynamic, technology-driven approaches in infectious disease surveillance.

BACKGROUND: Over the past few years, online record access (ORA) has been established through secure patient portals in various countries, allowing patients to access their health data, including clinical notes (\"open notes\"). Previous research indicates that ORA in mental health, particularly among patients with severe mental illness (SMI), has been rarely offered. Little is known about the expectations and motivations of patients with SMI when reading what their clinicians share via ORA.
OBJECTIVE: The aim of this study is to explore the reasons why patients with SMI consider or reject ORA and whether sociodemographic characteristics may influence patient decisions.
METHODS: ORA was offered to randomly selected patients at 3 university outpatient clinics in Brandenburg, Germany, which exclusively treat patients with SMI. Within the framework of a mixed methods evaluation, qualitative interviews were conducted with patients who chose to participate in ORA and those who declined, aiming to explore the underlying reasons for their decisions. The interviews were transcribed and analyzed using thematic analysis. Sociodemographic characteristics of patients were examined using descriptive statistics to identify predictors of acceptance or rejection of ORA.
RESULTS: Out of 103 included patients, 58% (n=60) wished to read their clinical notes. The reasons varied, ranging from a desire to engage more actively in their treatment to critically monitoring it and using the accessible data for third-party purposes. Conversely, 42% (n=43) chose not to use ORA, voicing concerns about possibly harming the trustful relationship with their clinicians as well as potential personal distress or uncertainty arising from reading the notes. Practical barriers such as a lack of digital literacy or suspected difficult-to-understand medical language were also named as contributing factors. Correlation analysis revealed that the majority of patients with depressive disorder desired to read the clinical notes (P<.001), while individuals with psychotic disorders showed a higher tendency to decline ORA (P<.05). No significant group differences were observed for other patient groups or characteristics.
CONCLUSIONS: The adoption of ORA is influenced by a wide range of motivational factors, while patients also present a similar variety of reasons for declining its use. The results emphasize the urgent need for knowledge and patient education regarding factors that may hinder the decision to use ORA, including its practical usage, its application possibilities, and concerns related to data privacy. Further research is needed to explore approaches for adequately preparing individuals with SMI to transition from their inherent interest to active engagement with ORA.
BACKGROUND: German Clinical Trial Register DRKS00030188; https://drks.de/search/en/trial/DRKS00030188.

BACKGROUND: Information regarding opioid use disorder (OUD) status and severity is important for patient care. Clinical notes provide valuable information for detecting and characterizing problematic opioid use, necessitating development of natural language processing (NLP) tools, which in turn requires reliably labeled OUD-relevant text and understanding of documentation patterns.
OBJECTIVE: To inform automated NLP methods, we aimed to develop and evaluate an annotation schema for characterizing OUD and its severity, and to document patterns of OUD-relevant information within clinical notes of heterogeneous patient cohorts.
METHODS: We developed an annotation schema to characterize OUD severity based on criteria from the Diagnostic and Statistical Manual of Mental Disorders, 5th edition. In total, 2 annotators reviewed clinical notes from key encounters of 100 adult patients with varied evidence of OUD, including patients with and those without chronic pain, with and without medication treatment for OUD, and a control group. We completed annotations at the sentence level. We calculated severity scores based on annotation of note text with 18 classes aligned with criteria for OUD severity and determined positive predictive values for OUD severity.
RESULTS: The annotation schema contained 27 classes. We annotated 1436 sentences from 82 patients; notes of 18 patients (11 of whom were controls) contained no relevant information. Interannotator agreement was above 70% for 11 of 15 batches of reviewed notes. Severity scores for control group patients were all 0. Among noncontrol patients, the mean severity score was 5.1 (SD 3.2), indicating moderate OUD, and the positive predictive value for detecting moderate or severe OUD was 0.71. Progress notes and notes from emergency department and outpatient settings contained the most and greatest diversity of information. Substance misuse and psychiatric classes were most prevalent and highly correlated across note types with high co-occurrence across patients.
CONCLUSIONS: Implementation of the annotation schema demonstrated strong potential for inferring OUD severity based on key information in a small set of clinical notes and highlighting where such information is documented. These advancements will facilitate NLP tool development to improve OUD prevention, diagnosis, and treatment.

BACKGROUND: Delirium in hospitalized patients is a syndrome of acute brain dysfunction. Diagnostic (International Classification of Diseases [ICD]) codes are often used in studies using electronic health records (EHRs), but they are inaccurate.
OBJECTIVE: We sought to develop a more accurate method using natural language processing (NLP) to detect delirium episodes on the basis of unstructured clinical notes.
METHODS: We collected 1.5 million notes from >10,000 patients from among 9 hospitals. Seven experts iteratively labeled 200,471 sentences. Using these, we trained three NLP classifiers: Support Vector Machine, Recurrent Neural Networks, and Transformer. Testing was performed using an external data set. We also evaluated associations with delirium billing (ICD) codes, medications, orders for restraints and sitters, direct assessments (Confusion Assessment Method [CAM] scores), and in-hospital mortality. F1 scores, confusion matrices, and areas under the receiver operating characteristic curve (AUCs) were used to compare NLP models. We used the φ coefficient to measure associations with other delirium indicators.
RESULTS: The transformer NLP performed best on the following parameters: micro F1=0.978, macro F1=0.918, positive AUC=0.984, and negative AUC=0.992. NLP detections exhibited higher correlations (φ) than ICD codes with deliriogenic medications (0.194 vs 0.073 for ICD codes), restraints and sitter orders (0.358 vs 0.177), mortality (0.216 vs 0.000), and CAM scores (0.256 vs -0.028).
CONCLUSIONS: Clinical notes are an attractive alternative to ICD codes for EHR delirium studies but require automated methods. Our NLP model detects delirium with high accuracy, similar to manual chart review. Our NLP approach can provide more accurate determination of delirium for large-scale EHR-based studies regarding delirium, quality improvement, and clinical trails.

OBJECTIVE: Estimates of parenthood in individuals with psychosis range from 27 to 63%. This number has likely increased due to the introduction of newer anti-psychotics and shorter hospital stays. The problems of psychosis can affect patients\' capacity to offer the consistent, responsive care required for healthy child development. The following research questions were assessed: (1) what proportion of these patients have their children correctly recorded in their clinical notes, (2) what proportion of patients in secondary care with a psychotic diagnosis have children, and (3) what sociodemographic characteristics are associated with parenthood in this population.
METHODS: This study used CRIS (Clinical Record Interactive Search) to search for patients with a diagnosis of non-affective or affective psychosis (F20-29, F31.2 or F31.5) within a UK NHS Trust. A binomial regression model was fitted to identify the variables associated with parenthood.
RESULTS: Fewer than half of the parents in the sample had their children recorded in the correct field in their clinical notes. Of 5173 patients with psychosis, 2006 (38.8%) were parents. Characteristics associated with parenthood included being female, older age, higher socioeconomic status, renting or owning, having ever been married, being unemployed, not being White (British) and not having a diagnosis of schizophrenia.
CONCLUSIONS: Over one-third of patients with psychosis were parents, and the study indicates that not all NHS Trusts are recording dependants accurately. Many variables were strongly associated with parenthood and these findings may help target interventions for this population.

BACKGROUND: Delayed start-of-care nursing visits in home health care (HHC) can result in negative outcomes, such as hospitalization. No previous studies have investigated why start-of-care HHC nursing visits are delayed, in part because most reasons for delayed visits are documented in free-text HHC nursing notes.
OBJECTIVE: The aims of this study were to (1) develop and test a natural language processing (NLP) algorithm that automatically identifies reasons for delayed visits in HHC free-text clinical notes and (2) describe reasons for delayed visits in a large patient sample.
METHODS: This study was conducted at the Visiting Nurse Service of New York (VNSNY). We examined data available at the VNSNY on all new episodes of care started in 2019 (N=48,497). An NLP algorithm was developed and tested to automatically identify and classify reasons for delayed visits.
RESULTS: The performance of the NLP algorithm was 0.8, 0.75, and 0.77 for precision, recall, and F-score, respectively. A total of one-third of HHC episodes (n=16,244) had delayed start-of-care HHC nursing visits. The most prevalent identified category of reasons for delayed start-of-care nursing visits was no answer at the door or phone (3728/8051, 46.3%), followed by patient/family request to postpone or refuse some HHC services (n=2858, 35.5%), and administrative or scheduling issues (n=1465, 18.2%). In 40% (n=16,244) of HHC episodes, 2 or more reasons were documented.
CONCLUSIONS: To avoid critical delays in start-of-care nursing visits, HHC organizations might examine and improve ways to effectively address the reasons for delayed visits, using effective interventions, such as educating patients or caregivers on the importance of a timely nursing visit and improving patients\' intake procedures.

BACKGROUND: Clinicians spend large amounts of their workday using electronic medical records (EMRs). Poorly designed documentation systems contribute to the proliferation of out-of-date information, increased time spent on medical records, clinician burnout, and medical errors. Beyond software interfaces, examining the underlying paradigms and organizational structures for clinical information may provide insights into ways to improve documentation systems. In particular, our attachment to the note as the major organizational unit for storing unstructured medical data may be a cause of many of the problems with modern clinical documentation. Notes, as currently understood, systematically incentivize information duplication and information scattering, both within a single clinician\'s notes over time and across multiple clinicians\' notes. Therefore, it is worthwhile to explore alternative paradigms for unstructured data organization.
OBJECTIVE: The aim of this study is to demonstrate the feasibility of building an EMR that does not use notes as the core organizational unit for unstructured data and which is designed specifically to disincentivize information duplication and information scattering.
METHODS: We used specific design principles to minimize the incentive for users to duplicate and scatter information. By default, the majority of a patient\'s medical history remains the same over time, so users should not have to redocument that information. Clinicians on different teams or services mostly share the same medical information, so all data should be collaboratively shared across teams and services (while still allowing for disagreement and nuance). In all cases where a clinician must state that information has remained the same, they should be able to attest to the information without redocumenting it. We designed and built a web-based EMR based on these design principles.
RESULTS: We built a medical documentation system that does not use notes and instead treats the chart as a single, dynamically updating, and fully collaborative workspace. All information is organized by clinical topic or problem. Version history functionality is used to enable granular tracking of changes over time. Our system is highly customizable to individual workflows and enables each individual user to decide which data should be structured and which should be unstructured, enabling individuals to leverage the advantages of structured templating and clinical decision support as desired without requiring programming knowledge. The system is designed to facilitate real-time, fully collaborative documentation and communication among multiple clinicians.
CONCLUSIONS: We demonstrated the feasibility of building a non-note-based, fully collaborative EMR system. Our attachment to the note as the only possible atomic unit of unstructured medical data should be reevaluated, and alternative models should be considered.

Introduction: Despite the growing efforts to standardize coding for social determinants of health (SDOH), they are infrequently captured in electronic health records (EHRs). Most SDOH variables are still captured in the unstructured fields (i.e., free-text) of EHRs. In this study we attempt to evaluate a practical text mining approach (i.e., advanced pattern matching techniques) in identifying phrases referring to housing issues, an important SDOH domain affecting value-based healthcare providers, using EHR of a large multispecialty medical group in the New England region, United States. To present how this approach would help the health systems to address the SDOH challenges of their patients we assess the demographic and clinical characteristics of patients with and without housing issues and briefly look into the patterns of healthcare utilization among the study population and for those with and without housing challenges. Methods: We identified five categories of housing issues [i.e., homelessness current (HC), homelessness history (HH), homelessness addressed (HA), housing instability (HI), and building quality (BQ)] and developed several phrases addressing each one through collaboration with SDOH experts, consulting the literature, and reviewing existing coding standards. We developed pattern-matching algorithms (i.e., advanced regular expressions), and then applied them in the selected EHR. We assessed the text mining approach for recall (sensitivity) and precision (positive predictive value) after comparing the identified phrases with manually annotated free-text for different housing issues. Results: The study dataset included EHR structured data for a total of 20,342 patients and 2,564,344 free-text clinical notes. The mean (SD) age in the study population was 75.96 (7.51). Additionally, 58.78% of the cohort were female. BQ and HI were the most frequent housing issues documented in EHR free-text notes and HH was the least frequent one. The regular expression methodology, when compared to manual annotation, had a high level of precision (positive predictive value) at phrase, note, and patient levels (96.36, 95.00, and 94.44%, respectively) across different categories of housing issues, but the recall (sensitivity) rate was relatively low (30.11, 32.20, and 41.46%, respectively). Conclusion: Results of this study can be used to advance the research in this domain, to assess the potential value of EHR\'s free-text in identifying patients with a high risk of housing issues, to improve patient care and outcomes, and to eventually mitigate socioeconomic disparities across individuals and communities.