OBJECTIVE: To use an artificial intelligence (AI)-powered large language model (LLM) to improve readability of patient handouts.
METHODS: Review of online material modified by AI.
METHODS: Academic center.
METHODS: Five handout materials obtained from the American Rhinologic Society (ARS) and the American Academy of Facial Plastic and Reconstructive Surgery websites were assessed using validated readability metrics. The handouts were inputted into OpenAI\'s ChatGPT-4 after prompting: \"Rewrite the following at a 6th-grade reading level.\" The understandability and actionability of both native and LLM-revised versions were evaluated using the Patient Education Materials Assessment Tool (PEMAT). Results were compared using Wilcoxon rank-sum tests.
RESULTS: The mean readability scores of the standard (ARS, American Academy of Facial Plastic and Reconstructive Surgery) materials corresponded to \"difficult,\" with reading categories ranging between high school and university grade levels. Conversely, the LLM-revised handouts had an average seventh-grade reading level. LLM-revised handouts had better readability in nearly all metrics tested: Flesch-Kincaid Reading Ease (70.8 vs 43.9; P < .05), Gunning Fog Score (10.2 vs 14.42; P < .05), Simple Measure of Gobbledygook (9.9 vs 13.1; P < .05), Coleman-Liau (8.8 vs 12.6; P < .05), and Automated Readability Index (8.2 vs 10.7; P = .06). PEMAT scores were significantly higher in the LLM-revised handouts for understandability (91 vs 74%; P < .05) with similar actionability (42 vs 34%; P = .15) when compared to the standard materials.
CONCLUSIONS: Patient-facing handouts can be augmented by ChatGPT with simple prompting to tailor information with improved readability. This study demonstrates the utility of LLMs to aid in rewriting patient handouts and may serve as a tool to help optimize education materials.
METHODS: Level VI.

BACKGROUND: Artificial intelligence (AI) is a burgeoning new field that has increased in popularity over the past couple of years, coinciding with the public release of large language model (LLM)-driven chatbots. These chatbots, such as ChatGPT, can be engaged directly in conversation, allowing users to ask them questions or issue other commands. Since LLMs are trained on large amounts of text data, they can also answer questions reliably and factually, an ability that has allowed them to serve as a source for medical inquiries. This study seeks to assess the readability of patient education materials on cardiac catheterization across four of the most common chatbots: ChatGPT, Microsoft Copilot, Google Gemini, and Meta AI.
METHODS: A set of 10 questions regarding cardiac catheterization was developed using website-based patient education materials on the topic. We then asked these questions in consecutive order to four of the most common chatbots: ChatGPT, Microsoft Copilot, Google Gemini, and Meta AI. The Flesch Reading Ease Score (FRES) was used to assess the readability score. Readability grade levels were assessed using six tools: Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG) Index, Automated Readability Index (ARI), and FORCAST Grade Level.
RESULTS: The mean FRES across all four chatbots was 40.2, while overall mean grade levels for the four chatbots were 11.2, 13.7, 13.7, 13.3, 11.2, and 11.6 across the FKGL, GFI, CLI, SMOG, ARI, and FORCAST indices, respectively. Mean reading grade levels across the six tools were 14.8 for ChatGPT, 12.3 for Microsoft Copilot, 13.1 for Google Gemini, and 9.6 for Meta AI. Further, FRES values for the four chatbots were 31, 35.8, 36.4, and 57.7, respectively.
CONCLUSIONS: This study shows that AI chatbots are capable of providing answers to medical questions regarding cardiac catheterization. However, the responses across the four chatbots had overall mean reading grade levels at the 11th-13th-grade level, depending on the tool used. This means that the materials were at the high school and even college reading level, which far exceeds the recommended sixth-grade level for patient education materials. Further, there is significant variability in the readability levels provided by different chatbots as, across all six grade-level assessments, Meta AI had the lowest scores and ChatGPT generally had the highest.

BACKGROUND: Understanding of health-related materials, termed health literacy, affects decision makings and outcomes in the treatment of bladder cancer. The National Institutes of Health recommend writing education materials at a sixth-seventh grade reading level. The goal of this study is to assess readability of bladder cancer materials available online.
OBJECTIVE: The goal of this study is to characterize available information about bladder cancer online and evaluate readability.
METHODS: Materials on bladder cancer were collected from the American Urological Association\'s Urology Care Foundation (AUA-UCF) and compared to top 50 websites by search engine results. Resources were analyzed using four different validated readability assessment scales. The mean and standard deviation of the materials was calculated, and a two-tailed t test for used to assess for significance between the two sets of patient education materials.
RESULTS: The average readability of AUA materials was 8.5 (8th-9th grade reading level). For the top 50 websites, average readability was 11.7 (11-12th grade reading level). A two-tailed t test between the AUA and top 50 websites demonstrated statistical significance between the readability of the two sets of resources (P = 0.0001), with the top search engine results being several grade levels higher than the recommended 6-7th grade reading level.
CONCLUSIONS: Most health information provided by the AUA on bladder cancer is written at a reading ability that aligns with most US adults, with top websites for search engine results exceeding the average reading level by several grade levels. By focusing on health literacy, urologists may contribute lowering barriers to health literacy, improving health care expenditure and perioperative complications.

BACKGROUND: Health literacy is a critical determinant of a patient\'s overall health status, and studies have demonstrated a consistent link between poor health literacy and negative health outcomes. The Centers for Disease Control and Prevention (CDC) and the National Institutes of Health (NIH) advise that patient educational materials (PEMs) should be written at an eighth-grade reading level or lower, matching the average reading level of adult Americans. The purpose of this study was to investigate the ability of generative artificial intelligence (AI) to edit PEMs from orthopaedic institutions to meet the CDC and NIH guidelines.
METHODS: PEMs about lateral epicondylitis (LE) from the top 25 ranked orthopaedic institutions from the 2022 U.S. News & World Report Best Hospitals Specialty Ranking were gathered. ChatGPT Plus (version 4.0) was then instructed to rewrite PEMs on LE from these institutions to comply with CDC and NIH-recommended guidelines. Readability scores were calculated for the original and rewritten PEMs, and paired t-tests were used to determine statistical significance.
RESULTS: Analysis of the original and edited PEMs about LE revealed significant reductions in reading grade level and word count of 3.70 ± 1.84 (p<0.001) and 346.72 ± 364.63 (p<0.001), respectively.
CONCLUSIONS: Our study demonstrated generative AI\'s ability to rewrite PEM about LE at a reading comprehension level that conforms to the CDC and NIH guidelines. Hospital administrators and orthopaedic surgeons should consider the findings of this study and the potential utility of artificial intelligence when crafting PEMs of their own.

Introduction The National Institutes of Health and the American Medical Association recommend patient education materials (EMs) be at or below the sixth-grade reading level. The American Cancer Society, Leukemia & Lymphoma Society, and National Comprehensive Cancer Network have accurate blood cancer EMs. Methods One hundred one (101) blood cancer EMs from the above organizations were assessed using the following: Flesch Reading Ease Formula (FREF), Flesch-Kincaid Grade Level (FKGL), Gunning Fog Index (GFI), Simple Measure of Gobbledygook Index (SMOG), and the Coleman-Liau Index (CLI). Results Only 3.96% of patient EMs scored at or below the seventh-grade reading level in all modalities. Healthcare professional education materials (HPEMs) averaged around the college to graduate level. For leukemia and lymphoma patient EMs, there were significant differences for FKGL vs. SMOG, FKGL vs. GFI, FKGL vs. CLI, SMOG vs. CLI, and GFI vs. CLI. For HPEMs, there were significant differences for FKGL vs. GFI and GFI vs. CLI. Conclusion The majority of patient EMs were above the seventh-grade reading level. A lack of easily readable patient EMs could lead to a poor understanding of disease and, thus, adverse health outcomes. Overall, patient EMs should not replace physician counseling. Physicians must close the gaps in patients\' understanding throughout their cancer treatment.

BACKGROUND: Dermatologic patient education materials (PEMs) are often written above the national average seventh- to eighth-grade reading level. ChatGPT-3.5, GPT-4, DermGPT, and DocsGPT are large language models (LLMs) that are responsive to user prompts. Our project assesses their use in generating dermatologic PEMs at specified reading levels.
OBJECTIVE: This study aims to assess the ability of select LLMs to generate PEMs for common and rare dermatologic conditions at unspecified and specified reading levels. Further, the study aims to assess the preservation of meaning across such LLM-generated PEMs, as assessed by dermatology resident trainees.
METHODS: The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. We prompted ChatGPT-3.5, GPT-4, DermGPT, and DocsGPT to \"Create a patient education handout about [condition] at a [FKRL]\" to iteratively generate 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs, evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 dermatology resident trainees.
RESULTS: The current American Academy of Dermatology PEMs had an average (SD) FKRL of 9.35 (1.26) and 9.50 (2.3) for common and rare diseases, respectively. For common diseases, the FKRLs of LLM-produced PEMs ranged between 9.8 and 11.21 (unspecified prompt), between 4.22 and 7.43 (fifth-grade prompt), and between 5.98 and 7.28 (seventh-grade prompt). For rare diseases, the FKRLs of LLM-produced PEMs ranged between 9.85 and 11.45 (unspecified prompt), between 4.22 and 7.43 (fifth-grade prompt), and between 5.98 and 7.28 (seventh-grade prompt). At the fifth-grade reading level, GPT-4 was better at producing PEMs for both common and rare conditions than ChatGPT-3.5 (P=.001 and P=.01, respectively), DermGPT (P<.001 and P=.03, respectively), and DocsGPT (P<.001 and P=.02, respectively). At the seventh-grade reading level, no significant difference was found between ChatGPT-3.5, GPT-4, DocsGPT, or DermGPT in producing PEMs for common conditions (all P>.05); however, for rare conditions, ChatGPT-3.5 and DocsGPT outperformed GPT-4 (P=.003 and P<.001, respectively). The preservation of meaning analysis revealed that for common conditions, DermGPT ranked the highest for overall ease of reading, patient understandability, and accuracy (14.75/15, 98%); for rare conditions, handouts generated by GPT-4 ranked the highest (14.5/15, 97%).
CONCLUSIONS: GPT-4 appeared to outperform ChatGPT-3.5, DocsGPT, and DermGPT at the fifth-grade FKRL for both common and rare conditions, although both ChatGPT-3.5 and DocsGPT performed better than GPT-4 at the seventh-grade FKRL for rare conditions. LLM-produced PEMs may reliably meet seventh-grade FKRLs for select common and rare dermatologic conditions and are easy to read, understandable for patients, and mostly accurate. LLMs may play a role in enhancing health literacy and disseminating accessible, understandable PEMs in dermatology.

UNASSIGNED: While the internet provides accessible medical information, often times it does not cater to the average patient\'s ability to understand medical text at a 6th and 8th grade reading level, per American Medical Association (AMA)/National Institute of Health (NIH) recommendations. This study looks to analyze current online materials relating to posterior cruciate ligament (PCL) surgery and their readability, understandability, and actionability.
UNASSIGNED: The top 100 Google searchs for \"PCL surgery\" were compiled. Research papers, procedural protocols, advertisements, and videos were excluded from the data collection. The readability was examined using 7 algorithms: the Flesch Reading Ease Score, Gunning Fog, Flesch-Kincaid Grade Level, Coleman-Liau Index, SMOG index, Automated Readability Index and the Linsear Write Formula. Two evaluators assessed Understandability and Actionability of the results with the Patient Educational Materials Assessment Tool (PEMAT). Outcome measures included Reading Grade Level, Reader\'s age minimum and maximum, Understandability, and Actionability.
UNASSIGNED: Of the 100 results, 16 were excluded based on the exclusion criteria. There was a statistically significant difference between the readability of the results from all algorithms and the current recommendation by AMA and NIH. Subgroup analysis demonstrated that there was no difference in readability as it pertained to which page they appeared on Google search. There was also no difference in readability between individual websites versus organizational websites (hospital and non-hospital educational websites). Three articles were at the 8th grade recommended reading level, and all three were from healthcare institutes.
UNASSIGNED: There is a discrepancy in readability between the recommendation of AMA/NIH and online educational materials regarding PCL surgeries, regardless of where they appear on Google and across different forums. The understandability and actionability were equally poor. Future research can focus on the readability and validity of video and social media as they are becoming increasingly popular sources of medical information.

Introduction Influenza is a major global health concern, with its rapid spread and mutation rate posing significant challenges in public health education and communication. Effective patient education materials (PEMs) are crucial for informed decision-making and improved health outcomes. This study evaluates the efficacy of online influenza PEMs using traditional readability tools and introduces the Contextual Health Education Readability Score (CHERS) to address the limitations of existing methods that do not capture the diverse array of visual and thematic means displayed. Materials and methods A comprehensive search was conducted to select relevant online influenza PEMs. This involved looking through Google\'s first two pages of results sorted by relevance, for a total of 20 results. These materials were evaluated using established readability tools (e.g., Flesch Reading Ease, Flesch-Kincaid Grade Level) and the Patient Education Materials Assessment Tool (PEMAT) for understandability and actionability. The study also involved the creation of CHERS, integrating factors such as semantic complexity, cultural relevance, and visual aid effectiveness. The development of CHERS included weighting each component based on its impact on readability and comprehension. Results The traditional readability tools demonstrated significant variability in the readability of the selected materials. The PEMAT analysis revealed general trends toward clarity in purpose and use of everyday language but indicated a need for improvement in summaries and visual aids. The CHERS formula was calculated as follows: CHERS = (0.4 × Average Sentence Length) + (0.3 × Average Syllables per Word) + (0.15 × Semantic Complexity Score) + (0.1 × Cultural Relevance Score) + (0.05 × Visual Aid Effectiveness Score), integrating multiple dimensions beyond traditional readability metrics. Discussion The study highlighted the limitations of traditional readability tools in assessing the complexity and cultural relevance of health information. The introduction of CHERS addressed these gaps by incorporating additional dimensions crucial for understanding in a healthcare context. The recommendations provided for creating effective influenza PEMs focused on language simplicity, cultural sensitivity, and actionability. This may enable further research into evaluating current PEMs and clarifying means of creating more effective content in the future. Conclusions The study underscores the need for comprehensive readability assessments in PEMs. The creation of CHERS marks a significant advancement in this field, providing a more holistic approach to evaluating health literacy materials. Its application could lead to the development of more inclusive and effective educational content, thereby improving public health outcomes and reducing the global burden of influenza. Future research should focus on further validating CHERS and exploring its applicability to other health conditions.

Immobilisation masks (IMs) are used for people with head and neck cancer (HNC) undergoing radiation therapy (RT) treatment to ensure accuracy and reproducibility between treatments. Claustrophobia-related mask anxiety in HNC patients is common and can compromise treatment due to patient distress. This scoping review aimed to describe the content of publicly available Patient Education Materials (PEMs) for people with HNC undergoing RT. Three search engines (Bing, Yahoo, and Google) were systematically searched using standard terms. PEMs in audio-visual or written formats were eligible for inclusion if the target readership was adults with HNC and included content on IMs for RT. Content was appraised using the Patient Education Materials Assessment Tool for Printable and Audio-Visual Materials to assess understandability and actionability. In total, 304 PEMs were identified of which 20 met the inclusion criteria. Sixteen PEMs were webpages, three were PDF format, and one was a standalone video. The understandability and actionability of PEMs ranged between 47 to 100% and 0 to 80%, respectively. PEMs authored by Foundations/Organisations scored higher in understandability (80-100%) and were more likely to discuss mask anxiety coping strategies. In comparison, News sites and IM manufacturers published PEMs with the lowest understandability scores (20-80%). The significant variations in the quality of IM PEMs identified suggest that some sources may be more effective at informing patients about IMs. Although multiple aspects of the PEMs were consistent across the reviewed materials, many PEMs lacked information, and a stronger focus on understandability and actionability is required.

BACKGROUND: Online patient education materials (OPEMs) exist to inform patient medical decisions, yet the average adult in the United States reads at an eighth-grade level and 50% of Medicaid patients read at or below a fifth-grade level. To appropriately meet US health literacy needs, the American Medical Association and National Institutes of Health recommend that patient education materials not exceed a sixth-grade level. The purpose of this study was to assess and compare the readability of English and Spanish online patient education materials pertaining to shoulder instability surgery.
METHODS: Google searches of the terms \"shoulder instability surgery\" and \"cirugía de inestabilidad de hombro\'\' were conducted to include 25 eligible online patient education materials OPEMs per language. English OPEM readability was calculated using Flesch-Kincaid Grade Level, Flesch Reading Ease, Flesch Reading Ease Grade Level, Gunning-Fog Index, Coleman-Liau Index, and Simple Measure of Gobbledygook. Spanish OPEM readability was assessed using Fernandez-Huerta Index (FHI) (the Spanish equivalent of Flesch Reading Ease), FHI Grade Level, Gutiérrez de Polini\'s Fórmula de comprensibilidad, and INFLESZ.
RESULTS: Readability index analysis revealed that the mean Flesch Reading Ease of English online patient education materials was significantly lower than the mean FHI of Spanish online patient education materials. English materials were also found to be written at a significantly higher grade level than Spanish materials.
CONCLUSIONS: Shoulder instability surgery online patient education materials in both English and Spanish are written at higher reading levels than recommended by the American Medical Association and National Institutes of Health, though Spanish online patient education materials were more readable on average.