BACKGROUND: Dermatologic patient education materials (PEMs) are often written above the national average seventh- to eighth-grade reading level. ChatGPT-3.5, GPT-4, DermGPT, and DocsGPT are large language models (LLMs) that are responsive to user prompts. Our project assesses their use in generating dermatologic PEMs at specified reading levels.
OBJECTIVE: This study aims to assess the ability of select LLMs to generate PEMs for common and rare dermatologic conditions at unspecified and specified reading levels. Further, the study aims to assess the preservation of meaning across such LLM-generated PEMs, as assessed by dermatology resident trainees.
METHODS: The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. We prompted ChatGPT-3.5, GPT-4, DermGPT, and DocsGPT to \"Create a patient education handout about [condition] at a [FKRL]\" to iteratively generate 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs, evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 dermatology resident trainees.
RESULTS: The current American Academy of Dermatology PEMs had an average (SD) FKRL of 9.35 (1.26) and 9.50 (2.3) for common and rare diseases, respectively. For common diseases, the FKRLs of LLM-produced PEMs ranged between 9.8 and 11.21 (unspecified prompt), between 4.22 and 7.43 (fifth-grade prompt), and between 5.98 and 7.28 (seventh-grade prompt). For rare diseases, the FKRLs of LLM-produced PEMs ranged between 9.85 and 11.45 (unspecified prompt), between 4.22 and 7.43 (fifth-grade prompt), and between 5.98 and 7.28 (seventh-grade prompt). At the fifth-grade reading level, GPT-4 was better at producing PEMs for both common and rare conditions than ChatGPT-3.5 (P=.001 and P=.01, respectively), DermGPT (P<.001 and P=.03, respectively), and DocsGPT (P<.001 and P=.02, respectively). At the seventh-grade reading level, no significant difference was found between ChatGPT-3.5, GPT-4, DocsGPT, or DermGPT in producing PEMs for common conditions (all P>.05); however, for rare conditions, ChatGPT-3.5 and DocsGPT outperformed GPT-4 (P=.003 and P<.001, respectively). The preservation of meaning analysis revealed that for common conditions, DermGPT ranked the highest for overall ease of reading, patient understandability, and accuracy (14.75/15, 98%); for rare conditions, handouts generated by GPT-4 ranked the highest (14.5/15, 97%).
CONCLUSIONS: GPT-4 appeared to outperform ChatGPT-3.5, DocsGPT, and DermGPT at the fifth-grade FKRL for both common and rare conditions, although both ChatGPT-3.5 and DocsGPT performed better than GPT-4 at the seventh-grade FKRL for rare conditions. LLM-produced PEMs may reliably meet seventh-grade FKRLs for select common and rare dermatologic conditions and are easy to read, understandable for patients, and mostly accurate. LLMs may play a role in enhancing health literacy and disseminating accessible, understandable PEMs in dermatology.

A cross-sectional study was conducted between March and August 2021; an electronic survey was administered to all dermatology residents (n=79) in all centers with the Saudi Board Dermatology Training Program in the western, eastern and southern regions. Participation was voluntary, and written informed consent was obtained before the study. The survey was completed by 70 dermatology residents, for an 88.61% response rate. All the participants were aware of the study\'s aims, and their informed consent was obtained. The study design complied with the Declaration of Helsinki ethical standards and was approved by the Institutional Review Board at Imam Muhammad Ibn Saud Islamic University in Saudi Arabia. A total of 70 respondents completed the questionnaire. High emotional exhaustion (EE) was present in 47.1% of respondents, low personal accomplishment (PA) was present in 65.7%, and high depersonalization (DP) was the least prevalent (24.3%) across all burnout dimensions of dermatology residents. Overall burnout was present in 21.4% of the dermatology residents. Multivariate analysis showed that the odds of EE were significantly lower in males than females [odds ratio (OR)=0.2, P=0.016] and the higher number of patients seen per clinic (OR=1.09, P=0.032) was associated with higher odds of having a high risk of EE, while higher satisfaction with work-life balance was associated with lower odds of a high risk of EE (OR=0.47, P=0.005). A higher number of study hours/week was associated with lower odds of low PA (OR=0.95, P=0.02). Similarly, higher satisfaction with career was associated with lower odds of low PA (OR=0.35, P=0.042). Multivariate analysis showed that only exercise (OR=0.21, P=0.05) and satisfaction with work-life balance (OR=0.42, P=0.008) were associated with a lower risk of burnout. Our study adds the burnout rates among dermatology residents in Saudi Arabia, in addition to possible risk factors that can predict burnout, to the literature. These findings can be applied to improve training programs and reduce the burnout rate among residents.