BACKGROUND: Considering the increase in the proportion of the older population worldwide, the demand for health system resources also arises. These tools optimize clinical decision-making, thus avoiding iatrogenesis and thus contributing to a better quality of life for the older population. In response, we created an online web application, the APIMedOlder, that provides access to healthcare professionals to allow healthcare professionals to access potentially inappropriate medication identification criteria through a useful tool with a simplified profile, allowing its applicability in clinical practice. This study aims to assess the usability of the APIMedOlder online web application by healthcare professionals.
METHODS: A questionnaire, based on the System Usability Scale, was distributed among 15 healthcare professionals (five pharmacists, four physicians, three pharmacy technicians, and three nurses), to fully explore the website.
RESULTS: Overall, healthcare professionals\' evaluation of the usability of the APIMedOlder online web application was rated as \"Best imaginable\" (mean score of 87.17 points), with individual scores ranging from 75 to 100 points. Internal consistency of α = 0.881 (CI 95%: 0.766 - 0.953) was achieved. Specific questionnaire items contributing to this high score included ease of use, learning efficiency, and integration of functions.
CONCLUSIONS: The overall evaluation of the developed tool was positive, with this online application being recognized as being easy to use and having well-integrated functions.

This paper presents a new edge detection process implemented in an embedded IoT device called Bee Smart Detection node to detect catastrophic apiary events. Such events include swarming, queen loss, and the detection of Colony Collapse Disorder (CCD) conditions. Two deep learning sub-processes are used for this purpose. The first uses a fuzzy multi-layered neural network of variable depths called fuzzy-stranded-NN to detect CCD conditions based on temperature and humidity measurements inside the beehive. The second utilizes a deep learning CNN model to detect swarming and queen loss cases based on sound recordings. The proposed processes have been implemented into autonomous Bee Smart Detection IoT devices that transmit their measurements and the detection results to the cloud over Wi-Fi. The BeeSD devices have been tested for easy-to-use functionality, autonomous operation, deep learning model inference accuracy, and inference execution speeds. The author presents the experimental results of the fuzzy-stranded-NN model for detecting critical conditions and deep learning CNN models for detecting swarming and queen loss. From the presented experimental results, the stranded-NN achieved accuracy results up to 95%, while the ResNet-50 model presented accuracy results up to 99% for detecting swarming or queen loss events. The ResNet-18 model is also the fastest inference speed replacement of the ResNet-50 model, achieving up to 93% accuracy results. Finally, cross-comparison of the deep learning models with machine learning ones shows that deep learning models can provide at least 3-5% better accuracy results.

In practical electrocardiography (ECG) interpretation, the scarcity of well-annotated data is a common challenge. Transfer learning techniques are valuable in such situations, yet the assessment of transferability has received limited attention. To tackle this issue, we introduce MELEP, which stands for Muti-label Expected Log of Empirical Predictions, a measure designed to estimate the effectiveness of knowledge transfer from a pre-trained model to a downstream multi-label ECG diagnosis task. MELEP is generic, working with new target data with different label sets, and computationally efficient, requiring only a single forward pass through the pre-trained model. To the best of our knowledge, MELEP is the first transferability metric specifically designed for multi-label ECG classification problems. Our experiments show that MELEP can predict the performance of pre-trained convolutional and recurrent deep neural networks, on small and imbalanced ECG data. Specifically, we observed strong correlation coefficients (with absolute values exceeding 0.6 in most cases) between MELEP and the actual average F1 scores of the fine-tuned models. Our work highlights the potential of MELEP to expedite the selection of suitable pre-trained models for ECG diagnosis tasks, saving time and effort that would otherwise be spent on fine-tuning these models.

A priority for machine learning in healthcare and other high stakes applications is to enable end-users to easily interpret individual predictions. This opinion piece outlines recent developments in interpretable classifiers and methods to open black box models.

CONCLUSIONS: In an effort to expedite the publication of articles, AJHP is posting manuscripts online as soon as possible after acceptance. Accepted manuscripts have been peer-reviewed and copyedited, but are posted online before technical formatting and author proofing. These manuscripts are not the final version of record and will be replaced with the final article (formatted per AJHP style and proofed by the authors) at a later time.
OBJECTIVE: Treatment for pulmonary hypertension includes medications with risk evaluation and mitigation strategy (REMS) programs. Health-system inpatient pharmacies dispensing these agents must comply with inpatient REMS dispensing criteria. Implementing a health-system policy with computerized provider order entry (CPOE) decision support may improve REMS compliance.
METHODS: This was a retrospective, quasi-experimental study comparing REMS compliance before and after development of a policy with CPOE decision support that was implemented in August 2019. Patients 18 years of age or older with a diagnosis of pulmonary hypertension were included if they received at least one dose of an endothelin receptor antagonist or riociguat while hospitalized. Patients were included in the preintervention group if they were hospitalized between August 1, 2017, and August 31, 2019, and in the postintervention group if they were hospitalized between September 1, 2019, and August 31, 2021. The primary outcome was the REMS compliance rate. Secondary endpoints included the time to REMS compliance and independent factors associated with failed or delayed REMS compliance.
RESULTS: A total of 150 patients were included, with 75 patients in both the pre- and postintervention groups. Compliance increased significantly from the preintervention (50%) to postintervention (92%) group (P < 0.001). Time to compliance was also significantly reduced from 770 minutes in the preintervention group to 140 minutes in the postintervention group (P = 0.031). Factors independently associated with REMS compliance were being in the postintervention group (odds ratio, 16.9; 95% confidence interval, 5.8-49.2) and being admitted to a pulmonary hypertension center for comprehensive care. (odds ratio, 7.8; 95% confidence interval, 2.9-21.2).
CONCLUSIONS: A health-system policy with CPOE decision support improved both the rate of and time to compliance with inpatient REMS dispensing procedures.

UNASSIGNED: The exponential growth in computing power and the increasing digitization of information have substantially advanced the machine learning (ML) research field. However, ML algorithms are often considered \"black boxes,\" and this fosters distrust. In medical domains, in which mistakes can result in fatal outcomes, practitioners may be especially reluctant to trust ML algorithms.
UNASSIGNED: The aim of this study is to explore the effect of user-interface design features on intensivists\' trust in an ML-based clinical decision support system.
UNASSIGNED: A total of 47 physicians from critical care specialties were presented with 3 patient cases of bacteremia in the setting of an ML-based simulation system. Three conditions of the simulation were tested according to combinations of information relevancy and interactivity. Participants\' trust in the system was assessed by their agreement with the system\'s prediction and a postexperiment questionnaire. Linear regression models were applied to measure the effects.
UNASSIGNED: Participants\' agreement with the system\'s prediction did not differ according to the experimental conditions. However, in the postexperiment questionnaire, higher information relevancy ratings and interactivity ratings were associated with higher perceived trust in the system (P<.001 for both). The explicit visual presentation of the features of the ML algorithm on the user interface resulted in lower trust among the participants (P=.05).
UNASSIGNED: Information relevancy and interactivity features should be considered in the design of the user interface of ML-based clinical decision support systems to enhance intensivists\' trust. This study sheds light on the connection between information relevancy, interactivity, and trust in human-ML interaction, specifically in the intensive care unit environment.

BACKGROUND: Emerging technologies such as artificial intelligence (AI) require an early-stage assessment of potential societal and ethical implications to increase their acceptability, desirability, and sustainability. This paper explores and compares 2 of these assessment approaches: the responsible innovation (RI) framework originating from technology studies and the co-design approach originating from design studies. While the RI framework has been introduced to guide early-stage technology assessment through anticipation, inclusion, reflexivity, and responsiveness, co-design is a commonly accepted approach in the development of technologies to support the care for older adults with frailty. However, there is limited understanding about how co-design contributes to the anticipation of implications.
OBJECTIVE: This paper empirically explores how the co-design process of an AI-based decision support system (DSS) for dementia caregivers is complemented by explicit anticipation of implications.
METHODS: This case study investigated an international collaborative project that focused on the co-design, development, testing, and commercialization of a DSS that is intended to provide actionable information to formal caregivers of people with dementia. In parallel to the co-design process, an RI exploration took place, which involved examining project members\' viewpoints on both positive and negative implications of using the DSS, along with strategies to address these implications. Results from the co-design process and RI exploration were analyzed and compared. In addition, retrospective interviews were held with project members to reflect on the co-design process and RI exploration.
RESULTS: Our results indicate that, when involved in exploring requirements for the DSS, co-design participants naturally raised various implications and conditions for responsible design and deployment: protecting privacy, preventing cognitive overload, providing transparency, empowering caregivers to be in control, safeguarding accuracy, and training users. However, when comparing the co-design results with insights from the RI exploration, we found limitations to the co-design results, for instance, regarding the specification, interrelatedness, and context dependency of implications and strategies to address implications.
CONCLUSIONS: This case study shows that a co-design process that focuses on opportunities for innovation rather than balancing attention for both positive and negative implications may result in knowledge gaps related to social and ethical implications and how they can be addressed. In the pursuit of responsible outcomes, co-design facilitators could broaden their scope and reconsider the specific implementation of the process-oriented RI principles of anticipation and inclusion.

OBJECTIVE: Conduct a scoping review of research studies that describe rule-based clinical decision support (CDS) malfunctions.
METHODS: In April 2022, we searched three bibliographic databases (MEDLINE, CINAHL, and Embase) for literature referencing CDS malfunctions. We coded the identified malfunctions according to an existing CDS malfunction taxonomy and added new categories for factors not already captured. We also extracted and summarized information related to the CDS system, such as architecture, data source, and data format.
RESULTS: Twenty-eight articles met inclusion criteria, capturing 130 malfunctions. Architectures used included stand-alone systems (eg, web-based calculator), integrated systems (eg, best practices alerts), and service-oriented architectures (eg, distributed systems like SMART or CDS Hooks). No standards-based CDS malfunctions were identified. The \"Cause\" category of the original taxonomy includes three new types (organizational policy, hardware error, and data source) and two existing causes were expanded to include additional layers. Only 29 malfunctions (22%) described the potential impact of the malfunction on patient care.
CONCLUSIONS: While a substantial amount of research on CDS exists, our review indicates there is a limited focus on CDS malfunctions, with even less attention on malfunctions associated with modern delivery architectures such as SMART and CDS Hooks.
CONCLUSIONS: CDS malfunctions can and do occur across several different care delivery architectures. To account for advances in health information technology, existing taxonomies of CDS malfunctions must be continually updated. This will be especially important for service-oriented architectures, which connect several disparate systems, and are increasing in use.

BACKGROUND: Childhood tumors in the central nervous system (CNS) have longer diagnostic delays than other pediatric tumors. Vague presenting symptoms pose a challenge in the diagnostic process; it has been indicated that patients and parents may be hesitant to seek help, and health care professionals (HCPs) may lack awareness and knowledge about clinical presentation. To raise awareness among HCPs, the Danish CNS tumor awareness initiative hjernetegn.dk was launched.
OBJECTIVE: This study aims to present the learnings from designing and implementing a decision support tool for HCPs to reduce diagnostic delay in childhood CNS tumors. The aims also include decisions regarding strategies for dissemination and use of social media, and an evaluation of the digital impact 6 months after launch.
METHODS: The phases of developing and implementing the tool include participatory co-creation workshops, designing the website and digital platforms, and implementing a press and media strategy. The digital impact of hjernetegn.dk was evaluated through website analytics and social media engagement.
UNASSIGNED: hjernetegn.dk was launched in August 2023. The results after 6 months exceeded key performance indicators. The analysis showed a high number of website visitors and engagement, with a plateau reached 3 months after the initial launch. The LinkedIn campaign and Google Search strategy also generated a high number of impressions and clicks.
CONCLUSIONS: The findings suggest that the initiative has been successfully integrated, raising awareness and providing a valuable tool for HCPs in diagnosing childhood CNS tumors. The study highlights the importance of interdisciplinary collaboration, co-creation, and ongoing community management, as well as broad dissemination strategies when introducing a digital support tool.

BACKGROUND: Although the use of artificial intelligence (AI)-based technologies, such as AI-based decision support systems (AI-DSSs), can help sustain and improve the quality and efficiency of care, their deployment creates ethical and social challenges. In recent years, a growing prevalence of high-level guidelines and frameworks for responsible AI innovation has been observed. However, few studies have specified the responsible embedding of AI-based technologies, such as AI-DSSs, in specific contexts, such as the nursing process in long-term care (LTC) for older adults.
OBJECTIVE: Prerequisites for responsible AI-assisted decision-making in nursing practice were explored from the perspectives of nurses and other professional stakeholders in LTC.
METHODS: Semistructured interviews were conducted with 24 care professionals in Dutch LTC, including nurses, care coordinators, data specialists, and care centralists. A total of 2 imaginary scenarios about AI-DSSs were developed beforehand and used to enable participants articulate their expectations regarding the opportunities and risks of AI-assisted decision-making. In addition, 6 high-level principles for responsible AI were used as probing themes to evoke further consideration of the risks associated with using AI-DSSs in LTC. Furthermore, the participants were asked to brainstorm possible strategies and actions in the design, implementation, and use of AI-DSSs to address or mitigate these risks. A thematic analysis was performed to identify the opportunities and risks of AI-assisted decision-making in nursing practice and the associated prerequisites for responsible innovation in this area.
RESULTS: The stance of care professionals on the use of AI-DSSs is not a matter of purely positive or negative expectations but rather a nuanced interplay of positive and negative elements that lead to a weighed perception of the prerequisites for responsible AI-assisted decision-making. Both opportunities and risks were identified in relation to the early identification of care needs, guidance in devising care strategies, shared decision-making, and the workload of and work experience of caregivers. To optimally balance the opportunities and risks of AI-assisted decision-making, seven categories of prerequisites for responsible AI-assisted decision-making in nursing practice were identified: (1) regular deliberation on data collection; (2) a balanced proactive nature of AI-DSSs; (3) incremental advancements aligned with trust and experience; (4) customization for all user groups, including clients and caregivers; (5) measures to counteract bias and narrow perspectives; (6) human-centric learning loops; and (7) the routinization of using AI-DSSs.
CONCLUSIONS: The opportunities of AI-assisted decision-making in nursing practice could turn into drawbacks depending on the specific shaping of the design and deployment of AI-DSSs. Therefore, we recommend considering the responsible use of AI-DSSs as a balancing act. Moreover, considering the interrelatedness of the identified prerequisites, we call for various actors, including developers and users of AI-DSSs, to cohesively address the different factors important to the responsible embedding of AI-DSSs in practice.