BACKGROUND: In the digital age, search engines and social media platforms are primary sources for health information, yet their commercial interests-focused algorithms often prioritize irrelevant content. Web-based health applications by reputable sources offer a solution to circumvent these biased algorithms. Despite this advantage, there remains a significant gap in research on the effective integration of content-ranking algorithms within these specialized health applications to ensure the delivery of personalized and relevant health information.
OBJECTIVE: This study introduces a generic methodology designed to facilitate the development and implementation of health information recommendation features within web-based health applications.
METHODS: We detail our proposed methodology, covering conceptual foundation and practical considerations through the stages of design, development, operation, review, and optimization in the software development life cycle. Using a case study, we demonstrate the practical application of the proposed methodology through the implementation of recommendation functionalities in the EndoZone platform, a platform dedicated to providing targeted health information on endometriosis.
RESULTS: Application of the proposed methodology in the EndoZone platform led to the creation of a tailored health information recommendation system known as EndoZone Informatics. Feedback from EndoZone stakeholders as well as insights from the implementation process validate the methodology\'s utility in enabling advanced recommendation features in health information applications. Preliminary assessments indicate that the system successfully delivers personalized content, adeptly incorporates user feedback, and exhibits considerable flexibility in adjusting its recommendation logic. While certain project-specific design flaws were not caught in the initial stages, these issues were subsequently identified and rectified in the review and optimization stages.
CONCLUSIONS: We propose a generic methodology to guide the design and implementation of health information recommendation functionality within web-based health information applications. By harnessing user characteristics and feedback for content ranking, this methodology enables the creation of personalized recommendations that align with individual user needs within trusted health applications. The successful application of our methodology in the development of EndoZone Informatics marks a significant progress toward personalized health information delivery at scale, tailored to the specific needs of users.

OBJECTIVE: Cough audio signal classification is a potentially useful tool in screening for respiratory disorders, such as COVID-19. Since it is dangerous to collect data from patients with contagious diseases, many research teams have turned to crowdsourcing to quickly gather cough sound data. The COUGHVID dataset enlisted expert physicians to annotate and diagnose the underlying diseases present in a limited number of recordings. However, this approach suffers from potential cough mislabeling, as well as disagreement between experts.
METHODS: In this work, we use a semi-supervised learning (SSL) approach - based on audio signal processing tools and interpretable machine learning models - to improve the labeling consistency of the COUGHVID dataset for 1) COVID-19 versus healthy cough sound classification 2) distinguishing wet from dry coughs, and 3) assessing cough severity. First, we leverage SSL expert knowledge aggregation techniques to overcome the labeling inconsistencies and label sparsity in the dataset. Next, our SSL approach is used to identify a subsample of re-labeled COUGHVID audio samples that can be used to train or augment future cough classifiers.
RESULTS: The consistency of the re-labeled COVID-19 and healthy data is demonstrated in that it exhibits a high degree of inter-class feature separability: 3x higher than that of the user-labeled data. Similarly, the SSL method increases this separability by 11.3x for cough type and 5.1x for severity classifications. Furthermore, the spectral differences in the user-labeled audio segments are amplified in the re-labeled data, resulting in significantly different power spectral densities between healthy and COVID-19 coughs in the 1-1.5 kHz range (p=1.2×10-64), which demonstrates both the increased consistency of the new dataset and its explainability from an acoustic perspective. Finally, we demonstrate how the re-labeled dataset can be used to train a COVID-19 classifier, achieving an AUC of 0.797.
CONCLUSIONS: We propose a SSL expert knowledge aggregation technique for the field of cough sound classification for the first time, and demonstrate how it can be used to combine the medical knowledge of multiple experts in an explainable fashion, thus providing abundant, consistent data for cough classification tasks.

Landslides are the most frequent and diffuse natural hazards in Italy causing the greatest number of fatalities and damage to urban areas. The integration of natural hazard information and social media data could improve warning systems to enhance the awareness of disaster managers and citizens about emergency events. The news about landslide events in newspapers or crowdsourcing platforms allows fast observation, surveying and classification. Currently, few studies have been produced on the combination of social media data and traditional sensors. This gap indicates that it is unclear how their integration can effectively provide emergency managers with appropriate knowledge. In this work, rainfall, human lives, and earmarked fund data sources were correlated to \"landslide news\". Analysis was applied to obtain information about temporal (2010-2019) and spatial (regional and warning hydrological zone scale) distribution. The temporal distribution of the data shows a continuous increase from 2015 until 2019 for both landslide and rainfall events. The number of people involved and the amount of earmarked funds do not exhibit any clear trend. The spatial distribution displays good correlation between \"landslide news\", traditional sensors (e.g., pluviometers) and possible effects in term of fatalities. In addition, the cost of soil protection, in monetary terms, indicates the effects of events.

Credibly estimating social-ecological relationships requires data with broad coverage and fine geographic resolutions that are not typically available from standard ecological surveys. Open and unstructured data from crowdsourced platforms offer an opportunity for collecting large quantities of user-submitted ecological data. However, the representativeness of the areas sampled by these data portals is not well known. We investigate how data availability in eBird, one of the largest and most popular crowdsourced science platforms, correlates with race and income of census tracts in two cities: Boston, MA and Phoenix, AZ. We find that checklist submissions vary greatly across census tracts, with similar patterns within both metropolitan regions. In particular, census tracts with high income and high proportions of white residents are most likely to be represented in the data in both cities, which indicates selection bias in eBird coverage. Our results illustrate the non-representativeness of eBird data, and they also raise deeper questions about the validity of statistical inferences regarding disparities that can be drawn from such datasets. We discuss these challenges and illustrate how sample selection problems in unstructured or semi-structured crowdsourced data can lead to spurious conclusions regarding the relationships between race, income, and access to urban bird biodiversity. While crowdsourced data are indispensable and complementary to more traditional approaches for collecting ecological data, we conclude that unstructured or semi-structured data may not be well-suited for all lines of inquiry, particularly those requiring consistent data coverage, and should thus be handled with appropriate care.

The role of schools in the spread of SARS-CoV-2 is controversial, with some claiming they are an important driver of the pandemic and others arguing that transmission in schools is negligible. School cluster reports that have been collected in various jurisdictions are a source of data about transmission in schools. These reports consist of the name of a school, a date, and the number of students known to be infected. We provide a simple model for the frequency and size of clusters in this data, based on random arrivals of index cases at schools who then infect their classmates with a highly variable rate, fitting the overdispersion evident in the data. We fit our model to reports from four Canadian provinces, providing estimates of mean and dispersion for cluster size, as well as the distribution of the instantaneous transmission parameter β, whilst factoring in imperfect ascertainment. According to our model with parameters estimated from the data, in all four provinces (i) more than 65% of non-index cases occur in the 20% largest clusters, and (ii) reducing instantaneous transmission rate and the number of contacts a student has at any given time are effective in reducing the total number of cases, whereas strict bubbling (keeping contacts consistent over time) does not contribute much to reduce cluster sizes. We predict strict bubbling to be more valuable in scenarios with substantially higher transmission rates.
During the COVID-19 pandemic, public health officials promoted social distancing as a way to reduce SARS-CoV-2 transmission. The goal of social distancing is to reduce the number, proximity, and duration of face-to-face interactions between people. To achieve this, people shifted many activities online or canceled events outright. In education, some schools closed and shifted to online learning, while others continued classes in person with safety precautions. Better information about SARS-CoV-2 transmission in schools could help public health officials to make decisions of what activities to keep in person and when to suspend classes. If safety measures lower transmission in schools considerably, then closing schools may not be worth online education\'s social, educational, and economic costs. However, if transmission of SARS-CoV-2 in schools remains high despite measures, closing schools may be essential, despite the costs. Tupper et al. used data about COVID-19 cases in children attending in-person school in four Canadian provinces between 2020 and 2021 to fit a computer model of school transmission. On average, their analysis shows that one infected person in a school leads to between two and three further cases. Most of the time, no more students are infected, indicating that normally infection clusters are small; and only rarely does one infected person set off a large outbreak. The model also showed that measures to reduce transmission, like masking or small class sizes, were more effective than interventions such as keeping students with the same cohort all day (bubbling). Tupper et al. caution that their findings apply to the variants of SARS-CoV-2 circulating in Canada during the 2020-2021 school year, and may not apply to newer, highly transmissible strains like Omicron. However, the model could always be adapted to assess school or workplace transmission of more recent strains of SARS-CoV-2, and more generally of other diseases. Thus, Tupper et al. provide a new approach to estimating the rate of disease transmission and comparing the impact of different prevention strategies.

BACKGROUND: Amazon\'s Mechanical Turk (MTurk) appears to be a reliable resource for studying clinical populations and accessing hard-to-reach populations. Recent research suggests that MTurk may also be a viable option for military recruitment.
OBJECTIVE: The goal of the current study was to examine the utility of collecting clinical data on military samples recruited via MTurk.
METHODS: Participants were 535 military veterans (Mage = 37.45; 71.8% men; 69.5% White) who completed measures assessing trauma and mental health.
RESULTS: Findings indicate that rates of military traumas and mental health diagnoses were higher than published comparisons; posttraumatic stress disorder (PTSD) and depression symptoms were found to be higher than values found in a nationally representative sample, lower than a treatment-seeking sample, and comparable to a MTurk-recruited military sample. Alcohol misuse was found to be higher than both nationally representative and treatment-seeking samples. Psychometric analyses indicated support for convergent validity of measures, and confirmatory factor analysis results demonstrated that empirically supported factor models of PTSD were replicated in the current sample; the hybrid model demonstrated the best fit.
CONCLUSIONS: Our findings support the utility of MTurk for collecting clinical data on military samples. Increasing access to and recruitment of military samples is important for advancing the field of military psychology. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The usage of crowdsourcing to recruit numerous participants has been recognized as beneficial in the human-computer interaction (HCI) field, such as for designing user interfaces and validating user performance models. In this work, we investigate its effectiveness for evaluating an error-rate prediction model in target pointing tasks. In contrast to models for operational times, a clicking error (i.e., missing a target) occurs by chance at a certain probability, e.g., 5%. Therefore, in traditional laboratory-based experiments, a lot of repetitions are needed to measure the central tendency of error rates. We hypothesize that recruiting many workers would enable us to keep the number of repetitions per worker much smaller. We collected data from 384 workers and found that existing models on operational time and error rate showed good fits (both R 2 > 0.95). A simulation where we changed the number of participants N P and the number of repetitions N repeat showed that the time prediction model was robust against small N P and N repeat, although the error-rate model fitness was considerably degraded. These findings empirically demonstrate a new utility of crowdsourced user experiments for collecting numerous participants, which should be of great use to HCI researchers for their evaluation studies.

Deep learning models can recognize the food item in an image and derive their nutrition information, including calories, macronutrients (carbohydrates, fats, and proteins), and micronutrients (vitamins and minerals). This technology has yet to be implemented for the nutrition assessment of restaurant food. In this paper, we crowdsource 15,908 food images of 470 restaurants in the Greater Hartford region on Tripadvisor and Google Place. These food images are loaded into a proprietary deep learning model (Calorie Mama) for nutrition assessment. We employ manual coding to validate the model accuracy based on the Food and Nutrient Database for Dietary Studies. The derived nutrition information is visualized at both the restaurant level and the census tract level. The deep learning model achieves 75.1% accuracy when compared with manual coding. It has more accurate labels for ethnic foods but cannot identify portion sizes, certain food items (e.g., specialty burgers and salads), and multiple food items in an image. The restaurant nutrition (RN) index is further proposed based on the derived nutrition information. By identifying the nutrition information of restaurant food through crowdsourced food images and a deep learning model, the study provides a pilot approach for large-scale nutrition assessment of the community food environment.

College campuses in the United States have begun implementing smoke and tobacco-free policies to discourage the use of tobacco. Smoke and tobacco-free policies, however, are contingent upon effective policy enforcement.
This study aimed to develop an empirically derived web-based tracking tool (Tracker) for crowdsourcing campus environmental reports of tobacco use and waste to support smoke and tobacco-free college policies.
An exploratory sequential mixed methods approach was utilized to inform the development and evaluation of Tracker. In October 2018, three focus groups across 2 California universities were conducted and themes were analyzed, guiding Tracker development. After 1 year of implementation, users were asked in April 2020 to complete a survey about their experience.
In the focus groups, two major themes emerged: barriers and facilitators to tool utilization. Further Tracker development was guided by focus group input to address these barriers (eg, information, policing, and logistical concerns) and facilitators (eg, environmental motivators and positive reinforcement). Amongst 1163 Tracker reports, those who completed the user survey (n=316) reported that the top motivations for using the tool had been having a cleaner environment (212/316, 79%) and health concerns (185/316, 69%).
Environmental concerns, a motivator that emerged in focus groups, shaped Tracker\'s development and was cited by the majority of users surveyed as a top motivator for utilization.

Causes of and treatments for long-COVID syndrome remain unknown. Drawing on uncertainty management theory (UMT), this study elucidates the communicative nature of crowdsourced medicine as a means by which COVID \"long-haulers\" respond to their poorly understood illness.
31,892 posts on the long-haulers subreddit (r/covidlonghaulers) were analyzed, starting with its creation date, July 24th, 2020, until January 7, 2021. The Meaning Extraction Method was used to identify clusters of words that mathematically group together across the text observations.
Analyses yielded 16 distinct factors of words, which we thematized based on their composition, the data, and UMT. The 16 themes encompassed symptoms (e.g., pain, respiratory, sensory), diagnostic concerns (testing, diagnosis), broad health concerns (immunity, physical activity, diet), chronicity, support, identity, and anxiety.
Findings provide a succinct, yet robust set of themes reflecting the information-seeking (i.e., \"This is happening to me\") and support-seeking functions of long-haulers\' talk (i.e., \"Is this happening to you?\"). Findings have implications for collective uncertainty management, online crowdsourcing, and patient advocacy.
We recommend that health care providers employ sensitivity when addressing the anxiety that long-haulers are experiencing while also validating that their physical symptoms are real. Online communities help long-haulers manage their uncertainty.