Developing a scalable framework for partnerships between health agencies and the Wikimedia ecosystem

In this era of information overload and misinformation, it is a challenge to rapidly translate evidence-based health information to the public. Wikipedia is a prominent global source of health information with high traffic, multilingual coverage, and acceptable quality control practices. Viewership data following the Ebola crisis and during the COVID-19 pandemic reveals that a significant number of web users located health guidance through Wikipedia and related projects, including its media repository Wikimedia Commons and structured data complement, Wikidata. The basic idea discussed in this paper is to increase and expedite health institutions' global reach to the general public, by developing a specific strategy to maximize the availability of focused content into Wikimedia’s public digital knowledge archives. It was conceptualized from the experiences of leading health organizations such as Cochrane, the World Health Organization (WHO) and other United Nations Organizations, Cancer Research UK, Nation al Network of Libraries of Medicine, and Centers for

(CDC)'s National Institute for Occupational Safety and Health (NIOSH). Each has customized strategies to integrate content in Wikipedia and evaluate responses.
We propose the development of an interactive guide on the Wikipedia and Wikidata platforms to support health agencies, health professionals and communicators in quickly distributing key messages during crisis situations. The guide aims to cover basic features of Wikipedia, including adding key health messages to Wikipedia articles, citing expert sources to facilitate fact-checking, staging text for translation into multiple languages; automating metrics reporting; sharing non-text media; anticipating offline reuse of Wikipedia content in apps or virtual assistants; structuring data for querying and reuse through Wikidata, and profiling other flagship projects from major health organizations.
In the first phase, we propose the development of a curriculum for the guide using information from prior case studies. In the second phase, the guide would be tested on select health-related topics as new case studies. In its third phase, the guide would be finalized and disseminated.
Keywords health communication; Wikipedia; consumer health information; health information systems; information networks; information science; information sharing; public health, health promotion; misinformation; infodemic.

Overview and background
In 2020, the WHO acknowledged it was not only fighting a pandemic caused by the SARS CoV-2 virus but was also engaged in an "infodemic". An infodemic occurs when an abundance of information, both accurate and misleading, spreads rapidly alongside an epidemic (World Health Organization 2020a). This type of information ecosystem makes it difficult for people to locate trustworthy information and reliable guidance. Infodemics have occurred before the COVID-19 pandemic; however, its scale has required public health agents to develop new strategies for managing them (World Health Organization 2020b). Chou et al. (2020) argue that in this type of information ecosystem, a traditional approach to misinformation centered on debunking would likely prove insufficient. These authors called for interdisciplinary research to develop new health strategies that identify the most effective timing, manner, and forums for responding to misinformation. They suggest that one such strategy could rely on getting ahead of misinformation, by developing proactive responses or inoculations against misinformation (Chou et al. 2020). Making health information easier to locate and understand is an initial and necessary step in science translation, so that evidence-based public health practices can be put into use (Elliott and Resnik 2019).
A July 2020 roundtable on health literacy convened by the National Academies of Sciences, Engineering, and Medicine explored an additional challenge resulting from the proliferation of health and medical misinformation and disinformation during the COVID-19 pandemic. The roundtable identified "midinformation" as a category distinct from misinformation and disinformation, being an information crisis where the public experiences a state of "informational ambiguity based on scant knowledge or emerging scientific evidence" (National Academies of Sciences, Engineering, and Medicine 2020). Combatting "midinformation" requires making sure that the information users see first is the information you want them to see (National Academies of Sciences, Engineering, and Medicine 2020. A possible strategy to reach audiences who rely on internet searches is to identify the information pathways used by them (National Academies of Sciences, Engineering, and Medicine 2020).
Wikipedia is recognized for advancing public health (Heilman et al. 2011, Mathew et al. 2013, Cohen 2014, Masukume et al. 2016, Morata and Chadha 2019, Smith 2020, Wikimedia Foundation 2020, McNeil 2020 and is one of the most accessed health information sources West 2015, Smith 2020), with over 30,000 medical articles on the English Wikipedia alone receiving over 5 million pageviews a day in April-May 2019 on average (Maggio et al. 2020). Wikipedia ranks highly in Google search, and users often select it (Calabrese et al. 2019). It can be used to deliver information quickly and effectively to large, established, relevant audiences. While commercial media platforms such as Twitter or Facebook make it easy to post messages and measure impact, Wikipedia is a volunteer nonprofit with strict and unique content guidelines that can be more challenging for expert use. However, science-based information on health conditions, therapies, and events (including epidemics or disasters) can be missing or underdeveloped on Wikipedia. Some health agencies developed strategies to integrate content in Wikipedia and evaluate readership and community response , Murphy et al. 2019. No one has yet produced generalized instructions specifically for health organizations on how to engage quickly, easily and effectively with Wikipedia's content and community.
Consequently, there is an as yet unmet need to develop a general-interest interactive guide on the Wikipedia and Wikidata platforms to support health professionals/communicators in quickly distributing key messages during crisis situations. Wikipedia has guides for publishing content such as biographies and specialized health information, but no guide for organizations to share updated population-level health messages. We propose creating this general-interest interactive guide using the Wikipedia platform's plentiful, developed native tools. The proposed infrastructure would be used to publish health information in Wikipedia to create a case study/demonstration and report pageviews. We aim to lower Wikipedia's barrier of accessibility to health communicators who wish to use Wikipedia for instant publication of institutionally backed messages. The proposed guide would cover basic features of Wikipedia: key messages, citations, and collecting readership metrics.

Objective
To increase and expedite health institutions' global reach to the general public, we propose the development of a general-interest interactive guide on Wikipedia to support health professionals/communication capacities to disseminate health-related information via the Wikipedia ecosystem.

Impact
The proposed plan would improve the timeliness and accuracy of communications and situational awareness regarding threats to the public's health and at-risk populations by expanding the delivery of health information to not only public health professionals, but also to large global audiences through Wikipedia. We expect it to achieve high communication impact, which we will evaluate through metrics reports. These reports can include a list of articles edited, photos and datasets shared, language communities served, publications cited, and pages viewed. Based on the precedent of classroom projects (e.g. Azzam et al. 2017, Weiner et al. 2019) and our review of traffic reports, we conservatively estimate the content would receive more than 10 million views in one year. Additionally, the guide and case studies developed under this project can make publishing to Wikipedia a more accessible option for health organizations globally. The guide would provide peer-topeer advice for professionals to orient themselves and consider options and opportunities for reaching an expansive audience.
Finally, the impact of this project is likely to extend beyond science communication to the general public. Newer systems that evaluate scholarly impact today recognize the importance of a publication appearing in Wikipedia. For example, Altmetric measures the influence and online reach of scholarly output through readership counts, mentions in news sites, blog posts, citation manager bookmarks, social media, and Wikipedia citations.
Contributing peer-reviewed scientific information into Wikipedia boosts not only the diffusion of an agency's knowledge, but also its scholarly recognition.

Implementation
Wikipedia serves a large number of readers who expect and need high quality information on specific topics, complete with bona-fide citations and links to reliable sources.  Fig. 1):

Schematic overview of the steps involved in developing a public health agency model for Wikipedia
Developing a scalable framework for partnerships between health agencies ...

Develop a Wikipedia case study for a specific situation or condition
• As background, note the precedent of Wikipedia's prior crowd organized projects including for Ebola, Zika, COVID-19 as well as Medicine and disasters in general. • Select a topic and analyze multiple communication models for Wikipedia. • Systematically improve Wikimedia content that overlaps and is broader than the target disease or disaster.
A good way to start is by selecting a focused health-related target topic to be used as a case study. Wikipedia has mechanisms to respond to highly dynamic situations, to assess the reliability of resources, to collaborate and to leverage expertise (Keegan and Tan 2020, Benjakob et al. 2021). The proposed concerted effort between health agencies and the Wikipedia community would strengthen Wikipedia content when quick reactions are needed. This can be achieved by systematically improving -ahead of time -Wikimedia content not only about the target topic but also "one step removed" from it, so that when the need to contextualize new information arises, it meets a scaffold (or "scientific backbone", in the language of Benjakob et al. 2021) of already partly contextualized preexisting knowledge ahead of public health emergency situations, thereby improving disaster preparedness, disaster literacy and disaster response. For example, in past epidemics, important complementary information included cultural context like burial practices for Ebola, microcephaly and mosquito control for Zika, or social distancing, face coverings and other non-pharmaceutical interventions for COVID-19.
Milestones in this phase of development include the following: • Compilation of a list of 20+ Wikipedia articles related to that topic, which would be the focus of development and consideration in the case study. • Identification of existing relevant publications on the chosen topic, from expert health organizations, from which we collect key messages to integrate into Wikipedia. • A list of 100 technical terms, curated as structured data in Wikidata, which would be targets for translation as needed. • A snapshot traffic report of all of this content measuring audience reach. • A preliminary engagement report surveying existing Wikipedia community stakeholder engagement in all of the elements above.

Collect communication impact metrics
• Present the background context that in digital publishing, there is an established culture of monitoring social media metrics to measure audience engagement as a way of demonstrating meaningful content delivery. Communication professionals engaged with platforms such as Twitter, Facebook, and YouTube do this. • Set up automated metrics reporting for selected Wikipedia articles and other Wikimedia resources, based on an inventory of key topics for each selected scenario. A survey of topics will make it possible to prioritize areas and materials of interest.
• Develop mechanisms to alert Wikipedians of new publications on the selected topic and alert the agency's staff which of their publications are mentioned in Wikipedia, noting if they are current or need to be updated. This process includes giving credit and doing disclosure of which organizations engaged in Wikipedia and how they interacted.
This phase of development establishes channels for exchanging information between expert organizations and Wikipedia. Expert organizations require traffic reports, community feedback, and alerts of media uptake from Wikipedia. The Wikipedia community requires documentation of institutional engagement, Wikipedia-aligned media contributions, and a forum for stakeholder conversation from organizational partners. This information exchange would occur in a WikiProject or similar Wikipedia ecosystem documentation model, where both staff of the organization and community stakeholders can meet to view all communication metrics, shared resources, and documentation. The project will build from precedents including the Wiki Education Foundation's process for monitoring classroom student editing, Consumer Reports' practice of measuring Wikipedia article pageviews, and Cochrane's interface presenting source publications to cite in Wikipedia articles. The initiative will support the inclusion of relevant evidence within Wikipedia emergency-related articles, as well as processes to help ensure that emergency preparedness and response information included in Wikipedia is of the highest quality and as accurate as possible. Trusted, evidence-based research can ultimately help people make informed decisions about their own health.
Milestones in this phase of development include the following: This phase of development reviews all previous activity and documents it, targeting health communication professionals who would replicate the activity while representing an expert health organization which is publishing in Wikipedia. While the guide will be general, we will get feedback from specific health organizations which express interest in exploring its future use.
Milestones in this phase of development include the following: • Finalize the development of the WikiProject or comparable on-Wikipedia documentation started in phase 1. • Finalize drafting of the white paper started and circulated in phase 2.
• Produce a snapshot report of metrics recorded in phase 3, with interpretation including an estimate of the hours spent to achieve each sort of communication impact. • Produce a standalone guide suitable for off-Wikipedia platform distribution, and which is in a format that health communication professionals would find familiar.

Resource considerations
Our proposal can only fully achieve success if carried out in coordination with support from Wikipedia community projects such as the ones mentioned here established Wikipedia community projects and staff from the participant organizations. A kick-off planning meeting with representatives of Wikimedia entities would be organized at the onset of the project to facilitate detailed conceptual and strategic planning. Recruiting public health informatics and data science expertise would be key in the first two phases of the project. Instructional design expertise should be considered for the third phase. Finally, minor purchases of supplies would be dedicated to customization of the offline Wikipedia through the Internetin-a-box platform.

Concluding remarks
At the moment of submission of this manuscript, our proposal is unfunded, but several of its elements are being executed by the agencies mentioned in the abstract and the WikiProjects aforementioned under Approach. The authors welcome the adoption of the proposed strategy or any of its elements by groups who share this project's goals.
In addition, citations to websites external to NIOSH do not constitute NIOSH endorsement of the sponsoring organizations or their programs or products. The description of the project idea is not associated with any specific funding opportunity, nor will it have any bearing in funding decisions. Furthermore, NIOSH is not responsible for the content of cited websites. All web addresses referenced in this document were accessible as of the publication date.