The School of Data Science at the University of Virginia received funding through an open call from the Wellcome Trust for this research and development proposal titled "FAIR and open multilingual clinical trials in Wikidata and Wikipedia". This document combines the concept note, full application, and supporting details in that application. Our intent in publishing this is to make the proposal available with free and open copyright licensing for other researchers to reuse and remix as they wish.

The project objective is to import clinical trials metadata from ClinicalTrials.gov into the Wikidata platform, and to curate the data using Wikidata workflows, and also to present the overall process as a model for using the Wikipedia ecosystem to share and remix data. If the project is successful to the extent of our wishes then our hope is that all sorts of researchers and the public will access and use clinical trials data for both traditional and new purposes.

Although we believe that researchers who currently use clinical trials data will find benefit from its curation through the Wikipedia platform development process, we also seek to promote access to this medical information to new audiences both within conventional analysis about clinical trials and in new and unexpected contexts. The new audiences which we anticipate are those whom we already know to browse Wikipedia's medical content, including researchers who prefer to access information outside of English language and non-researchers including students, journalists, and policy makers who previously would not have considered seeking this data were it not accessible through the familiar Wikipedia. By making the data much more accessible and also available for Wikipedia's style of crowdsourcing, we hope that others will develop and reuse this data including by linking trials to papers, people, and organizations; visualizing the trials with charts and maps; general curation of trials by keyword tagging or concept disambiguation; and language translation of technical terms in structured data collections such as Wikidata.

We are sharing this text in alignment with Wikipedia community values of openness and in a contemporary social context where sharing proposals is uncommon but which we wish were more routine. At the time of publishing this proposal we have developed the project but have not yet completed it. This document only presents the proposal, and we will publish our methods, results, and the overall model in a later paper. The term of the project is extended due to COVID-19, and we changed some of the project focus from the original proposal in response to the pandemic. The text here does not account for our response to COVID.

1. By default, adopt the established Wikipedia and Wikidata publishing and engagement practices for open, FAIR, documentation, receiving feedback in permanent public forums, and collaboration
2. Contribute to the documentation about the position of Wikipedia and Wikidata in the Linked Open Data ecosystem, particularly emphasizing university participation in import and export of research metadata in the Wikimedia platform and collecting impact metrics for doing so.
3. Within Wikidata, contribute to the WikiCite project which seeks to enrich data around citations and metadata, including PubMed research papers, and subsets of CrossRef, ORCID, and ClinicalTrials.gov. Explore and document possibilities to ingest non-United States clinical trial databases.
4. Identify the set of terms and concepts which are necessary to perform and visualize queries of medical research data, for example, "clinical research sites in a given country with the highest trial completion rates in infectious disease research"
5. Translate those terms to languages including Hindi, Bengali, and Swahili to the level of quality which is established as a norm by existing local community participants in Wikipedia and Wikidata
6. Use Wikipedia and Wikidata's native metrics reporting processes to measure the impact to users and the engagement of peer reviewers

The School of Data Science at the University of Virginia received funding through an open call from the Wellcome Trust for this research and development proposal titled "FAIR and open multilingual clinical trials in Wikidata and Wikipedia". This document combines the concept note, full application, and supporting details in that application. Our intent in publishing this is to make the proposal available with free and open copyright licensing for other researchers to reuse and remix as they wish.

The project objective is to import clinical trials metadata from ClinicalTrials.gov into the Wikidata platform, and to curate the data using Wikidata workflows, and also to present the overall process as a model for using the Wikipedia ecosystem to share and remix data. If the project is successful to the extent of our wishes then our hope is that all sorts of researchers and the public will access and use clinical trials data for both traditional and new purposes.

Although we believe that researchers who currently use clinical trials data will find benefit from its curation through the Wikipedia platform development process, we also seek to promote access to this medical information to new audiences both within conventional analysis about clinical trials and in new and unexpected contexts. The new audiences which we anticipate are those whom we already know to browse Wikipedia's medical content, including researchers who prefer to access information outside of English language and non-researchers including students, journalists, and policy makers who previously would not have considered seeking this data were it not accessible through the familiar Wikipedia. By making the data much more accessible and also available for Wikipedia's style of crowdsourcing, we hope that others will develop and reuse this data including by linking trials to papers, people, and organizations; visualizing the trials with charts and maps; general curation of trials by keyword tagging or concept disambiguation; and language translation of technical terms in structured data collections such as Wikidata.

We are sharing this text in alignment with Wikipedia community values of openness and in a contemporary social context where sharing proposals is uncommon but which we wish were more routine. At the time of publishing this proposal we have developed the project but have not yet completed it. This document only presents the proposal, and we will publish our methods, results, and the overall model in a later paper. The term of the project is extended due to COVID-19, and we changed some of the project focus from the original proposal in response to the pandemic. The text here does not account for our response to COVID.

Name: Lane Rasberry
Organization: University of Virginia
Department: School of Data Science
Division: Center for Ethics and Justice
Email: rasberry@virginia.edu
ORCID: 0000-0002-9485-6146

This project will use Wikipedia's own established processes for monitoring and evaluation of university projects to develop and publish general reference information in Wikipedia and Wikidata. We will evaluate this project in these established ways:

1. Content metrics - Report the standard publishing metrics as measured by Wikimedia's own native metrics suite for publishers
2. Diversity and inclusion - Partner with established Wikimedia community organizations; confirm their oversight and approval
3. Impact metrics – Report audience readership as measured by Wikipedia's own native metrics suite for users
4. Quality review - university student researchers will evaluate and publish an evaluation of the source research metadata and the translation process
5. Bias evaluation – we will subjectively publish our opinions on bias we identify and its cause. One obvious source of bias will be availability of open data, as much research indexed in PubMed and ClinicalTrials.gov is not compliant with recommended metadata standards. This project favors institutions which apply FAIR principles, and we will identify these practices.

Wikipedia as a publishing, technology, and community platform continually introduces processes for content development and evaluation. In 2012, with the establishment of the Wiki Education Foundation, there was a major cultural shift to make Wikipedia compatible with university education and research. Today, that precedent has developed into a suite of open evaluation tools for measuring audience size, levels of engagement, use of fact-checking processes, and a culture applying metrics to perform critical review of Wikipedia's quality. These measurements and processes establish a precedent for this project to follow in doing publishing and content development, operationalizing ethics in digital governance, and publicly demonstrating community conversation in seeking feedback on this project's activities in the context of global Wikimedia content development.

My name is Lane Rasberry and I am Wikimedian at the School of Data Science at the University of Virginia. The most important contribution I have made to research has been my advocacy for free and open media. I particularly do this in the context of Wikimedia platform publishing where I showcase Wikidata as a FAIR and open repository which is useful in itself and also a source from which anyone can export information for use in other platforms. For any other project seeking to be FAIR and open, I present Wikidata as a model project to emulate as the standard.

My usual talking point with Wikipedia is that it is the most requested, published, accessed, and consulted source of information in English on nearly every topic, including almost every topic of general interest in medicine. I advocate for institutional partnerships between expert health organizations, typically medical schools and media sources, and the Wikimedia platform. Uniformly all expert health organizations have great challenges in distributing their content to a relevant audience. In comparison, Wikipedia and Wikidata have the world's best available distribution channels and reach a massive audience through Internet search, personal assistants and similar mechanisms. My wish is to match Wikipedia's excellent distribution but lower quality content with the excellent content of expert organizations which have a relatively poorer distribution and media reach. Wikipedia's media reach is useful for any organization with an educational mission to distribute high quality information of general interest. I have managed various programs which integrate information into Wikipedia for the purpose of increased distribution.

Along with that distribution, I have designed policy and published documentation on various ethical considerations of online publishing, including making open media more accessible to diverse audiences who collaborate in friendly online virtual spaces. Such documentation and policy creates an environment where more people can participate and share in the benefits of the FAIR and open global digital commons.

1. Case studies of the student experience of publishing medical information in Wikipedia
   • Improving the Quality of Consumer Health Information on Wikipedia: Case Series (Weiner et al. 2019)
   • Why Medical Schools Should Embrace Wikipedia (Azzam et al. 2016)
2. Documentation and programming for the Scholia interface to Wikidata and the WikiCite project
   • Robustifying Scholia: paving the way for knowledge discovery and research assessment through Wikidata (Rasberry et al. 2019)
3. Promotion of diversity, inclusion, and friendly collaboration online
   • Automatic Detection of Online Abuse and Analysis of Problematic Users in Wikipedia (Rawat et al. 2019)
   • Queering Wikipedia (Wexelbaum et al. 2015)

I Lane Rasberry have been a highly active Wikimedia platform contributor since 2008. I have participated in Wikimedia institutional governance since 2012, including in the platform organizations Wikimedia Medicine, Wikimedia New York City, and Wikimedia LGBT+. I am as active as anyone else in the Wikimedia projects for university programs, the WikiCite project, designing institutional partnerships, and reporting communication metrics for evidence of impact. The Wikimedia platform opens research literature by being the most requested, published, accessed, and consulted source of information on all topics which it covers.

I have been a member of the HIV Vaccine Trials Network Community Advisory Board since 2007. In this role I have advocated for public access to medical research information for a long-running series of clinical trials.

Personally I document and publish notes from organizational meetings of all sorts and publish in the open as a default practice. I advocate for free and open licensing on any media output which anyone intends for distribution to the public.

School of Data Science, University of Virginia

Daniel Mietchen, data scientist

Lane Rasberry, Wikimedian in Residence

The School of Data Science at the University of Virginia is importing structured data into Wikidata, evaluating the quality of information in Wikipedia and Wikidata, and documenting best practices for university partnerships with the Wikimedia platform. This team identifies the research content for which there is a need in research discovery to be FAIR, available for query, and translated to promote global collaboration.

Daniel Mietchen, data scientist School of Data Science at the University of Virginia, Principle Investigator of the Scholia / WikiCite project to develop the Wikipedia / Wikimedia platform based interface for discovering and visualizing scholarly publications in a free and open system analogous to the popular but closed product Google Scholar. Dr. Mietchen's primary concern are the academic publications, whereas this proposal to Wellcome would integrate clinical trial data into this network and also localize the interface for non-English pilot languages relevant to the developing world. Another way to say this is that Dr. Mietchen operates a Wikipedia-based tool similar to PubMed and Google Scholar, and this project would collaborate with him to integrate ClinicalTrials.gov data into this and permit non-English language use.

UVA Global, University of Virginia

This is a language department at the university where faculty and classes will translate and publish structured data into Wikidata, where it will be FAIR and open in the semantic web.

Wikimedia community organizationsThese organizations provide community feedback on publication in Wikipedia and Wikidata and also on the translation process. These partnerships ensure participation among stakeholders and regional communities of users.

1. Primary deliverables (Linked Open Data for production)
   1. integrate clinical trials cataloging data from sites like ClinicalTrials.gov into Wikidata
   2. translate a limited vocabulary from ClinicalTrials.gov to Hindi, Bengali, and Swahili
2. Secondary deliverables (cultural products to promote diversity and good ethics)
   1. document Wikidata as a research interface for clinical trials (English, Hindi, Bengali, and Swahili)
   2. publish a general-interest essay on ethical considerations of making medical research data newly accessible

Wikidata is currently the central hub of the open Semantic Web. While various sources such as libraries, repositories, and ClinicalTrials.gov provide "open data", without integration into a Semantic Web portal like Wikidata, the information is largely inaccessible even to field experts. By mapping data to Wikidata, the information becomes accessible to the general public as well as professionals and researchers.

This project will integrate ClinicalTrials.gov into Wikidata, making it once and for all generally machine readable and free and open for export to any other platform. Furthermore, we will translate the search interface to three languages, Hindi, Bengali, and Swahili, bringing this data to those languages for the first time and as a precedent in global diversity.

If we are successful to the limits of our expectations, then all clinical trials data forever after will be free and open in the Semantic Web. Furthermore, we will set the precedent in this project that linguistic diversity must be central to open data projects of global interest. Finally, we will publish a case study of this project in advocacy of accessible open data.

Vision

Open practices

Monitoring

The short explanation of our output management plan is that we will publish everything into the Wikimedia platform, and deposit copies into our university institutional repository, and additionally publish a research paper in an indexed academic journal. All data from this project will have a Creative Commons Zero (CC0) dedication, and all other media will have a Creative Commons Attribution 4.0 International (CC BY 4.0) license. We intend for every part of this project to be open, FAIR, and accessible to a diverse audience of users.

There are two kinds of research outputs for this proposal: structured data and prose documentation. Our primary venue for publishing structured data is Wikidata, because there it becomes available for production or export in the Semantic Web and keeps metadata labels in multiple languages for its provenance and open licensing. We will additionally publish a copy of our data as its own dataset and media product, including in Zenodo and our university's institutional repository. We will set up a Wikimedia research project page, as is customary for any project in the Wikimedia platform, and either host or link out to all media projects from that page in the established way for this platform. This research page will be a multimedia interface for accessing the data, using the data, and browsing prose documentation.

Prose documentation will include instructions for using this data in Wikidata or exporting it for use in any other context. We will also publish information about the project to encourage broad social discourse in diverse academic fields about public access to clinical trials research. Audiences which we imagine include researchers, non-research professionals, and citizens with interest in clinical research, health care, public policy, corporate finance, research funding, public health, and social disparities. While our most detailed prose explanations will be in English language, as the structured data for this project is multilingual, we will also publish our portal interface in English, Hindi, Bengali, and Swahili.

This project has two salaries - one for a researcher to oversee integration of the content into Wikidata, and another for a data scientist to provide assistance with refining the ClinicalTrials.gov data to the Wikidata model.

Student researchers will conduct evaluation of data quality, characterization of this project's dataset within the Open Semantic Web, and critique the proposal for the ethical considerations which it raises. There are two research projects planned - one for a group of data science students whose analysis will include machine learning and probably include entity disambiguation and matching. In the summer project, the students will collaborate with the Center for Ethics and Justice at the university to document the data's accessibility and utility.

The translations will happen fairly early in the project, as they concern the labels and query terms for accessing the data, and usually not the data itself. This project begins with a substantial structured data corpus of the target languages and will build from that rather than originating any new system.

• Wikidata engagement: 6 weeks FTE
• data science: 6 weeks FTE
• equipment: 0
• graduate student research: 8 weeks FTE, university rates
• undergraduate student research: 8 weeks FTE, university rates
• translation: 4 weeks FTE, market rates through vendors
• other consulting: 4 weeks FTE, market rates through vendors

18 months

This project was funded by Wellcome Trust via the Open Research Fund 2019.

FAIR and open multilingual clinical trials in Wikidata and Wikipedia

School of Data Science, University of Virginia