Research Ideas and Outcomes :
Editorial
|
Corresponding author: Daniel Mietchen (daniel.mietchen@virginia.edu)
Received: 12 May 2021 | Published: 12 May 2021
© 2021 Daniel Mietchen, Lyubomir Penev, Teodor Georgiev, Boriana Ovcharova, Iva Kostadinova
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Citation:
Mietchen D, Penev L, Georgiev T, Ovcharova B, Kostadinova I (2021) Open science in practice: 300 published research ideas and outcomes illustrate how RIO Journal facilitates engagement with the research process. Research Ideas and Outcomes 7: e68595. https://doi.org/10.3897/rio.7.e68595
|
Since Research Ideas and Outcomes was launched in late 2015, it has stimulated experimentation around the publication of and engagement with research processes, especially those with a strong open science component. Here, we zoom in on the first 300 RIO articles that have been published and elucidate how they relate to the different stages and variants of the research cycle, how they help address societal challenges and what forms of engagement have evolved around these resources, most of which have a nature and scope that would prevent them from entering the scholarly record via more traditional journals. Building on these observations, we describe some changes we recently introduced in the policies and peer review process at RIO to further facilitate engagement with the research process, including the establishment of an article collections feature that allows us to bring together research ideas and outcomes from within one research cycle or across multiple ones, irrespective of where they have been published.
RIO’s mission is to publish the research process, to facilitate engagement with both the process and its outcomes and to highlight how the research published this way relates to societal challenges (
Since then, the research landscape has been evolving and the importance of a wider sharing of the substantial and diverse bits and pieces underlying the various research processes is receiving broader attention. While there are some steady developments in this direction, much of the observable progress was triggered by disruptive events, such as the Ebola and Zika epidemics and now the ongoing COVID-19 pandemic. This resonates with the observation, in a review of Hurricane Katrina, that "Open data matters most when the stakes are high". At RIO, we agree, but think that open data and open processes matter even when the stakes are not high or not (yet) known to be high.
We also observe some concerning trends, from increased “openwashing” to the continued “consolidation” of the publishing industry, where so-called “transformative agreements”, “read and publish” and other formats of large corporate deals between traditional publishers and well-funded research consortia eat up resources that could have instead been used to actually improve the research landscape. At RIO, we will continue to emphasise innovation for the benefit of the research ecosystem, rather than just a few individual players.
In this editorial, we explore some of the key developments in RIO over the last few years, how they relate to societal challenges and how RIO can continue to stimulate experimentation in this space by launching exciting new features and opportunities for researchers, projects, institutions, funders and readers.
There are various ways in which the content of RIO can be grouped. Here, we will look primarily at two of RIO’s key unique features: distribution by the various article types - which roughly correspond to different stages of the research cycle - as well as at the Sustainable Development Goals, to which RIO articles are mapped routinely. On that basis, we will highlight various ways in which RIO readers engage with RIO content and explore how these publications can serve as a resource for research projects.
As of 31 March 2021, RIO had published 300 articles. Of these, 32 (i.e. 11%) were traditional publication types from the end of a research cycle as published in most scholarly journals, while 132 (44%) were from early stages that rarely get published elsewhere and 136 (45%) from intermediate stages of the research cycle whose coverage in the scholarly record has traditionally been patchy (Table
Distribution of the first 300 RIO articles by research cycle stages: early in the research cycle (132), intermediate outcomes (136) and final outcomes (32). The “Grant proposals” row groups together all article types for grant proposals, including the two generic ones (Grant Proposal and Small Grant Proposal) and six funder-specific ones (for COST, Horizon 2020, NSF, NIH, FWF and DFG). The “Other” row groups all article types with just one example in RIO so far: two early types (Software Management Plan and PostDoc Project Plan) and six intermediate ones (Correspondence, Editorial, PhD Thesis, Monitoring Schema, Research Poster and Short Communication).
Article type |
Count |
Example |
Stage in the research cycle |
Grant Proposals |
76 |
Tracking Invasive Alien Species (TrIAS): Building a data-driven framework to inform policy ( |
early |
Project Reports |
38 |
Support Your Data: A Research Data Management Guide for Researchers ( |
intermediate |
Research Ideas |
33 |
Mental synthesis involves the synchronization of independent neuronal ensembles ( |
early |
Workshop Reports |
33 |
The London Workshop on the Biogeography and Connectivity of the Clarion-Clipperton Zone ( |
intermediate |
Research Articles |
23 |
Neurobiological mechanisms for nonverbal IQ tests: implications for instruction of nonverbal children with autism ( |
final |
Data Management Plans |
16 |
Data Management Plan for a Biotechnology and Biological Sciences Research Council (BBSRC) Tools and Resources Development Fund (TRDF) Grant ( |
early |
Policy Briefs |
13 |
Community engagement: The ‘last mile’ challenge for European research e-infrastructures ( |
intermediate |
Case Studies |
12 |
Case Study: Indigenous Knowledge and Data Sharing ( |
intermediate |
Review Articles |
9 |
Hazards and disasters in the geological and geomorphological record: a key to understanding past and future hazards and disasters ( |
final |
Methods |
7 |
Methods & Proposal for Metadata Guiding Principles for Scholarly Communications ( |
intermediate |
Commentaries |
5 |
Genetic Testing for Type 2 Diabetes in High-Risk Children: the Case for Primordial Prevention ( |
intermediate |
PhD Project Plans |
5 |
Physics of Laser in Contemporary Visual Arts: the research protocol ( |
early |
Single-media Publications |
4 |
EU BON’s contributions towards meeting Aichi Biodiversity Target 19 ( |
intermediate |
Software Descriptions |
4 |
PyLogFinder: A Python Program for Graphical Geophysical Log Selection ( |
intermediate |
Research Presentations |
3 |
Online direct import of specimen records into manuscripts and automatic creation of data papers from biological databases ( |
intermediate |
R Packages |
3 |
Novel pedagogical tool for simultaneous learning of plane geometry and R programming ( |
intermediate |
Conference Abstracts |
2 |
Induced pluripotent stem (iPS) cells and somatic cardiac regeneration — An exploratory bioinformatic analysis ( |
intermediate |
Data Papers |
2 |
Groundwater quality dataset of Semarang area, Indonesia ( |
intermediate |
Forum Papers |
2 |
Copyright and the Use of Images as Biodiversity Data ( |
intermediate |
Guidelines |
2 |
Foundational Practices of Research Data Management ( |
intermediate |
Other |
8 |
Benefits and costs of aphid phenological bet-hedging strategies ( |
2 early, 6 intermediate |
Research ideas can be conceived, developed and published without a specific funder in mind, but the implementation of such ideas typically requires some level of funding. Conversely, research funders have a mission to support research in various ways. While funders could, in principle, peruse public collections of research ideas to identify researchers, research infrastructure or research projects to fund, the predominant way of allocating research funds is for a funder to review a set of grant proposals submitted to them and to select some of them for funding.
Accordingly, early stages of the research cycle are particularly sensitive to priorities and policies of research funders. With that in mind, Suppl. material
The TableTable
A good number of examples in the Table are about global development funded out of Europe and North America, including through Canada’s International Development Research Center (IDRC, for example,
Besides opening up the research process, RIO emphasises the connection between research and societal challenges, in particular, by mapping its articles to the United Nations’ Sustainable Development Goals (SDGs, see
While papers published in RIO have addressed each of the 17 SDGs, some goals remain quite under-represented: five of the goals have 50 or more RIO articles to them, six of the goals have 10 or less (Table
Overview of the Sustainable Development Goals, with examples of RIO articles that have been mapped to them and the type of these articles.
No. of articles |
SDG |
Example article |
Article type |
111 |
SDG9: Industry, innovation & infrastructure |
Gentoo Linux for Neuroscience - a replicable, flexible, scalable, rolling-release environment that provides direct access to development software ( |
Software Management Plan |
92 |
SDG15: Life on land |
ConservePlants: An integrated approach to conservation of threatened plants for the 21st Century ( |
Grant Proposal |
91 |
SDG3: Good health & well-being |
Data Management Plan: Opening access to economic data to prevent tobacco related diseases in Africa ( |
Data Management Plan |
59 |
SDG14: Life below water |
World Register of marine Cave Species (WoRCS): a new Thematic Species Database for marine and anchialine cave biodiversity ( |
Project Report |
51 |
SDG4: Quality education |
Building and hacking open source hardware ( |
Workshop Report |
44 |
SDG17: Partnerships for the goals |
Widening the circle of care: An arts-based, participatory dialogue with stakeholders on cancer care for First Nations, Inuit, and Métis peoples in Ontario, Canada ( |
Small Grant Proposal |
28 |
SDG10: Reduced inequalities |
Eliminating disparities and implicit bias in health care delivery by utilizing a hub-and-spoke model ( |
Grant Proposal |
25 |
SDG13: Climate action |
The shadow of the future and the shadow of the past : Studying the impact of climate change on human behaviour ( |
Grant Proposal |
22 |
SDG11: Sustainable cities and communities |
A Political Ecology of Value: A Cohort-Based Ethnography of the Environmental Turn in Nicaraguan Urban Social Policy ( |
Data Management Plan |
15 |
SDG16: Peace and justice strong institutions |
Exploring the opportunities and challenges of implementing open research strategies within development institutions ( |
Grant Proposal |
12 |
SDG6: Clean water & sanitation |
Monitoring and risk assessment for groundwater sources in rural communities of Romania (GROUNDWATERISK) ( |
Grant Proposal |
10 |
SDG8: Decent work & economic growth |
Case Study: Strengthening the Economic Committee of the National Assembly in Vietnam ( |
Case Study |
10 |
SDG12: Responsible consumption & production |
The London Workshop on the Biogeography and Connectivity of the Clarion-Clipperton Zone ( |
Workshop Report |
4 |
SDG2: Zero hunger |
Supercritical carbon dioxide pasteurization to reduce the activity of muscle protease and its impact on physicochemical properties of Nile tilapia ( |
Research Article |
4 |
SDG5: Gender equality |
Rotatory role-playing and role-models to enhance the research integrity culture ( |
Grant Proposal |
2 |
SDG7: Affordable & clean energy |
Challenges in Swedish hydropower – politics, economics and rights ( |
Grant Proposal |
1 |
SDG1: No poverty |
Social processes in post-crisis municipal solid waste management innovations: A proposal for research and knowledge exchange in South Asia ( |
Grant Proposal |
We are experimenting with a more granular mapping between RIO articles and the SDGs and have thus introduced the possibility for articles published on SDG14 (Life below water) to also indicate which of the ten Targets under SDG14 they help to address. This way, six of these Targets have RIO articles associated with them, of which Target 14.a (Increase scientific knowledge, develop research capacities and transfer marine technology) with 17 articles (e.g.
While RIO uses English as its default language, we are aware that this is a barrier for some to engage with its content. Since all RIO content is openly licensed, anyone is free to translate any part of it into any languages they are interested in and this we encourage. We also support the publication of multilingual content, as long as an English version is available. For instance, Increasing understanding of alien species through citizen science (Alien-CSI) (
Most authors of RIO publications are based at universities and research institutions in Europe and North America, particularly in Germany, the United States and the United Kingdom. However, author affiliations in RIO publications are spread across over 70 countries and well beyond classical research institutions, which is a good basis for enhancing societal impact of research globally.
Such affiliations include a range of organizations that work in the field of sustainable development, for instance the Ministry of Environment, Energy and Climate Change of the Seychelles (Deep reef ecosystems of the Western Indian Ocean: addressing the great unknown (
RIO is open to submissions from all research fields and has published research from behavioural sciences (
This area is core to RIO’s mission and its support of open scholarship. Contributions to it include explorations of what the Scholarly Commons might look like (
Data management is central to contemporary research, open or not and, thus, important to RIO already from the perspective of improving scholarly practices, as discussed in the previous paragraph. Our emphasis on engagement with the research process and the various steps along the research cycle raise the importance of data management higher still, since some aspects of data management are important at every step.
It is thus not surprising that a full-text search for “data management” in RIO articles currently yields 210 results, i.e. 70% of the 300 articles. By comparison, a similar full-text search on PubMed Central yields 45915 hits for “data management” amongst 2671350 articles published since 17 December 2015 (the date when RIO published its first articles) - this is a rate of 1.7%.
This importance of data management to RIO manifests itself in various ways: besides the general importance of data management for all article types, there is a dedicated article type for Data Management Plans and several RIO collections have a strong focus on data management (e.g. Public Data Management Plans created with the DMPTool, Exploring the opportunities and challenges of implementing open research strategies within development institutions or Building the European Biodiversity Observation Network (EU BON) Project Outcomes).
Furthermore, data management plays an increasing role in Grant Proposals on any subject (see, for example, Work Package 1 in
Open science and citizen science have originated largely independently from each other, one emphasizing issues of reproducibility and transparency of research workflows, the other stimulating contributions to research workflows from outside the classical research ecosystem. Over time, they have evolved to interact in various fashions, and RIO is one of the contexts in which they do so.
Examples include Grant Proposals (
Another context in which open science and citizen science meet is provided by Wikimedia projects (
RIO’s founders all have a background in biodiversity research and many of the technical aspects of RIO’s publishing workflows have been prototyped with its sister journal, the Biodiversity Data Journal, which was launched two years earlier (
On that basis, biodiversity research is very visible in RIO all along the research cycle, especially via RIO collections for biodiversity-focused research projects like ICEDIG and SYNTHESYS+ (both about the digitization and interoperability of natural history collections) or EUBON (integration of biodiversity data across ground-based and remote sensing modalities).
Examples from outside these collections include Grant Proposals (
The ongoing COVID-19 pandemic has triggered a notable shift in the wider research landscape towards more open and more rapid sharing, thereby improving the alignment with RIO. Examples of RIO publications on the virus, the disease or the pandemic can be found in a dedicated collection, which includes Research Ideas (
A very basic form of engagement with a journal’s content is to access, browse, explore and read that content. Further forms of engagement may involve bookmarking, sharing, reviewing, annotating, commenting on, building on, reproducing, using or reusing the content.
All RIO articles and all of their components - for example, individual figures, tables or supplementary materials - are available via both the RIO website and Zenodo. Traffic via Zenodo can be substantial, for example, the Zenodo copy of the Software Management Plan “Gentoo Linux for Neuroscience - a replicable, flexible, scalable, rolling-release environment that provides direct access to development software” (
Traffic at the journal level over a given time period depends on a number of criteria, including the amount of its content, the degree to which the content matches interest in the group of potential readers during that timeframe, as well as findability and accessibility.
For RIO, journal-level traffic data has been in the order of just above 100,000 unique page views in 2020, at an average annual growth rate of about 20,000, for which 2021 is on track (cf. Fig.
For each article, RIO records the number of first-time (unique) and repeat visits (as identified via cookies) to the RIO website, as well as total views (i.e. the sum of unique and repeat visits) and makes these data available via an article’s Metrics tab. A list of the ten articles with the highest number of unique views as of 30 March 2021 is given in Table
Articles with the most unique views as of 30 March 2021, along with information about total views, the ratio between total and unique views (τ) and the article type.
Rank |
Unique views |
Total views |
Ratio τ of Total views/ Unique views |
Article |
Article type |
1 |
14684 |
23803 |
1.6 |
The influence of religion on science: the case of the idea of predestination in biospeleology ( |
Research Article |
2 |
10802 |
18250 |
1.7 |
A review of biodiversity-related issues and challenges in megadiverse Indonesia and other Southeast Asian countries ( |
Review Article |
3 |
10423 |
23624 |
2.3 |
Strategies and guidelines for scholarly publishing of biodiversity data ( |
Guidelines |
4 |
9258 |
16128 |
1.7 |
Vertical-Horizontal Regulated Soilless Farming via Advanced Hydroponics for Domestic Food Production in Doha, Qatar ( |
Methods |
5 |
8997 |
17404 |
1.9 |
Support Your Data: A Research Data Management Guide for Researchers ( |
Project Report |
6 |
8565 |
14156 |
1.7 |
The value of statistical tools to detect data fabrication ( |
Small Grant Proposal |
7 |
8182 |
33908 |
4.1 |
DNAqua-Net: Developing new genetic tools for bioassessment and monitoring of aquatic ecosystems in Europe ( |
Grant Proposal |
8 |
8149 |
14726 |
1.8 |
Language evolution to revolution: the leap from rich-vocabulary non-recursive communication system to recursive language 70,000 years ago was associated with acquisition of a novel component of imagination, called Prefrontal Synthesis, enabled by a mutation that slowed down the prefrontal cortex maturation simultaneously in two or more children – the Romulus and Remus hypothesis ( |
Research Article |
9 |
8004 |
16473 |
2.1 |
Enabling Open Science: Wikidata for Research (Wiki4R) ( |
H2020 Grant Proposal |
10 |
7410 |
17985 |
2.4 |
Controlling the taxonomic variable: Taxonomic concept resolution for a southeastern United States herbarium portal ( |
NSF Grant Proposal |
Inspecting the data in Table
Beyond per-article views, RIO also tracks sub-article view stats, which are likewise made accessible via an article’s Metrics tab. Suppl. materials
contain aggregate view stats for figures, subfigures, tables and supplementary materials across the current RIO corpus.
The most viewed figure - Figure 1 of
Many articles in RIO and elsewhere contain plates that combine several original images into one composite figure. RIO provides authors with the possibility to keep the original images individually accessible, which facilitates reuse. As of 31 March 2021, 28 of 300 RIO articles had composite figures that were set up this way.
The top 12 most viewed subfigures are all from just two articles,
The most viewed table - Table 3 of
Supplementary files receive less attention than main-text ones: six main-text figures have more views than the most viewed supplementary file, an Excel spreadsheet from
Materials published in RIO can be saved in various ways. For instance, an article can be included in a RIO collection, the article or any of its components can be downloaded from RIO or Zenodo, the URL of an article or any of its components can be bookmarked, or the metadata of any of these can be included in reference managers. In the following section, we will concentrate on downloads via the RIO website.
All RIO articles are available in HTML, PDF and JATS XML formats and they include figures - mostly in JPEG or PNG formats - as well as tables in CSV format. Some figures are composite figures. Figures and tables are optional and articles can further be complemented by supplementary materials that may come in any format, with common ones being spreadsheet (CSV, TSV, XLS, XLSX, ODS) and word processing formats (DOC, DOCX, TXT, ODT), as well as presentation formats (PPT, PPTX, ODP) and PDF again. Note that RIO encourages the deposition of data and other such materials in suitable repositories, as well as citation of such deposits.
Although RIO is designed for its content to be explored online, PDF downloads of its articles remain popular, clocking in at 10000 for the Review Article A review of biodiversity-related issues and challenges in megadiverse Indonesia and other Southeast Asian countries (
Beyond whole-article traffic, more granular content types also experience significant download activity across the research cycle, for example, Fig. 2 of the Review Article (
It is interesting to note that the most viewed and the most downloaded files typically differ. Similarly, the most viewed table received 375 total views, while dozens of tables have had more downloads than that, which indicates that users prefer to view tables offline (this preference is not news to us, but provided one of the major reasons why all tables in RIO and its sister journals can be downloaded as spreadsheets).
Suppl. material
Just as for views, supplementary files receive less attention than main-text ones in terms of downloads: the top 20 most downloaded main-text figures (Suppl. material
Some figures - for example, Figure 6 (a PERT diagram) of
With more of the research process becoming visible through publications at all stages of the research cycle, these publications become resources in and of themselves. While not all aspects of research (e.g. the development and provision of infrastructure) can be usefully cast in terms of projects, some key elements of project management are relevant across disciplines. Their traffic statistics indicate that sharing them is of particular interest to RIO readers, so we highlight some examples here, thereby complementing the project-related examples given earlier, especially in the section on data management above.
For instance, PERT charts explain the relationships between different components of a project and examples can be found in Fig. 1 of
Gantt charts visualize the timeline of activities within a project and examples are in Fig. 6 of
SWOT analyses highlight strengths and weaknesses of, as well as opportunities for and threats to a project, and examples can be found in Fig. 11 of
Risk assessments zoom in on potential threats, estimate their likelihood, assess their potential effects on the course of the project and outline mitigation measures. Examples can be found in Table 12 of
Another common element of project planning and management is an overview of the project’s governance structure and examples for that include Fig. 4 of
Budgets are a key element of project planning and management too and examples can be found in Supplementary Table 1 of
For a further key element of research projects - data management plans - RIO has a dedicated collection with examples including the management of legacy data (
Review reports of Grant Proposals are even more rarely available than Proposals themselves, but if they are, then that does receive attention, as is the case with the reviews for the Grant Proposal Heteroatom quantum corrals and nanoplasmonics in graphene (HeQuCoG) (
At RIO, one way to initiate discussion of an article is to invite others into the ARPHA drafting environment (
The Research Idea An oral live attenuated vaccine strategy against Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2/2019-nCoV) (
Grant Proposals are typically reviewed by funding agencies, yet the reviewers are generally anonymous and, thus, cannot be easily contacted for permission to publish the reviews along with the proposal. In some cases, though, this can be worked out. For instance, the Grant Proposal Heteroatom quantum corrals and nanoplasmonics in graphene (HeQuCoG) (
The Commentary Open comments on the Task Force SIRS report: Scholarly Infrastructures for Research Software (EOSC Executive Board, EOSCArchitecture) (
The Review Article A review of biodiversity-related issues and challenges in megadiverse Indonesia and other Southeast Asian countries (
Beyond those interactions taking place directly on the RIO website or via the ARPHA Writing Tool, discussion happens, of course, on many other channels, including social media where the community regularly tags us (thank you!) in discussions on open science matters, be they on unconventional publication types or peer review practices.
In
The act of citing a resource from a research publication is meant to indicate the flow of information, so as to allow others to trace it back when trying to build on something reported previously, to question or reframe it or otherwise engage with it. This has given rise to a whole industry of citation-based research evaluation that is hard to ignore in any research context, but much of what RIO publishes does not fit neatly into the current research evaluation landscape.
In this section, we focus instead on the original information flow aspects of citations and consider two kinds of flows - within a research cycle and across fields - and illustrate them with RIO examples.
Flow of information within a research cycle - even if it is slow as in the case of “sleeping beauties'' (
Within scientific projects, the progress of the research process - and even workshops - can also be conveniently tracked through a series of Workshop Reports, as in the case of the European Biodiversity Observation Network (EU BON), whose series of four stakeholder events were timely communicated one by one in
Research published in RIO has naturally been reused and built upon in other scholarly sources. For instance, an article investigating the role of religion in attitudes and responses to the climate crisis in Nigeria (
Interim and early outputs available from RIO have also served as stepping stones in unrelated studies published elsewhere. A paper outlining a framework for a reference database for images of marine taxa (
Besides continuity, RIO is interested in cross-fertilization between different parts of the research ecosystem. A RIO Research Article laying out a hypothesis about the neural basis of imagination (
When aggregating data at the journal level, other perspectives on citations emerge. For instance, Fig.
Partial citation graph around the Darwin Core paper (
Engagement with RIO content can go beyond the five mechanisms outlined by
RIO content keeps finding new uses: for instance, some job ads have begun to link to the proposals that triggered the grants providing the funding for the advertised positions (example based on
When authors add a reference to a RIO manuscript while drafting it through the ARPHA platform, the metadata of that reference is served to them via RefindIt. If it is not in there yet, authors should use the feature for entering the bibliographic information into the manuscript manually.
The Scholarly Publishing and Academic Resources Coalition found a unique way to engage with RIO by honouring it with its June 2016 SPARC Innovator Award (details).
In the HTML version of its articles, RIO provides a number of tabs, including article-level and citation metrics of the kind reported above, as well as ways to search for more publications by any of the authors (example).
Another form of engagement are updates. Authors can import any of their published RIO articles into the ARPHA drafting environment in order to create updates. For example, a Grant Proposal has been updated with the reviewer reports and information about the funding outcome (
Yet other ways to engage with RIO content are provided by collections.
As research progresses through the various steps within research cycles, collections provide a way to bundle the resulting outputs together, either within the same research cycle or across different ones (Table
Overview of RIO collections, ordered by total views, with the last three rows representing totals across all collections, as well as averages per collection and per article in collections (traffic data as of 8 April 2021).
Collection title |
Collection URL |
Collection editor |
Total PDF pages |
Total views |
Unique views |
Number of articles |
Building the European Biodiversity Observation Network (EU BON) Project Outcomes |
Florian Wetzel, Lyubomir Penev |
398 |
149832 |
66592 |
17 |
|
Exploring the opportunities and challenges of implementing open research strategies within development institutions: A project of the International Development Research Center |
Cameron Neylon, Leslie Chan |
168 |
89945 |
42967 |
17 |
|
Brainhack 2016 Project Reports |
Jörg Pfannmöller, Cameron Craddock, Pierre Bellec, Daniel Margulies, Nolan Nichols |
82 |
82344 |
42944 |
15 |
|
Public Data Management Plans created with the DMPTool |
Jennifer McWhorter, Jennifer Pannell, Josh Fisher, Jeri Fey |
73 |
65650 |
32085 |
9 |
|
Open Science |
Leo Lahti |
94 |
41732 |
20734 |
10 |
|
DNAqua-Net |
Florian Leese |
33 |
38522 |
10526 |
2 |
|
ICEDIG Project Outcomes |
Laurence Livermore, Anne Koivunen, Kari Lahti and Leif Schulman |
463 |
26735 |
11544 |
14 |
|
Open Biodiversity Knowledge Management System, PhD Project |
Viktor Senderov |
44 |
22480 |
11056 |
3 |
|
Observations, prevention and impact of COVID-19 |
Victor Padilla-Sanchez |
47 |
15703 |
9084 |
6 |
|
Metadata 2020 Project Outputs |
Laura Paglione, Ginny Hendricks |
59 |
11971 |
6251 |
4 |
|
SYNTHESYS+ Project Outcomes |
Laurence Livermore, Vince Smith, Katherine Dixey |
125 |
11069 |
5369 |
4 |
|
Path2Integrity Project Outcomes |
Julia Prieß-Buchheit, Oliver Claas, Iliyana Demirova, Agnieszka Dwojak-Matras, Lisa Häberlein, Belén López, Katharina Miller, Christiane Stock |
66 |
5945 |
2588 |
3 |
|
Selected papers of the SAVE-SD 2016 workshop on “Semantics, Analytics, Visualisation: Enhancing Scholarly Data” |
Silvio Peroni, Alejandra Gonzalez-Beltran, Francesco Osborne |
12 |
5737 |
2450 |
1 |
|
COST Action SAGA |
Carmen Cuenca-García |
25 |
5267 |
2324 |
1 |
|
OpenCon 2016: Empowering the Next Generation to Advance Open Access, Open Education and Open Data |
Joe McArthur |
6 |
5103 |
2535 |
1 |
|
ReNature: Promoting research excellence in nature-based solutions for innovation, sustainable economic growth and human well-being in Malta |
Anna Sapundzhieva, Mario Balzan |
31 |
4247 |
2009 |
3 |
|
Political Psychology |
Vincent Weidlich, Scott Bottorff, Jimmy Gustafsson, Kristian Åström, Gerhardt Fritzsche |
15 |
371 |
284 |
1 |
At the time of writing, these 17 collections collectively contained a total of 111 articles (i.e. 37% of all RIO articles) that had received about 580000 total page views (67% of all RIO page views), of which about 270000 were unique views (57% of all unique views). Per collection, there was an average of 6.5 articles, with about 34000 total views and 16000 unique views. Per article in a collection, this translates to 5200 total page views and 2400 unique views, as compared to the RIO averages of 2900 total views and 1600 unique views. Inclusion in a collection thus correlates with higher traffic and, while the reasons for this are not entirely clear, we are keeping a watchful eye on collections as we further develop RIO’s sociotechnical framework.
Over the last few years, the idea that the research cycle is worth sharing as a continuum, rather than as a scattering of standalone and supposedly final outcomes, has gained increased traction amongst researchers, research funders, research participants, research reusers and others who are involved in the development and cultivation of the research ecosystem (e.g.
In particular, the pandemics caused by the Ebola, Zika and SARS-CoV-2 viruses have acted as catalysts for changes in scholarly communication practices worldwide that emphasize the early sharing of results, along with a more comprehensive sharing of associated data, code and other materials (e.g.
Key ingredients of these workflows are the various article types - especially those still widely considered unconventional - as well as the societal challenges addressed by the underlying research and the various and continuously evolving forms of community engagement around that.
For instance, websites about research projects usually link prominently to publications that resulted from the project. If such publications come out when the project is winding down or has ended, the potential for engagement by others is limited. If, on the other hand, these publications come out during earlier phases in the project’s life cycle (e.g. grant proposals, data management plans, early reports), this gives current and potential collaborators or students, as well as journalists and the public, detailed insights into the project’s activities, which provide an excellent basis for meaningful engagement and collaboration.
For an example, see this call for proposals by the SYNTHESYS+ project, which links prominently to a report they published in RIO about a previous such call for proposals (
Taking into account our experience with RIO over the last five years and the insights and trends outlined above, we have made a number of changes to the way RIO operates. This section discusses the two main ones, which concern the scope of collections and the organization of peer review and briefly looks into a third - article types.
With the evolving range of uses of RIO materials in mind, permanent article collections in RIO have recently been upgraded such that they can not only show content published in RIO, but also metadata of materials published elsewhere, all in a consistent design that can be configured by the collection editors (cf. Fig.
A collection in RIO - for example for a topic, an event or a project - may include a diverse range of both traditional and unconventional research outputs, as well as links to publications from elsewhere (for details, see What can I publish on the RIO website).
This way, RIO collections can combine elements of traditional journal publishing (where the journal only publishes materials submitted to it) with elements of overlay journals (which pick some or all of their content from materials previously published elsewhere). This arrangement is not only interesting for projects, but also for events, organizations and communities centred around a specific topic or methodology. Apart from flexibility in terms of the source of the materials, RIO collections have also become more flexible in terms of the type of files they can accept: while articles published in RIO are natively available in the minable JATS XML, from which semantically enriched HTML and PDF versions can be generated, the inclusion of files published elsewhere into a RIO collection does not require those files to be available in XML or HTML (it can, in fact, be as easy as entering a DOI, which will then be used to fetch the relevant metadata). Thanks to the integration of the journal with the general-purpose open-access repository Zenodo, all items in a collection are automatically archived and indexed there, which further facilitates dissemination and citation.
In an example of a project collection, the EU-funded ICEDIG (Innovation and Consolidation for Large Scale Digitisation of Natural Heritage), led by several major natural history institutions, including the Natural History Museum of London, Naturalis Biodiversity Center, the French National Museum of Natural History and Helsinki University, brought together Policy Briefs, Project Reports, Research Articles and Review Papers, in order to provide a detailed overview of their own research continuum. As a result, future researchers and various stakeholders can easily piece together the key components within the project, in order to learn from, recreate or even build on the experience of ICEDIG.
Operating with a wide range of publication types and outputs originating from different fields, RIO has made use of several separate peer review paths to accommodate the specificity of frequent-use cases aiming at strengthening the role of the community in both pre-and post-publication peer review. While it only makes sense that contributions, such as Research Articles and Review Articles, are subject to pre-publication review, the situation is different for several non-conventional research outputs, such as Grant Proposals, Workshop or Project Reports, Policy Briefs, PhD Theses or more traditional ones, such as Conference Materials, as these have often already been assessed by a relevant institution, funder, scientific conference or another legitimate organisation before submission to a journal.
We have now streamlined these workflows by more clearly specifying the two main types of manuscripts with regard to the peer-review process (see How it works section of the journal’s website for more detail):
Regardless of whether or not a submission warrants mandatory peer review or not, each manuscript is subject to editorial evaluation, in order to ensure that the content is sound and meets RIO’s standards. In addition to that, all published articles in RIO can be subject to voluntary post-publication peer review. All reviews in RIO have always been signed and published alongside the reviewed article (under the Review tab) and with their own DOI - see
For articles submitted for inclusion in RIO collections, the collection editors decide on the mode of peer review, taking guidance from RIO’s default policies.
What we have changed in the default policies is that RIO editors will not organize pre-publication peer reviews anymore. For manuscript types requiring pre-publication peer review, authors are requested to suggest suitable reviewers and these reviewers will be invited automatically by the RIO editorial management system, rather than at the discretion of RIO editors. Authors of such manuscripts can also choose to have their manuscript published as is, at present, on ARPHA Preprints, subject to editorial screening. At any time of the peer review process, however, the RIO editors will be able to invite additional reviewers independently.
Once the manuscript has received at least two positive reviews pre-publication (and fewer negative than positive ones) or an endorsement from an editor, the manuscript will be accepted for publication, unless editorial evaluation of the manuscript in the context of its reviews finds a mismatch with RIO policies (e.g. in terms of data availability).
For article types where pre-publication peer review is not mandatory, RIO will not offer pre-publication peer review anymore. Instead, unless editorial evaluation indicates otherwise, we will publish the manuscript as is and encourage post-publication reviews and comments. This is in line with broader trends to preprints in multiple disciplines and with “publish, then review” approaches being adopted by other journals as well (e.g.
The article types already published in RIO show a great variety, but there are still more elements of the research process that could be shared more openly and RIO tries to facilitate that. We are thus exploring a range of potential new article types: a Registered Report could lay out the methodology for data yet to be acquired, an Ethics Management Plan article could outline the ethical approval process for a study, a Consent Form article could provide a blank version of a consent form used in a study, a Research Question article could zoom in on a single research question, while an Open Questions article could lay out a set of open questions in the context of a particular subject area and a Hypothesis article could be describing a single hypothesis, a Definition article a single definition, a Nanopublication article just a single factoid, a Call for Proposals article could invite funding proposals for a funding line or session proposals for an event, a Job Ad article could provide details for an open position and so on.
On the other hand, keeping information about these various - and often non-standard - article types in a structured format is not simple, so we are keeping an eye on how this could be streamlined further. This means that we are exploring mechanisms by which more generic article types - for example, for Grant Proposal articles - can be more readily adapted to specific use cases, for example, different funders or funding lines, in a way that is as much aligned with JATS best practices as possible.
The experience of these first 300 articles in RIO has demonstrated the multiple interconnected layers of actual or potential engagement through which these publications help enrich their respective research processes and lay a good foundation for reuse in research, education, sustainable development and beyond. Taken together, these articles cover a lot of ground and their aggregation highlights gaps and opportunities. Some of these have been addressed by new policies, some others need further attention. While this article is focused on the interaction between RIO and the research cycle in its many shades and forms, we are planning follow-ups that will situate RIO's efforts around publishing more of the research processes - and facilitating engagement with it - in the broader context of the evolving research landscape at large.
We would like to thank all authors, reviewers and editors of RIO manuscripts so far for providing the content on which this article is based and for their contribution to making the research landscape more open and collaborative. We would also like to thank the Advisory Board for guidance on strategic matters and Ivo Grigorov and Caterina Bergami for help with the pilot on granular tagging of articles pertaining to SDG14, Life under water.
Selection of RIO publications (as of 8 April 2021) arranged by funder, roughly sorted by whether their geographic scope is primarily international (top) or national (bottom).
Topics covered in RIO as of 8 April 2021.
Most viewed figures in RIO as of 8 April 2021.
Most viewed subfigures in RIO as of 8 April 2021.
Most viewed tables in RIO as of 8 April 2021.
Most downloaded figures in RIO as of 8 April 2021.
Most downloaded subfigures in RIO as of 8 April 2021.
Most downloaded tables in RIO as of 8 April 2021.
Most viewed supplementary materials in RIO as of 8 April 2021.
Most downloaded supplementary materials in RIO as of 8 April 2021.