Research Ideas and Outcomes (or RIO for short) is a new journal designed to communicate the nature of research as a process to engage with. RIO covers all areas of scholarly inquiry and links research ideas to their implementation through multiple steps that can be highlighted and scrutinized individually and transparently. Built on top of a collaborative online platform that handles every step from authoring through review and dissemination, RIO provides researchers with a rich set of publishing services to choose from, at competitive yet sustainable speed and pricing. It maps research activities to the societal challenges they help to address, and it facilitates the reuse of its materials in educational contexts through open licensing.

We are conscious that journals are just one way to organize scholarly publications and that the entire scholarly literature – formally spread across tens of thousands of journals – could be considered as just one journal (Gordon and Poulin 2008) if it weren’t for paywalls, reuse restrictions and a still significant body of research not available online. We also agree that the era of journals as we have known them for decades and centuries – e.g. as slow, expensive, static, limited in space and scope – is coming to an end (Krumholz 2015).

RIO is compatible with the currently prevailing journal concept in that it accepts submissions of research outcomes and offers to review, publish, disseminate and archive them. On that foundation, it significantly extends this journal concept and addresses many of its shortcomings, taking inspiration instead from constructive suggestions that have been made over the years about how to communicate research (cf. Table 1).

Table 1. Download as CSV 

Quotes timeline.

A chronological sequence of selected quotes that resonate with the ideas behind RIO

"I propose the creation […] of a newsletter or journal to be devoted exclusively to the publication of unexecuted research proposals." (Brender 1959)

“Ought not all scientists be concerned about rapid publication and wide distribution of results, and even of experiments under way, so as to avoid waste?” (Bahm 1971)

“funding should be proportional to the past productivity (i.e., the benefits returned to society)” (Roy 1985)

“Electronic documents give reproducible research a new meaning" (Claerbout and Karrenbach 1992)

“if databases and journals become more integrated, the way we do science could change significantly” (Bourne 2005)

“While scientists have gloried in the disruptive effect that the Web is having on publishers and libraries, with many fields strongly pushing open publication models, we are much more resistant to letting it be a disruptive force in the practice of our disciplines.” (Hendler 2007)

"There is little empirical evidence on the effects of grant giving peer review. No studies assessing the impact of peer review on the quality of funded research are presently available. Experimental studies assessing the effects of grant giving peer review on importance, relevance, usefulness, soundness of methods, soundness of ethics, completeness and accuracy of funded research are urgently needed. Practices aimed to control and evaluate the potentially negative effects of peer review should be implemented meanwhile." (Demicheli and Di Pietrantonj 2007)

“What if everyone in the world were in your lab – a ‘hive mind’ of sorts, but composed of countless creative intellects rather than mindless worker ants, and one in which resources, reagents and effort could be shared, along with ideas, in a manner not dictated by institutional and geographical constraints?” (Patil and Siegel 2009)

"I want publishers to publish my workflows" (Bourne 2010)

"an article about [a] computational result is advertising, not scholarship. The actual scholarship is the full software environment, code and data, that produced the result." (John Claerbout, quoted in Donoho 2010)

“collaborative work implies perpetual peer review” (Nowviskie 2011)

“We advocate scholarly communication à la carte—letting diners combine courses as they please so they get the meal that is most satisfying at the best price.” (Priem and Hemminger 2012)

“Publishing the final reports would harm neither successful grant applicants nor peer reviewers.” (Gurwitz et al. 2014)

“Once a good number of proposals were open, lots of other changes towards openness would follow across the entire research system.” (Mietchen 2014)

“Unfortunately, systematic study of peer review is difficult, largely because of the sensitive and confidential nature of the subject matter.” (Siler et al. 2014)

“Scientists need to support the SDGs. We must help to integrate monitoring and evaluation mechanisms into policy-making at all levels and ensure that information about our planet is easily available to all.” (Lu et al. 2015)

"we imagine a future environment for publishing that is considerably more dynamic and delivers information and resulting insights far more effectively" (Do and Mobley 2015)

Research is a process, and a cyclic one at that. With some variation across domains of inquiry, it spans from research ideas to gathering the means for putting them into practice to collecting and scrutinizing new evidence to contextualizing this with what we already know or do, and sparking new ideas.

By conventions and habits inherited from the paper era, most of this process is hidden from public view through multiple layers of obfuscation. In recent years, several of these layers have begun to be peeled off. For instance, the open access movement set out, with visible progress, to make publications actually public and to liberate them from reuse restrictions. Slowly but surely, it is followed by increasing efforts to reveal the data and code that support the results reported in publications, with a strong emphasis on reuse-friendly licenses. Reviews of such final research outputs are also starting to appear in steadily growing numbers.

Early stages of the research lifecycle have remained rather unaffected by these developments: out of the millions of research proposals written so far, very few have been shared publicly (probably very well under 0.1%, but that is hard to quantify with certainty), let alone under open licenses or in a machine readable manner. The situation is similar for a range of other works that researchers produce, e.g. grant reviews, data management plans, project reports or PhD plans.

Given the amount of effort that goes into producing these works, wouldn’t it make sense to share them more widely than we currently do? Wouldn’t a more open research process help address pending problems more efficiently than traditional approaches in which we closely guard information about ongoing research and only let the world know about the results, once we are done?

The few data points we have from researchers who tried to “think a thought at the world and have the world think back” (Patil and Siegel 2009) are encouraging: great responses have been received on public thoughts revolving around anything between mathematical conjectures (Gowers and Nielsen 2009) , the roles of individuals in history, detecting the Higgs boson, synthesizing malaria drugs (Woelfle et al. 2011) or deciphering Maya writings.

Researchers involved in projects of this kind have stated that “this process is to normal research as driving is to pushing a car” (Gowers 2009), that “open science is a research accelerator” (Woelfle et al. 2011).

RIO is designed to facilitate interactions around research ideas and to encourage engagement with these ideas as they are developed further. It does away with the major journal-side delays that authors experience elsewhere: the typesetting and proof stage as well as the usual positioning of the peer review process after submission and before publication (cf. Vale 2015).

Instead, it uses an entirely XML-based workflow, which renders the articles in final layout already during the drafting stage, and authors can configure a combination of pre-submission and post-publication peer review on the understanding that reviews will be public and the names of the reviewers revealed.

This way, journal-side delays in the publishing process are reduced to an absolute minimum on the order of days, mainly to ensure that the manuscript is within scope and meets technical and policy requirements.

The XML-based workflow, components of which have been tested for years in other Pensoft journals, also allows for automated submissions to RIO (as demonstrated by Blagoderov et al. 2010) as well as automated dissemination from it to content aggregators (e.g. Penev et al. 2011).

RIO is open for submissions from all domains of scholarly inquiry, and it will emphasize the role of research in addressing societal challenges. Both of these approaches provide new avenues for inter- and transdisciplinary interactions between researchers, research funders, students, communicators, decision-makers and others.

RIO provides researchers with the possibility to create a permanent public record for every step within their research cycles – including those not traditionally published – and to make these records discoverable both individually and as a group.

We have designed RIO for publishing a wide variety of possible research outputs, ranging from early stages in the research cycle like research ideas, grant proposals, data management plans or registered experimental designs to intermediate stages like data papers, case studies, software descriptions, single-figure publications, book reviews, research presentations or questionnaires to later stages like project reports, workflows, replication studies, Wikipedia articles or policy briefs and traditional research articles, reviews, or opinion pieces. A more comprehensive list of our publication types is available from the RIO website, and the complete list is provided to logged-in users through the menu for starting a new manuscript.

When several RIO publications originate from the same research idea, team, project or cycle, they can be linked through a RIO collection, which will contain pointers to the individual pieces – possibly including those published elsewhere – and help to contextualize them. Similarly, RIO services can be tailored to the needs of research funders, institutions, project coordinators, conference organizers, citizen science projects, and others.

For all publication types in RIO, we provide generic templates that assist the authoring system with producing machine-readable output. These templates can be customized to better fit the needs of individual research communities. For example, we have already implemented grant proposal templates for the National Science Foundation (NSF) and the National Institutes of Health (NIH) in the United States, for the European Commission’s Horizon 2020 Programme and for the German Research Foundation (DFG), amongst others. We welcome suggestions for additional publication types or for customizations of existing ones.

We think that the currently dominant pre-publication peer review system – "a model that simply may have run its course given societal and technological change" (Krumholz 2015) – unnecessarily delays decisions about publishing (in the case of articles; see Vale 2015 for discussion) or funding (in the case of grants, cf. Wessely 1998), while the secrecy around it is creating a whole range of other issues (Cicchetti 1991, Demicheli and Di Pietrantonj 2007, Wessely 1998), including duplicate funding (Reich and Myhrvold 2013).

Recognizing that “collaborative work implies perpetual peer review” (Nowviskie 2011), RIO’s multi-step approach to peer review starts early, already at the drafting stage. This is made possible by the ARPHA platform that handles all the steps from Authoring via Reviewing, Publishing and Hosting to Archiving.

ARPHA has been prototyped with the Biodiversity Data Journal since 2013 (Smith et al. 2013) and allows co-authors to work together on a draft and to invite colleagues, mentors, students, copyeditors and others to review, annotate, comment on or edit that very same draft. The platform assists the drafting process, e.g. by gathering bibliographic metadata on the basis of a publication’s identifier, and by checking that all references, tables and figures are actually cited in the text.

An additional layer of review is provided by pre-submission reviews, for which the authors invite peers from their fields who are not co-authors to assess the suitability of the manuscript for publication in RIO. This way, the authors do not depend on the journal for the timing of the reviews. To limit the potential for biased reviews at this stage, reviewer identities and their reviews are made public and permanently citable just like the manuscript itself. Pre-submission reviews are mandatory for the submission of some publication types (e.g. traditional articles or data papers) and optional for others (e.g. research ideas or software management plans).

Once the pre-submission reviews are in, or opted out from, the submission for a technical evaluation is just a click of a button on the part of the submitting author, which will trigger a manual review by RIO’s Editorial office to check compliance with those technical and policy requirements that are not baked into the automated validation by ARPHA (e.g., handling ethical issues, some unusual formatting, or detecting pseudoscience).

When these checks are completed, it is just another click of a button to submit the technically approved manuscript to the RIO channel within ARPHA. There, the manuscript and its associated pre-submission reviews will be published within days as HTML, PDF and Journal Article Tagging Suite (JATS) XML, and exposed to post-publication peer review by the research community, both on the RIO website and elsewhere, which we strongly encourage. In addition to that, authors can opt to have the journal organize a traditional round of invited reviews as well, albeit after publication.

An important element in increasing the transparency of peer review is the mandatory Author’s Statement, which authors are to fill in during submission to state whether their manuscript has been reviewed in ARPHA or elsewhere and to offer an explanation in case they have opted out from pre-submission review.

Finally, via its publisher Pensoft, RIO is also part of the Coalition on Annotating Scholarship (Perkel 2015), which aims to annotate the scholarly Web in a way that is consistent across publishers and publication formats. By narrowing the gap between publishing and research workflows, RIO could contribute to making annotation a part of the latter and consistent within and across research cycles.

Given that much of research is publicly funded and that public funding is limited, there is a growing interest in assessing the impact that research has on society beyond academia and in having this criterion influence decisions on whether and how public funds are to be spent on specific lines or fields of research (Roy 1985, Bornmann 2012, Reich and Myhrvold 2013).

Despite past criticisms of similar initiatives (e.g. Wright 2002), some researchers have called for support from the scientific community for the United Nations’ Sustainable Development Goals, seeing their role in “help[ing] to integrate monitoring and evaluation mechanisms into policy-making at all levels and ensure that information about our planet is easily available to all.” (Lu et al. 2015)

RIO addresses societal impact in several ways: (i) it is free to read, so that anyone interested can actually access it, (ii) it is openly licensed (CC BY 4.0 by default, with an option for CC0/Public Domain), so as to encourage the dissemination and reuse of its materials in other contexts, (iii) it is available in XML, which facilitates reuse by automated tools and integration with other platforms, (iv) it encourages authors to map their research to societal challenges it helps to address (and allows users to search and browse the journal by societal challenges they are interested in).

While the first three of these publishing practices are on the way to becoming standard in a growing range of disciplines, we are not aware of other journals to engage in the fourth one, but we encourage them to do so.

As another way to achieve societal impact, it has been suggested that researchers engage more in writing overview papers that summarize the state of knowledge in their field in a way that is accessible (in multiple senses of the word) to a broader audience, and that research evaluators should take such activities into account (Bornmann and Marx 2013). With that in mind, RIO offers the possibility to publish such overview papers as Policy Briefs.

When thinking of impact outside academia, another useful strategy is to bring research to places where non-academics might look for information. RIO will thus facilitate the creation of Wikipedia articles (Butler 2008, Logan et al. 2010), both on topics that have just been created through advances of scholarship (i.e. new methods or objects of study; e.g. RNA families, as in Daub et al. 2008) or on topics that have been studied for a while but not yet found decent coverage on the English Wikipedia (as pioneered for computational biology; Wodak et al. 2012).

Finally, RIO’s policies have been written with societal benefits in mind: they default to open sharing of all data and code underlying the research reported here and require public justification for exceptions to the open default. The primary effect of such an open default is an increase in the reproducibility and replicability and thus the reliability of research: the more of research workflows is being shared and the earlier the sharing occurs, the harder it will be for mistakes, systematic errors or fraud to go unnoticed. A welcome side effect of this is an increased educational value of the research and its documentation, and over time, we expect learners and educators, practitioners, journalists, artists, makers and others to engage with the research reported in RIO and with the associated data, code and materials.

A number of roadblocks still stand in the way of transitioning to such a more open system, and we are looking forward to work with the research community to overcome these barriers. For instance, grant proposals that have already been reviewed would be more complete and usable if the reviews could be published along with the manuscript, and under the same open license.

For this to happen, the reviewers would have to consent to the publication of their reviews and to the open licensing, but proposal authors do not normally know who the reviewers were, nor can they contact them to ask for these permissions, so these conversations require mediation on the part of the funding agencies involved, which do not necessarily have mechanisms to facilitate such interactions  at present. If the reviewers agree to their reviews being published, the next question is whether to identify themselves (in which case they can get credit or receive criticism for their review) or not.

To address such issues, we encourage funders and institutions to put in place mechanisms to the effect that authors wishing to make their proposals public do not require additional permissions to publish the reviews under the same license, and that reviewers are routinely asked whether they would like their identity revealed or not in such cases.

Similar concerns arise if grant proposals include figures or other materials for which the grant authors do not hold the copyright and that are not available under a license compatible with the license chosen for the publication of the proposal. If those materials have already been published in some way, the issue can often be circumvented by pointing to those earlier publications instead of including the materials directly.

For adaptations of prior works or for materials not previously published, the case easily becomes more complex. We thus advise authors who consider publishing their proposals to keep these issues in mind early on, in order not to delay that publication when they need it, e.g. when a deadline approaches, for which they plan to submit their proposal along with its DOI and possibly any associated reviews.

RIO is an attempt to bring the concepts of research publishing and open science closer together. It highlights not just the outcomes of research, but also the underlying process, thereby reducing author-side delays in making the individual steps of a research cycle public. Built around a platform that handles the entire manuscript life cycle in XML, RIO also reduces publisher-side delays to a minimum.

Through the combination of both of these approaches, RIO acts as an accelerator for the communication of research. In doing so, it actually moves the concept of a journal – to record thoughts and observations on a routine basis – closer to its etymological roots in the French word jour, which stands for “day”, rather than “months” or “years”, the currently prevailing time frames in which research is communicated.

To encourage quality submissions, RIO provides multiple avenues for peer review, allowing authors to opt in or out. It publishes the reviews and identifies the reviewers, so they can get credit for their efforts and are motivated to be constructive.

Providing authors with choices as to the services they want is another theme in RIO, and reflected in the pricing. Authors do not, however, have the option to put their content under restrictive licenses, since a further major theme is to encourage collaboration, reuse and public engagement all along the research cycle, with a keen eye on addressing societal challenges.

We would like to thank all those who have provided feedback on the ideas behind RIO, as well as suggestions and encouragement for their further development and refinement. Special thanks go to our Advisory Board members and Subject Editors, who were very supportive right from the start.