"WP No.";4;"Lead beneficiary";VUmc;"Start month";3;"End month";30 "WP title";"Community-driven design and piloting of reproducibility tools and practices" "ObjectivesThis WP uses future studies, interviews, focus groups & co-creation methods to engage the communities curated in Task 2.2 to steer & assist the development and/or adaptation of the reproducibility tools & practices in WP5. With a focus on the comparative analysis of reproducibility across different epistemological contexts, including barriers & drivers to reproducibility across full research lifecycle, we identify the gaps of knowledge for reproducibility across contexts, & design case-study pilots for the implementation & assessment of new tools. Main objectives:Use an innovative backcasting, co-design methodology to work with communities to sketch & select optimal scenarios for future reproducibility across contexts, & identify key areas for development to achieve these scenarios.Plan & design interventions that can help important stakeholders see what the costs & benefits are for reproducibility across epistemic contexts for researchers, funders & publishers.Prepare & run pilot scenarios for tools & practices developed in WP5 to showcase, verify, & evaluate them across methodological contexts" "Description of workTask 4.1. Future studies to identify priorities from the stakeholder community to predict future of reproducibility and identify actionable steps (M3-M9; VUmc [lead], KNOW, AU)Profound knowledge on potential reproducibility issues that are faced by researchers from different communities is essential to build a framework, create tools & make potential gains & costs visible. This task will first scope the ideal futures of reproducibility for 5 different stakeholders (soc, life, comp sci researchers, plus funders & publishers) in online scenario workshops. Using provisional results from the scoping work in Task 3.1 & 3.2, future scenarios will be depicted through 5 scenario workshops with backcasting & transcend methods. Based on these goals & ideas of the future, we will then work backwards to identify the key steps needed to reach these scenarios. In a next stage, these ideal scenarios will then be used to work with communities (3 cross-stakeholder focus groups & 14 interviews) to further identify drivers & barriers to realising these futures. Bottom-up coding of transcriptions will be conducted in Nvivo, synthesised & reported back to WP3 to further develop the framework (Deliverable 3.1), & carried forward to inform Task 4.2. Task 4.2. Requirements procurement and design (M8-M30; GESIS [lead], KNOW, ARC, VUmc, AU, UOXF, FLEMING)Building upon the priority action areas identified in T4.1, & leveraging the scoping work (inventory of tools & practices) from Task 3.2 & the epistemic contexts identified in Task 3.1, this task will work together with pilot community members to specify new interventions to increase reproducibility across all phases of the research lifecycle from ideation to assessment for different methodologies and epistemic contexts. For each case, a detailed plan of action will be created & the technical & social requirements mapped for development & implementation in WP5. New contexts for existing tools as well as the further development of tools in existing contexts will be considered. In the context of the pilots, we need to apply tools/practices to particular communities/scenarios. These communities have their own special needs & particularities & we here take them into consideration. While we have indicated priority tools for development based on our initial scoping work for this proposal (see section 1.2.2.2), in the spirit of co-creation we base the exact direction of this development on the input of our communities. We hence remain flexible in considering other tools/practices for adaptation or development. Hence this task does essential work in assessing community needs & steering accordingly the development efforts in WP5. Task 4.3 Pilots preparation activities (M13-M19; ARC [lead], VUmc)Piloting of new tools & practices is of great importance in TIER2 for validating, evaluating, & improving the set of tools & practices to facilitate the reproducibility of scientific results. Consequently, we implement a set of preparation activities to ensure successful planning & implementation. Although the pilot partners have contributed to the co-design of the tools & practices, various factors can influence their involvement in the piloting & evaluation activities having a large impact on the respective case studies. These factors are expected to be stakeholder-related (e.g., familiarity with technologies, software stacks), inherent to the project work (e.g., problematic/non-expected operation of components), or even unforeseen situations. Our goal is to plan ahead & implement all necessary steps to anticipate & mitigate any possible setbacks & problems. The task will also pre-register all methods to be used in pilot implementation, & design an evaluation framework for the assessment of WP5’s tools considering different epistemic contexts & specific stakeholder groups. In collaboration with the stakeholders, we will compile a list of performance indicators that can help us in unifying the assessment of the tools per research type & stakeholder group. During the proposal phase we have identified key elements of the pilots (listed in section 1.2.2.3 above), which will be validated, fine-grained & attuned to stakeholder requirements during the project, per the co-design approach we propose. Task 4.4 Pilot implementation and assessment (M20-M30; VUmc [lead], KNOW, ARC, AU, PENSOFT, GESIS, UOXF, FLEMING)This task involves execution of the pilot actions & the respective evaluation activities. The pilot actions involve use cases that showcase & apply in practice reproducibility-related tools & practices for researchers, publishers, & funders (details on the respective actions in section 1.2.2.3) involving all stakeholder communities (Social sciences, Life sciences, Computer Science researchers, plus funders & publishers). The stakeholders will also work towards the evaluation of the tools & practices according to the workflows designed in T4.3 (details on assessment methodology: section 1.2.2.4) to get insights into what works & what is not working for different epistemic diversities & what is needed to be able to successfully implement them in the research community, for funders & for publishers." "Deliverables:D4.1 Future reproducibility across epistemic contexts: Report on future studies/backcasting outcomes (VUmc, M9)D4.2 Pilot implementation and assessment plans (GESIS, M18)D4.3 Pilot implementation reflection report including assessment of efficacy & recommendations for future developments (VUmc, M30)Milestones:MS4.1: Pre-registration of protocol for future studies Task 4.1 (M4)MS4.2: Pre-registration of methods for pilot implementation/assessment (M18)MS4.3: Update briefing reports on pilot implementation progress via Reproducibility Hub (M26)"