Research Ideas and Outcomes : Workshop Report
Print
Workshop Report
Defining principles for mobile apps and platforms development in citizen science
expand article infoUlrike Sturm, Sven Schade§, Luigi Ceccaroni|, Margaret Gold, Christopher C. M. Kyba#, Bernat Claramunt¤, Muki Haklay«, Dick Kasperowski», Alexandra Albert˄, Jaume Piera˅, Jonathan Brier¦, Christopher Kullenberg», Soledad Lunaˀ
‡ Museum fuer Naturkunde, Leibniz Institute for Research on Evolution and Biodiversity, Berlin, Germany
§ European Commission, Ispra, Italy
| 1000001 Labs, Barcelona, Spain
¶ Natural History Museum London, London, United Kingdom
# GFZ German Research Centre for Geosciences, Potsdam, Germany
¤ Centre for Research on Ecology and Forestry Applications (CREAF), Barcelona, Spain
« University College London, London, United Kingdom
» University of Gothenburg, Gothenburg, Sweden
˄ University of Manchester, Manchester, United Kingdom
˅ Institute of Marine Sciences (ICM-CSIC), Barcelona, Spain
¦ University of Maryland, College Park, United States of America
ˀ European Citizen Science Association (ECSA), Berlin, Germany
Open Access

Abstract

Apps for mobile devices and web-based platforms are increasingly used in citizen science projects. While extensive research has been done in multiple areas of studies, from Human-Computer Interaction to public engagement in science, we are not aware of a collection of recommendations specific for citizen science that provides support and advice for planning, design and data management of mobile apps and platforms that will assist learning from best practice and successful implementations. In two workshops, citizen science practitioners with experience in mobile application and web-platform development and implementation came together to analyse, discuss and define recommendations for the initiators of technology based citizen science projects. Many of the recommendations produced during the two workshops are applicable to citizen science project that do not use mobile devices to collect data. Therefore, we propose to closely connect the results presented here with ECSA’s Ten Principles of Citizen Science.

Keywords

Citizen science, digital technologies, design, reuse, interoperability, sustainability

Contributors

Workshop participants, working group facilitators and organizers and their corresponding organizations are listed in Table 1.

List of participants of the workshops “Defining Principles for mobile apps and platforms development in citizen science”. The first workshop took place in December 2016 in Berlin, and the second in April 2017 in Gothenburg.

Name

Organization

Gaia Agnello

ECSA

Alexandra Albert

University of Manchester

Lina Andersson

Royal Institute of Technology, Sweden

Janice Ansine

The Open University

Leon Barthel

Leibniz Institute for Zoo and Wildlife Research (IZW)

Flavia Bartoccioni

University of Rome Tor Vergata

Rainer Borcherding

Project coordinator CS-Platform BeachExplorer.org

Jonathan Brier

University of Maryland

Daniela Campobello

University of Palermo

Sofia Capellan

BirdLife International

Luigi Ceccaroni

1000001 Labs

Bernat Claramunt López

CREAF

John Cornell

BirdLife International

Olha Danylo

International Institute for Applied Systems Analysis

Daniel Edler

University of Gothenburg

Carolin Ehmig

Universität Potsdam

Candan Eylül Kilsedar

Politecnico di Milano

Claudia Göbel

ECSA

Margaret Gold

Natural History Museum London

Volker Grescho

UFZ/iDiv

Gregor Hagedorn

Museum für Naturkunde Berlin

Niclas Hagen

University of Gothenburg

Muki Haklay

UCL

Giannis Haralabopoulos

University of Southampton

Jana Hoffmann

Museum für Naturkunde Berlin

Philipp Hummer

SPOTTERON

Clemens Jacobs

GIScience group, Heidelberg University

Kostas Karatzas

Aristotle University of Thessaloniki, Greece

Dick Kasperowski

University of Gothenburg

Itzhak Khait

Tel Aviv University’s Steinhardt Museum of Natural History & Cnature apps Inc

Guilherme Kodja Tebecherani

IPM - Iniciativa Pro Mar, Brazil

Renzo Kottmann

MPI for marine Microbiology

Lars Kristensen

University of Aarhus

Christopher Kullenberg

University of Gothenburg

Christopher Kyba

GFZ German Research Centre for Geosciences

Ola Langvall

Swedish University of Agricultural Sciences/Swedish National Phenology Network

Vanessa Lemos Campos

independent

Cecilia Lindhé

University of Gothenburg

Soledad Luna

ECSA

Sean Lynch

University College Cork

Agnes Mair

National History Museum Vienna

Åsa Mäkitalo

University of Gothenburg

André Mascarenhas

ECSA

Adam McMaster

Zooniverse, University of Oxford

Gerald Pape

senseBox

Patricia Paulsson

BioNote, Chalmers University of Technology

Chris Phethean

University of Southampton

Jaume Piera

Institute of Marine Sciences (ICM-CSIC)

Marisa Ponti

University of Gothenburg

Antonella Radicchi

Technical University Berlin

Robert Richter

Museum für Naturkunde Berlin

Veljo Runnel

University of Tartu Natural History Museum

Dilek Sahin

TEMA Foundation

Franziska Sattler

Museum für Naturkunde Berlin

Sven Schade

European Commission, Joint Research Centre (JRC)

Katrin Schneider

Unabhängiges Institut für Umweltfragen

Julia Schnetzer

MPI für marine Mikrobiologie Bremen

Wolfgang Schröder

netzweber GmbH

Andrea Sforzi

Museo Naturale Maremma

Jari Silander

Ymparisto, Finland

Michael Strohbach

TU Braunschweig

Ulrike Sturm

Museum für Naturkunde Berlin

Tobias Sturn

IIASA

Alda Terracciano

UCL / University of Gothenburg

Erik Thorelli

BioNote, University of Gothenburg

Jakub Trojan

Tomas Bata University in Zlin, Czech Republic

Gunilla Ullman

Ella Vogel

National Biodiversity Network (UK), and representing GBIF

Iris Wessolowski

3x3 :: Kommunikation, Analysen, Konzepte & Content fürs Web & Sciencekompass.de

Florian Wetzel

Museum für Naturkunde Berlin

Mirjana Zabic

University of Banja Luka, Bosnia and Herzegovina

David Ziegler

Museum für Naturkunde Berlin, German Citizen Science Platform

Introduction

In the field of citizen science, digital technologies are increasingly used (Bonney et al. 2014, Haklay 2012, Jennett et al. 2016, Newman et al. 2012, Raddick et al. 2013, Silvertown 2009, Wiggins and Crowston 2011, Higgins et al. 2016). Therefore, there is a need for support and advice for planning, design and data management of mobile apps and platforms in the citizen science community. Adriaens et al. 2015, for example, stress this need in relation to the monitoring of invasive alien species. Following these requests it is therefore time to join forces, and structure the growing number of new developments in order to prevent needless duplication of existing functionality, repetition of failure, and learn lessons from successful implementation and best practice. To jumpstart such collaboration, Ulrike Sturm (MfN) and Soledad Luna (ECSA) decided to organize two workshops with the aim of defining principles for the development of mobile apps and platforms in citizen science. Additionally, aiming to connect people working with apps and platforms, strengthen their networks, and foster synergies. As the European Citizen Science Association (ECSA) has a growing network of organizations and individuals from across Europe and beyond, who specialize in the field of citizen science across different cultures, national boundaries, and regulations, the workshops provided an opportunity to share the growing knowledge in this area.

This report summarizes the outcomes of the first and the second workshops. Detailed results, discussions and original material can be found in each workshop report (Sturm et al. 2017a, Sturm et al. 2017b) and in two online platforms that were used to allow for a remote and broader participation (Trello and Google folder).

Following a series of context-setting presentations, working group discussions were organized in six major areas: 1) Interoperability and data standardization; 2) User Interface & Experience Design; 3) Outreach, learning, education, and other rewards of participation; 4) Re-use; 5) Sharing of learning; and 6) Tracking participants’ contribution across different projects. In order to establish a common understanding, the participants agreed on a definition for the terms app, platform, portal and citizen science practitioner (Fig. 1).

Figure 1.  

Definitions in order to establish a common understanding for app, webpage, platform and portal and considering that the boundaries are not always clear and in some cases tools might act as a portal and a platform at the same time.

Key outcomes

The first and the second workshop were co-organized by Stadtnatur entdecken, a project of the Museum für Naturkunde Berlin (MfN) and the European Citizen Science Association (ECSA). The first workshop took place in December 2016, in Berlin, Germany. It had 46 participants and was supported by Bridging in Biodiversity Science (BIBS), Berlin-Brandenburg Institute of Advanced Biodiversity Research (BBIB) and the German Federal Ministry of the Environment, Nature Conservation, Building and Nuclear Safety. The second workshop took place in April 2017, in Gothenburg, Sweden and was co-organized with the University of Gothenburg. It had 31 participants, and was supported by the Marianne and Marcus Wallenberg Foundation.

In the first workshop, a number of projects were presented in order to share insights, success stories, and challenges. The apps and guidelines of the EU BON project were presented by Veljo Runnel (University of Tartu), Naturblick by Ulrike Sturm (Museum für Naturkunde Berlin), Loss of the Night/My Sky at Night by Christopher Kyba (Deutsches GeoForschungsZentrum Potsdam), NatureWatch by John Cornell (Bird Life International), Platform Natusfera by Bernat Claramunt (Ecological and Forestry Applications Research Centre CREAF) and Luigi Ceccaroni (1000001 Labs), and Fotoquest go by Olha Danylo from the International Institute for Applied Systems Analysis (IIASA). Bernat Claramunt (CREAF) also outlined the contribution of citizen science towards international biodiversity monitoring. Sven Schade from the Joint Research Center (JRC) reflected on the diverse landscape of citizen science apps and platforms and led the discussion to agree on a shared definition for apps, platforms and portals. He also highlighted the importance of integrating citizen’s contributions into the European policy making process. Renzo Kottmann (Max Planck Institute for marine Microbiology) expressed ideas about balancing open scientific data with closed personal data. And finally, Claire Born (Wigwam GmbH) guided the participants through a rapid-prototyping exercise to explore the applicability of the first identified principles.

During the second workshop, several presentations highlighted aspects of the diverse field of citizen science apps and platforms. Marisa Ponti (University of Gothenburg) presented Games in Citizen Science, Sven Schade (JRC) presented lessons learned on re-usability and how to define success, Adam McMaster (University of Oxford) presented the Zooniverse Platform, Jonathan Brier (University of Maryland) presented SciStarter and Ella Vogel (National Biodiversity Network) presented the UK National Biodiversity Network and GBIF. Christopher Kullenberg (University of Gothenburg) reflected on collecting social science data with smartphone apps and school children. Dick Kasperowski (University of Gothenburg) gave insight into the epistemological ideals of participation. Luigi Ceccaroni (1000001 Labs) reported on the ontology of citizen science. Posters on “Impacts of Design” by Jonathan Brier, “Building a Player Base” by Lars Kristensen (University of Aarhus), and the app “Marine Megafauna” by Guilherme KODJA Tebecherani (IPM - Iniciativa Pro Mar) were presented in the poster session.

At both workshops working groups were held in the following areas: 1) Interoperability and data standardization; 2) User Interface & Experience Design; 3) Outreach, learning, education, and other rewards of participation. The discussions of the second workshop built on the results of the first workshop. Additional working groups were organized for the second workshop with a focus on 4) Re-use; 5) Sharing of learning; and 6) Tracking participants’ contribution across different projects. Two online platforms allowed remote contribution to all working groups (Trello and Google folder).

Working Group 1: Principles for interoperability and data standardization

It was the scope of this working group to define principles in relation to openness, accessibility, reusability, data privacy, and back-end system design. The discussion outlined the need for a common structure and scheme for metadata. This includes e.g. metadata about: 1) citizen science projects, 2) tools used in projects, and 3) observations made by participants, including data quality information. It was stressed that the aspect of data privacy of citizen related data requires greater awareness and common standards. Recommendations were compiled for five main topics: the sharing of outcomes, legal and ethical issues, interoperability with projects that do not have a genuine science outcome or do not involve citizens, and feedback, acknowledgement and interoperability of quality assurance procedures.

Working Group 2: Principles for User Interface and Experience Design: Interaction, Mechanisms to support Motivation

This working group had the mandate to define principles in relation to usability, engagement (gamification, feedback) and procedure to improve data quality and assurance. The discussion of both workshops raised the overall question of whether design guidelines for citizen science would differ from industry guidelines. Therefore, people designing apps for citizen science should turn to the wealth of knowledge about effective app design in the technology industry. A general recommendation resulting from the workshop was that it would be useful to create a curated and easy accessible overview of this knowledge for citizen science practitioners.

Working Group 3: Principles for Outreach, Learning, Education, and other Rewards of Participation

It was the scope of this working group to define principles in relation to recruiting and retaining participants, as well as to learning and ethical considerations. Recruitment was defined as getting someone involved and motivated to make the first observation. Retainment was defined as keeping the participants engaged and using the system for the duration of the project. The group stated that one of the core issues for designers and project coordinators is to remember that citizen science projects are socio-technical, and therefore the social aspects require attention from the start as they should influence the design and the development decisions. Learning takes place at every step of the way, from the initial engagement with an app/platform to actually doing the tasks, and beyond. It was concluded that there is a gap in knowledge and research regarding learning and ethics, on which principles can be based.

Working Group 4: How to foster re-use?

This working group was a follow-up activity from the first workshop, in which participants identified this very specific and practical question: What is the necessary information to make a list of apps and platforms that can be re-used? The working group did not address specifically other challenges to foster re-use, such as reputation, recognition and reward systems. The main conclusions are that to be able to identify and reuse an existing app or platform the key metrics need to be known, including licensing, all relevant documentation and the development community that would be available for technical support or even to implement required modifications. Those may be defined as part of the standard metadata (see Working Group 1). Key issues were analysed such as the need for a list of apps and platforms that make it possible to find relevant tools and assess their characteristics. However, such a list needs to be curated and maintained or it will become obsolete with time. Without the dedicated effort of a central owner, an innovative and sustainable way to self-maintain it would be required. A tree structure with ranking aspects could be designed as a guide to apps and platforms according to the specific needs of any given citizen science project. Apps and platforms could be indexed similar to e.g. Scopus in scientific publishing, and ECSA could be acting similar to e.g. European Network of Living Labs (ENoLL ).

Working Group 5: How to share the learning of citizen science practitioners around the designing and building of apps and platforms, managing data and involving participants?

This working group was also a follow-up activity from the first workshop. The group discussed how to share and learn from each other regarding their experiences of developing and using apps and platforms for citizen science. The group recommended to recognize the exchange of knowledge moves in many directions. Also the documentation and sharing of both failures and successes needs to be enhanced. It was proposed to create an online body of knowledge about apps and platforms. This body of knowledge should include overall guidelines and best practices, as well as, lessons learned about opportunities and limitations so that it does not become obsolete over time.

Working Group 6: How to track participants’ contribution across different citizen science projects?

This working group was a follow-up activity to discussions concerning privacy and ethics in tracking participants’ contributions across projects, that took place during the first workshop. The discussion focussed on concerns regarding the ethics of sharing data about participants and the responsibilities that come with that. An ethics review of projects with 3rd party certification was discussed. The argumentation related to privacy focussed more on the possibilities to preserve anonymity and the right to be forgotten.

Even though the recommendations were discussed in working groups with different thematic focuses, the process clearly revealed that all aspects are connected. Therefore we developed joint recommendations for three crucial aspects of app and platform development in citizen science (Table 2).

Recommendations for app and web platform development in citizen science organized across three main aspects: 1) the sharing of outcomes and interoperability, including all recommendations to allow for different ways to share source code, the importance of licensing, data, metadata, documentation and methods, and the use of standards; 2) Communication and design, including specific recommendations for citizen science projects to support and promote learning; and 3) Ethical aspects, including project planning and legal aspects such as data privacy.

1. Sharing of outcomes and interoperability

Platforms, portals and apps should have an API to share as much data as possible, and open API standards should be followed where possible.

There should be transparency about what data are collected and where data is stored.

The licenses of data, source code and other resources should allow for re-use. Pay attention to the differing restrictions of different open licenses.

If sharing data and other resources with other countries, pay attention to national and international differences in licensing.
When writing the code, take into account the different levels of technical ability and keep reusability in mind (i.e. new projects can reuse the app or platform, and it should not be too hard to adapt the code for their purposes), especially coding conventions should be followed to ease take up.

It is preferable to open source the code base, but there are contextual situations where some limited closeness is necessary (e.g. long-term monitoring projects that require aggregation over time, and therefore consistency of the code base).

If sharing code, think about sustainability and discoverability. Examples are Github or the Astrophysics Source Code Library (ASCL) for source code in astrophysics.

Re-use also refers to the design (e.g. icons, and interaction), therefore, design should be based on existing standards if applicable (e.g. ISO/FDIS 9241-11 Ergonomics of human interaction). Design should be innovative but familiar. But be aware existing standards are in some instances not free to use and need additional investment of resources.

Existing fit-for-purpose platforms/project-portals/apps (and the communities using them) should be taken into account for re-use. We recommend that existing tools are built on and developed to the next level.

New developments should be well documented, if possible also in English (including the scope of use, any assumptions made, known limitations, and implications when using a particular solution).

If possible, use modular development of apps and platforms.

Consider multi-lingual translations of project documentation depending on the scale of the project.

If possible, use open standards for data and metadata, and use distribution services that make these accessible.

Examples include ALA - BioCollect, SciStarter, PPSR-CORE - CitSci.org, The US Federal Crowdsourcing and Citizen Science Catalog, Dublin Core, GBIF - IPT, Project Open Data Metadata Schema - POD v1.1, CKAN API, DCAT, Schema.org, OGC, CobWeb, ADIwg, ISO 19115/19110, Inspire) until a recommendation is made available by CSA-ECSA-ACSA (foreseen by October 2017).

Each observation site/data point should have a universally unique identifier (UUID coming from an existing standard).

Implement interoperable quality assurance procedures (especially including validation processes).

2. Communication and Design

Respect your participants by designing in a way that appreciates their time and lowers the barriers for entry. Think about ease of use, user friendliness, accessibility, and context.

Participants need to be able to communicate amongst themselves (e.g. in forums).

The design of apps and platforms for citizen scientists can be centered on a target group, or be developed to meet the needs of both communities in a co-creation process.

Define your target group (the ones you try to reach), and design for their needs to attract and support them.

Be as inclusive as possible. However, have in mind that the ideal target group size is project specific. In some cases, it is appropriate to narrow the target group in order to enhance recruitment and retention, or to maintain a higher quality of data.

Think about the participant’s journey: “take care”/ consider participants throughout the processes of your project, think about their motivation.

Plan and understand the required level of engagement for the project: is it fine to have many on-off participants, or does the nature of the project require deeper long-term engagement?

Check your assumptions about the participants, e.g. do the people you want to reach have e-mail? Internet? Smartphones? Are they comfortable with technology? Do they have Wi-Fi connectivity or data in the field?

Consider the context in which participants are using the app or the website during the design phase. For example, consider the weather (will the participants be using mittens?) and the natural environment (is it wet?).

Take different levels of physical ability into account when designing the interface.

Take other aspects of inclusivity and accessibility into account. In particular design for people with visual difficulties.

Balance designing for citizens with designing for scientists, keeping in mind who the central users of the app or platform will be.

Whenever possible, citizen scientists should be the co-creators of apps, contributing to the structure and design from the very start of development (i.e. not limited to testing).

Design to prevent bias both in the participants demographic composition, as well as in the data collection, coverage, consistency.

Communicate constantly with the participants and react to feedback and ideas.

Design for communication: start by deciding on the right “tool” (app, platform, website, non- digital tools, or a mix).

Communication should be as direct as possible, and both sides should profit. Develop meaningful thanks/giveback within the app itself.

Unstructured learning and communication should be recognized as valuable. Allow unstructured space for informal learning and reflection among citizen scientists.

3. Ethical aspects

If you are planning a long-term project, you need to assess the need for a long-term sustainability strategy (that includes tech support communication, an appropriate resources over the life of the project) and put it into place if needed. Otherwise, do not even start!

Plan the lifetime of the project and the legacy of the project: does it have a natural ending point? (not just the end of funding, but an end to the research goal). Communicate that ending point throughout. If the project can continue beyond the end of funding, plan for longevity and sustainability, commit to on-going support or a hand-over. Data should be stored long-term and must therefore be planned for from the beginning.

Remove your app (from app stores or other online repositories) when it is no longer useful (and if you can’t analyze or store the collected data).

Fully respect privacy of personal data. Take only as much personal data as required according to the objectives of the project, and delete personal data as soon as possible if they are not needed anymore in relation to project objectives.

Provide participants ways to decide about the privacy of their data.

Ensure that secure data transmission and storage are in place.

Particularly with regard to personal data, encryption is recommended, and only a minimum of App Rights Requests should be made on mobile devices.

Consider data privacy and intellectual property rights (IPR) that apply for your country or region, including also those at international level.

Give transparent and easy to access information about the app and project. People should not need to download the app to find out what it does and how it relates to them.

Include details about data protection, ethical use of data, and contact information within the app, and on the website or platform.

Discussion and Conclusions

Many of the recommendations produced during the two workshops are applicable to any citizen science project. Therefore, we propose to connect the results presented here directly to ECSA’s Ten Principles of Citizen Science, and not create a specific principle(s) for the development of mobile apps and platforms in citizen science. ECSA’s Ten Principles of Citizen Science cover in a broad manner many aspects of citizen science projects, including specific guidelines and recommendations that are applicable to the development of mobile apps and web platforms, serving as a principled guide for all citizen science project developers and practitioners.

We emphasise the importance of a participant-centred approach to the design of any citizen science project and the tools that it uses, such as apps and platforms, including sensitivity to cultural values. We therefore recommend that co-creation and co-design approaches to app development be incorporated. We propose that this will lead to wider and deeper engagement with participants, as well as facilitate learning and the sharing of knowledge in an accessible and inclusive manner. Similarly, tools that support two-way and constant communication should be embedded in the project such that participants’ ideas and feedback are gathered throughout the project.

In line with the notion of Open Science (see, for example, Wikipedia), the scope of sharing should be envisioned beyond data and metadata, to also include other outcomes such as source code, relevant documentation, training material and contact information about those who develop a certain project or tool. Appropriate human and machine readable licenses should be put into place.

Interoperability should be supported among projects in a way that they become able to “interact towards mutually beneficial goals, involving the sharing of information and knowledge between these [projects], through the business processes they support, by means of the exchange of data between their ICT systems” (adopted from the European Interoperability Framework). This should not be restricted exclusively to citizen science projects but to any project, even regardless of having a genuine scientific outcome or not. This should help to embrace the outcomes of citizen science projects more widely, for example, by projects targeted at policy making.

We also propose to further specify and extend the principle regarding distinct issues of privacy, security and ethics. Privacy issues can, for example, be addressed by requesting only the essentially required information and careful user management. Security might be ensured by data encryption, App Rights Requests, etc. On the ethical dimension this included, for example, the need to provide transparent and easy access to information about the project, tools used, and data sharing policies.

In general, the workshops pointed out that further discussion and research are needed to propose evidence based recommendations on the topics of ethics in citizen science, learning in citizen science, and app / platform re-use.

Finally, we suggest, to the citizen science community to consider and further discuss the recommendations that resulted of the analysis carried out during these two workshops. And, if wished, to add interpretations of ECSA’s Ten Principles of Citizen Science in the context of apps and platforms, for example, in the form of implementation guidelines. We also recommend to discuss and consider the building, together with a sustainable maintenance method, an index for citizen science apps and platforms similar to e.g. Scorpus in scientific publishing, with ECSA acting similar to e.g. ENoLL, and a cross collaboration of the Australian Citizen Science Association (ACSA), ECSA, and the US-based Citizen Science Association (CSA) for defining criteria for ethics evaluation and certification.

Acknowledgements

We would like to thank all of the workshop participants for their contributions, and their organizations for making that participation possible. We wish to particularly acknowledge the Bridging in Biodiversity Science (BIBS) and Berlin-Brandenburg Institute of Advanced Biodiversity Research (BBIB), the Federal Ministry of the Environment, Nature Conservation, Building and Nuclear Safety, the University of Gothenburg and the Marianne and Marcus Wallenberg Foundation. The publication of this article was funded by the Open Access fund of the Leibniz Association. We also thank Jamie Williams for his review of an earlier version of this article.

Sven Schade wants to underline that the views expressed are purely those of the author and may not in any circumstances be regarded as stating an official position of the European Commission.

References