This project report was written as a formal Deliverable (D1.3) of the DiSSCo Prepare Project (Koureas et al. 2023) and was previously made available to project partners and submitted to the European Commission as a report. While the differences between these versions are minor, the authors consider this the definitive version of the report.

The following text is the formal task description (Task 1.3) from the DiSSCo Prepare project's Description of the Action (workplan):

"Based on the analysis of previous studies, relevant criteria will be identified and developed into a basic model for the prioritisation of digitisation of objects held in NSCs. Criteria to be considered include scientific relevance, user needs, socioeconomic impact, specialisation, technical feasibility and cost".

At the onset of this project, two core studies were available on the topic of digitisation. The most recent work was carried out in the ICEDIG project and reported in the final deliverable “Inventory of criteria for prioritisation of digitisation of natural history collections” (Bakker et al. 2018). This work complemented the study by Krishtalka et al. (2016) on how to accelerate the discovery of biocollections data. The most important points made in these studies have been summarised in Suppl. material 1 and they were the inspiration for our literature investigations. Two literature reviews were carried out, the first in 2021 and the second in 2022. Based on the results, a corpus of previous studies on prioritisation of digitisation was compiled, covering the period from 2018 until June 2022. The list of relevant references found during the 2021 survey was included in a previous report (Suppl. material 2) and those found during the 2022 survey are listed in Suppl. material 3.

For the 2021 survey, works deemed to be relevant were scored (1-3), based on relevance for the investigation with 1 being most relevant. The searches were carried out in Google Scholar with the following search parameters:

1. Search: ”natural history collections” ”prioritisation” since 2017;
2. Search: ”natural history collections” ”digitisation” since 2017;
3. Search: ”digitisation” ”prioritisation” since 2017;
4. Search: ”natural history collections” ”digitisation” ”prioritisation” since 2017.

In comparison to the results presented by Bakker et al. (2018), a total of 12 new publications deemed to be relevant were identified from the four search compilations (April 2021). In the additional analysis carried out in June 2022, a total of 14 new publications deemed to be relevant were identified from the four search compilations (see Table 1).

The 2022 survey was carried out under much broader criteria and resulted in a large number of publications (see Suppl. material 3).

In addition to the literature study, two surveys were carried out amongst DiSSCo partners: one coverng their digitisation strategy if present and one covering the prioritisation criteria they used for digitisation completed or in progress.

In addition to the prior costbook work (Hardisty et al. 2020) and transcription cost work (Walton et al. 2020a) in the ICEDIG project, we asked all partners in the task for detailed and complete information on digitisation costs. Such information was not readily available for most projects, but we present two detailed case studies obtained from NHMD and UniFi. Both projects were externally funded and prioritised because there was internal research relevance (e.g. staff undertaking active research on the collections) and because they were considered relevant and impactful to the external funders.

The most significant results obtained through the literature review were reports carried out by GBIF (2016) and within the DiSSCo-related project ICEDIG*1. These two publications will, therefore, be summarised here (extended summary in Suppl. material 1); the additional relevant publications are listed in Suppl. material 3.

Two surveys were carried amongst DiSSCo partners on their digitisation strategy (if existing), as well as on which prioritisation criteria they employed for digitisation which had already been done or was in progress. The main findings have been summarised here and the complete responses can be found in Suppl. material 2 and Suppl. material 4.

Cost is an important consideration in any digitisation project, it often constitutes a criterion overruling other considerations, either because projects are not considered to be affordable (they cannot be achieved within available resources) or, perhaps, because the value for money of pursuing them is not considered sufficient. We found that most of the published cost analyses of digitisation, including the in-depth analysis made in the context of the ICEDIG project (Hardisty et al. 2020) did not comprehensively consider all the costs involved in pre-digitisation, digitisation sensu strictu and post-digitisation. In the two examples summarised below, we have tried to include all stages in the process, from the moment a sample has left the cabinet until it has been safely returned. Perhaps the most important function of the examples is to serve as a checklist of cost items to keep in mind. See also the list of questions to be considered in the Conclusion and Recommendation section.

The results from both the essay-based and the multiple-choice questionnaire, like the results from the literature studies, highlighted the extreme complexity of prioritisation. Fulfilling the ambition of DiSSCo, to digitise millions of specimens in all possible shapes, sizes, origins, ages, state and value, is indeed a daunting task. The very high number of prioritisation criteria that have been suggested may appear as a barrier to progress for many institutions or may need to be balanced at an organisational level for example, to meet strategic or funding opportunities, while also carrying out projects to develop new digitisation workflows or to meet the needs of particular users. An organisation planning a digitisation project needs to consider whether, for example, scientific relevance should be a guiding principle (and define what this means in their specific case) and/or what the funding opportunities are and/or what data quality can be obtained with the resources at hand and/or what the societal interest in the digital specimens to be created is.

With the aim to facilitate decisions about prioritisation of digitisation to be taken by DiSSCo or by individual institutions, we here offer a classification of the multitude of possible criteria into four main categories. Based on our literature study and the results of our surveys, we propose the following four categories:

• Relevance;
• Data quality;
• Cost;
• Feasibility.

All criteria that have been suggested previously fall into one (or more) of the four groups which are, thus, not new criteria, but are meant as an aid to reduce the multi-dimensionality of the “criterion space” during the first steps in the prioritisation process.

The categories of criteria are not completely mutually exclusive. For example, “Cost” may be seen as a component of “Feasibility” an indeed, cost considerations often overrule other criteria. In spite of the somewhat simplistic classification of prioritisation criteria presented above, prioritisation remains a very complex task. It is important to bear in mind that considering just one criterion or just one category of criteria in isolation, will not result in a sound prioritisation. All categories need to be considered, as visualised in Fig. 3. It is also worth remembering that prioritisation is not an exact science, nor is prioritisation constant, but may vary over time, for example, as policies or funding opportunities change.

Figure 3.  

Interrelation of the four main categories of criteria. Data quality and cost are represented on the horizontal and vertical axes (axis values are arbitrary). Relevance is represented by the size of the circles and feasibility by the intensity of their colour. Project A and B will both deliver data of high quality and high relevance. Although Project B data will be of slightly lower quality and slightly higher cost, this project may be chosen because of higher feasibility. Project C has little to recommend it, whereas Project D (low data quality, medium relevance and feasibility and low cost) might be prioritised depending on what the data will primarily be used for.

Relevance may be seen as the primary criterion for prioritising digitisation. If the digitised specimens to be generated are of low relevance, i.e., will lead to no benefit or have no impact, other types of criteria (data quality, cost, feasibility) become almost irrelevant.

Different kinds of users have different needs: what is seen as most relevant for one may not be most relevant for another. According to the comprehensive ICEDIG study (Bakker et al. 2018), scientific relevance is deemed most important, at least amongst the respondents to the ICEDIG’s survey, but collection relevance is also important, whereas social and economic relevance are less so. However, depending on the nature of the specimens to be digitised, on the funding possibilities etc., none of these categories of relevance can be neglected - and they are likely to overlap, if, for instance, scientific relevance is in a discipline which addresses societal and economic challenges, such as biodiversity loss. Concerning social and societal relevance, see the report by Figueira et al. (2023), as well as the “Discussion and outlook” chapter in Fitzgerald et al. (2021) and von Mering et al. (2021). The GBIF study (Krishtalka et al. 2016) agreed with ICEDIG in finding research most important, but disagreed in finding funding/grant opportunities,and taxonomic priorities second and third. Even “scientific relevance” is a complex concept. See Table 5 for an attempt to visualise the different needs of different scientific disciplines.

There are two further complexities in relation to using scientific relevance as a guide to prioritisation in DiSSCo. Firstly, it is likely that almost all collection objects where sufficient data are present have scientific relevance against one or more of the types of research mentioned above. Deciding which of these purposes are ‘most’ important or relevant is extremely challenging. Secondly, this relies on our current understanding of what is important, relevant and useful - but a key benefit sought through digitisation is to unlock new avenues and paradigms of research, for example joining up collections data to other data sources in ways which have not previously been explored. Again, this makes judgements of scientific relevance, based on today’s evidence inherently flawed, although still worthwhile as one of the criteria to provide information on prioritisation. Irrespective of how carefully relevance criteria are analysed, nothing is immutable. Like prioritisation in general, scientific relevance may change over time as institutions and researchers change their focus.

Much of the existing research prioritisation focuses on scientific research. The low prioritisation of 'social-relevant criteria' or social relevance (Bakker et al. 2018) may seem surprising, but are at least, in part, the result of limitations in the current scale and scope of digitised material and existing patterns of usage. Bakker et al. (2020) describe the change in use of natural history collections from their original taxonomic focus to a much broader, interdisciplinary use, including climate change, human health and food security. Recent work by Popov et al. (2021) and Hardy et al. (2023) reports on the financial benefits of digitising collections and the growing demand for socially-relevant data with cross-domain approaches, such as using computer vision on natural history collections for climate change research (Wilson et al. 2022) and supporting conservation assessments for wild relatives of important agricultural crops (Khoury et al. 2020).

As a thought experiment, consider two digitised collection: one with 100,000 digitised specimens and a second with 1,000,000 digitised specimens. At first glance we might consider the latter more advanced in terms of quantity of digital specimens. However, what is the quality of the digital specimens in the two collections? When planning and assessing digitisation, data quality needs to be taken into consideration although this aspect has not been very much considered in previous studies. See Chapman (2005a) for a thorough treatment of the data quality concept.

There are two main dimensions of data quality:

• How much information is there in each digital specimen (Information level)?
• How reliable is that information?

A third essential aspect of data quality is potential for validation and improvement:

• How can we know how reliable is our data and how can we improve it?

Discussion of data quality is also not independent of the relevance criteria discussed above - the reason data quality is important has to do with whether data are ‘research-ready’ and impactful. There may be areas of data quality, such as high quality geo-referencing, that are relevant to widespread fields of research; but other areas of detail which are critical for particular studies, but less valuable to widespread users. It is also often the case that a few key data fields from a large volume of specimens may be more valuable than deep and detailed data on just a handful of objects - again, it depends on the potential uses and users. Ultimately, however, it is reasonable to say that if data about specimens are clearly poor or lacking (e.g. labels are missing, damaged etc.), those specimens are unlikely to achieve much impact through digitisation. These points are explored further below.

A digitised specimen may be anything from a textual record with minimal information (e.g. species name) to an extended digital specimen represented by full collection information, illustrations in the form of photos and CT scans, morphometric data, DNA sequences, sound recordings, chemical profiles and with links to related data and resources.

In order to quantify the information level of digital specimens, a digitisation standard has been developed. The Minimum Information about a Digital Specimen (MIDS) standard (Hardisty and Haston 2021) comprises three main levels of digitisation plus an initial ‘pre-digitisation’ level. These levels provide a framework for prioritising, planning, costing and monitoring a digitisation programme for collections. Using the MIDS standard, the digitisation level of a collection can be scored and changes can be tracked. The four MIDS levels are shown in Table 6.

The level of information required varies significantly depending on what the data are being used for. Planning and costing a digitisation programme potentially requires a low level of information; some ‘big data’ analyses, including species distributions, require an additional set of data; whilst taxonomic research may require all the data that are available on the specimen. Mass digitisation programmes are commonly taking a staged approach to capturing information, starting at the basic level (MIDS, Level 1) and using a range of options, including outsourcing and crowdsourcing, to transcribe additional data and reach a higher digitisation level. The extended record (MIDS, Level 3) equates to the DiSSCo open Digital Specimen specification (Hardisty and Haston 2021).

An example of a digitised specimen with a very high information level can be considered a digital surrogate. This concept was described by Godfray (2007) for the digitisation of type specimens made available online. Considering that requests for access to type specimens constitute a significant fraction of requests for access to natural history specimens, these digital surrogates may save travel and shipment expenses, as well as time. For example, Akkari et al. (2015) described a new species of millipede and in addition to the physical type specimen, they published interactive CT scans of the same specimen (Fig. 4). The scans have subsequently been used by Naumann et al. (2019) for a study on millipede feeding mechanisms.

Figure 4.  

A CT scan of the millipede described by Akkari et al. (2015) (© 2015 Akkari et al. used under the CC BY 4.0 license). The image shows the anterior part of the body with mouthparts and copulatory organs highlighted. The scan may be manipulated to show details important for, for example, taxonomy.

However, while digitisation of type specimens to a high level of detail has many benefits, it does not enable 'big data' type analyses, such as species distributions which are critical to understanding environmental change - it is likely that a balance is required in prioritisation between detailed data on some specimens and lower levels of data on many specimens.

Reliability (data quality in the strict sense) was treated in detail by Chapman (2005a). The data that DiSSCo deals with to a high degree includes species-occurrence information, i.e., records of a particular species from a particular place. A typical species-occurrence data-point includes taxonomic/nomenclatural information (which species, subspecies or other taxon), geographical information, collector and collecting date information and often also other descriptive data, such as habitat, host plant etc.

For all these components of a data-point, but especially obvious for spatial data, their accuracy and precision need to be considered. Accuracy and precision are often confused: accuracy refers to the closeness of measured values, observations or estimates to the real or true value, whereas precision includes statistical precision (the closeness with which repeated observations conform to themselves) and numerical precision (the number of significant digits that, for example, decimal latitude/longitude is recorded in) (Chapman 2005a). The difference between accuracy and precision of species-occurrence data is shown in Fig. 5. The accuracy and precision can also be applied to non-spatial data. For example, a collection may have an identification to subspecies level (i.e. have high precision), but be the wrong taxon (i.e. have low accuracy) or be [correctly] identified only to family level (high accuracy, but low precision) (Chapman 2005a).

Figure 5.  

The differences between accuracy and precision in a spatial context. The red spots show the true location, the black spots represent the locations as reported by a collector. Far left - High precision, low accuracy. Middle left - Low precision, low accuracy showing random error. Middle right - Low precision, high accuracy. Far right - High precision and high accuracy. From Chapman (2005a) (© 2005 Chapman et al. used under the CC BY 4.0 license).

Ideally, all data-points would have high accuracy and high precision. However, for some purposes, high precision is not necessary for the data to be “fit for use”. This is illustrated in Fig. 5. The figure refers to spatial data, but “fitness for use” considerations also apply to other types of information. For example, for some purposes, identification to subspecies level is necessary, whereas for others, species level is sufficient. Additionally, for some purposes, year of collection is sufficient, whereas for others, the exact date or, at least, month is required.

Irrespective of how carefully a dataset has been prepared, very few datasets – if any at all – are guaranteed error-free. Therefore, quality assessment and data cleaning are important aspects of digitisation.

For DiSSCo, four types of information are particularly relevant: 1) taxonomic and nomenclatural information, 2) spatial information (georeferencing), 3) collection date and 4) image quality. For fossils, 5) geological age is also essential. Concerning types 1–3, data cleaning was treated in detail by Chapman (2005b), with emphasis on 1) and 2). Just as the digitisation process itself needs prioritisation according to the four main categories of criteria, the data validation and cleaning process needs to be prioritised according to criteria of relevance, cost and feasibility.

Quality control should be done by experts with access to both the physical and digitised collections. When voucher specimens are kept in a collection, the accuracy and precision of the taxonomic/nomenclatural information can be checked by a specialist at any time, but this seldom applies to the accuracy and precision of data on location, date, collector, habitat etc. Hence a great responsibility for accuracy and precision in recording rests on the collectors themselves. An alternative approach is to use a range of online tools, such as the data quality control checks within aggregators, such as the Global Biodiversity Information Facility (GBIF) and SpeciesLink, which include checks on geocoordinates, taxon names and date formats. GBIF also provides a list of tools which include some that support assessing and improving biodiveristy data quality (https://www.gbif.org/resource/search?contentType=tool)*10. Bionomia is an online resource which has automated the process of parsing and cleaning names of collectors and determiners and finding associated specimens, using integrations with GBIF, Wikidata, ORCID and Zenodo. This enables the discovery of errors or inconsistencies in specimen data relating to collectors and determiners (https://bionomia.net/) (Shorthouse 2020).

Manual data cleaning, for example, by taxonomic specialists or curators, will continue to be important. For example, the identification of collectors’ itineraries allows for checking for possible errors if, for example, the date of collection does not fit the particular pattern of that collector (Chapman 2005b).

In the framework of the SYNTHESYS+ project, Walton et al. (2020b) made a “landscape analysis” for the Specimen Data Refinery that will become one of DiSSCo’s e-services. Chapter 3 of Dillen et al. (2021) deals with the semantic enhancement of digital specimens, with emphasis on taxonomic names, geographical features of the specimen and names of persons (collectors, identifiers etc.) associated with the specimen.

Finally, as always, a balanced view is recommendable. It is better to release imperfect data than to hold data back in the pursuit of (impossible?) perfection. Releasing (imperfect) digital data can help to improve data quality, for example, by opening it up to comment from international experts remotely.

Cost considerations, including funding opportunities and the affordability of projects within available resources, will have a big impact as to what is prioritised in a digitisation project. The cost of digitisation has been the subject of many analyses – recent examples are Tegelberg et al. (2017), Hardisty et al. (2020), Medina et al. (2020), Walton et al. 2020a and also the costbook of DiSSCo (Landel et al. 2023. A general lesson from these analyses is that it is impossible to give a simple figure for “What does it cost to digitise a specimen”? The desired data quality, the level of infrastructure already available, as well as salary levels for different categories of people in different countries, all play a role in cost considerations.

Hardisty et al. (2020) analysed the different types of costs, based on information from seven natural history collection institutes in Europe and described the different types of costs to be considered:

• Capital costs, such as the purchase of equipment, buildings;
• Fixed operating costs (i.e. operating costs which are not dependent on the level of usage of the facility), such as maintenance contracts, some salaries, building/floor rental, heating and lighting etc.;
• Variable operating costs (i.e. operating costs which depend on the level of activity), such as per hour costs of staff carrying out digitisation tasks, barcode labels and other consumable materials.

Another useful classification described by Hardisty et al. (2020) divides costs into:

• Establishment costs, meaning the upfront costs of building and equipping a digitisation facility;
• Costs of digitising specimens;
• Costs of preserving the digitised data and making it findable, accessible, interoperable and re-usable (i.e. ‘FAIR’).

In particular, the costs of preserving digitised data are often neglected or underestimated, although they may constitute a very significant part of digitisation costs. See, for example, the case studies of costs in the present report. While cost, including funding opportunities, is likely to be critical to any decision to undertake digitisation, focusing on this cost alone is problematic if DiSSCo only prioritises specimens which are cheapest to digitise. Cost needs to be taken into account alongside the other criteria and is perhaps better expressed and understood as ‘value for money’ - the most advantageous combination of cost and quality (or likely impact) or, in other words, whether it is cost-effective to digitise certain things, because there is a feasible workflow; scientific or other relevance that will make the data impactful; sufficient data available; and funding to meet the expected costs. Cost data will be added to some of the workflows in DiSSCo’s digitisation guides website (https://dissco.github.io/) and to the “digit-key” (https://digit.naturalheritage.be/digit-key) being developed by the Royal Belgian Institute of Natural Sciences.

The feasibility of a digitisation project is, of course, dependent on available funds. In other words, cost might be seen as one aspect of feasibility. However, cost considerations aside, there are other factors that determine a project’s feasibility: Is the collection ready to be digitised? Are skilled staff available? Is the IT and other technical infrastructure geared to the task? Has a digitisation workflow been tested and established at a suitable scale?

De Smedt et al. (2022) provide a useful checklist for “pre-digitisation curation” as a contribution to the DiSSCo Digitisation Guides website (https://dissco.github.io/). “Skilled staff” not only refers to the people who do the digitisation. These people should, of course, know how to handle the sometimes fragile specimens; ideally, they would also possess some knowledge of the organisms they are digitising and of the collection in which the specimens reside. In addition to the “hands-on” digitisation staff, it is important that people with extensive knowledge of the organisms to be digitised are available, in order to ensure a high quality of the digitised data. For historical collections, knowledge on the relevant collections, collectors, expeditions etc. is also necessary.

“IT and other technical infrastructure” includes such things as cameras/scanners, conveyor belts etc., but also computing power, appropriate software, storage space and back-up options.

The human and other resources necessary for a successful project vary according to the type of specimen and the project scale. It has become known that digitisation (including at mass scale) of herbarium sheets is relatively easy. For collections of dried insects (which in terms of sheer specimen numbers constitute a very large, if not the largest part of DiSSCo’s collections), methods are being developed for efficient mass digitisation of the specimens and the associated labels (Tegelberg et al. 2017, Price et al. 2018, Wu et al. 2019). Additionally, an automated mass digitisation workflow for microscope slides has been prepared (Allan et al. 2019). Wet-preserved specimens, such as invertebrates stored in jars with alcohol or in glass tubes which are, in turn, stored in jars, pose a huge challenge in terms of human and other resources, but see Dupont et al. (2020).

The human and other resources necessary for a successful project also vary according to the desired level of data quality, including information level (e.g. MIDS), accuracy and precision.

Many, especially smaller, institutions will have difficulties mustering the necessary resources to make a digitisation project feasible. Collaboration may ameliorate this situation. DiSSCo provides a unique opportunity, not only for sharing and learning from best practice workflows which can improve feasibility, but also for direct collaboration on digitisation. The efficiency and potential impact of the digitisation of natural history collections will be immensely higher if DiSSCo-wide agreements can be made. At the DiSSCo level, it may also be possible to apply for European funds to carry out large-scale digitisation projects. DiSSCo-wide digitisation targets could be of the following types (hypothetical examples):

• X% of all herbarium sheets in DiSSCo collections databased and imaged before 20XX;
• All primary types of insects in DiSSCo collections databased to MIDS level X before 20XX;
• All African birds in DiSSCo collections databased and imaged before 20XX.

Despite the complicated nature of the matter, the “academic” presentation of various types of criteria for prioritisation is relatively straightforward. In contrast, their practical implementation is anything but straightforward. All analyses show that there is no such thing as one primary criterion taking precedence over others. Bakker et al. (2018) outlined three methods to implement prioritisation criteria for digitisation:

1. A decision tree (not a tree, but an electronic multi-entry key), focusing on practical (feasibility) and funding (cost) criteria;
2. A scoring method;
3. A panel review.

Concerning the decision tree, Bakker et al. (2018) referred to an “Appendix 6” which, however, is not included in their report. We have had access to an incomplete draft of this appendix in the form of an extensive Excel sheet. It is obvious that constructing an operational decision tree or multi-entry key will be extremely complicated, if possible at all, even if the scope of the tree/key will be limited to feasibility and cost criteria. Therefore, we have focused on the scoring and panel methods. As pointed out by Bakker et al. (2018), these can be used one at a time or in combination and, based on the experience from the SYNTHESYS projects*3, a combination does indeed look like the best solution.

To gather the information required for prioritisation, whether for evaluation or preparation of project proposals or for preparing an internal strategy, the following questions are recommended:

RELEVANCE:

• What is the scientific relevance of the project? (Which types of research will be facilitated by the generated digital data)?
• What is the socio-economic relevance of the project? (Which economic and social benefits will result from the project? Will the project support national/European/global political goals, including the 17 Sustainable Development Goals of the UN)?

COST:

• Is the cost/benefit ratio of the project reasonable (“value for money”)?
• Are all steps in the digitisation process considered?
• Is sufficient funding available (affordability)?
• If not, is there a realistic plan for obtaining sufficient funding?

QUALITY:

• Is the level of information (e.g. MIDS) of the generated digital data sufficient for the purpose of the project?
• Is the accuracy and precision of the generated digital data sufficient for the purpose of the project?
• Is long-term storage and FAIR availability of the digital data ensured?
• Is there a plan for data validation/quality control/data enhancement?

FEASIBILITY:

• Is the necessary IT infrastructure available?
• If not, is there a realistic plan for gaining access to the necessary IT infrastructure?
• Is the necessary technical infrastructure (e.g. cameras, scanners, conveyor belts) available?
• If not, is there a realistic plan for gaining access to the necessary technical infrastructure?
• Is the necessary scientific (e.g. taxonomic experts, curators) and technical (e.g. IT) staff available?
• If not, is there a realistic plan for making such staff available?
• Is there scope for joining forces with other projects?

Finally, whereas prioritisation of digitisation is the subject of the present report, it is important to remember that the digital specimens that have been and will be created, still need links to the physical specimens since physical specimens always will be the ultimate (potential) validators (or 'vouchers') for digital data. Irrespective of the “digital revolution” in which DiSSCo takes part, physical collections, therefore, will need continued funding, including funding for skilled curators. This priority for digitisation of natural history collections is as high as any other.