This project report was written as a formal Milestone (MS44) as part of Task 7.2 of the ICEDIG Project. It was previously made available to project partners and submitted to the European Commision as a report on 31 March 2019. While the differences between these versions are minor the authors consider this the definitive version of the report.

The original data were collected at the beginning of 2019 and do not necessarily reflect the current capacity of any institute included in the report.

Due to the low number of institutions taking part in this task we combined the survey technique of data collection with an interview technique. We sent out a survey to six institutions:

Botanic Garden Mesie (APM)

We refer to each institute using the abbreviated name given in brackets.

In addition, there have been reports and surveys completed previously with some information on technical capacity. Where possible, we used this existing information to make a first attempt to complete relevant parts of the survey. The initial approach taken for this analysis was the extraction of digital components from institutional strategy documents. Information on institutional priorities and digitisation capacity was extremely limited from these sources. The next focus was searching for relevant information in other recent EC project documents such as the SYNTHESYS+ proposal (Smith et al. 2019). However, this information was not provided specifically for this survey, and so was not suitably structured to answer many of our questions. To fill in the remaining gaps we emailed the survey to each institution, and provided the option of having a meeting to either go through the survey together or answer any remaining questions.

Interpretation can be a challenging topic when discussing digitisation and what is meant by fully digitising an object. For the purpose of this report and to create clarity, we decided to use the DiSSCo survey definitions for digitisation.

The survey (Suppl. material 1) looks at three main areas within digitisation that would provide useful information around institutional technical capabilities; ‘Current Status’, ‘Workflow’ and ‘Resources’.

‘Current Status’ has been included as it will give us a quick overview of what stage institutions are at in digitising their collections. We have the current status data from the previous DiSSCo survey, however this will give us the opportunity to update this information if needed. ‘Workflow’ will allow us to see what methods each institution uses to digitise their collections and how they prioritise what to digitise. We can then compare and contrast what works well, what could be improved and any potential barriers. Lastly, ‘Resources’ will reveal what institutions have at their disposal for digitisation and how this relates to workflow. The survey consists of open-ended and tick box questions. All questions have space in the last column for free text to ensure all possible data is captured.

The ‘Current Status’ section of the survey was pre-populated as much as possible with data from the DiSSCo survey. Each institution was asked to update the current status of digitised collections, if new data was available. This section was divided into three sections (along with the percentages for total collections):

Number of specimens catalogued (i.e. records exist on in-house collection management system).

As previously mentioned, we used the DiSSCo definitions for all of the above. This remains a challenging subject, as interpretation of the terms (‘catalogued’, ‘digitised’ and ‘fully digitised’) can vary drastically and this was raised by most institutions.

Current status is summarised in Table 1 below.

In order to understand each institution’s digitisation process, we examined workflows to better understand the different work streams and tasks involved in digitising collections. We wanted to see how each institution streamlines the process in order to facilitate the complexities of digitisation.

Where does your digitisation take place?

All responding institutions currently digitise in-house, although APM also employs external contractors to digitise specimens on-site, noting that this makes it easier to accurately track progress in the project and minimise the risk of damage in transporting specimens. Several institutions (Naturalis, MNHN, NHM) have in the past outsourced digitisation to contractors such as Naturalis for off-site digitisation of herbarium sheets, the former two at scale and the latter as a smaller pilot project.

What collections are you currently able to digitise?

Each institution has a variety of different collections which in turn creates a variety of expertise when it comes to digitisation. Table 2 shows the collections that each institution can currently digitise.

All institutions can digitise herbarium sheets, showing considerable expertise in this area across ICEDIG partner institutions, but also emphasising the relative simplicity of the workflow. Similarly, six out of seven institutions have the facility to digitise microscope slides. MNHN selected all collection types listed in the survey, suggesting capacity to digitise in some form a wide variety of collections. RBGK also included additional collections that they are able to digitise that weren’t explicitly separated in the list: ‘mycology’, ‘seed’, ‘DNA and Tissue Bank’ and ‘in vitro’. LUMOUS added in the ‘other’ option the ability to digitise galls with stacking equipment.

What specimen handling techniques are used?

Two institutions use either automated or assisted positioning of specimens. APM use a template that guides technicians on where to position the specimen before imaging. LUOMUS uses automated conveyor belt techniques as well as manual. RBGK stated that it mainly uses manual handling techniques, however the microscope slide workflows are slightly more automated with the use of a Zeiss Axio Scan. MNHN, NHM and UTARTU all use manual handling techniques.

Do you currently have any high throughput digitisation activities going on?

Five out of seven institutions (APM, Naturalis, RBGK, LUOMUS and NHM) reported to currently be running high throughput digitisation activities. Naturalis are working on digitising bagged butterfly collections, RBGK are currently digitising 220 herbarium specimens per day and the NHM are currently working on high throughput projects for pinned insects, herbarium sheets and microscope slides. MNHN have finished their latest high throughput project, and have completed digitising their herbarium sheets.

What formal processes are used to prioritise digitisation? For example, is there a scoring system in place?

In the past, some institutions surveyed had designed formal prioritisation frameworks for collections digitisation. However, in some cases (NHM, APM) those institutions have not persisted with using them for various reasons. Naturalis have adopted a collection evaluation system that was developed by the Smithsonian which provides structured data to underpin the prioritisation process. A variant of this system (‘Join the Dots’) has been rolled out at the NHM, and is beginning to be incorporated into the prioritisation process.

Institutions which do not currently use a formal scoring process still use various criteria on a less formal basis to help them prioritise which collections to digitise (Table 3). APM are currently working on African and Belgian collections, as their scientists are working on these collections. The Belgian collections were also chosen due to being native, and the African collection due to their colonial past. RBGK align with their science strategy and/or their collections strategy. MNHN prioritise type specimens. In UTARTU it is the curators who make decisions on what to digitise, however they are currently in the process of formalising this procedure. At the NHM projects are assessed on a case by case basis, looking at a set of criteria, with external funding being the most important factor.

Research benefits were a consideration for all seven institutions when prioritising collections to digitise, with funding also being a factor for six out of seven institutions. Public engagement was the least popular criterion with only Naturalis taking this into consideration. This was followed by technical innovation (3 of 7 institutions) and curatorial benefit (4 of 7 institutions). RBGK added an additional criterion: the time required for specimen selection, due to how their collections are arranged. They noted that it is prohibitively slow to select material by collector or even country, so taxonomic grouping or large geographical regions are preferred.

What information resources are available to support digitisation activities?

Five out of seven institutions (APM, RBGK, Naturalis, LUOMUS and NHM) provide written guidelines for handling specimens, while only two out of seven (APM and Naturalis) have guidelines on mounting/rehousing specimens. In the latter case, LUOMUS noted that their guidelines were not written down, while NHM commented that mounting and rehousing is commonly carried out by curators rather than digitisers (and where not, digitisers receive specialist training prior to the project). MNHN do not have written guidelines, but note that for both activities any manipulation is carried out by specialized technicians.

Six out of seven institutions have other relevant guidelines for digitisation processes. APM have guidelines for transcription of label information and for in-house imaging. Naturalis have health and safety guidelines. RBGK have data entry and also imaging guidelines that are adapted for each project. LUOMUS have frequently asked questions, and guides to georeferencing and transcription. The NHM have guidelines on recording information and semi-automated ingestion of data into the collection management system.

Do you have mobile digitisation stations?

Only one institution, RBGK, reported having a fully mobile digitisation station. This includes an adjustable desk, Mac, camera with adjustable column and a one sided open box with LED lighting at the top. This mobile station can be moved close to wherever the collections are being digitised. Some institutions have digitisation equipment that could technically be moved if needed, however it would not be an easy process and so remains static.

How do you track movement of specimens to ensure they are returned to the correct location?

A variety of answers were given for this question. At APM, the external company they hire to digitise collections have their own tracking system that uses QR codes. In the APM herbarium storage facility, the cupboards are divided into pigeon holes (64 per cupboard). When a pile of specimens are taken out of a pigeon hole and placed onto a trolley, a sheet is added to the pile with a QR code. An identical sheet with the same QR code is left in the empty pigeon hole.

Naturalis have coded their storage locations. All digitised specimens have this code with their standard location in the registration system, so they are always returned to the correct location after digitisation.

At RBGK, when collecting individual specimens for digitisation, tags are placed in the cupboard where the specimen is stored in place of the specimen so that it can then be placed back in the correct location. There are also collection Excel spreadsheets that are completed by the digitiser. Here the name, region and folder information are recorded. If entire cupboards are being moved, the boxes of specimens are numbered for order. When there are new digitisers working, more qualified staff check that the specimens have been put back in the correct location.

LUOMUS have a drawer number tracking system. Curators at UTARTU, similar to RBGK, place markers on specimens to show to where it should be returned. MNHN have a tracking system (details on this system was not provided). The NHM were the only institution not to have a tracking system however specimens taken from storage are returned on a regular basis and most collections are indexed based on their taxonomic name or have location labels.

Which protocols for barcoding are used in specimen digitisation?

Six out of seven institutions used barcodes in some way, with UTARTU being the exception. At APM every specimen (even if there is more than one) on each herbarium sheet gets a barcode, and similarly at Naturalis each digitised object is given a barcode.

At RBGK, most specimens are barcoded. Microscope slides are manually barcoded before digitisation, then the software used during the digitisation process automatically reads the barcode and creates filenames containing these barcodes for images produced. Economic botany and spirits are not barcoded but numbered. Not all new herbarium accessions are barcoded however all new fungarium accessions are databased and there is also software currently being developed to print out barcodes on fungarium labels.

LUOMUS use encoded CETAF stable identifiers. MNHN have protocols for barcoding botany, QR codes for entomology and marine invertebrates. NHM have protocols for barcodes including generating and purchasing labels, attaching them to specimens and reading/recording them into the CMS.

Quality Assurance policy/standards in place?

Four out of seven institutions reported that quality assurnace (QA) policy or standards were in place on some level. APM were the only institution that did not report any conditions for QA being implemented. Circumstances that affected the degree of QA included the particular demands of the project (Naturalis) and resources available (RBGK), for both images and transcribed data.

RBGK also added that QA guidelines are defined on a project by project basis, and tailored to the size of each project. Checks by digitisers and peer reviews are carried out with an aim to check about 5% of work completed for a project. However, constraints of project demands and funding can again limit the amount of QA activities.

NHM conducts QA on images at a basic level, mostly looking at file names along with some random checks. For transcribed data, controlled lists are generated where possible along with manual checks of the data.

Image elements included when digitising

Three out of seven institutions include all of the five elements (colour calibration chart, scale bar, labels, barcodes, institute name) when digitising specimens (Table 4). No institution named any additional elements to those listed. RBGK include their institution name on the scale bar so that those two criteria are always paired. UTARTU stated that curators do not always include a colour calibration chart and scale bar, so this is only sometimes included when digitising.

How do you track time and costs of digitisation workflows?

Five out of seven institutions reported using some kind of method to track time and cost of collection digitisation. Naturalis have team leaders for each digitisation team, whose responsibility it is to monitor and report results and capacity. At RBGK, digitisers fill out progress reports and flexi timesheets which record each day’s activities along with timestamps on data entry, where possible. RBGK are currently using Toggl to categorise activities to see how long each section of the workflow takes. At LUOMUS they track time and cost on mass digitisation lines using logbooks. NHM also manually enter time and cost in spreadsheets, and MNHN complete a monthly evaluation of production by unit which is used in their global annual report.

How can digitised collections be accessed by researchers or the general public?

All institutions have made their digitised collections easily accessible to researchers and the general public. The main insitutional or national online collection is listed first, followed by their insitutional contributions to other large aggregator sites:

APM

Institutional Collections: www.botanicalcollections.be

Naturalis

Institutional Collections: http://bioportal.naturalis.nl

RBGK

Institutional Collections: https://www.kew.org/science/collections-and-resources/data-and-digital/collections-catalogues

LUOMUS

Institutional (National) Collections*1 : https://laji.fi/en

MNHN

Institutional Collections : https://science.mnhn.fr/ and https://www.mnhn.fr/fr/collections/ensembles-collections

UTARTU

Institutional Collections: UTARTU contribute to national aggregators listed below with an overview here: https://www.natmuseum.ut.ee/en/biodiversity-informatics

NHM

NHM collections via Data Portal: http://data.nhm.ac.uk/

Do you have any planned or current crowdsourcing projects?

Three out of seven institutions are currently running active crowdsourcing projects, while Naturalis, LUOMUS, UTARTU and NHM do not have any current or planned crowdsourcing projects.

APM run and support the DoeDat project, a multilingual crowdsourcing platform based on the code of DigiVol and the Atlas of Living Australia.

RBGK are involved in DigiVol and always aim to have a number of crowdsourcing projects on herbarium and fungarium transcription each year. They complete around 30,000 specimen records per year through crowdsourcing. Overall on DigiVol RBGK have now completed over 40,000 tasks in 38 expeditions with 502 volunteers.

MNHN have a crowdsourcing programme called Les herbonautes which has been operational since 2013. It was developed for the transcription of herbarium labels and is now also used for paleontology.

Do you support Digitisation on Demand requests?

Six out of seven institutions support Digitisation on Demand requests. At APM, if there is a request for a physical loan and the specimens have not yet been digitised, they are first digitised so that the researcher can select from them for the physical loan. At the RBGK they have a maximum of 10 images per request, and specimens requested are logged through a loan management system. LUOMUS supports Digitisation on Demand but currently has no formal procedures in place. At MNHN Digitisation on Demand requests are processed on a web demand management tool (MNHN collections - Requests) and validation is completed by the collection manager. NHM do not currently support Digitisation on Demand, but are actively working on it.

Do you have one or more specialised digitisation teams and what number of dedicated digitisers do you currently have?

Six out of seven institutions report having a specialist digitisation team. Naturalis have two team leaders who are dedicated digitizers, and NHM has a specialised digitisation team consisting of five personnel. UTARTU do not have a specialised team or dedicated digitisers.

APM have their technicians complete two hours a week (ca. 150 images) of digitisation. They also have 13 dedicated digitisers, including 10 technicians, and three volunteers who each spend roughly three hours imaging per week.

At RBGK they currently have three core staff in the digital collections team, however within this team effectively only one person spends 0.25 of their time imaging. They also have one dedicated person for scanning microscope slides. Team numbers are variable and depend on what projects are going on, funding and what interns and volunteers they have.

LUOMUS have a new team that started this February consisting of personnel from four teams and 13 people. Permanent personnel consist of seven people. Two of these employees are full-time (1.0 FTE) and four employees are part-time (0.5 FTE), with six additional full time employees.

MNHN have specialised digitisation teams consisting of a systematic collection management team, and specialised platforms for 3D and CT scans. Within this team there are 4 personal linked to 3D platforms and CT scans and 80 technicians in the collection management team.

What are staff trained in?

Staff working on digitising collections are a valuable resource. We wanted to briefly look at the main skill set staff working within digitisation have (see Table 5). The skills of staff will obviously be tailored to what collections institutions hold. The area most covered by training was photography with six out of seven institutions having staff trained in this area. This was followed by scanning with five out of seven institutions having staff trained in this area. Only two out of seven institutions had members of staff trained in machine learning. RBGK added another area not listed in the survey which was training on R and SQL queries along with how to test new products and tools introduced. For example, a new platform such as Transkribus.

What equipment is available?

A vital part of assessing technical capacity for digitisation is examining what equipment institutions have access to. Table 6 shows a list of equipment available at each institution.

MNHN and NHM reported the most diverse lists of digitisation equipment, which is likely to be a reflection in part of the scope and heterogeneity of those collections. All institutions have access to a macro photography set up, showing that a basic capacity for 2D imaging is now quite ubiquitous across institutions.

Naturalis also described a setup of three conveyor belts with innovative software that enables rapid digitization while still ensuring high quality. This allowed for more than 4 million herbarium sheets to be scanned over a 10 month period.

What types of software do you currently use in the digitisation workflows?

Within digitisation workflows, a variety of software solutions may be used for to support the various steps such as imaging and transcription (Table 7).

Six out of seven institutions reported using commercial software as part of their workflows, including Aetopia, BGBase, Capture One, EMu, Microsoft Access, Photoshop, Sketchfab and the conveyor belt control system Digitarium (Tegelberg et al. 2017).

Four out of seven institutions used open source software, and five out of seven develop software products in house. LUOMUS have developed their own collections management system (Heikkinen et al. 2019), APM have developed their own automated barcode recognition software, as have NHM with image segmentation functionality also included (Hudson et al. 2015). RBGK reported that they have developed all of their collections management systems in-house, but are now looking to combine these and will probably not take the bespoke development approach for this.