Conventional zoological and botanical collecting activities usually preserve target organisms as representative biological individuals (in unitary organisms), clones (in modular organisms) or as fragments thereof. Such preserved organisms, conventionally referred to as voucher specimens (e.g. Culley (2013), Martin (1990)), are generally assumed to possess a single taxonomic identity, i.e. to belong to a certain species (whether known or not). [Voucher] specimens were regarded as key elements of biorepository-underpinned genomic diversity research at large (Hanner and Gregory 2007) and recognised as central to the logistical framework of DNA barcoding workflows (Borisenko et al. 2009) where reference DNA sequences were derived from isolated, taxonomically identified organisms. This voucher-centric framework remains salient for many current genomic studies and initiatives (Buckner et al. 2021, Lewin et al. 2018).

Recent syn-ecological advances, aided by rapidly developing DNA technologies, are expanding the perception of an organism beyond its core taxonomic identity. Instead, organisms are increasingly recognised as hosts to diverse microbiomes (Gibson et al. 2014), pathobiomes (Vayssier-Taussat et al. 2014) or symbiomes (Thompson et al. 2021) whose metagenomes harbour genetic information from a consortium of multiple taxa, which may be studied in the context of their trophic (Anonymous 2013, Wells et al. 2022), mutualistic (Bell et al. 2017, Pornon et al. 2017) or other ecological relationships. This paradigm shift extends to the corresponding collection voucher specimens, obscuring their operational distinction from other types of biological materials discussed below.

Much of ecological genomic research deals with field-collected aggregations of multiple uncounted, sometimes undiscernible organisms of different, often unknown taxonomic identities. Such aggregations are commonly referred to as “bulk samples” (Gibson et al. 2014, Leray and Knowlton 2015, Taberlet et al. 2018). Although increasingly used in genomic literature, the term “bulk sample” has not been applied in the broader environmental sampling context (Hoffmann 1994); instead, it has been used to describe aggregate non-biological samples (Zhang 2007). To avoid terminological confusion, we suggest that the qualifier “bulk” is best reserved for cases when unaltered portions of substrate or water are taken for analysis; however, the composition and concentration of biological materials within that “sample” remains the same as in the environment from which it has been taken. By contrast, most biological research involves some form of differential extraction and/or condensing of organisms or their derivatives, such as DNA, metabolites or other organic matter, relative to their original occurrence or concentration in the environment. For example, the contents of a collecting container in a Malaise trap, Berlese funnel, plankton tow or organic slough on an eDNA water filter represent concentrated organisms or organic matter derived from the air, water or substrate. Thus, we suggest that it is conceptually important to discern extracted/concentrated biological materials from unaltered bulk portions removed from the environment.

The term “sample” has been widely used to describe organismal parts or pieces of tissue destined for laboratory analysis (e.g. Kilian et al. (2015), Plitzner et al. (2017)). This terminological overlap is sometimes obviated by using “subsample” to discern lab-derived materials from field-sourced “bulk samples” (Gibson et al. 2014, Leray and Knowlton 2015); however, the term “subsample” may also be used to characterise portions of an organismal tissue sample, adding to further terminological ambiguity. Thus, we propose to avoid using the term “sample” to describe materials sourced from the field, at least as a technical definition for operational entities in the data schema (see below).

The term “lot” is commonly used in biological collection management to categorise a batch of multiple organisms derived from a single collecting event. It is sometimes restricted to characterise taxonomically sorted aggregations of specimens and juxtaposed to unsorted “bulk samples”, such as trap contents sourced from the field (Anonymous 2016). We think that the distinction between “sorted” and “unsorted” collections is arbitrary because the very act of field collecting (including eDNA filtration) is, in essence, a form of initial sorting imposed by the chosen collecting methodology. To account for these collecting/sampling scenarios and to obviate the potential terminological confusion between field-sourced and laboratory-derived biological materials, we propose to expand the term “Lot” to characterise all types of field-sourced biological materials, including eDNA; but to restrict the term “Sample” to its laboratory-derived partition, which may include DNA filtered from the preservation medium. Correspondingly, within our proposed data architecture, we will apply the term “collecting” to field sourcing of biological materials and will restrict the term “sampling” to processing/partitioning these materials for laboratory analysis. Finally, within this terminological context, we see no need to define “subsamples” or “subsampling” as distinct operational categories.

We posit that, despite the fundamental biological difference between lots, individual organisms and environmental DNA, the logistics of field sourcing, processing and analysing biological materials of different nature are fundamentally similar. For example, molecular analytical protocols applied in environmental DNA research can also be used for DNA-based biodiversity analysis of aggregate specimen collections, such as arthropod traps or plankton tows. Analyses of such lots can be done by picking out and sequencing individual specimens (e.g. Young et al. (2017)), by homogenising the entire contents of the storage container for bulk DNA extraction, followed by metabarcoding (e.g. Leray and Knowlton (2015)) or by filtering and metabarcoding the organic suspension diffused into the fixative (Milián-García et al. 2021, Milián‐García et al. 2021, Zizka et al. 2019). In the latter case, preserved specimens remain relatively intact, allowing for subsequent morphological or organism-based DNA analyses. A robust data management approach should accommodate all these research designs, whether DNA is taken directly from the environment, the storage medium or extracted from preserved organisms.

Data records hosted by biodiversity data aggregators, such as GBIF, are centred around “species occurrences” or “observations” (sensu Lindström (2006)), which report taxonomically identified biological organisms collected or otherwise detected in a certain location at a certain time. This is also known as the Occurrence Core (Wieczorek et al. 2014) approach. The material entity upon which the Occurrence record is based is known as MaterialSample, defined as a “physical result of a sampling (or subsampling) event” in the Darwin Core data schema—https://dwc.tdwg.org/terms/#materialsample (Mayfield-Meyer et al. 2022, Wieczorek et al. 2014). The different categories of biological objects discussed above are all examples of the MaterialSample class. Although not a necessary component of an observational record, the MaterialSample is a critical element of both collection specimen-based and eDNA-inferred taxonomic occurrences.

Once a MaterialSample is collected in the field (Lot), it may be processed/subdivided (Sample) and transformed, for example, through DNA extraction (Aliquot, see below). It may further be transferred between agents, research teams, labs, institutions etc. during different phases of the analytical process. During each of these stages, associated data must “pass through” the data management system of the next processing facility efficiently and without information loss. A laboratory should be able to use the same data management system to track eDNA research, to facilitate metagenomic analyses of lots (e.g. invertebrate trap contents) and to contribute reference DNA sequences derived from taxonomically curated voucher specimens. In a “simple” eDNA research scenario, the same water filter with field-collected organic slough may be registered as a Lot or as a Sample, depending on its processing stage. The proposed data management framework provides sufficient flexibility required to accommodate the various collection processing pathways for eDNA research and other emerging fields of enquiry. At the same time, it conforms to the logic model of conventional collection-based biodiversity research, which reduces potential connectivity issues during future “crosswalking” with data schemas used in natural history collection databases (Thomer et al. 2017).

The second, operationally critical part of an occurrence record is the Event, broadly defined within Darwin Core as an “action that occurs at some location during some time” — https://dwc.tdwg.org/terms/#event (Wieczorek et al. 2014). Within the narrower context of biological collecting, an Event, sometimes defined as “gathering” (Lindström 2006), could be characterised as an action aimed at acquiring a MaterialSample. If successful, the event can, thus, be regarded as the source or origin of derived MaterialSample(s) and the corresponding occurrence record(s); however, even if unsuccessful, it provides an important methodological context on the broader collecting effort deployed to obtain an occurrence dataset.

The taxonomic identity of the MaterialSample constitutes the central piece of information contained in an occurrence record; however, it is of tangential importance to the logistics of an eDNA research project. The detection of certain taxa in a sample depends on the sampling methodology used (e.g. study site choice, filtration technique, preservation parameters) and is derived from a certain procedural outcome (e.g. targetted PCR detection, Sanger sequencing or metabarcoding). Each sample can be subdivided and processed using different analytical and/or bioinformatic pipelines or as several replicates using the same pipeline. As the limit of detection for different taxa may vary between methods and/or analytical parameters used, these analyses may yield varying taxonomic outcomes. Thus, although taxonomic occurrence records are the end-point of many eDNA research projects, they are best treated as context-dependent annotations of a MaterialSample and only meaningful if underpinned by a robust and adequately parametrised “Event—MaterialSample” data dyad. This data management approach is congruent with the emerging Collecting Event Core concept (Kissling et al. 2018, Wieczorek et al. 2014, M and RJ 2017) that emphasises the critical role of methodologies in producing specific occurrence records.

In eDNA research, as with other taxonomic inferences derived from collected and analysed biological objects, it is practical to shift the emphasis of the data model from Occurrence Core to Event Core. Under the Event Core logic model (Kissling et al. 2018), biodiversity survey activities (e.g. expeditions or field trips) can be broken down into a series of [collecting] Events, each of which typically results in a MaterialSample that, in turn, undergoes a series of analytical procedures to infer biodiversity information (occurrences). Additional data elements are necessary to attain Event Core parametrisation. For example, geospatial localisation of collecting events is needed to link a MaterialSample with a pre-defined collecting locality (e.g. site, station), corresponding to Darwin Core’s Location Class (https://dwc.tdwg.org/terms/#location).

From a pragmatic laboratory data management point of view, it is important to acknowledge that the Darwin Core schema employed by GBIF was designed to facilitate biodiversity data publication (Robertson et al. 2014, Wieczorek et al. 2014), rather than data capture. While it is an important minimalist, universal data standard, it is not necessarily sufficient to accommodate all data elements pertinent to particular use cases (Chapman et al. 2020) and does not offer the relational structure required to track the nested hierarchy of field collecting, laboratory operations or other logistical aspects of biodiversity research. For example, both MaterialSample and Event classes have been used to characterise field collecting efforts and outcomes (e.g. Kissling et al. (2018), Wieczorek et al. (2014)). As mentioned earlier, DNA-inferred taxon detections are also dependent on the outcome of molecular analyses of derived DNA aliquots, which should be regarded as separate “analytical events” and “material sub-samples”, respectively. Within this context, the Collecting Event would be distinct from the Analytical Event. This approach mirrors the distinction between two different classes of events characterising the “material sampling process” and the “identification process” recognised within the Biological Collections Ontology (Walls et al. 2014).

To maintain semantic distinction between field collecting and laboratory analyses, we will refer to the former as Events and to the latter as Analyses, each characterised by a defined methodology and localised in space and time. Operationally, this allows breaking the data entry process into stages corresponding to phases of field collecting, post-field processing and laboratory analyses. Keeping track of unsuccessful Events and Analyses (“negative results”) further parametrises the methodological context for the sought taxonomic occurrence outcomes. For example, it may be useful to know that the detection of a certain taxon in a certain locality is linked to several unsuccessful attempts to recover its sequence using alternative collecting protocols or analytical parameters. Darwin Core does not accommodate for this relational complexity (Walls et al. 2014), although it offers semantic provisions for reporting genomic information through its associatedSequences term—https://dwc.tdwg.org/list/#dwc_associatedSequences and associated terms.

It is important to contextualise our ontological framework by providing semantic clarification on our use of the terms “data” and “metadata”. We apply the original and currently predominant definition of the term “metadata” as “data about data” (Furner 2020). Applied to ecological datasets, metadata would, thus, be restricted to information describing the content, context, structure, quality and accessibility of data (Hampton et al. 2017, Michener 2006, Michener and Jones 2012).

Several recent works have confounded the scope of the term “metadata” to denote sampling and provenance information (e.g. ten Hoopen et al. (2022)), to define data on environmental conditions and other circumstantial parameters of field sampling (e.g. Bockrath et al. (2022), Felczykowska et al. (2015)) and to include methodological information on sampling and laboratory analyses (Nicholson et al. 2020). In our view, this approach obscures the otherwise clear semantic distinction between data and metadata or even makes it very context-based, for example, sequencing or qPCR data, vs. all other information (Abbott et al. 2021). To avoid this “terminological creep”, we will refer to information about the provenance of biological materials, parameters of the collecting effort or analytical methods as different categories of data proper. For example, geocoded location or water quality measurements from the collecting station from which the samples originated would constitute eDNA-associated provenance data, but not metadata.

Within the context of eDNA data ontologies and within the scope of data associated with natural objects or observations, we can define three major categories characterised by the nature of data (Table 1): provenance, attributes and history. It is important to note that ontological categories should be discerned from operational entities, such as Events or MaterialSamples. Neither should these abstract groupings be conflated with specific tables in the data schema that will be discussed below. The three ontological categories defined below are best construed as classes of data that could characterised using sets of data fields within the hierarchy of tables in the data management system. As such, they constitute important dimensions or qualifiers that can help to define the operational entities and to identify their relationships within the data architecture.

Provenance circumscribes the spatiotemporal and circumstantial properties of the collecting or analytical events. This is the core part of the biodiversity ontology, providing details on the origin and transformations of biological objects and inferred taxonomic occurrences. Provenance data can be grouped into three broad categories of properties that describe the collecting event’s localisation in space (“where?”), time (“when?”) and the method used (“how?”). This information should be recorded at the time when the collecting or analytical event occurs and applies by extension to all biological objects (MaterialSamples) that are collected or produced as a result: lots, specimens, samples, aliquots and their derivatives.

Attributes characterise intrinsic (e.g. organismal) or relational (e.g. ecological) properties of the MaterialSample (“what?”) or related circumstantial properties of their origin. Unlike provenance information, which applies to an entire event and all derived materials, attributes may characterise a collection lot as a whole or may be restricted to individual biological objects or their derivatives (e.g. size of an organism or form of sample preservation). Data acquired during subsequent analysis, such as DNA concentration, sequence quality and interpretation of analytical results (e.g. presence/absence of target taxa) will fall into this category as well. Relevant information may be recorded at the time of collecting or during subsequent processing and analysis and may be stored in the form of structured data fields or file attachments. In the context of eDNA research, field-collected data may include a description and/or images of the filter containing the water sample.

Once a biological object is removed from nature and is transferred into human custody, it also becomes a cultural object. Historic context provides an account of agents (persons) and organisations behind the events, for example, staff undertaking the sampling activities and performing subsequent processing/analyses of MaterialSample. Thus, “historic” properties record and contextualise human interactions with biological objects, rather than their natural origin or intrinsic properties. This information provides background on the purpose of the events and overall experimental design (“why?”), the actors involved (“who?”), a record of transactions (e.g. change of ownership), processing status, storage conditions and physical location(s) of materials. It should be stressed that any information about the biological object constitutes an integral part of its research value to the scientific enterprise and, thus, by extension, of its cultural value to society at large.

Certain data types may fall into a “grey area”. For example, photos taken at the collection site can be used to parametrise provenance data; however, they also depict attributes of the collecting station and/or collecting event (see below). Likewise, a scanned page from a field journal may depict provenance information, attributes of the materials collected and historic context of the collecting process.

The following two data entities provide the geographic reference for the Collecting Event Core. Although field adoption of GIS-based data capture in wildlife census has been proposed early on (Travaini et al. 2007), its application in sample-based operations is far from common. The database prototype contains Visual Basic modules allowing real-time GPS data capture using MS Windows-enabled tablets, but is predominantly intended for manual data entry or batch conversion from existing records.

Sites

A Site is a medium-high level of geographic localisation of project activities. Using Fisheries and Oceans Canada (DFO) standard terminology (Abbott et al. 2021), it is defined as “a specific area, within a selected sample location, where water (or other environmental substrate) will be collected”. A site represents the general geographic location of the research area; therefore, it is not necessarily defined by a research project. This table also incorporates information relevant to the DFO “geographic location” and “regions” data definitions (Abbott et al. 2021). Typically, the site would be associated with a named geographic area, limited by geomorphological characteristics (e.g. water body, valley or catchment basin) or administrative boundaries (e.g. conservation area or municipality). If justified by experimental design, it may represent a more restricted research area within this geographic location. Within the proposed data architecture, several projects may use the same site, while a single project may contain field activities conducted at multiple sites.

The Site registry in the current database prototype can be cross-referenced against automatically downloadable official gazetteers of geographic localities for Canada and the United States:

the Canadian Geographical Names Database (CGNDB) provided by the Geographical Names Board of Canada – https://natural-resources.canada.ca/earth-sciences/geography/download-geographical-names-data/9245

and the USGS Geographic Names Information System (GNIS) provided by the U.S. Board on Geographic Names – https://prd-tnm.s3.amazonaws.com/index.html?prefix=StagedProducts/GeographicNames/DomesticNames/

Stations

A Station identifies an exact geolocated spot where field samples are taken. As per DFO terminology (Abbott et al. 2021), it “refers to spatially distinct sampling locations within a site”. It is generally research project-specific (each station is linked to only one project) and is characterised by distinct GPS coordinates, which may or may not be tied to a geographically defined landmark. Each station is further parametrised by fine-grained habitat characteristics. Multiple stations may be established within each site, whereas the same station may be surveyed one or multiple times; therefore, it may be associated with one or several sampling events. Built-in functionality within the database prototype allows real-time direct capture of geographic coordinates using a GPS-enabled Windows laptop or tablet; additional scripts further allow reverse geocoding of the geographic location of each station based on the coordinates entered. Finally, several built-in tools allow individual or batch validation of these geographic coordinates by plotting them in Open Street Map, Microsoft Maps or by generating a KML file for displaying in Google Earth.

The following database tables contain information directly related to the Darwin Core Event data class. Note that only one of them (Events proper) is directly linked to MaterialSamples, whereas the remainder are used to provide further parametrised context (Readings and Observations) and to help structure this information into the experimental logic model (Activities).

Activities

An Activity represents a series of collecting events, measurements or observations undertaken as part of a project within a specified site, usually over a restricted timespan (e.g. one to several days); for example, a field trip or short-term expedition. Each Activity is associated (unambiguously linked) to one project (through the reference Project ID) and to one site (through the reference Site ID). An Activity is carried out within a specified Collecting Site as part of a single Project.

[Collecting] Events

The Events table characterises the specific targetted field collecting effort that results in the acquisition of biological materials (Lot; see below) at a particular Station over a specified time interval. As the name implies, it is the key element of the event-based data management schema; it is also the key point of reference linking biological materials with their provenance information. Each Event is linked to a single parent Activity and Station. Within the eDNA research context, a typical example would be the collection of aquatic DNA on to a water filter. Each sampling replicate, repeat or replication, as per DFO definition (Abbott et al. 2021), should be recorded as a distinct Event. If successful, it would typically be associated with a single eDNA Lot (see below) and all its derivatives.

Readings (Instrumental Reads)

Many ecological sampling activities involve recording chemical, physical or other parameters of the environment (water, soil, air) at the locality where sampling occurs. These measurements are usually taken with specialised equipment, using a set of standards established as part of the study design. An example would be water quality measurements taken with a digital probe. The Readings table is designed to accommodate this information. Although often considered part of sample “metadata”, this information does not fit the strict metadata definition (see discussion above). It is not necessarily linked to any particular sampling Event; but may be indirectly associated with one or several Events through the corresponding Station and collection date.

Observations

Although sometimes used as an alternative name for the occurrence record (Lindström 2006), the Observations table is used here as a collection of optional ancillary data pertaining to any of the other tables. Observation notes generally are not part of the standard data recorded, but may affect the outcome of the analyses. Examples include phenology, weather conditions, wildlife presence etc. Unlike Instrumental Reads, an Observation may pertain to a specific Event, Site, Station or Activity on a certain date or within a specified date range. By extension, it may provide further parametrised context to all corresponding MaterialSample units.

The following tables characterise operational relationships within the MaterialSample Core data class, operationally separated into three categories: Lots (field-derived MaterialSamples), Samples (resulting from concentrating, subdividing or otherwise processing Lots at the research facility) and Aliquots (laboratory derivatives of samples destined for analysis).

Lots

The Lots table houses a registry of field-sourced biological materials (Lots) originating from a field collecting Event. As mentioned previously, the term has been co-opted from natural history collection management practice where it is used to define a set of specimens and/or samples from one or multiple organisms originating from the same collecting event that are catalogued and stored together as a single unit (Anonymous 2016), but extended here to eDNA samples taken from the environment. Each Lot must be unambiguously linked to a single collecting Event, which provides the necessary provenance context. An additional module incorporated in the database prototype allows users to register a predefined quota of Lot ID numbers before a field trip and to pre-print sticky lot labels in several common formats. If a Lot is later subdivided into sub-lots, this can be accommodated within the proposed data architecture by adding new Lot records linked to the parent Lot ID through a corresponding foreign key field.

*Specimens

For research operations focused on building genomic reference collections linked to preserved voucher specimens (as discussed in the Operational Framework section), it may be optimal to designate a separate Specimens table within the proposed data architecture. However, for the purpose of most field-based eDNA research, the Lots table can accommodate essential provenance information on voucher specimens (e.g. opportunistically collected organisms), without the need to designate a separate data entity. The table is, therefore, not implemented in the prototype data schema.

Samples

The Samples table stores information about field- or laboratory-derived MaterialSamples prepared and preserved for archival storage and/or partitioned for laboratory analysis. In cases when DNA is filtered from the Lot preservation medium, the Sample would constitute a portion of that parent Lot; however, under many eDNA research scenarios, it may represent the same physical object as the entire Lot (e.g. DNA filter). In some cases, Samples may originate from external collaborators and not directly from the field. Samples are often grouped together into Containers (see below) for processing or storage efficiency; however, the latter should not be conflated with Lots. The table accommodates subdividing each sample into subsamples by creating new records linked to the parent Sample ID via the designated foreign key field.

Aliquots

Aliquots are laboratory derivatives of Samples: DNA extracts, PCR products etc. In many cases, these are transient substances that are used up during analyses. The purpose of an Aliquot is to identify the portion of each sample that is destined for a specific analytical pipeline (e.g. to sequence a particular gene region). Aliquots are arranged into Arrays (see below) for streamlined batch processing. Similar to Lots and Samples, sub-aliquots can be accounted for by creating new records linked to the parent Aliquot ID through a foreign key field.

The following two data entities are used to organise MaterialSample units for storage, batch processing and/or analysis. Unlike MaterialSample units proper, these entities are provenance-agnostic, allowing us to aggregate materials from multiple collecting Events, Activities, Sites etc., provided that each associated MaterialSample and, if applicable, its position (e.g. processing order) within the batch are unambiguously tracked.

Containers

Containers are physical objects used to store biological materials, which can be archived, relocated or processed as a single unit. Each container can be used to house one or many biological collection items (e.g. whole Lots, Samples, Aliquots or portions thereof). Containers are designed to facilitate organisation of samples together within a processing or storage batch, their localisation within the research facility and their transfer within or outside the lab. Examples include tube racks, boxes, trays, Tupperware, removable drawers etc.

Arrays

Arrays, or processing batches, are operational (logistical) counterparts of Containers that are used to organise Aliquots or their virtual derivatives in sequential order for laboratory analyses. As such, they may be somewhat “ephemeral” as physical objects, for example, PCR plates that are used up and discarded after DNA sequencing. They may also be purely virtual, for example, batches of raw DNA sequence files run through an informatics pipeline. Additional built-in functionality in the prototype database allows the user to map aliquots within an array (processing batch) and display these maps in several common formats, for example, 12 × 8 wells in a microplate or 10 × 10 sample tubes in a rack.

The following two data entities do not conform to any existing Darwin Core data classes; however, they are operationally essential for laboratory information management. As discussed earlier, we use the term “Analytical Event” to emphasise that they represent a separate category of “events”, which corresponds to the “identification process” category recognised within the Biological Collections Ontology (Walls et al. 2014). Together with the Collecting Event Core information, it constitutes a critical piece of information for inferring taxonomic Occurrences from the MaterialSample.

Analyses

The Analyses table provides a registry of analytical procedures and stages used in laboratory analyses, for example, DNA extraction, PCR reactions, sequencing runs etc. The prototype data schema provides for many-to-many relationships between Analyses registry and associated Arrays, thereby allowing flexibility in tracking the processing of a single Array through multiple analytical stages or assembling multiple Arrays for a single analytical procedure, for example, multiplexing several PCR plates for the same sequencing run. Analyses table fields are further parametrised by ancillary registries for target Markers, PCR Primer combinations and a Multiplexing schema to map the Aliquots used in Next-Generation sequencing runs.

Experiments

Experiments represent sets of Analyses aimed at a particular research goal; for example, grouping together sets of Analyses that use the same protocols. As such, they represent abstract entities used to facilitate operational logistics and may be placed in the category below.

The following tables serve to facilitate overall operational logistics of research projects, thereby parametrising the historic component of the ontological framework. Portions of the information contained in these tables fall into the metadata category, as related to Event and MaterialSample Core components.

Projects

The Project table houses records pertaining to the logistics of administration and/or management of research and survey activities; therefore, it is not subordinate to any other database module. Projects are registered before the beginning of any related field activities or laboratory experiments. All activities, Collecting Event Core and MaterialSample Core tables are associated with the respective Project by linking each of them to the corresponding Project ID. However, Projects may have a many-to-many relationship with laboratory Analyses and Experiments, depending on experimental design and laboratory management logistics. As such, the Project may be considered as the logistical counterpart of the Experiment.

Agents

The Agents table hosts names, institutional affiliations and contact details of persons recorded in other database tables (collectors, data recorders, processing staff, collaborators, project managers, expedition leads etc.). Information from this table is linked to agent drop-down menus available in other tables.

Organizations

The Organizations table holds information about institutions, laboratories, companies and other organisations affiliated with or responsible for different projects, experiments, corresponding activities and analytical stages.

Accessions

The Accessions table is adopted from biological collection management practices (Berendsohn et al. 2011, Miller et al. 2020) to document batches of biological materials (e.g. Lots or Samples) received together on the same day under the auspices of the same project. Information contained within an accession record thus extends to each associated MaterialSample (Lot, Sample and all derivatives). While bearing no direct relevance to research design, accessions are important for tracking the administrative aspects of biological material transactions, such as ownership, destination, mutually agreed terms and applicable restrictions on analysis or data publication. Tracking this information is particularly important if biological materials are sourced from another institution and, especially, from another country.

*Loans

Although not currently implemented in the prototype database, loans (batches of biological materials dispatched to external users) constitute an important component of collection management logistics (Berendsohn et al. 2011). If a laboratory plans to engage in biological material transactions with third parties, it should consider incorporating a separate registry of loans in its data management strategy. The corresponding change can be easily implemented in the database prototype.

[Storage] Units

Storage Units are items of furniture and/or equipment used for material storage (freezers, refrigerators, cabinets, shelving units etc.). Typically, they have a fixed location in a specific building, floor, room etc. within an organisation. Each Storage Unit is linked to multiple Storage Locators (see below).

[Storage] Locators

Storage Locators are fixed compartments within Storage Units housing various physical or biological objects, specifically, collection items (Containers with Lots, Samples or Aliquots). Examples include fixed drawers, shelves or slots within freezers, shelving units or storage cabinets. Locators are important in ensuring that biological materials housed and processed by lab members can be easily found and tracked within the laboratory or collection facility. Locators have a one-to-many relationship with storage Containers and, by extension, with all associated Lots, Samples and/or Aliquots.

Equipment

Most research activities use specialised equipment, which may impact field collecting and analytical outcomes. An Equipment inventory helps to control for biases that may be introduced by using generic equipment types (e.g. technical specifications of different brands of eDNA samplers) or particular equipment items (e.g. working condition or calibration). Individual Events, Instrumental Reads and Analyses could be linked to utilised Equipment items through dedicated foreign key fields. Depending on laboratory setting, this module could be further parametrised by adding separate registries of calibration, maintenance or sign-out for use by laboratory staff and/or external collaborators.

*Supplies

Basic information on standard Supplies used in particular Events (e.g. eDNA filters) and Analyses (e.g. PCR reagents) is incorporated within the respective Events, Analyses and other data tables. For larger-scale operations, it may be useful to establish a separate registry of supplies and/or reagents that would allow evaluating the relative performance of separate supply batches or reagent stocks over time. Although not implemented in the prototype database, this data module could be added and further parametrised by logging accrued stock and its use for field or laboratory work, linked to Activities, Events, Experiments or Analyses. It could also be integrated with other enterprise resource planning modules, such as a registry of purchase orders. Such modules could be custom-built or adopted from existing off-the-shelf enterprise resource planning solutions.

The following tables provide annotations for files associated with existing data records that are either unstructured (e.g. raster images) or cannot be adequately parsed and incorporated into existing data fields without significant information loss (e.g. original Excel tables). Each data entry includes a reference (foreign key) linking it to the “parent” record (e.g. collecting Event ID) and an absolute URL to the online resource where the file is hosted. By default, attachment files are named in a self-explanatory way (i.e. by incorporating the foreign key into the file name) and are hosted in a designated folder on SharePoint or other cloud server that ensures reliable data hosting for the project’s life cycle. This allows effective retrieval of external files associated with each database record, as well as direct browsing through data folders on the cloud server and, as necessary, batch processing, backup or migration of these files. Built-in database functionality allows the user to perform batch renaming of files and automated generation of links, based on a set of standard algorithms.

Attachments

The Attachments table provides annotation for generic file attachments, such as documents, digital images or collaborator-provided Excel spreadsheets. Attachments may be linked to any record in any of the core tables within the prototype data schema using their primary ID as a foreign key. If the same primary ID is used to identify records in two or more different tables (e.g. if the syntax of the collecting Event ID is identical to the derived Lot ID), then the same attachment file (e.g. photo of the collected water filter) is linked to all corresponding database records. Hosting these files in dedicated folders on the database SharePoint server allows direct batch viewing and download through the online SharePoint interface or using OneDrive file manager applications.

Sequences

The Sequences table is used specifically to annotate DNA sequence file attachments (e.g. FASTA and FASTQ files) linked to records of individual Aliquots from which they have been generated. In addition to providing links to the Aliquot ID and file URL, this table includes fields that provide additional parametrisation, in line with the Darwin Core’s AssociatedSequences (https://dwc.tdwg.org/list/#dwc_associatedSequences) and related fields. While the database prototype does not offer built-in functionality for analysing stored sequence data files, it facilitates their direct download and processing using external software applications.

Protocols

The Protocols table provides a registry of Analytical Protocols and SOPs used in the organisation’s research operations linked to Collecting and Analytical Event Core modules. This module currently provides only basic annotation functionality; however, it offers potential for future parametrisation of research outcomes by adding custom tables with project- or laboratory-specific qualitative or quantitative metrics that vary, according to the collection or analytical protocols selected.

Several additional tables and fields within the prototype data schema allow basic taxonomic annotation for the MaterialSample (Lot, Sample or Aliquot), including modules that validate the taxonomy used against existing taxonomic references (currently, GBIF and NCBI taxonomy). The NGS_Taxonomy table provides a detailed breakdown of taxonomic occurrence records inferred from analysing Aliquot-derived raw sequence data using different informatics pipelines. By extension, these results are linked to field-sourced Lots with associated Collecting Events and other provenance information. They are also linked to laboratory-assembled arrays and associated analytical protocol parameters (Analytical Events), allowing us to backtrace the field provenance and/or methodological and procedural origin of each taxonomic occurrence record.

Table 2 provides a rough breakdown of what happens to biological materials (MaterialSample) and associated data during typical phases of the research project. It is intended to provide context for best practices outlined below.

A good practice with respect to ensuring the uniqueness of the identifiers used as primary keys (e.g. Lot ID or Container ID numbers) is to generate them in advance of a field trip or experiment using a dedicated module of the data management system. This will ensure both uniqueness and accuracy of the syntax used for any given activity and will also “preoccupy” this syntax pattern and not allow it to be registered accidentally by another user or field crew.

We have developed an MS Excel template that could be pre-filled and printed in a 4.625” x 7” weatherproof 5-inch binder format where core blocks of data and data fields are structured similarly to the database, allowing for subsequent manual database entry from hand-filled templates. The template is organised as a set of predefined forms, rather than as a non-relational spreadsheet. Each form mirrors the operating procedure performed in the field: arriving on site, confirming the location, identifying and characterising sampling stations, performing water quality tests, sample collection and recording ancillary observations.

The database prototype offers a suite of pre-defined MS Excel templates to assist users with standardised field data capture, batch data conversions and validation. The tools are being constantly updated to address emerging user needs. Several modules currently available or under development are listed below:

• Batch geographic coordinate conversions from degree, minute, second (DMS) and Universal Transverse Mercator (UTM) formats to decimal degrees;
• Batch reverse geocoding script to provide geographic annotations for sets of coordinates;
• Batch data converter from an MS Excel generic non-relational field data template;
• Batch data converter from an MS Excel standard non-relational data template used by the GEN-FISH network — https://gen-fish.ca/;
• Batch data converter from Excel table output from the Sample and Field data collection Information System for the Hanner Lab (sFISH) prototype mobile field app — https://github.com/HannerLab/sFish;
• Batch data converter from the database to the Molecular Detection Mapping and Analysis Platform for R (MDMAPR) Shiny web app (Yu et al. 2020);
• Batch Excel conversion file for manual 96-well plate array assembly from isolated samples;
• Batch data converter from custom Next-Generation Sequencing pipeline outputs — under development.

Data management systems benefit from active engagement of users in the process of their development (Glöckler et al. 2020). This engagement is particularly important for new databasing projects aiming to facilitate emerging and actively developing research areas, such as eDNA. Active input is sought from current and prospective database users and collaborators to strengthen the system’s ontological and semantic conceptual framework, to review and improve its architecture and data schema, to improve its user interface, built-in tools and additional functionality.

Continued curation and management are essential for maintaining a database’s utility over time (Blair et al. 2020). Although the database and its additional data conversion tools are designed to maximise efficiency in data upload and validation, its use requires a significant time commitment to familiarise with the user interface and custom functions and continued attention towards data curation.

Sustained efforts should be devoted towards building robust, standardised, logically consistent and intuitively comprehensible naming conventions for natural Primary Keys used throughout the data management system, especially when digital records refer to analogue biological objects that are being collected, stored and analysed.

Curation of unstructured and/or analogue data (e.g. images, hand-written field notes) requires digital capture of representative data files (e.g. photos or scans), which are then appended to core database records as annotated attachments. Associated metadata, if available, could be used to parametrise such files. As mentioned earlier, the database prototype allows storing and annotating diverse types of file attachments; however, detailed user input and continued curation are required to ensure that archived files remain properly organised, referenced and readily accessible through individual database records or directly from the hosting server.

To date, robust standards have been developed for biodiversity data (Berendsohn et al. 1999, Chapman et al. 2020) and specifically for DNA-derived occurrence data (e.g. Finstad et al. (2023)); and best practices have been established for making them findable, accessible, interoperable and reusable, or FAIR (Reyserhove et al. 2020, Wilkinson et al. 2016). Tools are being developed to facilitate adherence of eDNA data to FAIR principles (e.g. Kimble et al. (2022)), with similar developments underway addressing taxonomic occurrence and biodiversity data at large (Sandall et al. 2022, Reyserhove et al. 2020). Unfortunately, there is currently limited awareness and/or slow uptake of these principles amongst field and laboratory biologists. In practice, adherence to these standards is difficult, particularly for small research groups lacking integrative data management systems and dedicated databasing staff. We hope that the proposed data management solution will be a step towards facilitating the generation of FAIR eDNA data records. Amongst the priority areas for future development of the database prototype are further mapping of the data schema on to Darwin Core and the development of a semi-automated data submission pipeline from the database to GBIF and other biodiversity data aggregators.