In Machine Learning (ML), representative training sets (e.g. Aversa et al. 2018b) are usually composed by combining data from various sources. The data selection process includes data search, access, and evaluation. The data search is usually carried out in databases, platforms, and repositories, which may follow different data management standards mainly because of the different purposes (e.g. institutional repositories versus discipline-specific databases). A user-friendly information system to ease the search of domain repositories and databases is the NFDI4Ing Data Collections Explorer (KIT Data Manager 2022), while the Metadata Standards Catalog (HMC 2021) provides a continuously extensible list of existing metadata standards and schemas.

Once suitable data sets are found and accessed, they need to be further analyzed by the scientist in order to evaluate their usability for training a ML model on the chosen task. In particular, when dealing with supervised learning, additional preprocessing is required, e.g. the assignment of labels in image recognition methods. As the data is collected from different sources, these labels need to be aligned, e.g. according to semantically similar categories; this way, images can be grouped together and relabeled. After a sufficient amount of images has been collected and relabeled, the composed training data set can be further prepared for ML using other techniques, e.g. resizing, or rescaling. The previously described preprocessing steps may be laborious and time-consuming, as they are usually performed manually by scientists, preventing them from spending their valuable time on the actual ML task, i.e., model training and analysis of results.

A possible solution to overcome the heterogeneity of data and repositories, and to reduce the amount of manually performed actions, is to apply the FAIR principles (GO-FAIR 2018) to the data to enable its machine-actionability. This can be realized with the use of FAIR Digital Objects (FDOs) (FAIR Digital Objects Forum 2022). A FDO is a digital representation of data as a sequence of bits, identified by a globally-unique, persistent, and resolvable Identifier (PID), described by an information record and classified by a type that determines the operations a machine can perform on the data. The FDOs may contain in their information record a reference to the repository where the data is deposited. This approach enables repository-agnostic (meta)data interpretation on a common basis for machines, without making any direct changes to the original data. This abstraction layer of FDOs in turn allows different data assets to be connected and related, regardless of where they are stored.

In this work we present the application of FDOs to Scanning Electron Microscopy (SEM) images labeled with a term related to their content. In this context, we refer to metadata as either administrative (bundled in the information record and necessary to manage the data, e.g. to identify its format) or scientific (information about the data in the context of a specific scientific question, e.g. labels describing the image content, which are represented by FDOs themselves). We explain the design of the FDOs and the requirements for their implementation in this context. Finally, we discuss the benefits of representing SEM image data as FDOs to compose new training data sets for further ML applications.

The FDO concept, which provides a generic framework, is not directly applicable as it covers a broad scope, requires contextual interpretation, and domain-specific expertise along with decision-making. Therefore, it must be implemented into an architecture of components that enable its use. An essential aspect that has been established in all FDO implementations is the employment of repositories as trustworthy data storage.

The approach of the Research Data Alliance (RDA) (European Commission et al. 2013) community for implementing the FDO concept is built on existing systems: the PID-Service of the Handle Registry (DONA-Stiftung 2014) is used to create, update and resolve the PID assigned to a data object in order to access it; the ePIC (European Persistent Identifier Consortium 2009) Data Type Registry (DTR) contains the Kernel Information Profiles (KIP), where the attributes of the information records are provided, defined, typed and assigned a machine-readable PID as well as a human-readable string name. Based on this structure, a minimum set of attributes (including the type of a FDO), which we call administrative metadata, required for machine-actionable decisions can be provided to clients.

Further ways to model the FDO concept have been proposed: one of them is the FAIR Digital Object Framework (FDOF). Its implementation was shown in the frame of a test case from NFDI4DS, where ML data components (e.g. images as training data, source code, and publications) were connected to each other by representing them as FDOs (Boukhers et al. 2022). Another solution is offered by the Distributed System of Scientific Collections (DiSSCo) research infrastructure, which is implementing FDO via PID records (Handle), also called FDO records, and JSON-LD representation of Linked Data. This can enable different services, such as Graph Machine learning on a biodiversity knowledge graph (Grieb et al. 2023).

Situation before the use of FDOs

Metadata curation

Implementation of the FDO concept

We created several FDOs to represent the data landscape, i.e., images and labels, as well as their types (Fig. 1). For the sake of clarity, we will refer to them in the text using different prefixes (i.e., image-, label-, type- FDO). We created the PIDs using the Handle Registry PID-Service. This was facilitated by a Typed PID Maker (TPM) instance (Pfeil and Jejkal 2020) that was implemented and connected to the Handle Registry and the ePIC DTR. The TPM covers validation of the FDOs information records against their KIPs during creation, and can also be used to update and retrieve them.

Figure 1.  

The design of the FDO represents SEM images with their labels. The arrows that go from the PID record attributes point to the different objects either via URI, or PID, i.e., to the KIP, the types, the locations, the license, or another FDO.

To fill the information record of the image- and label- FDOs, we used the Helmholtz KIP (Jejkal et al. 2022), which was developed by the community of the Helmholtz Metadata Collaboration (HMC) and refers to the RDA Kernel Type WG recommendations on KIPs (Weigel et al. 2019). It contains a set of 15 basic attributes that cover the recommended minimum, and additional information about a FDO. It is registered and defined in the ePIC DTR test instance we used for our work (Jejkal and Schweikert 2022). For our application case, we used 10 of the 15 attributes, including a reference to the location where the content is stored.

In our implementation, each image- and label- FDO has a reference (i.e., PID) to a type- FDO, which contains information such as MIME-type, related metadata schema, and version. The type- FDO, which is based on the File Type KIP (Pfeil 2022) and typed, determines the operations that can be performed on the data represented by the image- and label- FDOs. After the type- FDOs were created, their PIDs were included in the attributes of the information records for the image- and label- FDOs, together with the repository location (URI) of the data objects, and the cross-references to the related FDOs. We yielded a total of 7 PIDs, 3 for the type- FDOs (i.e., 21.11152/c7009b0f-3cbd-41fa-9e42-cecf0d1f3552, 21.11152/1bd1fab9-2d13-4710-84ff-0aed44097fee, and 21.11152/de2d965b-8941-46f1-b0f0-94e2ca41c18c) and 4 for the image- and label- FDOs (i.e., 21.11152/37833c54-1d36-42e4-858d-831447122863, 21.11152/219ca65d-a534-4f44-aec5-a98be36f9f56, 21.11152/324ef76e-25f7-4f8c-89aa-992ae6996d1a, 21.11152/b0b5de04-6e11-480b-ab66-2d4a5f42ea9e).

Implementation of a FDO client

We implemented a Python client (Blumenröhr 2023b) to exploit our machine-readable and interpretable FDOs in order to perform the relabeling task. The client is a semi-automatic tool that enables the user to dynamically select the attributes of the FDO's information record needed for a given operation, which is implemented as a class method locally. For operations that need to communicate with an API endpoint on the server-side, we employed the Python 'requests' package, which uses the HTTP protocol. In general, the class methods use as input the selected attributes, defined in the ePIC DTR, and return the result of the processed attribute value. Some methods are FDO type-agnostic, whilst others are FDO type-specific. As an overview, the steps the client performs are:

1. receive the PID of a FDO
2. send a request to the Handle Registry using the TPM to resolve the PID and retrieve the information record
3. validate the PIDs of the record attributes against their profile using the ePIC DTR
4. in case of FDO type-specific methods, evaluate the FDO type and execute the corresponding task (Fig. 2).

Figure 2.  

The architecture of a client that performs operations on the FDO representation of the SEM images to carry out the relabeling task.

In our specific use case of the relabeling task, w e provided the PIDs of the image FDOs to the client, which then performed the sequence of operations shown in Fig. 3. It is worth noticing that we developed a FDO type-specific method to retrieve the label information from a JSON file: the method assesses similarities between label terms from different FDOs by reading the vocabulary definitions of the ML-basic schema (Blumenröhr 2022) and by using the related, broader, and narrower concepts. The latter defines how terms are connected to each other and can be easily processed by machines, as the UNESCO Thesaurus provides a JSON-LD representation of them.

Figure 3.  

The sequence (i.e., workflow) in which the operations shown in Fig. 2 are performed by the client. For simplicity, we only show the scenario where the first PID provided to the client corresponds to the image FDO.

Situation with the use of FDOs

Relabeling ML data with FDOs

Our work successfully shows the feasibility of using FDOs in the context of an ML application case to automate the time-consuming relabeling task as part of the training data composition. It is worth remarking that our approach implements harmonized data descriptions using schemas and vocabularies to facilitate machine-readability and -interpretability. We decided to represent each data component, i.e., SEM images data sets and metadata documents containing the labels, as separate FDOs. This allows to use each FDO independently from this application case, and to link new FDOs to the already existing ones through their PIDs. The latter matches our scenario, where scientific metadata was added after data curation. The introduction of a type- FDO enables additional enrichment of FDO type-specific attributes in the information record, e.g., MIME-Type and metadata schema. Moreover, it facilitates a one-to-many relation, where all data components of the same type point to the same type- FDO, i.e., to the same PID.

The strength of our design stays in its high flexibility: the attributes in the FDO information records are standardized and reusable, being defined in a DTR; other metadata schemas and vocabulary specifications can be implemented to extend the FDO content; a client with modified features can be easily realized to perform different tasks with respect to the one presented in this work.

As a future perspective, interesting aspects can be explored: which particular attributes of the FDO information record are required to enable machine-actionable decisions in order to fulfill a given task? What is the most efficient level of granularity to represent the data in a given application case? Our FDO design can surely pave the way for further development and applications to support the answers to these questions and beyond.

This work has been supported by the research program 'Engineering Digital Futures' of the Helmholtz Association of German Research Centers and the Helmholtz Metadata Collaboration Platform. This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 101007417 within the framework of the NFFA-Europe Pilot (NEP) Joint Activities. We acknowledge support by the KIT-Publication Fund of the Karlsruhe Institute of Technology.