Research Ideas and Outcomes : Research Article
PDF
Research Article
HafsAuga MobileEM: mobile electronic monitoring for fisheries management and research
expand article infoLachlan Fetterplace, Emilia Benavente Norrman§, Kristin Öhman, Filip Bohlin|, Lisa Sörman|, Daniel Rooth, Peter Ljungberg, Sara Königson
‡ Institute of Coastal Research, Department of Aquatic Resources, Swedish University of Agricultural Sciences, Uppsala, Sweden
§ NIRAS, Göteborg, Sweden
| Institute of Marine Research, Department of Aquatic Resources, Swedish University of Agricultural Sciences, Lysekil, Sweden
Open Access

Abstract

Electronic monitoring (EM) using video cameras is valuable for documenting fisheries catch and bycatch, but it remains challenging to implement in small-scale fisheries. Current barriers include high costs, technical installation needs and limited power supply on small vessels. In addition, as most EM systems on the market are difficult to quickly move between vessels, they do not allow for random data collection, which may be required to obtain reliable estimates of bycatch across a fleet. Basic EM systems available, designed for use in small-scale fisheries, are image-based, have low frame rates and are not always capable of recording in high enough video quality to identify species with high precision.

The Swedish small-scale fishery consists of over 700 boats (under 12 m length), with key target species including cod, herring, sprat and flatfish. To meet monitoring requirements and to gather sufficient data for machine-learning applications, we created the HafsAuga MobileEM: a low-cost mobile multi-camera, GPS and remote data offload system for recording data on fisheries catch, bycatch and effort. It records video (up to 60 fps), is compact (~ 2 kg) and deployable in under 30 minutes. Designed to be simple to operate and install, it is modifiable and allows users to connect to a vessel's 12v power or to an internal battery to record high-quality video footage continuously for over a week. This system is ideal for use in small-scale fisheries and also well-suited to situations where fleets need to be randomly sampled by quickly moving EM systems between vessels.

Here, we describe the HafsAuga MobileEM system and outline its use in Sweden, where it has been in use since 2020. To date, twenty Swedish vessels have had mobile systems mounted on them and over 1000 fishing days have been successfully recorded. The HafsAuga MobileEM provides an innovative new EM tool with potential applications in fisheries in other regions.

Keywords

protected species, fisheries monitoring, small-scale fisheries

Introduction

Electronic monitoring (EM) using cameras, often combined with various data sensors, is a method that is increasingly, albeit slowly, being implemented to document fisheries catch, bycatch (unintentional or unwanted catch) and effort (Michelin 2018, Helmond 2020). The EM systems in use vary widely in design and components; however, they generally consist of camera(s), data storage and GPS at a minimum. These base components are then often further enhanced with computer hardware, Wi-Fi offload capabilities and numerous data loggers and activity sensors (see Fujita (2018)). The video and sensor data can then be analysed later and various metrics recorded for example, species identity, total catch, retained catch versus discard catch. The majority of EM systems have been designed for permanent or semi-permanent installation on commercial fishing boats with power supply and cabin for placement of EM system components. As a result, the majority of fully implemented EM programmes are in fisheries with vessels over 12 m (see Helmond (2020) for details of fully implemented EM programmes).

Despite small-scale fishing vessels constituting the vast majority of the global fishing fleet by number and accounting for around 40% of the total global fisheries catch (FAO 2023), the widespread adoption of EM in these fisheries has been sluggish. An impediment to increased uptake of EM in small-scale fisheries is the unsuitability of large, hard-wired EM systems for these vessels. This incompatibility arises from practical constraints, including limited power supply, space restrictions and the high costs associated with large-scale EM systems. In some instances, the cost of buying and implementing such systems exceeds the economic value of the small-scale fishers' yearly catch or, in some, surpasses the value of the vessel. The need for specialist technicians to install and provide maintenance further increases the associated costs. While recent efforts have been made to develop EM systems tailored to small-scale fisheries, the majority of these systems are image-based rather than video, rely on a single camera, have slow frame-rates or need to be hard-wired into a vessel (Helmond 2020). Whilst these are welcome additions to the EM arsenal, they do not always meet the needs of many fisheries.

The problem remains: in situations where monitoring is carried out on small vessels, the availability of on-board power is uncertain and high video resolution and fast frame-rates are needed to identify many similar-looking species with high precision or to capture sufficient data for use in machine-learning development and the available EM systems are not always suitable. This underscores the need for a broader range of EM options to cater to the diverse needs of small-scale fisheries.

Here we describe one of these options: HafsAuga MobileEM, a system particularly suitable for use in small-scale fisheries and where random monitoring of fleets is required. The system was developed at the Swedish University of Agricultural Sciences (SLU) in collaboration with Spot X Underwater Vision and subsequently improved over time in response to field use, as well as evolving needs identified in Swedish small-scale fisheries. We briefly summarise our use of HafsAuga MobileEM systems in Sweden over the last three years.

Electronic Monitoring in the Swedish context

The EU Habitats Directive (Directive, Habitats 1992) obliged Member States to establish a system to monitor the incidental capture and killing of the animal species listed in Annex IV (Articles 6.2 and 12.4 of the Habitats Directive). As a result, in Sweden, there is growing impetus to expand EM research and increase monitoring coverage in small-scale fisheries, with a focus on addressing bycatch. Swedish small-scale fisheries, like many small-scale fisheries globally, have a wide variety of vessel types in use (see Boonstra and Hentati‐Sundberg (2016)), which complicates implementing an EM programme. In the Swedish small-scale gill net fishery specifically, vessels in use include everything from fully-powered cabined boats through to small tenders with hand-pulled nets and no cabins or power source for EM systems. Fishers go out over a day and return to their home harbour, geographically limiting the fishing areas each boat fishes. Swedish small-scale fishers often use several types of gear to target a mixture of species over several fishing seasons. Commonly-used gear types in Swedish small-scale fisheries are gillnets and pots (Königson 2022), as well as pound nets and set longlines. Of these, gillnets have the highest risk of bycatch of protected species (ICES 2019). A wide range of species are bycaught in gillnets in the Swedish small-scale fishery; however, the bycatch of marine mammals (including the critically endangered Baltic porpoise Phocoena phocoena), numerous seabirds (see Glemarec et al. (2021), for an in-depth review of bycatch rates in the Baltic Sea) and elasmobranchs (e.g. spurdog Squalus acanthias in the Skagerrak and Kattegat; Börjesson et al. (2022)) are of particular concern for fisheries management. Since the small-scale fleet is geographically dispersed across Sweden, fishing patterns are heterogeneous and vessels are small so do not always easily fit observers; monitoring with observers is logistically challenging and, therefore, a small cost-effective mobile EM system is particularly beneficial in this context.

The process of setting up EM in the Swedish small-scale fisheries began with pilot projects, starting in 2017, initially using various camera types and ad-hoc EM systems cameras developed in-house to collect data on mitigation measures to reduce bycatch (Björklund Aksoy 2020, Häberle 2021, Fetterplace 2023, Königson 2023). These early works demonstrated that using camera systems to collect bycatch data on protected, endangered and threatened species, could be cost effective and provide reliable data. An initial list of requirements that an EM system needed to best suit the Swedish fleet was then compiled, based on results from these pilot projects, earlier Swedish EM work (e.g. Tilander and Lunneryd (2009)), communication with fishers and experiences from other research institutes (e.g. The Danish Technical University).

Ongoing trials then aimed to further refine the initial list of EM requirements by testing camera placement, video specifications, addressing power issues, determining data requirements, establishing data retrieval processes and evaluating the acceptance of the system amongst fishers. The final refined requirements for an effective EM system tailored to Swedish conditions were found to be numerous and specific (as outlined in Table 1). Key amongst these requirements was the necessity for adaptability across a diverse range of vessels, ease of moving the system between vessels and the ability to customise frame-rate and video quality (so that the optimal rate could be determined through testing, rather than dictated by the equipment).

Table 1.

Initial requirements identified for electronic monitoring systems for use in bycatch research and monitoring in Swedish small-scale fisheries.

Requirement

Description

Cost-effectiveness

The system should be cost effective compared to permanently wired large-scale systems (Low cost to install and maintain).

Portability

It must be compact, easily transported by a single person and be easily installed on different vessels.

User-Friendly Interface

An intuitive interface for fishers and operators, requiring minimal technical expertise.

High-Quality Video and Image Capture

Capable of capturing clear video footage for improved species identification and documentation. Capture sufficient quality (minimum 1080p) and quantity of data for use in machine-learning development.

Configurable Video Resolution and Frame Rate

Adjustable video resolution and frame-rates to meet specific user needs, ranging from high definition to low resolution, with preconfigured frame-rates of 10 fps that can be easily adjusted to a maximum of 60 fps or as low as required.

Multiple Camera Support

Ability to accommodate multiple cameras, such as two waterproof cameras to capture key on-board points of interest (e.g. net return point and catch sorting table), with the option to upgrade with additional cameras if required.

Camera Cable Flexibility

Cameras should have flexibility of placement and include multiple placement options.

Optional On-board Monitor

An optional monitor for on-board checks or live feeds to facilitate real-time observation.

Battery Life

Battery power options and extended battery life to ensure continuous monitoring over longer fishing trips.

Power Source Flexibility

Compatibility with the vessel's 12V power source or be adapted easily to fit other less common power sources.

Environmental Durability

Built to withstand exposure to water, weather and marine conditions.

Remote Data Offload

Capability to transmit data remotely via Wi-Fi, 4G/5G or other networks.

Integration with Analysis Software

Compatibility with existing analysis software for data processing and species identification.

Data Storage Options

Adequate storage options for video and image data with adjustable capacity and easily manually downloadable and swap out and go hard drives.

GPS Integration

GPS functionality for accurate location and speed logging. Integrated GPS and film capabilities when using the latest Linux-based system option.

Random Sampling Capability

Ability to be easily moved between vessels for randomised data collection across the fleet.

Despite extensive efforts to identify suitable commercially available systems, our investigation revealed a lack of systems that met the particular requirements identified by Swedish researchers and management for application in small-scale fisheries (refer to Table 1 and Suppl. material 1.1). Whilst some systems met a subset of requirements, common shortcomings included failure to meet frame-rate or resolution needs and the provision of only a single camera, necessitating the use of two systems on each boat. As a result, we developed and tested a number of mobile EM system variants in-house at SLU, including LPScam mobile EM (Johannesson, Sörman et al., unpublished) and the HafsAuga MobileEM described here.

Description of the EM System

HafsAuga MobileEM is a low-cost portable multi-camera, GPS and remote data offload system for recording fisheries catch, bycatch and effort data. Developed in collaboration with Spot X Underwater Vision, the system has been designed to be simple to operate, easily configurable and modifiable, have a small space requirement and allow users to record high quality video footage continuously for over a week on a single charge. The system can be easily carried and set up by a single person with no technical expertise needed, whilst still enabling high quality footage and sensor data to be collected over long periods on battery or via simple plug-in power. The system is configurable to meet the differing needs of fishers, management and researchers. The system features are outlined in detail in Table 2.

Table 2.

Features of the current HafsAuga MobileEM systems.

Feature

Description

Camera

Two waterproof video cameras designed to capture critical on-board locations, such as the net return point and catch sorting board. Option to custom upgrade to four or more cameras if needed. Cameras are cabled for easy placement and come with various cable length options.

Monitoring and Live Feed

The system provides an optional monitor for on-board checks and live feed, allowing users to view real-time footage as needed. Cloud-based live feed upgrade available.

Video Quality and Frame Rate

Option in build stage available for either fixed frame-rate or alternatively configurable video resolution (1920 x 1080 standard) and frame-rate settings – allowing the preconfigured frame-rate of 10 fps or 30 fps to be modified to reach a maximum of 60fps or reduced to the desired rate.

Size

The system is lightweight and comes in a waterproof case that can be carried by a single person. Two case sizes are available based on battery or charging requirements: a small battery and DC optional powered unit (6 kg, 45 x 45 x 45 cm) or an extra small DC-powered unit (2 kg, 36 x 26 x 14.5 cm). The extra small unit can also be connected to a small portable battery internally if needed or a large battery externally. The camera and cable weigh approximately 1.7 kg, based on a 10 m cable.

Power Options

Users can choose between battery-powered units with plug-in power and charging capability or a plug-in only version.

Remote Data Offload

Supports remote data offload to the cloud via WIFI, 4G/5G and phone networks.

Data Storage Options

The system offers configurable data storage options and comes standard with 1TB of swap in and out storage, providing flexibility in managing recorded data.

Continuous GPS Monitoring

The GPS provides continuous tracking data, whilst the system is on and can potentially be upgraded in the build stage to include geo-fencing to, for example, stop the video recording in port.

Field Testing

In July of 2020, we began the transition from various test EM camera systems to the first prototypes of HafsAuga MobileEM (Fig. 1). On most vessels, the system case was placed inside the small vessel cabin or on the outer cabin wall and each system was deployed with two cabled cameras – one looking over the side and the other looking directly down on to the sorting table. The over-the-side camera was positioned on a custom extendable mount (Bohlin Unpublished, see Suppl. material 2) to overlook the point where the net is leaving the water when being retrieved and to minimise the field of view being fully or partially obscured by the hull and railings (Fig. 2A). The second camera was positioned to overlook the sorting table where the catch is removed from the net, whilst as much as possible, minimising the inclusion of the crew in the field of view (Fig. 2B and 2C). This specific combination of camera placement balanced the desire for crew member privacy, whilst giving an overview of the critical on-board locations and minimising the risk of missing bycatch dropping out of the gillnets as they were breaking the surface. The day-to-day operation (turning on and off, cleaning lenses and replacing data storage) was handled by the commercial fishers themselves, with visits by field technicians to collect data and discuss any changes based on the fishers ongoing experience with the system and its setup.

Figure 1.  

This image shows the first prototype of the HafsAuga MobileEM. Two of these systems were first deployed in July 2020 and are still in use today. This system version includes a waterproof storage space sufficient to hold a car battery and maintain power for over a week. A plug-in screen can be added for checking camera angles and changing settings.

Figure 2.  

Images obtained from electronic monitoring with HafsAuga MobileEM on small-scale Swedish gillnet vessels show (A) a bycaught harbour porpoise (Phocoena phocoena) recorded on the over-the-side camera, (B) a great cormorant (Phalacrocorax carbo) and (C) a harbour seal (Phoca vitulina) recorded on the sorting table camera. The standard HafsAuga MobileEM setup comprises two cameras; however, the system is capable of supporting additional cameras, which can be included during the build phase if needed.

The first HafsAuga MobileEM prototypes were successful in collecting over 100 days of bycatch data from three vessels in the first year of their deployment (Fig. 3). Commercial fisher and field staff feedback on these systems gave rise to a second set of prototypes, with some modifications, that were added to the programme in 2021 to 2022. For example, on vessels that were able to power the EM system, the removal of battery storage space in the first system housings would save significant space. The second prototypes, therefore, were significantly smaller (Fig. 4A). Further, modifications on later systems added to the programme in 2022 included a more streamlined build, more easily removable data storage (plug and go solid state hard-drives), more robust cable connection options and switch to a Linux-based operating system that allows better function customisation (Fig. 4B) (see Suppl. material 1.2 for some lessons learnt). Further improvements or modifications will continue into the future as new solutions and needs arise (see the Potential Future System Improvements section for some examples).

Figure 3.  

The number of boats (A) in the Swedish small-scale gillnet fishery with mobile electronic monitoring (EM) systems on-board and the total number of fishing days filmed per year (B). From July of 2020 to July 2021, the EM systems in use were transitioned from a mixture of ad hoc systems to the HafsAuga Mobile and the LPScam systems developed at SLU.

Figure 4.  

A) The 2nd prototypes of the HafsAuga MobileEM were plug-in power variants that can be plugged into the vessel's 12v power or an external battery. Ten of these systems were built in early 2021 and all were deployed by July 2021. B) A new further streamlined and upgraded prototype was built in early 2022 and included a more easily accessible and swappable solid state hard-drive (SATA-SDD). Four of these were deployed by July 2022.

Participation in the EM programme has steadily increased since the first systems were rolled out (Fig. 3) with the main barrier to further uptake being the limited availability of EM systems, which resulted in fishers having to wait to join the programme. In 2023, the HafsAuga MobileEM system was integrated in the national data collection framework in Sweden. Since then, the system has been deployed on 20 vessels (Fig. 3) and mobile EM systems have become the primary monitoring method within the EU Data Collection Framework (DCF) for tracking protected species bycatch in Swedish small-scale gillnet fisheries.

Potential Future System Improvements

There are several areas where the HafsAuga MobileEM system itself could be further improved or adapted to suit particular needs. These include:

  1. Increased File Format Options: Adding a range of video file types that can be selected directly on the system would reduce the need for file conversion. Currently, the file output type has to be selected in the build stage and this can create additional data handling steps if other file formats are needed. For instance, our latest builds record using .ts file format and some commercial analysis programmes require MP4 or other format types. Ideally, we could select the video format on the HafsAuga system in the field in response to changing needs.
  2. Improved Power Management: Modifying the power buttons so that the "off" button powers down the system only after data has been copied to the exchangeable SATA-SSD. This would eliminate the need for an intermediate step involving a delayed power down. Additionally, in systems not powered by battery, making the system less sensitive to power changes or cuts by adding a small back-up battery or tide-over mechanism would enhance reliability.
  3. Data Upload and Recording: Implementing a feature that allows the camera recordings to be stopped and the remote data uploads to continue, would significantly enhance efficiency. Currently the systems are designed so that turning off the cameras also shuts down the systems, which pauses any backlogged remote uploads instead of allowing them to continue. Related to this, geo-fencing so the systems automatically cease recording when the vessel returns to the home harbour, but continues uploading any remaining data, would further enhance efficiency and minimise unnecessary data handling and storage.

Future Directions

In addition to any small system modifications, based on new deployment scenarios or based on feedback from ongoing field deployment outlined above, machine learning and additional novel EM underwater camera angles are areas we would like to explore further.

Machine Learning: Since early 2022, we have been working towards the semi-automation of bycatch data collected with HafsAuga Mobile using machine-learning techniques (Basheer and Muhammad 2022, Svensson 2023). We have demonstrated that it is possible to use machine learning to detect bycatch of porpoise in EM data on gillnet fisheries and that high quality footage and images, such as those collected using the HafsAuga system make the process more efficient and effective (Svensson 2023). To date, all of the machine-learning process has been undertaken remotely from the fishing vessels.

A significant cost associated with EM is the transfer and storage of data, either via manual collection or remote transfer. Reducing the size of files to be transferred would provide considerable savings and, if we are able to avoid sending raw film, there is the added benefit of not needing to transfer sensitive data (e.g. potential identification of fishers and their boats). In the future, we aim to test machine learning on site (i.e. onboard the boat in the HafsAuga system), before sending the results to a central server and we built the HafsAuga system with this future requirement in mind. We have also begun exploring the use of hierarchical federated machine-learning framework (edge learning) using local models in Sweden and Denmark to improve overall bycatch detection models (Basheer and Muhammad 2022) and are investigating the feasibility of implementing local models on HafsAuga systems onboard individual vessels.

Underwater EM Camera Angles: Bycatch often drops out of gill-nets between the water surface and the on-board net hauler (Course 2021) and EM can provide better estimates of drop-outs than on-board observers (see Kindt-Larsen (2012)). Some drop-outs also likely occur whilst the net is still under the water (Kindt-Larsen 2012) or larger animals may be released underwater at the side of a vessel (Neidig 2024) and, in both cases, the number or species of bycatch is likely to be missed by conventional EM systems. We plan to test the use of an underwater camera to record bycatch that drops out of the net before reaching the surface – either by adding another camera to the HafsAuga mobile system during the build stage and deploying on a mount similar to that developed by Neidig et al. (2024) or alternatively running a specialised underwater system such as CatchCam (SafetyNet Technologies 2024) or a generalist low cost underwater camera (e.g. Mouy (2020), Fetterplace (2023)) alongside the HafsAuga mobile system.

Conclusions

The HafsAuga MobileEM system was developed as a practical solution to the specific needs of Swedish small-scale fisheries. It offers a flexible, portable and cost-effective monitoring option. The system has evolved and continues to evolve through close collaboration amongstn researchers, manufacturers and fishers, ensuring it meets the varied requirements for capturing reliable bycatch data. The successful and ongoing deployment of a mobile EM system marks a significant step forward in Swedish small-scale fisheries bycatch monitoring. The system’s adaptability and portability offer a model that could be applied in other regions and fisheries facing similar challenges.

Acknowledgements

We thank the participating commercial small-scale fishers for their ongoing input and for making field testing, often in challenging conditions, possible. We also thank the technicians, researchers and IT staff at SLU involved with the DCF programme and fisheries bycatch research for their assistance and valuable input into the project.

Author contributions

LF: Conceptualisation; Investigation; Writing – original draft; Writing – review and editing; Formal Analysis; Methodology. EBN: Methodology; Conceptualisation; Investigation; Writing – review. KÖ: Methodology; Investigation; Writing – review and editing. FB: Methodology; Investigation; Writing – review. LS: Methodology; Conceptualisation; Investigation; Writing – review. DR: Investigation; Writing – review and editing. PL: Methodology; Investigation; Writing – review. SK: Supervision; Conceptualisation; Project Administration; Funding Acquisition; Writing – review and editing.

Conflicts of interest

The authors have declared that no competing interests exist.

References

Supplementary materials

Suppl. material 1: Additional EM system requirements, Lessons Learnt 
Authors:  Lachlan Fetterplace, Kristin Öhman, Sara Königson
Data type:  Additional information (text)
Brief description: 

Additional details on the key Electronic monitoring system requirements for Swedish Small-scale fisheries and on lessons learnt during development and prototype iterations.

Suppl. material 2: Custom Camera Mounts 
Authors:  Filip Bohlin,Lachlan Fetterplace
Data type:  Images
Brief description: 

Examples of custom extendable mounts to hold cameras on small-scale vessels.

login to comment