Jellyfish Identification Software for Underwater Laser Cameras (JTRACK)

Jelly ﬁ sh can form erratic blooms in response to seasonal and irregular changes in environmental conditions with often large, transient e ﬀ ects on local ecosystem structure as well as e ﬀ ects on several sectors of the marine and maritime economy. Early warning systems able to detect conditions for jelly ﬁ sh proliferation can enable management responses to mitigate such e ﬀ ects providing bene ﬁ t to local ecosystems and economies. We propose here the creation of a research team in response to the EU call for proposal under the European Maritime and Fisheries Fund called “Blue Labs: innovative solutions for maritime challenges”. The project will establish a BLUELAB team with a strong cross-sectorial component that will bene ﬁ t of the expertise of researchers in IT and Marine Biology, Computer Vision and embedded systems, which will work in collaboration with Industry and Policy maker to develop an early warning system using a new underwater imaging system based on Time of Flight Laser cameras. The camera will be combined to machine learning algorithm allowing autonomous early detection of jelly ﬁ sh species (e.g. polyp, ephyra and planula stages). The team will develop the system and the companion software and will demonstrate its applications in real case conditions.


1.
To establish a cross-disciplinary working team (experts and young-scientists) on underwater image acquisition and object recognition, including engineers, mathematicians, marine ecologists and economists from industry and academia (TN1).

2.
To design, develop and commercially assess efficient and reliable software (called JTRACK) for automatic detection, identification and tracking of jellyfish (ST1).

3.
To tailor JTRACK to specific stakeholder needs and to images obtained by the next generation underwater LiDAR imaging technologies (ST2).

4.
To demonstrate to selected stakeholders the jelly identification system in the field along with conventional oceanographic methods to allow for direct comparison (MO1).

5.
To identify go-to-market strategies and any additional technical requirement to commercialize the software (MO2).

6.
To effectively disseminate project results to the large scale marine and maritime stakeholders (MO3).
Other goals Blue Lab. The interaction within the established Blue team will produce research development and innovation with long-lasting benefit at regional and trans-national scales. The team will have regular (weekly and monthly) meetings (including teleconferences) and it will be composed by: 1. a selected group of post-graduate scientists coming from DTU (DK), UniPARTH (IT) and GEOMAR (GER); 2.
senior researchers from the involved Universities; 3.
industrial tutors from BIORAS (DK) and advisors from SINTEF (NO).
An internship program with short-term mobility for students will be established at the beginning of the program and will involve all Partners; MSc projects linked to action will

Contribute to the objectives set in the Call for Proposals
The proposal contributes to all the relevant objectives set in the call text Table 1.

Scope as defined in the call for proposals Objectives and specific activities in JTRACK
Priorities … 4. Invasive alien species and jelly fish proliferation:… Obj. ST1 Design, develop and commercially assess an efficient and reliable software (called JTRACK) for automatic detection, identification and tracking of jellyfish Work Packages 2,3 and 4 …this action aims at promoting innovative "laboratories" (hereinafter called Blue Labs) to pilot new and viable solutions addressing selected maritime and marine challenges and opportunities in the blue economy. The focus of this action is to support a novel way of working, where young scientists supported by researchers, industry and local stakeholders, team up to develop innovative solutions to support the development of a sustainable blue economy, while preserving marine resources and ecosystems.
Obj. TN1 Establish a cross-disciplinary working team (experts and youngscientists) on underwater image acquisition and object recognition, including engineers, mathematicians, marine ecologists and economists from industry and academia Work Package 1 -task 1.2 The Blue Labs concept works at the interface between research and commercial exploitation so that results from research are taken forward to the market. By doing so, it aims at contributing to generate jobs and business opportunities in the blue economy Obj. MO2 Identify go-to-market strategies and any additional technical requirement to commercialize the software Work Package 4 -task 4.3 Table 1.
Comparison of call scope vs. JTRACK objectives and project design.
Jellyfish Identification Software for Underwater Laser Cameras (JTRACK)

Scope as defined in the call for proposals
Objectives and specific activities in JTRACK leveraging young peoples' skills and creativity and increasing awareness of marine challenges and opportunities Obj. TN1 as above, and Obj. MO1 : Demonstrate to selected stakeholders the jelly identification system (camera and JTRACK) in the field Work task 4.2 supporting pioneering partnerships between maritime stakeholders and fostering multidisciplinary approaches by combining competences from businesses, the public sector and research bodies    and dramatic changes to the local food web (Daskalov et al. 2007). Jellyfish can cause direct damages to tourism, infrastructure and fishing gears ) and have impacts on food production, as jellyfish and forage fish (small pelagic fish) feed at similar trophic level (Azaña Schnedler-Meyer et al. 2016). Hence, jellyfish proliferation is a common concern in all the European seas and detection and management of these species are acknowledged in the present call text as priority theme (priority 4). Jellyfish management is identified as one of the main sub-objectives to maintain a "Rich and healthy wildlife" of the Baltic as indicated in the European Union Strategy for the Baltic Sea Region (COM 2015). Quantitative systematic data on the species are largely lacking since sampling methods for jellyfish require filtration of large water volumes as well as analyses of the samples right after the catch since most species are either too large or do not withstand conventional preservation methods (Engell-Sørensen et al. 2009). An image based quantification system would circumvent the current constrain on high quality data acquisition for small and large jellyfish and will significantly contribute to main policy areas (MSFD and CFP) providing an early-detection system for these species. Jellies and mussels farming: high concentrations of jellies could lower the production of mussels through reducing zooplankton abundance allowing phytoplankton to proliferate to harmful levels including oxygen depletion in near bottom areas, as has been shown due to jellyfish blooms in Limfjorden, Denmark (Riisgård et al. 2015). In such cases, it is relevant to move the mussels to more favorable places. A jellyfish detection system in mussels farm may therefore alert the mussel farm producers, so they can make actions to mitigate the problem.

General description of the action
Jellies and shrimp production/fish farming: A jellyfish bloom completely damaged the Norwegian shrimp fishery in 2011 because of clogging of the fishing nets. Similarly, a large jellyfish bloom negatively impacted salmon aquaculture revenues in Ireland (Baxter et al. 2011). Detection system and species identification could prevent negative economic impacts.
Jellies and coastal management: In southeastern Sweden, one of the three reactors that make up the Oskarshamn nuclear power plant was temporary shut down for one day in 2013 as the intake pipes was felled by jellyfish. For Japanese waters it has been shown that during the 2000´s, incidences between jellyfish and power stations drastically increased compared to earlier years (Purcell 2012).

JTRACK: technical advantages.
Limitations of present-day organisms identification systems are linked to the poor imaging system, difficulties in object detection and low accuracy algorithms. Imaging quality of common underwater cameras are exposed to the common rapid changes in underwater light conditions due to water turbidity, surface waves, cloud cover, etc. These limits make the object identification challenging especially when dealing with mimetic or transparent objects (e.g., fish and jellyfish). Moreover the difficulties in sharp object detection and the little information about the object when detected (i.e., morphology and color) makes the training of the machine-learning algorithm difficult. JTRACK will focus on detection, tracking and identification solutions tailored for the next generation range-gated camera technology based on state-of-the-art "time of flight" image sensors and innovative pulsed laser illumination (LiDAR). Those new cameras provide undoubted advantages respect to regular cameras, since the autonomous light source makes them independent from local underwater light conditions. Moreover the range-gating technology is able to minimize the effects of water turbidity allowing for larger observed volumes (by factor 2 to 3 over regular cameras) making them much more suitable for early detection systems of marine species. Finally the camera can provide three-dimensional information about the observed objects including their size, shape, and speed. Species identification can then be largely improved when those additional data are integrated in the machine-learning algorithm. While ST and MO activities will be mainly implemented in Denmark (DTU, BIORAS) TN tasks will directly involve different Countries and especially Italy (UniPARTH) in dissemination and transferability of the results, Norway (SINTEF) for liasoning with other projects and advise in go-to-market strategies. Additionally, broad dissemination to the public and policy makers via open science days and demonstration of the system is planned in Denmark (DTU) and Germany (GEOMAR). During the 24 months life of the project the Blue Lab team will have dedicated physical office-space and facilities available at DTU for daily work and extended group meetings.
JTRACK: added value. The innovative software would be the first of this kind in fully exploiting the advantages of LiDAR cameras for object detection and identification. We expect to produce new solutions in machine-learning algorithm to be efficiently designed to utilize the additional imaging features provided by these cameras. The project will clearly build upon H2020 UTOFIA project that is developing state-of-the art LiDAR cameras which will be make available to develop JTRACK's activities. JTRACK will significantly expand the commercial exploitation for the UTOFIA cameras adding significant value to the project including new markets.
The working team includes experts on: computer vision, jellyfish ecology, tracking technology, imaging systems, economy and industry. Moreover the overlapping interests of many of the post-graduate students associated to the project will guarantee communications within the team to efficiently exploit the expertise available in JTRACK and reach all the goals. Students will have tutors from University (DTU, UniPARTH), research institute (GEOMAR) and Industry (BIORAS) with additional industrial partner (SINTEF) providing advice on imaging system, identification algorithm and go-to-market strategies.
The Norwegian research institution SINTEF is vital to the implementation of the project since it will give access to the UTOFIA camera (prototype and final systems), and will be able to provide advice on both imaging systems and interpretation of such images. The University of Naples "Parhtenope" will help the technological advance in WP3, and the transfer of knowledge generated in the project to other regions: both policy makers and a larger group of stakeholders during the dissemination activities. Additionally, on-going monitoring activities and free (in-kind) access to different research vessels at GEOMAR allow for testing and quantitative ground-truth the detection system throughout the project.

Expected Results
The main results of the project are: • Compilation of a new annotated training database for jellyfish identification that includes: taxonomic names, sizes and shape categories of the jellies • Release of the software JTRACK version 1 for jellyfish detection, identification and tracking with underwater time-of-flight and LiDAR image acquisition systems • An international cross-disciplinary long-lasting team between academia and industry on underwater marine species identification will be established • Establishment of a long-lasting forum for Baltic Sea marine and maritime stakeholders • Development and test of new and alternative sampling technology for delicate, nonpreservable plankton organisms The project will build upon on-going EU projects such as H2020 UTOFIA, which is developing time-of-flight imaging systems based on LiDAR and the completed project FP7 VECTORS in identifying suitable Baltic regions for jellies imaging. Results of the project will have major impacts in the implementation of the EU Strategy for the Baltic Sea Region (EUSBSR) as reported in Table 2 Technology Readiness Level (TRL) From already available basic technology on machine learning algorithm (TRL 1) during the course of the project we will research the feasibility for jellyfish identification software (TRL 2) producing a first proof of concept (JTRACK version 0; TRL 3) within 6 months of the project. The version 0 of JTRACK will then be integrated with object identification and tracking using regular cameras in experimental setup (TRL 4, Month 12). LiDAR cameras will then be used in the field to acquire images that will be processed by an extended version of the identification software (JTRACK version 1, TRL 5). Finally the system camera JTRACK will be fully integrated and the demo system demonstrated to stakeholders in relevant marine environment (TRL 6, Month 22).

Composition of the Consortium and Blue Lab Team
We have 5 applicant Organisations (DTU, GEOMAR, UniPARTH, BIORAS Aps, SINTEF ICT) from 4 different Countries (Denmark, Germany, Norway and Italy). The team has all the necessary skills and expertize to successfully implement the project providing results that will have impacts at Baltic Sea regional scale and beyond. We describe below the specific people involved in the Blue Lab Team from the different Organizations and provide in each WP (described in section "f") specific role and responsibilities. In Table 3 we provide a synthetic overview on the relevant profiles for the different Organizations and their involvement in the specific Work Packages (WP) with a general task description.
Coordinator: Dr. Patrizio Mariani is Senior Researcher at DTU Aqua and will be the project coordinator. He is an expert marine ecologist and is involved as PI in several technology projects at DTU including underwater visualization systems, tracking algorithm for fish behaviour and underwater robotics. He is chair of the Steering Committee of EUROMARINE EUROMARINE 2014), a European marine science network with 72 Member Organizations with focus on foresight science, science-governance interface and marine technology development. He is main PI in COLUMBUS and UTOFIA, member of the European Marine Board and involved in several working groups including the CIESM Table 2.
Results, impacts and indicators of the JTRACK project.
(Mediterranean Science Commission) working group on marine connectivity. He is main PI of the Centre of Ocean Life (CentreForOceanLife 2012 a centre of Excellence with currently about 25 PhDs and 10 Postdocs; he is leading a lab in Marine Ecosystems and Technology (BlueLab 2017) with 2 MSc students, 1 PhD and 1 postdoc.
P1-Technical University of Denmark (DTU) www.dtu.dk -is the leading technical university in the Nordic Countries and is among the most innovative Universities globally. DTU is characterized by research-based education at the highest international level, with close interaction between researchers, Authorities and business, as well as strategic alliances with several international partners. Two Departments at DTU will participate to this action: DTU Aqua, the National institute for Aquatic Resources that is a leading European institute in marine and maritime science and a major player for Baltic Sea fisheries; DTU Compute, the department for applied mathematics and computer science is an internationally leading research department within e.g. machine learning, computer vision and statistical image analysis. Both departments have very-long records in management of large EU projects and world class expertize in generating innovative products for public sector consultancy and to the benefit of both the industry and public sector institutions.

WP1
30% DTU Aqua will coordinate the project and will provide logistic and IT support. The coordinator will establish the management body of the project, will organize the meetings and communicate with EC to report on activities; together with the Steering Committee will implement specific actions (e.g. mobility plan) and will manage risks. DTU Aqua will also be responsible of setting up the website and its administration.

WP2
10% DTU Aqua will support the field and laboratory activities for imaging acquisition and compilation of the training database. DTU (both Aqua and Compute) will be responsible for the testing of the complete demo system.

WP3
30% DTU Compute will lead the final software development and interact with UniPARTH to optimize the machine-learning algorithm for the LiDAR camera.

WP4
30% DTU Aqua (supported by Compute) will guarantee coordination with other on-going EU projects (namely: COLUMBUS, UTOFIA) and marine science networks (e.g. EUROMARINE) and coordinate the outreach and dissemination activities supporting the stakeholder training with the complete demo system. Table 3.
Partners' description, role and specific tasks in the JTRACK.
Jellyfish Identification Software for Underwater Laser Cameras (JTRACK) P2-GEOMAR www.geomar.de -is the Helmholtz Centre for Ocean Research and has an excellence expertise in ocean research within the Baltic Sea -as exemplified by its coordination of the Bonus EU Project BIO-C3. GEOMAR covers all aspects of ocean science from chemical, physical and biological oceanography to geology as well as ocean atmosphere interactions. With this breadth of research, GEOMAR is unique in Germany, combining coastal ecosystems with blue water research. It has an annual budget of ca. 72 Mio. EUR, ca. 950 employees and 4 research vessels. The personnel contributing to JTRACK include the post-graduate student Dr. Cornelia Jaspers, who is a world known expert on gelatinous zooplankton and jellyfish ecology and Dr. Catriona Clemmesen-Bockelmann, expert on fish larvae ecology and the Kiel Fjord system and will provide valuable background information for JTRACK.

WP Time Description of work
WP2 60% GEOMAR will lead all the laboratory and field activities to perform jellyfish imaging. GEOMAR will provide shipof opportunities and all the knowledge of jellyfish taxonomy and ecology to compile the training database.

WP3
10% GEOMAR contributes to jellyfish identification WP4 30% GEOMAR will participate to all the dissemination and outreach activities providing link to their network of stakeholders in the Baltic Sea Region, providing content for e-news and website and disseminating project results in conferences and meetings. Moreover GEOMAR will lead the stakeholder training activities providing ship of opportunities and coordination with the "Future Ocean" cluster in Germany.
P3-BIORAS Aps www.bioras.com -is a Danish software development company, with many years of experience in integrating instrumentation and optical systems. The company supplies advanced specialized digital image analysis solutions for analysis of still images and video sequences in 2D or 3D. Bioras develops laboratory equipment and monitoring systems: FishGuard system for monitoring fish behavior in commercial aquaculture plants; BallastWISE for quantifying the number of living micro-organisms in water samples from ballast water tanks for the shipping industry; an automatic plate counter, branded as PetrilyzerTM. Bioras is involved in a range of development projects, and has established an extensive network of scientific and industrial collaborators. Bioras has provided training and internship opportunities for a number of students and graduates.

WP Time Description of work
WP1 15% BIORAS will be the main industrial tutor for post-graduate students and help in defining the mobility plan.
WP2 15% BIORAS will support the final demo system test in the field.
WP3 30% BIORAS will contribute in the development of the object detection and tracking.
WP4 40% BIORAS will participate to the stakeholder training and will lead the task identifying a go-to-market strategy including a business model and business plan which can guarantee the future sustainability of JTRACK.
P4-University of Naples "Parthenope" (UniPARTH) www.uniparthenope.it -The Science and Technology Department of University of Naples "Parthenope" (UniPARTH) develops activity of basic and applied research for the knowledge advancement, for new methodologies and tools definition and the development of meaningful applications in the sectors included in the following research areas: Computer science, Physical Sciences, Earth Sciences, Geodetic Sciences, Maritime and Aerial Navigation Sciences and technologies. UniPARTH is active in the base and applied research for the disciplines of Computational Mathematics, Artificial Vision and Forms Recognition and Classification, Images Elaboration and in the associated education.

WP Time Description of work
WP3 60% UniPARTH will lead WP3 and drive design and implementation of the JTRACK algorithm including testing different methodologies for segmentation and tracking of the objects.
WP4 40% UniPARTH will contribute to the dissemination of the results interacting with the Mediterranean Region networks, organizing a stakeholder meeting and contributing to website and news digest.
P5-SINTEF www.sintef.no -SINTEF is Scandinavia's largest independent research organisation, and is a nonprofit research foundation. We employ 2100 people and more than 90% of our annual turnover derives from contract research for industry and the public sector. We receive minimal state funding (around 6%). SINTEF is multidisciplinary, with international top-level expertise in a wide range of technological and scientific disciplines, medicine and the social sciences. Our company vision is "technology for a better society", and it is an important aspect of our societal role to contribute to the creation of more jobs. SINTEF has 20+ years of experience developing optical measurement systems for industrial problems and interpreting the data from these sensor systems. They have worked with core UTOFIA technology -Time-of-Flight sensors -for the past ten years, with a particular focus on characterisation and usage towards robotics.

WP Time Description of work
WP2 10% SINTEF will through UTOFIA provide the LiDAR camera for WP2 work.
WP3 30% SINTEF will advice on best practice for integration of software and LiDAR camera system. WP4 60% SINTEF will advice in producing a business plan for JTRACK coherent with LiDAR camera market potential; will participate to the kickoff and final meeting and contribute to the dissemination of the results.

Added value
The Blue Lab established in this project is an international and interdisciplinary team providing innovation via research and technology-development by designing, testing and demonstrating a jellyfish detection systems, which will have direct impacts across several Blue Growth Sectors*1: monitoring and observation, marine environment, coastal tourism, aquaculture, fisheries, maritime transport. As described above we can group the activities in JTRACK around three main themes: Scientific and Technical (ST); Training and Networking (TN); Market and Outreach (MO).
-Ecological ST activities include: jellyfish ecology, execute laboratory-and field-work.
All Partners are main world lead players in marine and maritime sectors. There is a strong regional component in our consortium with DTU and GEOMAR being main actors in the Baltic Sea Regional strategies. Moreover the transnational component in the Consortium is essential for the efficient execution of ST and TN activities. The composition of the Consortium guarantees proper capacity building and training opportunities for all the postgraduate students participating to the project. As formation of students is at the centre of JTRACK, students will be main players in stakeholder management and interactions, as well as the main engine for the ST aspects. BIORAS participation in tutoring and marketing strategies will give both the commercial dimension to the students (which is nonetheless already traditionally in focus at DTU) and provide the best possible framework for efficient commercialization of the software after the end of the project since it matches the core activities of this SME.

RISK MANAGEMENT PLAN:
Major technical risks connected to the individual work packages and phases of work have been identified in the course of project preparation. Risks listed have been selected based on the probability and consequence for the project; only high-probability and high-consequence risks are listed. This has been summarized in Table 4.

1: Administration, self-assessment and risk management (M1 -M24).
Ensure the distribution of responsibilities and the flow of information and to provide the technical and organisational guidance to the project and coordinate the research efforts. Delivery of reports and deliverables in time and in the expected quality. Establish the management structure and schedule central meetings (kickoff, midterm and final) to get feedback on the project planning and progress to make sure the project will give maximum impact on marine science and technology. Establish a clear conflict management to ensure fast and acceptable conflict resolution while reducing the risks of escalations of disputes. Do regular assessment and the analysis of potential risks to prepare corrective actions.
Organisational structure Several management bodies will be implemented in the project Table 5. In particular, a Steering Committee (SC) will be established to help monitor the overall technical progress and to ensure that the decisions are taken on a sound technical and scientific basis. The risk management plan defined in point e) of this proposal will guide discussions about possible mitigation actions, which are taken by the SC. WP leaders and a representative of the post-graduate students compose the SC and its decision taken with verbal consensus or explicit majority vote when needed. A procedure for internal review of each deliverable is established with responsible for review indicated along the deliverables. A support team will help the Coordinator (Dr. Patrizio Mariani) in the management of the project. The support team will be composed by personnel at DTU Aqua: Ivo Grigorov from the Research Office, Lilian Andersen from the secretariat and Dr. Thomas Lindeberg Thørgersen, who has a formation in economy, he is post-graduate students at DTU Aqua and will be associated to JTRACK. He has experience in project management, stakeholder management (node leader in COLUMBUS) and will assist in the management of JTRACK.

Internal communication.
Kick-off (Month 3) mid-term (Month 12) and final meeting (Month 22) will be organized by the Project Management team. Regular monthly meetings with relevant partners will be scheduled as needed. The Project Management team will organize weekly Blue Lab meetings around core activities and will compile minutes for each meeting and make them available on the webpage.

Responsibility level Management Body Composition Main Responsibilities
Task Task leader One person from Partners · Execute the actions described in the task · Report to the responsible WP leader risks and impediments in the execution of the task  M24). Implement weekly meetings of the working team and organize the SC monthly meetings. Establish the stakeholder forum and provide methods for stakeholder engagement on a regular basis to guide stakeholder interaction in WP4 and technical development in WP3. Support tutoring of the postgraduate and their interactions with the industrial tutor, internal advisors and stakeholders, including entrepreneurship training provided by FOUNDERS A/S (please refer to Letter of Sponsors). Deliver a plan -agreed among all involved Partners -and monitor the internship and mobility program for post-graduate students, which will constitute the basis for a preliminary Memorandum of Understanding between partners for long term mobility.

Task 1.3: Creation and maintenance of the website (M1 -M24).
Design and develop a webpage for the BlueLab team with dedicated sections for partners contributes (e.g., a blog) and ensure each participants, advisors and stakeholder to contribute to the content including publication of: ideas, results, data, test cases, demo. The task responsible will also publish bi-monthly electronic newsletters to update on project status. All generated image data by this project will be openly accessible via our homepage and stored in PANGAEA to ensure long-term accessibility.

Role of Partners:
DTU will be the Coordinator of JTRACK and will lead this WP and be responsible for administration of resources, funding and effort put in the project, coordination of reporting to the EC (deliverables, milestones, periodic reports). This includes the setup and maintenance of IT services like Web Content Management systems (CMS), internal wiki pages, and Mailing list Services. DTU will also be in charge of the scientific leadership and is therefore responsible for the guidance and controlling of technical effort, reaching the targeted objectives and performance of partners. Most of the technical coordination will be carried out as part of work in WP3, DTU work in WP1 will only relate to overall technical management.
BIORAS will be the main industrial tutor for the post-graduate students in the working team and act as facilitators to stakeholder meetings. BIORAS will provide support in the technical coordination (in WP3) and technical management of the project.

Scientific and Technical, Training and Networking
Objectives: • Perform testing of the imaging system under controlled laboratory conditions • Establish a database on jellyfish imaging and features from the Baltic Sea Region • Quantitative comparison between camera and conventional sampling methods

Description of Work:
Task 2.1: System testing and parameters definition (M1 -M6). Design and perform laboratory experiments where different jellyfish groups (with regard to shape and size) will be observed and analysed under controlled conditions. Classical cameras e.g., GoPro and machine vision cameras in underwater housings, and a LiDAR camera will be employed simultaneously to allow for direct comparison of different technologies along with actual size and shape measurements of the organisms. Further, different turbidity levels will be simulated to compare cameras' performances. Images and data will contribute to the training database developed in Task 2.2 and will define parameters range for jellyfish observations by the LiDAR camera. Experiments will be supported by the National Aquarium in Denmark "Den BlåPlanet" that is a sponsor of JTRACK (please ref. Letter of Sponsors) and provide in-kind free access to their facilities.

Task 2.2: Imaging acquisition and database compilation (M3 -M24).
Weekly or biweekly monitoring cruises in the Kiel Bight will be used to gather data from spring to late autumn. Data acquisition will consist of regular cameras and LiDAR type camera systems along with classical net borne methods for inter-calibration and method comparison. We envision to follow the population development of the common moon jellyfish (Aurelia aurita) (from early stage to adult stages) as well as detecting first occurrence and population dynamics of the invasive comb jelly Mnemiopsis leidyi, which covers a size range from 1 to 60 mm. Additionally other jellyfish that might occur will be imaged and annotated with taxonomic names, size and shape categories. Depending on inflow events of higher saline waters from the Kattegat potential species present in the area are i.e. Cyanea capillata, Obelia octopunctata, Sarsia tubulosa. Further, to have a large cover of the Baltic Region, cruises to offshore waters in the central Baltic Basins will be performed in March, April, May and September of the second year (2018) collecting images both at night and during daylight. An annotated database to be used in WP3 will be compiled. Previous sampling showed abundance of jelly species in the order of 50 per cubic meter in the area. Hence we expect to collect on the order of tens of thousands of images which will be included in the annotated training database.

Task 2.3: Field test of a complete demo system (M18 -M24).
The final JTRACK demo will be tested in a workshop organized back-to-back with the stakeholder training session in WP4 (Task 4.2). Field test will be conducted using the R/V Littorina and using the demosystem mounted and deployed on available ROV. We will test the suitability of the system in contrasting ecosystems i.e., turbid coastal waters (Kiel Fjord) in comparison to more offshore waters (Arkona Basin). The system will be used in combination with regular sampling and conventional underwater cameras to establish level of accuracy of jellyfish detection.

Role of Partners:
GEOMAR will lead this WP and ensure that laboratory and field data will be gathered and is responsible for the quality control of jellyfish identification. This includes the test of the camera and laser system in the field as well as in the laboratory under controlled conditions with cultured jellyfish species of different size and shape. GEOMAR will arrange the workshop and ensure that all Partners will have access to the gathered data and training sets that will be compiled. GEOMAR will give advice on the relevant biology, taxonomy and ecology of jellyfish and will offer ship of-opportunities.
BIORAS will aid in expertise regarding acquisition of real time video using proprietary underwater housings together with camera-and optical configurations.
DTU will assist the experimental studies at "Den BlåPlanet" managing the logistic and will ensure that the data collection is suitable for the algorithm development implemented in WP3. DTU will be main responsible for testing the final demo-system SINTEF will provide access to the LiDAR cameras for laboratory and field-work. Monitoring the development of jellyfish populations from image data requires a method for quantifying the jellyfish in the recorded images and a model that relates this quantity to the population. Detection of jellyfish and identification of their species from image data typically involves detection of individual jellyfish, segmenting them to measure their size and shape, tracking individuals through consecutive frames, and classifying them to species based on image features. The initial method development will be based on images of fish from 1) LiDAR and 2) regular underwater cameras, which have been recorded in 2016 while images containing jellyfish will first be available in the project in early 2017. This initial model development will allow gaining experience on segmentation, size measurements, tracking and feature extraction. This analysis will provide information about the sampling possibilities with the system, i.e. what sizes that can be measured at given depths. The depth measurements obtained by LiDAR provides a very good basis for segmentation and accurate size measurements. Features of both desirable objects and non-desirable objects will be assimilated for classification using feature learning mechanisms based on deep neural networks, capable of reaching high identification percentages in most computer vision tasks, and in particular Deep Belief Networks.

Task 3.2: Development and test of minimum viable product (JTRACK version 0) (M6-M15).
Based on experience gained in Task 3.1 a plan for data collection will be coordinated with Task 2.2 in order to obtain data for model development including both training and test data. The design of the algorithm will be based on the acquired data, where image features including shape, texture and size features will be extracted. Tracking can be used to extract movement patterns and for offering optimal frames and subsequences for feature extraction. All these features will be used in a classification method trained from the image data collected from regular cameras (early stage) and when possible from LiDAR imaging under controlled conditions. Reference data to train and verify the jellyfish classification will be based on expert annotations. Relating the measurements obtained from image data to population data requires an additional model. Reference data for the population model will be obtained from field-sampling. Robustness of the chosen algorithm will be verified through images acquired both from laboratory and data acquired from the sea. Tracking objects is challenging because the system must deal with changes of appearance, illuminations, occlusions, out-of-plane rotations and real-time processing requirements. The main challenge is the difficulty to handle the appearance changes of the target object. The appearance changes can be caused by intrinsic changes such as pose, scale and shape variation and by extrinsic changes such as illumination, camera motion, camera viewpoint, and occlusions. A preliminary but fully functional version of the software JTRACK version 0 will be presented at the International ASLO conference in February 2018.

Task 3.3: Operational version of the model and assessment in operation (JTRACK version 1) (M14-M24).
Bringing an algorithm from an experimental stage to operation typically requires modifications and adjustments. This can both be caused by change in the characteristics of the data and in the requirements of the requests by the user of the equipment. The purpose of this task is to adjust and optimize the chosen algorithm to meet these changes and requests. This involves an implementation for real-time analysis of the acquired image data, and a reporting module that allows easy interpretation of the measurements including quantity and a possibility for an uncertainty estimate. This will be specifically tailored for the needs of the stakeholders and therefore developed in close collaboration with the end-users. In particular we will follow advice coming from task 4.2 and after the first stakeholder meeting (Month 12).

Role of Partners:
UniPARTH will lead this WP and be the main responsible for the initial algorithm development providing an early proof of concept that will be further developed at DTU. UniPARTH will then advice on system developing for JTRACK version 0 and JTRACK version 1 assisting post-graduate students work and providing specific capacity building and training to them.
DTU will be responsible for the release of JTRACK algorithm version 0 and version 1. Will guide post-graduate students' work and design, test and optimize machine-learning algorithm for detection, tracking and identification.
BIORAS will provide advice on all steps of the technical development. They will be primary responsible for algorithm development for regular camera images and will provide totoring to all post-graduate students.
GEOMAR will provide expert annotation on images and provide guidance and advise on aspects of jellyfish ecology relevant to algorithm development (e.g., shape modification, drifting with currents, etc.) SINTEF will advice on the integration of the identification software and LiDAR camera settings.

Market and Outreach, Training and Networking
Objectives: • Promote and support dissemination activities of the project results and generate the content for internal communication flow and webpage • Identify best go-to-market strategies for JTRACK

Description of Work:
Task 4.1: Outreach and international awareness raising (M1 -M24). Information and awareness raising activities regarding the project to the general public and stakeholders include (see also Table 5): participation to conferences (e.g. ASLO meeting 2018 see task 3.2), presentation of the technology to science forum (e.g. EUROMARINE, Marine Board), liasoning with H2020 COLUMBUS project (Knowledge transfer CSA) and engaging the "Future Ocean" cluster in Germany. A dissemination plan will be prepared at Month 3 by the task leader and updated at Month 15 of the project. These activities with stakeholders will be mainly developed by the post-graduate students of the group in order to promote acquisition of transferable skills in communication and to expose them to the challenges and opportunities present in marine and maritime sectors. A post-graduate will be also responsible of updating the e-news and blog sections of the Blue Lab website developed in WP1. . Early (JTRACK version 0) and final (JTRACK version 1) versions of JTRACK software will be presented to stakeholders in dedicated meeting at month 12 and Month 22. The early meeting will collect feedbacks on technical aspects (e.g. graphical user interface, identification system, etc) following guidelines provided in task 1.2 and to support WP3 algorithm development in task 3.3. The final JTRACK version 1 demo will be presented to stakeholders in a workshop and training session organized in the last quarter of the project (e.g., October-November 2018). We envision conducting a 3-day event with the first day dedicated to equipment preparation and software testing while two days organized around a training session with invited stakeholders and other scientists. The training session will introduce to system deployment and operation (e.g., using a ROV) and data acquisition and processing. A day cruises with the R/V Littorina, will allow the participation of ca. 25 stakeholders from different groups that will be invited to join. We will demonstrate the suitability of the system in contrasting ecosystems i.e., turbid coastal waters (Kiel Fjord) in comparison to more offshore waters (Arkona Basin). Post-graduate students will organize the event introduce the system to stakeholders and demonstrate the functionality of the system.

Deliverables: D4.4
Responsible: GEOMAR, Partners DTU, BIORAS, UniPARTH Task 4.3: Evaluation of go-to-market strategies for JTRACK (M3 -M24). The leader of this task will act as Exploitation Manager and during the life of the project will implement activities to generate a comprehensive business model and business plan agreed among all Partners and including the following elements: Universities and research institutions and government bodies and agencies (environmental agencies, public marine labs, Navies, civil security, Coast guards, etc.) are expected to purchase through public call for tenders. In order to achieve an efficient market introduction and a global distributor network, we are planning to contact distributors specialized in underwater technology and marine survey equipment with a global network, towards the end of the project when we have fully functional prototype and the patentability has been clarified.
Examples of potential distributors might be: Eiva A/S (www.eiva.com), a Danish company specialized in marine subsea engineering and marine survey equipment; McArtney (http:// www.macartney.com) a global supplier of underwater technology specializing in design, manufacture, sales and service of a wide range of systems to offshore operators, ocean science institute).

JTRACK business plan.
A business plan will be written based the IP management plan, market potential analysis and the go to market strategy. If a distributor has been identified, the distributor will be invited to contribute to the marketing section of the business plan.

Role of Partners:
BIORAS will coordinate this work package and ensure delivery and reporting being on time. Moreover, BIORAS will lead task 4.3 identifying during the duration of the project all the elements and ideas contributing to the definition of the business model and business plan. BIORAS will also contribute to the communication and dissemination activities and demonstration and training of stakeholders.
DTU will lead the dissemination and communication activates providing overview of the opportunities and facilitating interactions with on-going EU projects (e.g., COLUMBUS) and marine science networks (e.g., EUROMARINE) as well as motivating post-graduate students and the entire Blue Lab team to provide content for the webpage. DTU will also participate to the identification of business model in task 4.3.
UniPARTH will contribute to dissemination and communication activities providing access to stakeholders from the Mediterranean Sea Region. They will organize the first stakeholder meeting in Month 12 in task 4.2 and will contribute to all aspect of task 4.3.
GEOMAR will be the main leader for stakeholder interactions in task 4.2, providing access and facilities for the demonstration and training. They will be responsible of the organization of the final stakeholder meeting and contribute to all communication and dissemination activities in task 4.1.
SINTEF will advice on an effective business plan for JTRACK to be coherent with present market potential of LiDAR cameras and contribute to dissemination and communiation activities.

Timeline
GANTT chart with task number and name in each WP, months duration since the start of the project, duration of the task (in blue) and deliverables number (in green with the number).