Research Ideas and Outcomes :
Grant Proposal
|
Corresponding author: Patrizio Mariani (pat@aqua.dtu.dk)
Received: 27 Feb 2018 | Published: 28 Feb 2018
© 2018 Patrizio Mariani
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Citation:
Mariani P (2018) Jellyfish Identification Software for Underwater Laser Cameras (JTRACK). Research Ideas and Outcomes 4: e24716. https://doi.org/10.3897/rio.4.e24716
|
Jellyfish can form erratic blooms in response to seasonal and irregular changes in environmental conditions with often large, transient effects on local ecosystem structure as well as effects on several sectors of the marine and maritime economy. Early warning systems able to detect conditions for jelly fish proliferation can enable management responses to mitigate such effects providing benefit to local ecosystems and economies. We propose here the creation of a research team in response to the EU call for proposal under the European Maritime and Fisheries Fund called “Blue Labs: innovative solutions for maritime challenges”. The project will establish a BLUELAB team with a strong cross-sectorial component that will benefit of the expertise of researchers in IT and Marine Biology, Computer Vision and embedded systems, which will work in collaboration with Industry and Policy maker to develop an early warning system using a new underwater imaging system based on Time of Flight Laser cameras. The camera will be combined to machine learning algorithm allowing autonomous early detection of jellyfish species (e.g. polyp, ephyra and planula stages). The team will develop the system and the companion software and will demonstrate its applications in real case conditions.
Machine Learning, Lidar, Species Identification, Computer Vision
The overall aim of the project is to enable automatic jellyfish species identification and recognition necessary for early-detection of harmful proliferation of jellyfish and the general management of alien species, through exploiting state-of-the-art underwater imaging technologies based on innovative pulsed laser illumination (LiDAR).
This goal will be reached through six distinct and achievable objectives in terms of Scientific and Technical Excellence (ST), Training and Networking Excellence (TN) and Market and Outreach Excellence (MO). Those objectives are:
To establish a cross-disciplinary working team (experts and young-scientists) on underwater image acquisition and object recognition, including engineers, mathematicians, marine ecologists and economists from industry and academia (TN1).
To design, develop and commercially assess efficient and reliable software (called JTRACK) for automatic detection, identification and tracking of jellyfish (ST1).
To tailor JTRACK to specific stakeholder needs and to images obtained by the next generation underwater LiDAR imaging technologies (ST2).
To demonstrate to selected stakeholders the jelly identification system in the field along with conventional oceanographic methods to allow for direct comparison (MO1).
To identify go-to-market strategies and any additional technical requirement to commercialize the software (MO2).
To effectively disseminate project results to the large scale marine and maritime stakeholders (MO3).
Other goals
Blue Lab. The interaction within the established Blue team will produce research development and innovation with long-lasting benefit at regional and trans-national scales. The team will have regular (weekly and monthly) meetings (including teleconferences) and it will be composed by:
An internship program with short-term mobility for students will be established at the beginning of the program and will involve all Partners; MSc projects linked to action will alos be supported during the life of the project. Postgraduate students will allow knowledge transfer and capacity building within the consortium.
Regional approach, cooperation and coordination amongst marine stakeholders. Preliminary contacts with stakeholders have been established and specific needs have been already identified. Some of the stakeholders will also sponsor the action (please refer to the sponsor letters). One-to-one tailored meetings with relevant stakeholders will be planned during the lifetime of the project and a larger forum for a final general assembly meeting will be established, supporting the EU Strategy for the Baltic Sea Region by enhancing the coherence of the cooperation within the region.
Exploiting previous results. JTRACK will build upon an on-going project and a previous project, namely H2020 UTOFIA (
The proposal contributes to all the relevant objectives set in the call text Table
Scope as defined in the call for proposals |
Objectives and specific activities in JTRACK |
Priorities … 4. Invasive alien species and jelly fish proliferation:… |
Obj. ST1 Design, develop and commercially assess an efficient and reliable software (called JTRACK) for automatic detection, identification and tracking of jellyfish Work Packages 2, 3 and 4 |
…this action aims at promoting innovative "laboratories" (hereinafter called Blue Labs) to pilot new and viable solutions addressing selected maritime and marine challenges and opportunities in the blue economy. The focus of this action is to support a novel way of working, where young scientists supported by researchers, industry and local stakeholders, team up to develop innovative solutions to support the development of a sustainable blue economy, while preserving marine resources and ecosystems. |
Obj. TN1 Establish a cross-disciplinary working team (experts and young-scientists) on underwater image acquisition and object recognition, including engineers, mathematicians, marine ecologists and economists from industry and academia Work Package 1 – task 1.2 |
The Blue Labs concept works at the interface between research and commercial exploitation so that results from research are taken forward to the market. By doing so, it aims at contributing to generate jobs and business opportunities in the blue economy |
Obj. MO2 Identify go-to-market strategies and any additional technical requirement to commercialize the software Work Package 4 – task 4.3 |
leveraging young peoples’ skills and creativity and increasing awareness of marine challenges and opportunities |
Obj. TN1 as above, and Obj. MO1 : Demonstrate to selected stakeholders the jelly identification system (camera and JTRACK) in the field Work Package 4 – task 4.1, task 4.2 |
supporting pioneering partnerships between maritime stakeholders and fostering multidisciplinary approaches by combining competences from businesses, the public sector and research bodies |
Obj. TN1 and Obj. MO1 as above and Obj. MO3 Effectively disseminate project results to the large scale marine and maritime stakeholders Work Package 1 – task 1.2 |
supporting enhanced cooperation and coordination amongst maritime stakeholders at local and sea basin levels (including at sub-region level of the sea basin), whilst capitalising on local stakeholders’ knowledge |
Obj. MO1 and Obj. MO3 as above and Obj. ST2 Tailor JTRACK to specific stakeholder needs and to images obtained by the next generation underwater LiDAR imaging technologies Work Package 1 – task 1.2 |
the "Blue Labs" action under this call for proposals, aims to build on the results of relevant projects and support innovative and multidisciplinary approaches that can help move those results forward closer to the commercial stage |
Obj. TN1 and Obj. MO2 Identify go-to-market strategies and any additional technical requirement to commercialize the software Work Package 4 – task 4.3 |
State-of-the-art. Within the variety of life forms populating the oceans jellyfish are certainly among the most intriguing because of their inflated body plan, diverse life cycles, and ecological role. During the last few decades, this species has raised public concerns about their impact on our societies (
Knowledge gaps and opportunities. Present-day technologies are unable to provide reliable early detection systems of jellyfish invasion in most real-life situations. Indeed there is a need to develop fast-accurate and reliable identification system for marine species as indicated by a dedicated FAO working group (
Jellies and mussels farming: high concentrations of jellies could lower the production of mussels through reducing zooplankton abundance allowing phytoplankton to proliferate to harmful levels including oxygen depletion in near bottom areas, as has been shown due to jellyfish blooms in Limfjorden, Denmark (
Jellies and shrimp production/fish farming: A jellyfish bloom completely damaged the Norwegian shrimp fishery in 2011 because of clogging of the fishing nets. Similarly, a large jellyfish bloom negatively impacted salmon aquaculture revenues in Ireland (
Jellies and coastal management: In southeastern Sweden, one of the three reactors that make up the Oskarshamn nuclear power plant was temporary shut down for one day in 2013 as the intake pipes was felled by jellyfish. For Japanese waters it has been shown that during the 2000´s, incidences between jellyfish and power stations drastically increased compared to earlier years (
JTRACK: technical advantages. Limitations of present-day organisms identification systems are linked to the poor imaging system, difficulties in object detection and low accuracy algorithms. Imaging quality of common underwater cameras are exposed to the common rapid changes in underwater light conditions due to water turbidity, surface waves, cloud cover, etc. These limits make the object identification challenging especially when dealing with mimetic or transparent objects (e.g., fish and jellyfish). Moreover the difficulties in sharp object detection and the little information about the object when detected (i.e., morphology and color) makes the training of the machine-learning algorithm difficult. JTRACK will focus on detection, tracking and identification solutions tailored for the next generation range-gated camera technology based on state-of-the-art “time of flight” image sensors and innovative pulsed laser illumination (LiDAR). Those new cameras provide undoubted advantages respect to regular cameras, since the autonomous light source makes them independent from local underwater light conditions. Moreover the range-gating technology is able to minimize the effects of water turbidity allowing for larger observed volumes (by factor 2 to 3 over regular cameras) making them much more suitable for early detection systems of marine species. Finally the camera can provide three-dimensional information about the observed objects including their size, shape, and speed. Species identification can then be largely improved when those additional data are integrated in the machine-learning algorithm.
JTRACK: project activities. The project is developed along different activities:
Scientific and Technical (ST) – (1) Design and execute laboratory and field work for jellyfish imaging. (2) Develop, test and apply software for jellyfish species detection, tracking, identification and sizing. (3) Apply a demo JTRACK system (camera + software) in the field (see WP 2 and 3).
Training and Networking (TN) – (1) Postgraduate training in: ST aspects, stakeholder management. (2) Training of stakeholders using the JTRACK system. (3) Facilitate transferability of project results to the other regions. (4) Identify synergies with relevant on-going EU H2020 projects (e.g.,
Market and Outreach (MO) – (1) define go-to-market strategy including a detailed business plan. (2) Plan actions to ensure project sustainability after the end of JTRACK. (3) Dissemination actions (website, etc.) and build-upon COLUMBUS team activities and the “Future Ocean” cluster in Germany (see WP 4).
While ST and MO activities will be mainly implemented in Denmark (DTU, BIORAS) TN tasks will directly involve different Countries and especially Italy (UniPARTH) in dissemination and transferability of the results, Norway (SINTEF) for liasoning with other projects and advise in go-to-market strategies. Additionally, broad dissemination to the public and policy makers via open science days and demonstration of the system is planned in Denmark (DTU) and Germany (GEOMAR). During the 24 months life of the project the Blue Lab team will have dedicated physical office-space and facilities available at DTU for daily work and extended group meetings.
JTRACK: added value. The innovative software would be the first of this kind in fully exploiting the advantages of LiDAR cameras for object detection and identification. We expect to produce new solutions in machine-learning algorithm to be efficiently designed to utilize the additional imaging features provided by these cameras. The project will clearly build upon H2020 UTOFIA project that is developing state-of-the art LiDAR cameras which will be make available to develop JTRACK’s activities. JTRACK will significantly expand the commercial exploitation for the UTOFIA cameras adding significant value to the project including new markets.
The working team includes experts on: computer vision, jellyfish ecology, tracking technology, imaging systems, economy and industry. Moreover the overlapping interests of many of the post-graduate students associated to the project will guarantee communications within the team to efficiently exploit the expertise available in JTRACK and reach all the goals. Students will have tutors from University (DTU, UniPARTH), research institute (GEOMAR) and Industry (BIORAS) with additional industrial partner (SINTEF) providing advice on imaging system, identification algorithm and go-to-market strategies. The Norwegian research institution SINTEF is vital to the implementation of the project since it will give access to the UTOFIA camera (prototype and final systems), and will be able to provide advice on both imaging systems and interpretation of such images. The University of Naples “Parhtenope” will help the technological advance in WP3, and the transfer of knowledge generated in the project to other regions: both policy makers and a larger group of stakeholders during the dissemination activities. Additionally, on-going monitoring activities and free (in-kind) access to different research vessels at GEOMAR allow for testing and quantitative ground-truth the detection system throughout the project.
The main results of the project are:
Compilation of a new annotated training database for jellyfish identification that includes: taxonomic names, sizes and shape categories of the jellies
Release of the software JTRACK version 1 for jellyfish detection, identification and tracking with underwater time-of-flight and LiDAR image acquisition systems
An international cross-disciplinary long-lasting team between academia and industry on underwater marine species identification will be established
Establishment of a long-lasting forum for Baltic Sea marine and maritime stakeholders
Development and test of new and alternative sampling technology for delicate, non-preservable plankton organisms
The project will build upon on-going EU projects such as H2020 UTOFIA, which is developing time-of-flight imaging systems based on LiDAR and the completed project FP7 VECTORS in identifying suitable Baltic regions for jellies imaging. Results of the project will have major impacts in the implementation of the EU Strategy for the Baltic Sea Region (EUSBSR) as reported in Table
Main result |
Impact |
Indicator |
Compilation of a new annotated training database for jellyfish identification |
Set baseline for MSFD indicators supporting the HELCOM Baltic Sea Action Plan ( |
Database available on PANGAEA ( |
Release of the JTRACK software for jellyfish detection, identification and tracking with underwater time-of-flight image acquisition systems |
Filling gap in present species identification methods; support Policy Areas on Ship, Hazards and Nutrients in e.g., Managing the invasive alien species risks at the Baltic Sea, Development of Baltic Marine Environment Protection Commission (HELCOM) core set indicators, Baltic Blue Growth (BBG) ( |
Software available for download on gitHub and project webpage |
Establishing an international cross-disciplinary long-lasting team between academia and industry on underwater marine species identification |
Support Policy Area Education of the EUSBSR and Integrated Maritime Policy |
Signature of a Memorandum of Understanding for internship student mobility between Partners |
Establishing a long-lasting forum for Baltic Sea marine and maritime stakeholders |
Supporting several Policy Area of the EUSBSR |
Website with a dedicated section for stakeholder news and comments |
New sampling technology for delicate, non-preservable plankton organisms |
Baseline for MSFD indicators supporting the HELCOM Baltic Sea Action Plan ( |
Demonstration to stakeholders month 22 |
Technology Readiness Level (TRL)
From already available basic technology on machine learning algorithm (TRL 1) during the course of the project we will research the feasibility for jellyfish identification software (TRL 2) producing a first proof of concept (JTRACK version 0; TRL 3) within 6 months of the project. The version 0 of JTRACK will then be integrated with object identification and tracking using regular cameras in experimental setup (TRL 4, Month 12). LiDAR cameras will then be used in the field to acquire images that will be processed by an extended version of the identification software (JTRACK version 1, TRL 5). Finally the system camera JTRACK will be fully integrated and the demo system demonstrated to stakeholders in relevant marine environment (TRL 6, Month 22).
We have 5 applicant Organisations (DTU, GEOMAR, UniPARTH, BIORAS Aps, SINTEF ICT) from 4 different Countries (Denmark, Germany, Norway and Italy). The team has all the necessary skills and expertize to successfully implement the project providing results that will have impacts at Baltic Sea regional scale and beyond. We describe below the specific people involved in the Blue Lab Team from the different Organizations and provide in each WP (described in section “f”) specific role and responsibilities. In Table
P1- Technical University of Denmark (DTU) www.dtu.dk - is the leading technical university in the Nordic Countries and is among the most innovative Universities globally. DTU is characterized by research-based education at the highest international level, with close interaction between researchers, Authorities and business, as well as strategic alliances with several international partners. Two Departments at DTU will participate to this action: DTU Aqua, the National institute for Aquatic Resources that is a leading European institute in marine and maritime science and a major player for Baltic Sea fisheries; DTU Compute, the department for applied mathematics and computer science is an internationally leading research department within e.g. machine learning, computer vision and statistical image analysis. Both departments have very-long records in management of large EU projects and world class expertize in generating innovative products for public sector consultancy and to the benefit of both the industry and public sector institutions. |
|||
WP |
Time |
Description of work |
|
WP1 |
30% |
DTU Aqua will coordinate the project and will provide logistic and IT support. The coordinator will establish the management body of the project, will organize the meetings and communicate with EC to report on activities; together with the Steering Committee will implement specific actions (e.g. mobility plan) and will manage risks. DTU Aqua will also be responsible of setting up the website and its administration. |
|
WP2 |
10% |
DTU Aqua will support the field and laboratory activities for imaging acquisition and compilation of the training database. DTU (both Aqua and Compute) will be responsible for the testing of the complete demo system. |
|
WP3 |
30% |
DTU Compute will lead the final software development and interact with UniPARTH to optimize the machine-learning algorithm for the LiDAR camera. |
|
WP4 |
30% |
DTU Aqua (supported by Compute) will guarantee coordination with other on-going EU projects (namely: COLUMBUS, UTOFIA) and marine science networks (e.g. EUROMARINE) and coordinate the outreach and dissemination activities supporting the stakeholder training with the complete demo system. |
|
P2- GEOMAR www.geomar.de - is the Helmholtz Centre for Ocean Research and has an excellence expertise in ocean research within the Baltic Sea – as exemplified by its coordination of the Bonus EU Project BIO-C3. GEOMAR covers all aspects of ocean science from chemical, physical and biological oceanography to geology as well as ocean atmosphere interactions. With this breadth of research, GEOMAR is unique in Germany, combining coastal ecosystems with blue water research. It has an annual budget of ca. 72 Mio. EUR, ca. 950 employees and 4 research vessels. The personnel contributing to JTRACK include the post-graduate student Dr. Cornelia Jaspers, who is a world known expert on gelatinous zooplankton and jellyfish ecology and Dr. Catriona Clemmesen-Bockelmann, expert on fish larvae ecology and the Kiel Fjord system and will provide valuable background information for JTRACK. |
|||
WP |
Time |
Description of work |
|
WP2 |
60% |
GEOMAR will lead all the laboratory and field activities to perform jellyfish imaging. GEOMAR will provide ship-of opportunities and all the knowledge of jellyfish taxonomy and ecology to compile the training database. |
|
WP3 |
10% |
GEOMAR contributes to jellyfish identification |
|
WP4 |
30% |
GEOMAR will participate to all the dissemination and outreach activities providing link to their network of stakeholders in the Baltic Sea Region, providing content for e-news and website and disseminating project results in conferences and meetings. Moreover GEOMAR will lead the stakeholder training activities providing ship of opportunities and coordination with the “Future Ocean” cluster in Germany. |
|
P3- BIORAS Aps www.bioras.com - is a Danish software development company, with many years of experience in integrating instrumentation and optical systems. The company supplies advanced specialized digital image analysis solutions for analysis of still images and video sequences in 2D or 3D. Bioras develops laboratory equipment and monitoring systems: FishGuard system for monitoring fish behavior in commercial aquaculture plants; BallastWISE for quantifying the number of living micro-organisms in water samples from ballast water tanks for the shipping industry; an automatic plate counter, branded as PetrilyzerTM. Bioras is involved in a range of development projects, and has established an extensive network of scientific and industrial collaborators. Bioras has provided training and internship opportunities for a number of students and graduates. |
|||
WP |
Time |
Description of work |
|
WP1 |
15% |
BIORAS will be the main industrial tutor for post-graduate students and help in defining the mobility plan. |
|
WP2 |
15% |
BIORAS will support the final demo system test in the field. |
|
WP3 |
30% |
BIORAS will contribute in the development of the object detection and tracking. |
|
WP4 |
40% |
BIORAS will participate to the stakeholder training and will lead the task identifying a go-to-market strategy including a business model and business plan which can guarantee the future sustainability of JTRACK. |
|
P4- University of Naples “Parthenope” (UniPARTH) www.uniparthenope.it -The Science and Technology Department of University of Naples “Parthenope” (UniPARTH) develops activity of basic and applied research for the knowledge advancement, for new methodologies and tools definition and the development of meaningful applications in the sectors included in the following research areas: Computer science, Physical Sciences, Earth Sciences, Geodetic Sciences, Maritime and Aerial Navigation Sciences and technologies. UniPARTH is active in the base and applied research for the disciplines of Computational Mathematics, Artificial Vision and Forms Recognition and Classification, Images Elaboration and in the associated education. |
|||
WP |
Time |
Description of work |
|
WP3 |
60% |
UniPARTH will lead WP3 and drive design and implementation of the JTRACK algorithm including testing different methodologies for segmentation and tracking of the objects. |
|
WP4 |
40% |
UniPARTH will contribute to the dissemination of the results interacting with the Mediterranean Region networks, organizing a stakeholder meeting and contributing to website and news digest. |
|
P5- SINTEF www.sintef.no - SINTEF is Scandinavia's largest independent research organisation, and is a non-profit research foundation. We employ 2100 people and more than 90% of our annual turnover derives from contract research for industry and the public sector. We receive minimal state funding (around 6%). SINTEF is multidisciplinary, with international top-level expertise in a wide range of technological and scientific disciplines, medicine and the social sciences. Our company vision is "technology for a better society", and it is an important aspect of our societal role to contribute to the creation of more jobs. SINTEF has 20+ years of experience developing optical measurement systems for industrial problems and interpreting the data from these sensor systems. They have worked with core UTOFIA technology – Time-of-Flight sensors – for the past ten years, with a particular focus on characterisation and usage towards robotics. |
|||
WP |
Time |
Description of work |
|
WP2 |
10% |
SINTEF will through UTOFIA provide the LiDAR camera for WP2 work. |
|
WP3 |
30% |
SINTEF will advice on best practice for integration of software and LiDAR camera system. |
|
WP4 |
60% |
SINTEF will advice in producing a business plan for JTRACK coherent with LiDAR camera market potential; will participate to the kickoff and final meeting and contribute to the dissemination of the results. |
Coordinator: Dr. Patrizio Mariani is Senior Researcher at DTU Aqua and will be the project coordinator. He is an expert marine ecologist and is involved as PI in several technology projects at DTU including underwater visualization systems, tracking algorithm for fish behaviour and underwater robotics. He is chair of the Steering Committee of EUROMARINE
Added value
The Blue Lab established in this project is an international and interdisciplinary team providing innovation via research and technology-development by designing, testing and demonstrating a jellyfish detection systems, which will have direct impacts across several Blue Growth Sectors*
- Ecological ST activities include: jellyfish ecology, execute laboratory- and field-work.
All Partners are main world lead players in marine and maritime sectors. There is a strong regional component in our consortium with DTU and GEOMAR being main actors in the Baltic Sea Regional strategies. Moreover the transnational component in the Consortium is essential for the efficient execution of ST and TN activities. The composition of the Consortium guarantees proper capacity building and training opportunities for all the post-graduate students participating to the project. As formation of students is at the centre of JTRACK, students will be main players in stakeholder management and interactions, as well as the main engine for the ST aspects. BIORAS participation in tutoring and marketing strategies will give both the commercial dimension to the students (which is nonetheless already traditionally in focus at DTU) and provide the best possible framework for efficient commercialization of the software after the end of the project since it matches the core activities of this SME.
RISK MANAGEMENT PLAN: Major technical risks connected to the individual work packages and phases of work have been identified in the course of project preparation. Risks listed have been selected based on the probability and consequence for the project; only high-probability and high-consequence risks are listed. This has been summarized in Table
Risk no. |
Description |
WP involved |
Possible mitigation measures |
R01 |
Postgraduates moving to other jobs before the end of the project |
WP 1-4 |
Discuss with EC possibility of substituting them with other postgraduates Distribute the work among the expert PIs of the working team |
R02 |
Delay in recruiting the postgraduate needed at DTU |
WP 1-4 |
Identify potential candidates before the kickoff meeting and possibly among those already working in the different Labs WP leaders and task leaders to advance the work until positions are fulfilled |
R03 |
Commercial LiDAR cameras not available during the life of the project |
WP 1-4 |
Use prototypes developed in the UTOFIA project (system 0, 1, 2) |
R04 |
LiDAR camera offers poor imaging of jellies |
WP2-4 |
Early test of the system Algorithm developed for regular cameras |
R05 |
Driver and software interface development more complex than originally estimated |
WP 2-3 |
Review of interface requirements early within each project phase |
R06 |
Algorithm development does not identify substantial improvements |
WP 3 |
Early and regular review of algorithm scope and implementation |
R07 |
Underperforming partners |
WP 1 |
Close contact between WP Leaders and Technical Leader, short feedback loops and personal contact |
R08 |
Stakeholders not interested in participating in the project |
WP 4 |
Define strategy for one-to-one meetings with stakeholders during Trades, Conferences and other meetings |
R09 |
Mis-match between jellyfish presence in the field and system development and testing |
WP 2-3 |
Apply for more ship-of-opportunities and regular review the workplan based on observational evidences on jellyfish presence |
R10 |
Demo-test application delayed |
WP 2, 4 |
Revise dissemination and exploitation plans. Define alternate application cases |
Risks will be reviewed in Steering Committee meetings, to both monitor their status and to identify new possible risks found during development.
P1 Project Management
Work Package Leader - DTU
Participant Partners - BIORAS
Type of Activity - Training and Networking, Market and Outreach
Objectives:
Description of Work:
Task 1.1: Administration, self-assessment and risk management (M1 – M24). Ensure the distribution of responsibilities and the flow of information and to provide the technical and organisational guidance to the project and coordinate the research efforts. Delivery of reports and deliverables in time and in the expected quality. Establish the management structure and schedule central meetings (kickoff, midterm and final) to get feedback on the project planning and progress to make sure the project will give maximum impact on marine science and technology. Establish a clear conflict management to ensure fast and acceptable conflict resolution while reducing the risks of escalations of disputes. Do regular assessment and the analysis of potential risks to prepare corrective actions.
Organisational structure Several management bodies will be implemented in the project Table
Responsibility level |
Management Body |
Composition |
Main Responsibilities |
Task |
Task leader |
One person from Partners |
· Execute the actions described in the task · Report to the responsible WP leader risks and impediments in the execution of the task |
WP |
WP leader |
One person from Partners |
· Co-ordinate and report on progress of detailed work in the WP · Responsible for the timely delivery of all the results from their work packages, as defined in the project work plan. |
Project as a whole |
Steering Committee |
- Work package leaders - Project Coordinator - Postgraduate representative |
· Make strategic decisions concerning project co-ordination, direction, and overall management and planning. · Project Risk Management · Plan and implement the internship and short mobility program |
Exploitation manager |
Leader within WP4 (e.g., BIORAS, DTU) |
· Provide an exploitation and business plan · Oversees IPR management |
|
Project management Team |
Project coordinator + support team (see below) |
· Organize the SC meetings and prepare the agenda and report the minutes · Implement decision of the Steering Committee · Assist all other management bodies |
|
Project coordinator |
Dr. Patrizio Mariani – DTU Aqua |
· Co-ordinate and report on progress of JTRACK · Engage with stakeholders |
Internal communication. Kick-off (Month 3) mid-term (Month 12) and final meeting (Month 22) will be organized by the Project Management team. Regular monthly meetings with relevant partners will be scheduled as needed. The Project Management team will organize weekly Blue Lab meetings around core activities and will compile minutes for each meeting and make them available on the webpage.
Deliverables: D1.1, D1.5, D1.6, D1.7, D1.8
Responsible: DTU
Task 1.2: Mobility, networking and tutoring (M1 – M24). Implement weekly meetings of the working team and organize the SC monthly meetings. Establish the stakeholder forum and provide methods for stakeholder engagement on a regular basis to guide stakeholder interaction in WP4 and technical development in WP3. Support tutoring of the post-graduate and their interactions with the industrial tutor, internal advisors and stakeholders, including entrepreneurship training provided by FOUNDERS A/S (please refer to Letter of Sponsors). Deliver a plan – agreed among all involved Partners – and monitor the internship and mobility program for post-graduate students, which will constitute the basis for a preliminary Memorandum of Understanding between partners for long term mobility.
Deliverables: D1.2, D1.3
Responsible: DTU, Partner BIORAS
Task 1.3: Creation and maintenance of the website (M1 – M24). Design and develop a webpage for the BlueLab team with dedicated sections for partners contributes (e.g., a blog) and ensure each participants, advisors and stakeholder to contribute to the content including publication of: ideas, results, data, test cases, demo. The task responsible will also publish bi-monthly electronic newsletters to update on project status. All generated image data by this project will be openly accessible via our homepage and stored in PANGAEA to ensure long-term accessibility.
Deliverables: D1.4
Responsible: DTU with contributes from all Partners
Role of Partners:
DTU will be the Coordinator of JTRACK and will lead this WP and be responsible for administration of resources, funding and effort put in the project, coordination of reporting to the EC (deliverables, milestones, periodic reports). This includes the setup and maintenance of IT services like Web Content Management systems (CMS), internal wiki pages, and Mailing list Services. DTU will also be in charge of the scientific leadership and is therefore responsible for the guidance and controlling of technical effort, reaching the targeted objectives and performance of partners. Most of the technical coordination will be carried out as part of work in WP3, DTU work in WP1 will only relate to overall technical management.
BIORAS will be the main industrial tutor for the post-graduate students in the working team and act as facilitators to stakeholder meetings. BIORAS will provide support in the technical coordination (in WP3) and technical management of the project.
Deliverables (Lead Partner, delivery month, reviewer)
D1.1 Project internal management structure and communication infrastructure established and central meetings planned (kickoff, midterm, final) (DTU, M3, SINTEF)
D1.2 Report on plan for post-graduate internship and mobility and draft MoU among partners for long term mobility (DTU, M6, GEOMAR)
D1.3 Guidelines for stakeholder interactions and questionnaire on technical feedbacks (DTU, M10, BIORAS)
D1.4 Website and data repository available (DTU, M3, UniPARTH)
D1.5 Progress report month 6 (DTU, M6, UniPARTH)
D1.6 Interim report month 12 (DTU, M12, GEOMAR)
D1.7 Progress report month 18 (DTU, M18, BIORAS)
D1.8 Final report (DTU, M24, BIORAS)
WP2 - Jellyfish imaging, training database, field testing and quantitative ground truthing
Work Package Leader - GEOMAR
Participant Partners - DTU, BIORAS, SINTEF
Type of Activity
Scientific and Technical, Training and Networking
Objectives:
Description of Work:
Task 2.1: System testing and parameters definition (M1 – M6). Design and perform laboratory experiments where different jellyfish groups (with regard to shape and size) will be observed and analysed under controlled conditions. Classical cameras e.g., GoPro and machine vision cameras in underwater housings, and a LiDAR camera will be employed simultaneously to allow for direct comparison of different technologies along with actual size and shape measurements of the organisms. Further, different turbidity levels will be simulated to compare cameras’ performances. Images and data will contribute to the training database developed in Task 2.2 and will define parameters range for jellyfish observations by the LiDAR camera. Experiments will be supported by the National Aquarium in Denmark “Den BlåPlanet” that is a sponsor of JTRACK (please ref. Letter of Sponsors) and provide in-kind free access to their facilities.
Deliverables: D2.1
Responsible: GEOMAR, Partners DTU, BIORAS, SINTEF
Task 2.2: Imaging acquisition and database compilation (M3 – M24). Weekly or bi-weekly monitoring cruises in the Kiel Bight will be used to gather data from spring to late autumn. Data acquisition will consist of regular cameras and LiDAR type camera systems along with classical net borne methods for inter-calibration and method comparison. We envision to follow the population development of the common moon jellyfish (Aurelia aurita) (from early stage to adult stages) as well as detecting first occurrence and population dynamics of the invasive comb jelly Mnemiopsis leidyi, which covers a size range from 1 to 60 mm. Additionally other jellyfish that might occur will be imaged and annotated with taxonomic names, size and shape categories. Depending on inflow events of higher saline waters from the Kattegat potential species present in the area are i.e. Cyanea capillata, Obelia octopunctata, Sarsia tubulosa. Further, to have a large cover of the Baltic Region, cruises to offshore waters in the central Baltic Basins will be performed in March, April, May and September of the second year (2018) collecting images both at night and during daylight. An annotated database to be used in WP3 will be compiled. Previous sampling showed abundance of jelly species in the order of 50 per cubic meter in the area. Hence we expect to collect on the order of tens of thousands of images which will be included in the annotated training database.
Deliverables: D2.2
Responsible: GEOMAR, Partner DTU
Task 2.3: Field test of a complete demo system (M18 – M24). The final JTRACK demo will be tested in a workshop organized back-to-back with the stakeholder training session in WP4 (Task 4.2). Field test will be conducted using the R/V Littorina and using the demo-system mounted and deployed on available ROV. We will test the suitability of the system in contrasting ecosystems i.e., turbid coastal waters (Kiel Fjord) in comparison to more offshore waters (Arkona Basin). The system will be used in combination with regular sampling and conventional underwater cameras to establish level of accuracy of jellyfish detection.
Deliverables: D2.3, D2.4
Responsible: DTU Partners GEOMAR, BIORAS
Role of Partners:
GEOMAR will lead this WP and ensure that laboratory and field data will be gathered and is responsible for the quality control of jellyfish identification. This includes the test of the camera and laser system in the field as well as in the laboratory under controlled conditions with cultured jellyfish species of different size and shape. GEOMAR will arrange the workshop and ensure that all Partners will have access to the gathered data and training sets that will be compiled. GEOMAR will give advice on the relevant biology, taxonomy and ecology of jellyfish and will offer ship of-opportunities.
BIORAS will aid in expertise regarding acquisition of real time video using proprietary underwater housings together with camera- and optical configurations.
DTU will assist the experimental studies at “Den BlåPlanet” managing the logistic and will ensure that the data collection is suitable for the algorithm development implemented in WP3. DTU will be main responsible for testing the final demo-system
SINTEF will provide access to the LiDAR cameras for laboratory and field-work.
Deliverables (Lead Partner, delivery month, reviewer)
D2.1 Report on system comparison under different turbidity levels (GEOMAR, M6, DTU)
D2.2 Training database for jellyfish identification available on open access data repository (GEOMAR, M12, UniPARTH)
D2.3 Report on demo performance (DTU, M22, UniPARTH)
D2.4 Manuscript on recent developments in jellyfish quantification (GEOMAR, M24, DTU, + all partners invited to co-authors)
WP3 - Detection, identification and tracking algorithm
Work Package Leader - UniPARTH
Participant Partners - DTU, BIORAS, SINTEF, GEOMAR
Type of Activity
Scientific and Technical
Objectives:
Description of Work:
Task 3.1: Initial model development (M1-M6). Monitoring the development of jellyfish populations from image data requires a method for quantifying the jellyfish in the recorded images and a model that relates this quantity to the population. Detection of jellyfish and identification of their species from image data typically involves detection of individual jellyfish, segmenting them to measure their size and shape, tracking individuals through consecutive frames, and classifying them to species based on image features. The initial method development will be based on images of fish from 1) LiDAR and 2) regular underwater cameras, which have been recorded in 2016 while images containing jellyfish will first be available in the project in early 2017. This initial model development will allow gaining experience on segmentation, size measurements, tracking and feature extraction. This analysis will provide information about the sampling possibilities with the system, i.e. what sizes that can be measured at given depths. The depth measurements obtained by LiDAR provides a very good basis for segmentation and accurate size measurements. Features of both desirable objects and non-desirable objects will be assimilated for classification using feature learning mechanisms based on deep neural networks, capable of reaching high identification percentages in most computer vision tasks, and in particular Deep Belief Networks.
Deliverables: D3.1
Responsible: UniPARTH, Partners DTU, BIORAS
Task 3.2: Development and test of minimum viable product (JTRACK version 0) (M6-M15). Based on experience gained in Task 3.1 a plan for data collection will be coordinated with Task 2.2 in order to obtain data for model development including both training and test data. The design of the algorithm will be based on the acquired data, where image features including shape, texture and size features will be extracted. Tracking can be used to extract movement patterns and for offering optimal frames and subsequences for feature extraction. All these features will be used in a classification method trained from the image data collected from regular cameras (early stage) and when possible from LiDAR imaging under controlled conditions. Reference data to train and verify the jellyfish classification will be based on expert annotations. Relating the measurements obtained from image data to population data requires an additional model. Reference data for the population model will be obtained from field-sampling. Robustness of the chosen algorithm will be verified through images acquired both from laboratory and data acquired from the sea. Tracking objects is challenging because the system must deal with changes of appearance, illuminations, occlusions, out-of-plane rotations and real-time processing requirements. The main challenge is the difficulty to handle the appearance changes of the target object. The appearance changes can be caused by intrinsic changes such as pose, scale and shape variation and by extrinsic changes such as illumination, camera motion, camera viewpoint, and occlusions. A preliminary but fully functional version of the software JTRACK version 0 will be presented at the International ASLO conference in February 2018.
Deliverables: D3.2
Responsible: DTU, Partners BIORAS, UniPARTH, GEOMAR
Task 3.3: Operational version of the model and assessment in operation (JTRACK version 1) (M14-M24). Bringing an algorithm from an experimental stage to operation typically requires modifications and adjustments. This can both be caused by change in the characteristics of the data and in the requirements of the requests by the user of the equipment. The purpose of this task is to adjust and optimize the chosen algorithm to meet these changes and requests. This involves an implementation for real-time analysis of the acquired image data, and a reporting module that allows easy interpretation of the measurements including quantity and a possibility for an uncertainty estimate. This will be specifically tailored for the needs of the stakeholders and therefore developed in close collaboration with the end-users. In particular we will follow advice coming from task 4.2 and after the first stakeholder meeting (Month 12).
Deliverables: D3.3, D3.4
Responsible: DTU, Partners UniPARTH, BIORAS
Role of Partners:
UniPARTH will lead this WP and be the main responsible for the initial algorithm development providing an early proof of concept that will be further developed at DTU. UniPARTH will then advice on system developing for JTRACK version 0 and JTRACK version 1 assisting post-graduate students work and providing specific capacity building and training to them.
DTU will be responsible for the release of JTRACK algorithm version 0 and version 1. Will guide post-graduate students’ work and design, test and optimize machine-learning algorithm for detection, tracking and identification.
BIORAS will provide advice on all steps of the technical development. They will be primary responsible for algorithm development for regular camera images and will provide totoring to all post-graduate students.
GEOMAR will provide expert annotation on images and provide guidance and advise on aspects of jellyfish ecology relevant to algorithm development (e.g., shape modification, drifting with currents, etc.)
SINTEF will advice on the integration of the identification software and LiDAR camera settings.
Deliverables (Lead Partner, delivery month, reviewer)
D3.1 Technical report on performance and importance of different image features for quantifying fish: comparison between LiDAR and regular cameras (UniPARTH, M6, DTU)
D3.2 Release of the minimum viable software product (JTRACK version 0) allowing quantification of jellyfish (DTU, M15, UniPARTH)
D3.3 Release operational version of the algorithm (JTRACK version 1) – (DTU, M22, BIORAS)
D3.4 Manuscript on recent developments in jellyfish detection, tracking and identification (UniPARTH, M24, + all partners invited to co-authors)
WP4 - Dissemination activities and go-to-market plan
Work Package Leader - BIORAS
Participant Partners - UniPARTH, DTU, GEOMAR, SINTEF
Type of Activity
Market and Outreach, Training and Networking
Objectives:
Description of Work:
Task 4.1: Outreach and international awareness raising (M1 – M24). Information and awareness raising activities regarding the project to the general public and stakeholders include (see also Table 5): participation to conferences (e.g. ASLO meeting 2018 see task 3.2), presentation of the technology to science forum (e.g. EUROMARINE, Marine Board), liasoning with H2020 COLUMBUS project (Knowledge transfer CSA) and engaging the “Future Ocean” cluster in Germany. A dissemination plan will be prepared at Month 3 by the task leader and updated at Month 15 of the project. These activities with stakeholders will be mainly developed by the post-graduate students of the group in order to promote acquisition of transferable skills in communication and to expose them to the challenges and opportunities present in marine and maritime sectors. A post-graduate will be also responsible of updating the e-news and blog sections of the Blue Lab website developed in WP1.
Table 5. Communication and dissemination activities
Deliverables: D4.1, D4.3
Responsible: DTU, Partners UniPARTH, GEOMAR, BIORAS, SINTEF
Task 4.2: Demonstration and training of stakeholders (M10 – M24). Early (JTRACK version 0) and final (JTRACK version 1) versions of JTRACK software will be presented to stakeholders in dedicated meeting at month 12 and Month 22. The early meeting will collect feedbacks on technical aspects (e.g. graphical user interface, identification system, etc) following guidelines provided in task 1.2 and to support WP3 algorithm development in task 3.3. The final JTRACK version 1 demo will be presented to stakeholders in a workshop and training session organized in the last quarter of the project (e.g., October-November 2018). We envision conducting a 3-day event with the first day dedicated to equipment preparation and software testing while two days organized around a training session with invited stakeholders and other scientists. The training session will introduce to system deployment and operation (e.g., using a ROV) and data acquisition and processing. A day cruises with the R/V Littorina, will allow the participation of ca. 25 stakeholders from different groups that will be invited to join. We will demonstrate the suitability of the system in contrasting ecosystems i.e., turbid coastal waters (Kiel Fjord) in comparison to more offshore waters (Arkona Basin). Post-graduate students will organize the event introduce the system to stakeholders and demonstrate the functionality of the system.
Deliverables: D4.4
Responsible: GEOMAR, Partners DTU, BIORAS, UniPARTH
Task 4.3: Evaluation of go-to-market strategies for JTRACK (M3 – M24). The leader of this task will act as Exploitation Manager and during the life of the project will implement activities to generate a comprehensive business model and business plan agreed among all Partners and including the following elements:
1. Intellectual Property (IP) protection and management plan agreement between partners will be discussed and agreed upon at project start.
2. Identify the market and potential customers (exploitation plan). In general, companies that could have an interest in this invention could be (i) government bodies and agencies (environmental agencies, public marine labs, Navies, civil security, Coast guards, etc.); (ii) engineering consulting companies responsible for monitoring of water quality for example near aquaculture facilities, or near water intake for cooling of power plants or atomic reactors; (iii) research institutes and universities; (iv) sensor manufacturing companies interested in purchasing the rights to manufacture and commercialize the sensor. We will contact at least 3 potential customers from each segment to analyze current interest.
3. Testing of the patentability of JTRACK software as a whole or parts of the system. If the JTRACK as a whole or parts of the system are patentable, a patent application will be written during the project period. DTU has a patent office, which can assist during this process. An alternative to patenting is to license the software. The project partners will discuss possible licensing strategies.
4. Define a go to market strategy. We are expecting an interest in the JTRACK system globally from:
Universities and research institutions and government bodies and agencies (environmental agencies, public marine labs, Navies, civil security, Coast guards, etc.) are expected to purchase through public call for tenders. In order to achieve an efficient market introduction and a global distributor network, we are planning to contact distributors specialized in underwater technology and marine survey equipment with a global network, towards the end of the project when we have fully functional prototype and the patentability has been clarified.
Examples of potential distributors might be: Eiva A/S (www.eiva.com), a Danish company specialized in marine subsea engineering and marine survey equipment; McArtney (http://www.macartney.com) a global supplier of underwater technology specializing in design, manufacture, sales and service of a wide range of systems to offshore operators, ocean science institute).
5. JTRACK business plan. A business plan will be written based the IP management plan, market potential analysis and the go to market strategy. If a distributor has been identified, the distributor will be invited to contribute to the marketing section of the business plan.
Deliverables: D4.2, D4.5
Responsible: BIORAS, Partners DTU, SINTEF
Role of Partners:
BIORAS will coordinate this work package and ensure delivery and reporting being on time. Moreover, BIORAS will lead task 4.3 identifying during the duration of the project all the elements and ideas contributing to the definition of the business model and business plan. BIORAS will also contribute to the communication and dissemination activities and demonstration and training of stakeholders.
DTU will lead the dissemination and communication activates providing overview of the opportunities and facilitating interactions with on-going EU projects (e.g., COLUMBUS) and marine science networks (e.g., EUROMARINE) as well as motivating post-graduate students and the entire Blue Lab team to provide content for the webpage. DTU will also participate to the identification of business model in task 4.3.
UniPARTH will contribute to dissemination and communication activities providing access to stakeholders from the Mediterranean Sea Region. They will organize the first stakeholder meeting in Month 12 in task 4.2 and will contribute to all aspect of task 4.3.
GEOMAR will be the main leader for stakeholder interactions in task 4.2, providing access and facilities for the demonstration and training. They will be responsible of the organization of the final stakeholder meeting and contribute to all communication and dissemination activities in task 4.1.
SINTEF will advice on an effective business plan for JTRACK to be coherent with present market potential of LiDAR cameras and contribute to dissemination and communiation activities.
Deliverables (Lead Partner, delivery month, reviewer)
D4.1 Dissemination and exploitation plan (DTU, M3, UniPARTH)
D4.2 Exploitation plan (BIORAS, M8, DTU)
D4.3 Final revised dissemination and exploitation plan (DTU, M15, BIORAS)
D4.4 Report on JTRACK system for application (GEOMAR, M23, SINTEF)
D4.5 Business model and business plan beyond the end of the project (BIORAS, M24, SINTEF)
GANTT chart with task number and name in each WP, months duration since the start of the project, duration of the task (in blue) and deliverables number (in green with the number).
EMFF Work Programme 2016 Call for Proposals EASME/EMFF/2016/1.2.1.4
STATUS: Reserve List
Technical University of Denmark, DTU Aqua
Blue Growth Sectors as identified by the EU H2020 (