Research Ideas and Outcomes : Data Management Plan (NSF Generic)
|
Corresponding author: Jennifer McWhorter (jmcwhorter@ucsd.edu), Julie Thomas (jothomas@ucsd.edu)
Received: 12 Apr 2016 | Published: 15 Apr 2016
© 2016 Jennifer McWhorter, Darren Wright, Julie Thomas.
This is an open access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Citation: McWhorter J, Wright D, Thomas J (2016) Coastal Data Information Program (CDIP). Research Ideas and Outcomes 2: e8827. doi: 10.3897/rio.2.e8827
|
![]() |
Since its inception in November 1975, the Coastal Data Information Program (CDIP) collects near real-time physical environmental data mostly in the coastal US and South Pacific, with a focus on waves. CDIP has many partners including industry, federal and state agencies and academia.
In all cases, these data are transmitted from the station location to CDIP at the Scripps Institution of Oceanography (SIO), La Jolla, CA where the data are processed and disseminated to interested parties.
waves, data, quality control, real-time, long-term time series, sea-surface temperature
Contextual statement describing what data are collected and relevant URL (IOOS Certification, f 1. ii)
The program captures wave, wind, and temperature data in real-time, updating every 30 minutes.
Describe how the data are ingested (IOOS Certification, f 2.)
The data are collected by several redundant pathways.
Describe how data are managed (IOOS Certification, f 2.)
The data are managed at the SIO/CDIP server. Once ingested, CDIP processes and quality controls these data. The data are stored on disk in ASCII, NetCDF, and SQL formats. Back-up occurs hourly locally, daily offsite at the UCSD Supercomputer Center and biannually to Amazon Cloud.
Describe the data quality control procedures that have been applied to the data. (IOOS Certification, f 3.)
A sophisticated suite of automated and human quality control procedures are developed, as defined in the QARTOD manual (http://www.ioos.noaa.gov/qartod/waves/welcome.html). In addition, CDIP has also developed further instrument and site specific tests. The tests are summarized in the following table: http://cdip.ucsd.edu/documents/index/product_docs/qc_summaries/waves/waves_table.php?&xtab=CDIP
All errors causing an exception are handled by the following:
When there are critical errors involving a buoy offsite or a station that has not updated within 3 hours, the software team is not only notified via email but, a designated watch person is also paged.
Only those data that pass all the QC tests are transmitted to the National Data Buoy Center (NDBC) & the National Weather Service (NWS).
The above quality control procedure can be monitored at: http://cdip.ucsd.edu/diag
N/A
Adheres to the NOAA Data Sharing Procedural Directive. The System is an operational system; therefore the RICE should strive to provide as much data as possible, in real-time or near real-time, to support the operation of the System. (IOOS Certification, f. 4.)
Once data have been acquired, processed, and quality controlled, CDIP makes the complete data set available. (Near-real time, approximately 3 minutes after the data are transmitted)
N/A
N/A
N/A
How can the information be accessed? (IOOS Certification, f 1. ii)
CDIP Shares data in a variety of file formats.
All of the above.
All of CDIP's data sets are described by detailed metadata, which is continuously updated and available online in a number of formats. FGDC-compliant metadata are included, in both HTML and XML formats. The metadata for any specific data set are accessible from the station pages in the historic section of the website. In addition to the standard web pages, static XML metadata files are available for download or harvesting from a web-accessible folder (http://cdip.ucsd.edu/data_access/metadata). The NetCDF files also include metadata and are available in ISO 19115 XML from the CDIP THREDDS catalog (http://thredds.cdip.ucsd.edu)
FGDC metadata consists of seven main sections, five of which do not need to be included if they do not apply to the data set in question. For CDIP metadata, two sections are omitted Spatial_Reference_Information and Spatial_Data_Organization_Information - because they only apply to datasets that include spatial data. (Although CDIP's metadata contains spatial info - deployment positions - the data sets themselves do not.)
Thus CDIP metadata consists of five sections:
Many of the fields in the content standard are defined as free text, and can contain links to other resources. CDIP's metadata takes full advantage of this fact, linking to relevant documents and pages on the CDIP website wherever possible. This is the most efficient and effective approach because CDIP's online documentation is extensive and covers most of the topics addressed in the FGDC standard. By linking directly to CDIP's web resources redundancy is avoided and the metadata are ensured to be up-to-date. This same approach is used in defining CDIP's entity and attribute information.
CDIP's FGDC metadata is generated by querying our 'archive' MySQL database and passed through the US Geological Service’s utility 'mp': http://geology.usgs.gov/tools/metadata/tools/doc/mp.html The mp program verifies that the metadata is FGDC-compliant, and then outputs it in the desired format, either html or xml.
CDIP’s NetCDF files have ISO 19115 compliment metadata which are generated with custom FORTRAN scripts.
CDIP’s data sets are described by detailed metadata in a number of formats:
FGDC and ISO 19115 metadata are both accepted standards and mandated by the US Federal Government.
Points of contact- Individuals responsible for the data management and coordination across the region (CV’s attached); (IOOS Certification f 1. i)
Julie Thomas - Employee 39 years, Principal Investigator/Program Manager
Darren Wright - Employee 10 years, Programmer/Analyst
Jen McWhorter - Employee 1 year, Administrative Analyst
Identify the procedures used to evaluate the capability of the individual (s) identified in subsection 997.23(f)(1) to conduct the assigned duties responsibly. (IOOS Certification, f 1. iii)
The University of California has a process in place for personnel evaluation. These evaluations are on file with UC San Diego Human Resources. All personnel listed have received excellent evaluations.
Documents of the RICE’s data archiving process or describes how the RICE intends to archive data at the national archive center (e.g., NODC, NGDC, NCDC) in a manner that follows guidelines outlined by that center. Documentation shall be in the form of a Submission Agreement, Submission Information Form (SIF) or other, similar data producer-archive agreement (IOOS Certification, f 6.).
National Centers for Environmental Information (NCEI) is the federal archive repository. Historic data from CDIP stations are archived monthly and available at NCEI (http://www.nodc.noaa.gov/access/index.html). The archive process was established with the NCEI Submission Information Form (https://goo.gl/AmX8F8).
Local redundant HDD storage at the CDIP Lab, the UCSD Supercomputer center, Amazon Glacier and NCEI.
Data are indefinitely stored.
All data are publicly available and preserved.
Raw data are decoded and formatted, analyzed and quality controlled.
FGDC standard metadata are available per deposit and transformation. NetCDF files have complete metadata and quality control flags.
Time series and spectral files.
Describe how data are distributed including a description of the flow of data through the RICE data assembly center from the source to the public dissemination/access mechanism. (IOOS Certification, f. 2.)
CDIP Access to Data (http://cdip.ucsd.edu/?nav=documents&xitem=product#access)
CDIP data and products are freely available for public use. When referenced, please provide a link to the CDIP homepage.
Examples:
Data are publicly available.
In general, a data sharing agreement will not be required. However, data should be properly acknowledged.
The one exception is with NOAA Physical Ocean Real Time System (PORTS). A Memorandum of Understanding (MOU) between NOAA PORTS and the US Army Corps, representing CDIP as the funding agency, is signed.
N/A
The funding agency & the University of California, San Diego through a contractual agreement.
Articles:
Storm wave induced mortality of giant kelp, Macrocystis pyrifera, in Southern California Seymour, R.J., M.J. Tegner, P.K. Dayton, and P.E. Parnell, Estuarine, Coastal and Shelf Science, Vol. 28, pp. 277-292, 1989
Unusual marine erosion in San Diego County from a single storm Dayton, P.K., R.J. Seymour, P.E. Parnell, and M.J. Tegner, Estuarine, Coastal and Shelf Science, Vol. 29, pp. 277-292, 1989
COASTAL FORUM: Unusual damage from a California storm Seymour, R.J., Shore & Beach, Vol. 57, No. 3, July 1989, p. 31, 1989
[Editorial] The great storm of January 1988 Seymour, R.J., Shore & Beach, Vol. 57, No. 4, October 1989, p. 2., 1989
A Comparison of Spectral Refraction and Refraction-Diffraction Wave Propagation Models William C. O'Reilly and R. T. Guza, J. Waterway, Port, Coastal and Ocean Eng., 117, (3), pp199-215, 1991.
A Comparison of Two Spectral Wave Models in the Southern California Bight William C. O'Reilly and R. T. Guza, Coastal Eng., 19, pp263-282, 1993.
Wave Monitoring in The Southern California Bight William C. O'Reilly, R. J. Seymour, R. T. Guza, D. Castel, Ocean Wave Measurement and Analysis, Proc. 2nd Int. Symp. July 25-28, 1993, pp448-457.
New Technology in Coastal Wave Monitoring Richard Seymour, David Castel, David McGehee, Julianna Thomas, and William O'Reilly, Ocean Wave Measurement and Analysis, Proc. 2nd Int. Symp. July 25-28, 1993, pp105-123.
Field Wave Gaging Program, Wave Data Analysis Standard Marshal D. Earle, David McGehee, and Michael Tubman, USACE Instruction Report CERC-95-1, March 1995.
Effects of Southern California Kelp Beds on Waves M. Hany S. Elwany, William C. O'Reilly, Members, ASCE, Robert T. Guza, and Reinhard E. Flick, J. Waterway, Port, Coastal and Ocean Eng., 121,(2), pp143-150, 1995.
A Comparison of Directional Buoy and Fixed Platform Measurements of Pacific Swell W. C. O'Reilly, T. H. C. Herbers, R. J. Seymour and R. T. Guza, J. Atmos. and Ocean. Technol., 13, (1), pp231-238, 1996.
Observations of Seiche Forcing and Amplification in Three Small Harbors Okihiro, M. and Guza, R., J. Waterway, Port, Coastal, Ocean Eng., 122(5), 232-223, 1996.
Effects of El Nino on the West Coast Wave Climate Seymour, R.J., Shore & Beach, Vol. 66(3): 3-6, 1998
Assimilating Coastal Wave Observations in Regional Swell Predictions. Part 1: Inverse Methods W. C. O'Reilly and R. T. Guza, J. Physical Oceanography, 28, (4), pp679-691, 1998.
The Relationship Between Incident Wave Energy and Seacliff Erosion Rates: San Diego County, California Bunumof, B.T., Storlazzi, C.D., Seymour, R.J., Griggs, G.B., California Journal Coastal Research, Vol. 16, No. 4, 1162-1178, 2000
Evidence for Changes to the Northeast Pacific Wave Climate Seymour, R.J., Journal of Coastal Research, Vol. 27, Issue 1: pp. 194-201, 2001
Rapid Erosion of a Southern California Beach Fill Seymour, R.J., R.T. Guza, W. O'Reilly and Steve Elgar, J. Coastal Engineering, 52, (2), pp151-158, 2004.
Application of Airborne LIDAR for Seacliff Volumetric Change and Beach-Sediment Budget Contributions Adam P. Young and Scott A. Ashford, J. Coastal Research, 22, (2), pp307-318, 2006.
Performance Evaluation of Seacliff Erosion Control Methods Adam P. Young and Scott A. Ashford, Shore and Beach, 74, (4), pp16-24, 2006.
Evolution of Surface Gravity Waves Over a Submarine Canyon R. Magne, K.A. Belibassakis, T.H.C. Herbers, Fabrice Ardhuin, W.C. O'Reilly, and V. Rey, J. Geophysical Research, 112, C01002, pp1-12, 2007.
A Technique for Eliminating Water Returns from Lidar Beach Elevation Surveys Yates, M.L., R.T. Guza, R. Gutierrez, and R.J. Seymour, J. Atmos. and Ocean. Techol., 25, pp1671-1682, 2008.
Seasonal Persistence of a Small Southern California Beach Fill M.L. Yates, R.T. Guza, W.C. O'Reilly, R.J. Seymour, J. Coastal Engineering, 56, pp559-564, 2009.
Overview of seasonal sand level changes on southern California beaches Yates, M.L., R.T. Guza, W.C. O'Reilly, and R.J. Seymour, Shore & Beach, 77(1), pp39-46, 2009.
Comparison of short-term seacliff retreat measurement methods in Del Mar, California Adam P. Young, R.E. Flick, R. Gutierrez, and R.T. Guza, Geomorphology, 2009
Coarse Sediment Yields from Seacliff Erosion in the Oceanside Littoral Cell Adam P. Young, J.H. Raymond, J. Sorenson, E.A. Johnstone, N.W. Driscoll, R.E. Flick, and R.T. Guza, Journal of Coastal Research, Vol 26, No. 3, pp. 580-585, May 2010.
Rain, Waves, & Short-Term Evolution of Composite Seacliffs in Southern California Adam P. Young, R.T. Guza, R.E. Flick, W.C. O'Reilly, and R. Gutierrez, Marine Geology, 2009
Comparison of Airborne and Terrestrial LIDAR Estimates of Seacliff Erosion in Southern California Adam P. Young, M.J. Olsen, N. Driscoll, R.E. Flick, R. Gutierrez, R.T. Guza, E. Johnstone, and F. Kuester, Photogrammetric Engineering & Remote Sensing, April 2010.
A Portable Airborne Scanning Lidar System for Ocean and Coastal Applications Benjamin D. Reineman, Luc Lenain, David Castel, and W. Kendall Melville, 2009
Equilibrium shoreline response: Observations and modeling Yates, M.L, R.T. Guza, and W.C. O’Reilly, J. Geophys Res, 114, C09014, doi: 10.1029/2009JC005359, 2009
Short-term coastal cliff retreat statistics at Sunset Cliffs - Point Loma, California, USA Young, A.P., R.T. Guza, W.C. O'Reilly, R.E. Flick, and R. Gutierrez, Natural Hazards and Earth System Sciences, 11, 205-217, Jan 2011
The Effect of Temporal Wave Averaging on the Performance of an Empirical Shoreline Evolution Model M.A. Davidson,M.A., I.L. Turner and R.T. Guza, Coastal Engineering, 58, 802–805, 2011
Equilibrium Shoreline Response of a High Wave Energy Beach Yates, M.L, R.T. Guza, W.C. O'Reilly, P. Barnad and J Hansen, J. Geophys. Res., 116, C04014, doi: 10.1029/2010JC006681, 2011
Coastal cliff ground motions from local ocean swell and ifragravity waves in southern California Adam P. Young, P.N. Adams, W.C. O'Reilly, R.E. Flick, R.T. Guza, J. Geophys Res., 116, C09007, doi: 10.1029/2011JC007175, Sep 2011
National Oceanic and Atmospheric Administration (NOAA) Template
Public Data Management Plan created with the DMPTool: https://dmptool.org/plans/15485.pdf