Skip to Main content Skip to Navigation
Conference papers

Quality Assurance in the Ingestion of Data into the CDS VizieR Catalogue and Data Services

Abstract : VizieR is a reference service provided by the CDS for astronomical catalogues and tables published in academic journals (Ochsenbein et al. 2000), and also for associated data. Quality assurance is a key factor that guides the operations, development and maintenance of the data ingestion procedures. The catalogue ingestion pipeline involves a number of validation steps, which must be implemented with high efficiency to process the 1200 catalogues per year from the major astronomy journals. These processes involve integrated teams of software engineers, specialized data librarians (documentalists) and astronomers, and various levels of interaction with the original authors and data providers. Procedures for the ingestion of associated data (Landais 2016) have recently been improved with semi-automatic mapping of metadata into the IVOA ObsCore standard, with an interactive tool to help authors submit their data (images, spectra, time series etc.). We present an overview of the quality assurance procedures in place for the operation of the VizieR pipelines, and identify the future challenges of increasing volumes and complexity of data. We highlight the lessons learned from implementing the FITS metadata mapping tools for authors and data providers. We show how the quality assurance is an essential part of making the VizieR data comply with FAIR (Findable, Accessible, Interoperable and Re-useable) principles, and the necessity of quality assurance in for the operational aspects of supporting more than 300,000 VizieR queries per day through multiple interactive and programmatic interfaces.
Document type :
Conference papers
Complete list of metadata

https://hal.archives-ouvertes.fr/hal-02993648
Contributor : Mark Allen Connect in order to contact the contributor
Submitted on : Friday, November 6, 2020 - 9:36:56 PM
Last modification on : Saturday, November 7, 2020 - 3:30:17 AM

Identifiers

Collections

Citation

Gilles Landais, Pierre Ocvirk, M Allen, Marianne Brouty, Emmanuelle Perret, et al.. Quality Assurance in the Ingestion of Data into the CDS VizieR Catalogue and Data Services. Astronomical Data Analysis Software and Systems XXVIII, Oct 2018, College Park, United States. ⟨hal-02993648⟩

Share

Metrics

Record views

24