ESA's Gaia astronomy mission, that was launched 19th December 2013, aims at mapping more than one billion stars and objects. The scientific data processing has been delegated to the Data Processing and Analysis Consortium (DPAC), which is composed of more than 400 people all across Europe. This consortium, organized in 9 scientific coordination units across 6 data processing centers, is presented herein. Responsible for a data processing centre (DPCC) that will run 3 scientific coordination units, CNES is responsible for a large part of the Gaia data processing. This paper focuses on the organization of the development and operations of this processing center inside the DPAC, dealing with challenging constraints: multiple heterogeneous teams involved without contractual relationships, poor technical interface and planning definition. We also present the technical features that have helped us to deal with the evolving interfaces, the number of developers, the large volume of data to process (PetaByte order), the huge processing resources (up to 6000 processor cores): collaborative environment, common tools and baseline for our developers, scalable data management technology, hardware cluster. After several months of operation, we provide some feedback on the beginning of the processing of real Gaia data and on the plan to end the development, transitioning to operations: updating scientific code already in operation in parallel with the development and integration of the scientific code that will be run later in the mission.
展开▼
