0.gif

 What is the ESO Data Processing System (EDPS)?

EDPS is a system to automatically organise data from ESO instruments for pipeline processing and running the pipeline on these data. It is used for quality control at ESO. The current public release is a beta version without a GUI. A GUI is being developed and the system is meant to  eventually replace the older EsoReflex environment. If you make use of EDPS for research that leads to a scientific publication, we request to reference  Freudling, Zampieri, Coccato et al 2024, A&A 681, A93.

EsoReflex is currently the recommended environment for the reduction of VLT data with ESO pipelines because of its rich GUI, interactive features and detailed documentation including tutorials. However, for some users and in limited circumstances, EDPS is already now the best way to run ESO pipelines. We recommend to use EDPS for users

  • who want to learn and experiment with the future environment to run ESO pipelines.
  • who want to process large amounts of data in batch mode mostly using standard pipeline setup, in particular if these data are taken over an extended period of time.The advantage of EDPS in this situation is:
    • EDPS is written in Python and is much more robust than EsoReflex.
    • EDPS takes better advantage of available computational resources. In particular, recipes are run in parallel when possible, taking into account the internal parallelization of the recipes. Depending on circumstances, this might lead to an order of magnitude faster executions.
    • Unlike EsoReflex, EDPS takes into account changes in the instrument hardware when organizing the data ("break points").
  • who want to (re-)process calibration data independent of science data. EsoReflex is not set up to do this.

 

Installation

  Prerequisites

  The basic prerequisites for installing and running EDPS are:

  • Python 3.9 or higher
  • A working installation of at least one ESO pipeline that supports EDPS workflows. Currently the following pipeline include an EDPS workflow: ESPRESSO, UVES, and KMOS.
  • The esorex command to run pipelines needs to be in the path.

 

  Instructions

  EDPS can be installed using the following command in most Linux and MacOS versions:

            pip install --extra-index-url https://ftp.eso.org/pub/dfs/pipelines/libraries edps adari_core==0.6.0

   Detailed installation instructions are in the EDPS Tutorial.

Important Note

EDPS is already used for quality control at ESO. For this reason, interfaces and general features are considered stable and well tested. However, we can not guarantee at this point that there will not be some changes. In particular, workflows written for the current version of EDPS might need to be modified to run with future version of EDPS.

Documentation

The general principles, workflow designs and workflow syntax have been  described in  Freudling, Zampieri, Coccato et al 2024, A&A 681, A93. We recommend users to first look at this paper. A tutorial on how to install EDPS and run the published workflows should provide all the information to get started and reduce data with EDPS. A tutorial on how to write EDPS workflows is also available. 

Feedback

Please direct any and all feedback concerning installing or using EDPS to the EDPS support team.