p1 API Python Tutorial
Introduction
The Phase 1 Application Programming Interface (API) is a programming-language independent, HTTPS-based interface that uses the lightweight JSON standard as data format to exchange information between your application and the phase 1 server at ESO. In order to program more conveniently against the API, it is helpful to create small, language-specific bindings to the API. We provide a simple example binding for Python, i.e. p1api.py.
Please note:For now, this binding does not cover all approximately 70 API endpoints, but focuses on those endpoints considered relevant for semi-automated preparation of proposals (rather than using the user interface), for instance for large proposals with many observing runs, targets and observations. Entering other information such as text fields, investigators, scientific keywords and the actual submission of the proposal still has to be carried out with the user interface.
In the following sections, we discuss basic proposal programming using this binding with hands-on coding examples. Note that the data exchanged in the various API calls is often instrument-specific. In order to fully understand the data required and returned by the various API calls and their detailed behaviour, please consult the Phase 1 API Specification.
Tutorial
We use Python 3 for this tutorial. Please ensure that it is installed and the executable python3 is on your command line search path. The p1api is published via the Python Package Index. We recommend also installing pretty printing support. The following command will install p1api and pretty printing in your account for python3.
python3 -m pip install --upgrade --user phase1api prettyprint
This will also install p1api's dependencies. For this tutorial we will use the interactive Python3 interpreter. Please start it and type the listed commands as we go through this tutorial. Fear not, you are safe to play in the demo environment. Please start the interactive Python 3 interpreter by typing [i]python3 into your command-line shell.
Python 3.11.3 (v3.11.3:f3909b8bc8, Apr 4 2023, 20:12:10) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>>
Pretty printing support
In order to print JSON data returned from the various API calls in a somewhat more readable way, we first add pretty printing support to our code. Type
import p1api import pprint p = pprint.PrettyPrinter(indent=4)
To get an overview of the available API calls, you can now type
help(p1api.p1api)
Log in
Now let's get access to the API and log in using the 'demo' environment. The other supported environment is 'production' for actual proposal preparation and submission. Run the following code, you should see not output.
api = p1api.ApiConnection('demo', '52052', 'tutorial')
Creating a proposal
When creating a new proposal, the ID of a "proposal submission cycle" has to be given. We therefore first have to find a cycle that is in the right state.
cycles, _ = api.getCycles() openCycles = [c for c in cycles if c['step'] == 'ProposalSubmission'] p.pprint(openCycles)
This will print cycles available for proposal submission similar to this
[ ...,
{ 'cycleId': 2696,
'dateCreated': '2024-02-12T10:05:17Z',
'description': 'DEMO: ESO Regular Call for Proposals for Period P114',
'lastUpdated': '2024-02-21T16:38:11Z',
'lastUpdater': 83142,
'name': 'P114',
'startingPeriod': 114,
'step': 'ProposalSubmission',
'type': 'Regular',
'version': 36 }]
Let us take note of the cycleId, and create a normal proposal.
cycleId = 2696 proposal, _ = api.createProposal(cycleId, 'Normal', 'My Test Proposal')
Creating targets
Targets are created as a pool of targets on proposal-level. Only later are they assigned to observing runs as science targets or potentially as reference stars to observations. Let's define a few targets in code. Of course this information could also be read from an externally generated file.
targets = [ { 'brightness': [ {'band': 'B', 'magnitude': 14.4},
{'band': 'R', 'magnitude': 14.3},
{'band': 'I', 'magnitude': 14.4},
{'band': 'J', 'magnitude': 15.402},
{'band': 'H', 'magnitude': 14.491},
{'band': 'K', 'magnitude': 15.036},
{'band': 'u', 'magnitude': 16.596},
{'band': 'g', 'magnitude': 14.943},
{'band': 'r', 'magnitude': 14.343},
{'band': 'i', 'magnitude': 13.962},
{'band': 'z', 'magnitude': 13.88}],
'comment': '',
'coordinateSystem': 'J2000',
'dec': '-07:50:43.389',
'inSolarSystem': False,
'movement': { 'epoch': 2000.0,
'parallax': 0.0,
'properMotionDec': 0.0,
'properMotionRa': 0.0,
'radialVelocity': 3728.0,
'redShift': 0.0,
'useRedShift': False},
'name': 'NGC 1234',
'ra': '03:09:39.000',
'sed': 'none' },
{ 'brightness': [ {'band': 'U', 'magnitude': 6.37},
{'band': 'B', 'magnitude': 7.03},
{'band': 'V', 'magnitude': 6.48},
{'band': 'G', 'magnitude': 4.035},
{'band': 'R', 'magnitude': 4.9},
{'band': 'I', 'magnitude': 4.41},
{'band': 'J', 'magnitude': 1.658},
{'band': 'H', 'magnitude': 1.073},
{'band': 'K', 'magnitude': 0.171}],
'comment': '',
'coordinateSystem': 'J2000',
'dec': '-59:41:04.051',
'inSolarSystem': False,
'movement': { 'epoch': 2000.0,
'parallax': 0.0,
'properMotionDec': 0.0041,
'properMotionRa': -0.011,
'radialVelocity': -25.0,
'redShift': 0.0,
'useRedShift': False},
'name': 'eta Car',
'ra': '10:45:03.537',
'sed': 'none' },
{ 'brightness': [ {'band': 'B', 'magnitude': 7.89},
{'band': 'V', 'magnitude': 6.94},
{'band': 'G', 'magnitude': 15.131},
{'band': 'J', 'magnitude': 4.76},
{'band': 'H', 'magnitude': 4.09},
{'band': 'K', 'magnitude': 3.831},
{'band': 'u', 'magnitude': 11.673},
{'band': 'g', 'magnitude': 9.635},
{'band': 'r', 'magnitude': 8.708},
{'band': 'i', 'magnitude': 8.373},
{'band': 'z', 'magnitude': 7.921}],
'comment': '',
'coordinateSystem': 'J2000',
'dec': '69:03:55.062',
'inSolarSystem': False,
'movement': { 'epoch': 2000.0,
'parallax': 0.149,
'properMotionDec': -7e-05,
'properMotionRa': 0.00028,
'radialVelocity': -47.0,
'redShift': 0.0,
'useRedShift': False},
'name': 'M 81',
'ra': '09:55:33.172',
'sed': 'none' }]
Now we can add those targets to the proposal one by one and verify they were created properly.
proposalId = proposal['proposalId']
for t in targets:
api.createTarget(proposalId, t)
targets, _ = api.getTargets(proposalId)
p.pprint(targets)
Note that if you want to change additional target properties, you can the target using api.getTarget() to retrieve its exact JSON structure, modify that structure and update it using api.updateTarget().
targetId = targets[0]['targetId'] target, version = api.getTarget(targetId) target['comment'] = 'Highest priority' target, version = api.updateTarget(target, version)
Versioning of created entities
Whenever you get, create or update an entity (e.g. observing run, target, observing setup, observation) the corresponding API call will return two items: (1) the existing, created or updated entity and (2) its version. This version must be used for any further updates or deletion of the given entity. This mechanism (optimistic locking) is required to protect against concurrent modification errors. If you get an error during update or deletion of an entity, it usually means that the entity was modified in the meantime, e.g. in the user interface or by some other tool. To fix the problem, simply get the entity and its version again.
target, version = api.getTarget(targetId)
Creating an observing run
We can create an observing run by providing a run title, instrument, period number, observing mode, telescope setup and run type.
run, _ = api.createRun(proposalId, 'My First Run', 'XSHOOTER', 114, 'SM', 'UT3', 'Normal') p.pprint(run)
Which will produce output similar to
{ 'instrument': 'XSHOOTER',
'observingConstraints': { 'airmass': 2.8,
'atm': '50% (Seeing < 0.8 arcsec)',
'fli': 1.0,
'skyTransparency': 'Clear',
'waterVapour': 30.0},
'observingMode': 'SM',
'periodNumber': 114,
'proprietaryTime': 12,
'runId': 28005,
'runType': 'Normal',
'telescopeSetup': 'UT3',
'timeConstraints': '',
'title': 'My First Run'}
Note that if you want to change additional run properties, you can get the run using api.getRun() to retrieve its exact JSON structure, modify that structure and update it using api.updateRun(). For instance, in order to change the fractional lunar illumination constraint, you would do the following
runId = run['runId'] run, version = api.getRun(runId) run['observingConstraints']['fli'] = 0.5 run, version = api.updateRun(run, version)
Creating observing setups under a run
Now that we have a run, we can create observing setups under it. Observing setups are "prototype observations", i.e. they describe an observation but do not have a target. We will associate targets later. Observing setups only have a name and serve as a container for a sequence of acquisition template and one or more science templates to be observed.
obsSetup1, _ = api.createObservingSetup(runId, 'Setup 1') obsSetup2, _ = api.createObservingSetup(runId, 'Setup 2') obsSetups, _ = api.getObservingSetups(runId) p.pprint(obsSetups)
Creating templates under an observing setup
Now we can create an acquisition and a science template under the observing setups.
obsSetupIds = [obsSetup1['obsSetupId'], obsSetup2['obsSetupId']]
for s in obsSetupIds:
api.createTemplate(s, 'XSHOOTER_slt_acq')
api.createTemplate(s, 'XSHOOTER_slt_obs_Stare')
templates, _ = api.getTemplates(s)
p.pprint(templates)
Creating observations
We create observations by associating some of the previously defined targets to a run. For each observing setup that exists under the run, an observation is automatically created. If we associate n targets and there are m observing setups, we end up with n * m observations. The example below associates all 3 targets we defined earlier to our run.
# ----- get the IDs of the targets to associate targets, _ = api.getTargets(proposalId) targetIds = list(map(lambda t: t['targetId'], targets)) p.pprint(targetIds) # ----- get version for current run-targets association _, version = api.getRunTargets(runId) # ----- associate targets to the run targetIds, version = api.updateRunTargets(runId, targetIds, version) p.pprint(targetIds)
Now we can verify the created observations.
for t in targetIds:
for s in obsSetupIds:
observation, _ = api.getObservation(t,s)
p.pprint(observation)
Which prints the individual observations as follows
{ 'etcCertified': False,
'obsSetupId': 27004,
'referenceTargets': [],
'repeatCount': 1,
'targetCount': 1,
'telescopeTime': 0,
'templates': [ { 'etcCertified': False,
'integrationTime': 0,
'overheads': 360,
'parameters': [],
'snr': 0.0,
'templateId': 55565,
'templateName': 'XSHOOTER_slt_acq',
'type': 'acquisition'},
{ 'etcCertified': False,
'integrationTime': 0,
'overheads': 0,
'parameters': [],
'snr': 0.0,
'templateId': 55567,
'templateName': 'XSHOOTER_slt_obs_Stare',
'type': 'science'}]}
{ 'etcCertified': False,
'obsSetupId': 27005,
'referenceTargets': [],
'repeatCount': 1,
'targetCount': 1,
'telescopeTime': 0,
'templates': [ { 'etcCertified': False,
'integrationTime': 0,
'overheads': 360,
'parameters': [],
'snr': 0.0,
'templateId': 55566,
'templateName': 'XSHOOTER_slt_acq',
'type': 'acquisition'},
{ 'etcCertified': False,
'integrationTime': 0,
'overheads': 0,
'parameters': [],
'snr': 0.0,
'templateId': 55568,
'templateName': 'XSHOOTER_slt_obs_Stare',
'type': 'science'}]}
{ 'etcCertified': False,
'obsSetupId': 27004,
'referenceTargets': [],
'repeatCount': 1,
'targetCount': 1,
'telescopeTime': 0,
'templates': [ { 'etcCertified': False,
'integrationTime': 0,
'overheads': 360,
'parameters': [],
'snr': 0.0,
'templateId': 55565,
'templateName': 'XSHOOTER_slt_acq',
'type': 'acquisition'},
{ 'etcCertified': False,
'integrationTime': 0,
'overheads': 0,
'parameters': [],
'snr': 0.0,
'templateId': 55567,
'templateName': 'XSHOOTER_slt_obs_Stare',
'type': 'science'}]}
{ 'etcCertified': False,
'obsSetupId': 27005,
'referenceTargets': [],
'repeatCount': 1,
'targetCount': 1,
'telescopeTime': 0,
'templates': [ { 'etcCertified': False,
'integrationTime': 0,
'overheads': 360,
'parameters': [],
'snr': 0.0,
'templateId': 55566,
'templateName': 'XSHOOTER_slt_acq',
'type': 'acquisition'},
{ 'etcCertified': False,
'integrationTime': 0,
'overheads': 0,
'parameters': [],
'snr': 0.0,
'templateId': 55568,
'templateName': 'XSHOOTER_slt_obs_Stare',
'type': 'science'}]}
{ 'etcCertified': False,
'obsSetupId': 27004,
'referenceTargets': [],
'repeatCount': 1,
'targetCount': 1,
'telescopeTime': 0,
'templates': [ { 'etcCertified': False,
'integrationTime': 0,
'overheads': 360,
'parameters': [],
'snr': 0.0,
'templateId': 55565,
'templateName': 'XSHOOTER_slt_acq',
'type': 'acquisition'},
{ 'etcCertified': False,
'integrationTime': 0,
'overheads': 0,
'parameters': [],
'snr': 0.0,
'templateId': 55567,
'templateName': 'XSHOOTER_slt_obs_Stare',
'type': 'science'}]}
{ 'etcCertified': False,
'obsSetupId': 27005,
'referenceTargets': [],
'repeatCount': 1,
'targetCount': 1,
'telescopeTime': 0,
'templates': [ { 'etcCertified': False,
'integrationTime': 0,
'overheads': 360,
'parameters': [],
'snr': 0.0,
'templateId': 55566,
'templateName': 'XSHOOTER_slt_acq',
'type': 'acquisition'},
{ 'etcCertified': False,
'integrationTime': 0,
'overheads': 0,
'parameters': [],
'snr': 0.0,
'templateId': 55568,
'templateName': 'XSHOOTER_slt_obs_Stare',
'type': 'science'}]}
Editing an observation
We can edit the telescope time or repeat count of a particular observation as follows
observation, version = api.getObservation(targetIds[0], obsSetupIds[0]) p.pprint(observation) observation['repeatCount'] = 42 observation['telescopeTime'] = 1000 observation, _ = api.updateObservation(targetIds[0], observation, version) p.pprint(observation)
If you have any questions or suggestions for improvements, please don't hesitate to get in touch with us via Helpdesk portal.