The 4MOST Operation System: instrument and science optimization; MPE software and management contribution

J. Laas, A. Merloni, W.-F. Thi

 

Following an ESO call for proposals, a consortium led by AIP Potsdam proposed the building of the 4MOST fiber spectrograph comprising 1600 low resolution and 800 high resolution fibers feeding 3 spectrograph units. 4MOST (homepage: 4most.eu) will comprise a combination of 70% consortium surveys and 30% external community surveys, with ESO overseeing the call for proposals that was closed in December 2020. The main consortium goal in the application of this instrument is the spectroscopy of interesting targets from various space missions, in particular the GAIA and eROSITA surveys.

MPE’s contribution to this project has been that of instrument and science optimization during the design and planning stage, and now currently the design and implementation of remote operations. This is provided in the form of a Work Package within the 4MOST project named Operations System (OpSys), which is hosted at MPE but also consists of many other external members, most recently from Durham University and AIP.

Previously, as part of the preparation phase of the project, a survey simulator software system was built and hosted at MPE, for the development of algorithms to optimize survey strategy. This has continued to be an iterative and ongoing activity, tied closely with the continued development of individual consortium surveys. Based on the scope of the consortium surveys, OpSys has also been responsible for verifying that 4MOST, as designed, can indeed achieve its primary science goals.

The design and implementation of a fully-automated system to drive remote observations is now the main focus of OpSys. Using ESO’s terminology, remote observations is based on their “Designated Visitor Mode” and will consist of: i) the complete collection (and processing) of input files from each science survey such that ii) a set of components defining both the survey planner and survey scheduler may operate in an automated manner, such that iii) the facilities at ESO’s Paranal may simply routinely observe the individual Observation Blocks (OBs) defined remotely by OpSys.

Such operations is not a one-way process, as OpSys must both a) maintain an interface for constant monitoring of the instrument’s status and ambient conditions in Paranal, and b) routinely collect feedback from 4MOST’s Data Management System (DMS) to be able to determine which individual observational targets must be re-observed and take care to correctly reschedule them.

This is best illustrated in the schematic diagram here.

As was designed and implemented previously, OpSys must first begin with the input files provided by the individual surveys. This is done using the MPE-hosted 4FS Web Interface (note: access is mostly restricted only to designated survey users).

Secondly, our survey planning software must then be able to apply algorithms which implement the survey strategy, based on the well-defined input files. The strategy survey is the product of a working group dedicated to survey simulations.

Finally, our survey scheduling software must be able to a) rank and choose optimal visits in the sky which best agree with both the long-term survey strategy and the ambient conditions in Paranal, and b) assign individual objects to each of the ca. 2400 fibres so as to maximize the science products. Each chosen visit is clarified to ESO in the form of an observing queue, which the observatory is intended to follow strictly, excepting technical issues or poor weather.

From the diagram above, the OpSys acronyms are: Visit Planner (VP), Fibre-/tile-to-target Probabilities (FTP/TTP), Long-Term Scheduler (LTS), Short-Term Scheduler (STS), Fibre-Target Allocation (FTA), OB Builder (OBB), and the Survey Progress Monitor (SPM). The ESO acronyms are: Telescope Control System (TCS), Instrument Control System (ICS), and the Detector Control System (DCS).

All OpSys software at MPE is proudly based on open-source tools and libraries, notably: Apache Airflow, Django, the Astropy Project, healpy, astroplan, and PostgresQL. The individual software components are either being written from scratch or being adopted/reworked from previous software developed during earlier phases of the project’s design and analysis.