Numina Pipeline Environment¶
This guide is intended as an introductory overview of pipeline creation with Numina. For detailed reference documentation of the functions and classes contained in the package, see the Numina Reference.
Warning
This “Pipeline Creation Guide” is still a work in progress; some of the material is not organized, and several aspects of Numina are not yet covered sufficient detail.
Execution environment of the Recipes¶
Recipes have different execution environments. Some recipes are designed to process observing modes required for the observation. These modes are related to visualization, acquisition and focusing. The Recipes are integrated in the GTC environment. We call these recipes the Data Factory Pipeline, (DFP).
Other group of recipes are devoted to scientific observing modes: imaging, spectroscopy and auxiliary calibrations. These Recipes constitute the Data Reduction Pipeline, (DRP). The software is meant to be standalone, users shall download the software and run it in their own computers, with reduction parameters and calibrations provided by the instrument team.
Users of the DRP will use the simple Numina CLI. Users of the DFP shall interact with the software through the GTC Inspector.
Recipe Example¶
In order to make the new Pipeline visible to Numina you have to:
- Create a configuration yaml file.
- Create a loader file.
- Link the entry_point option in the setup.py with the loader file.
- Create the Pipeline’s Recipes.
In the following we will continue with the same example as previously.
Configuration File¶
Create a new yaml file in the root folder named drp.yaml with the following information:
name: CLODIA
configurations:
default: {}
modes:
- date: 2015-11-12
description: Description of CLODIA Instrument
key: clodia_key
name: Clodia_Name
reference: numina
status: FINAL
summary: Summary of CLODIA Instrument
tagger: null
pipelines:
default:
recipes:
clodia_key: clodiadrp.recipes.name_of_recipe
version: 1
#products:
#- name: clodiadrp.products.TraceMap
# alias: TraceMap
Loader File¶
Create a new loader file in the root folder named loader.py with the following information:
from numina.core import drp_load
def drp_load():
'''Entry point to load CLODIA DRP.'''
return drp_load('clodiadrp', 'drp.yaml')
Link Files¶
Once we have created the loader.py file, the only thing we have to do to make CLODIA visible to Numina is to modify the setup.py file
from setuptools import setup
setup(name='clodiadrp',
entry_points = {
'numina.pipeline.1': ['CLODIA = clodiadrp.loader:drp_load'],
},
)
Recipes Creation¶
All new Recipes inherit from BaseRecipe class that can be found in numina.core.recipes so following with the example, we want to create a new recipe in the folder recipes of clodiadrp: clodiadrp.recipes.name_of_recipe with the following code:
from numina.core import Result
from numina.core.recipes import BaseRecipe
from numina.core.requirements import ObservationResultRequirement
import numpy as np
from astropy.io import fits
class name_of_recipe(BaseRecipe):
obresult = ObservationResultRequirement()
output_image = Result()
def run(self, rinput):
n = np.arange(50.0)
hdu = fits.PrimaryHDU(n)
result = self.create_result(output_image=hdu)
return result
Directory structure¶
At the end, our files in the example should follow the next directory structure:
clodiadrp
|-- recipes
| |-- __init__.py
| |-- name_of_recipe.py
|-- products
| |-- __init__.py
|-- setup.py
|-- loader.py
|-- drp.yam
Note that it is quite important to specify all required arguments (obresult) and the output (output_image). When the reduction is run from the command line using Numina CLI, it checks that the required values are provided or have default values.
Furthermore, if the Recipe is properly configured it is executed with an observing block data structure as input. Otherwise, when it is run using Numina CLI, the data structure is created from a text file.