Uploading a Dataset

From Web-App

You can easily load a dataset using the web app.

You will first need to select a CatalogTag.

Use the load button in the Dataset section of the sidebar and the use the control panel to provide a name and file to upload.

The currently selected CatalogTag will be associated with the Dataset.

Alternatively, you can use the create button in the Dataset section of the sidebar and the use the control panel to provide a name and data values.

From python on client side

rail_pz_service.client.load.PZRailLoadClient.dataset(self, **kwargs)

Load a Dataset into the database

Parameters:: **kwargs (Any) – Input parameter. Must match LoadDatasetQuery
Returns:: Newly created and loaded dataset
Return type:: models.Dataset

Example

client = RZRailClient()
new_dataset = client.load.dataset(
    name='my_com_cam_dataset',
    path='local_version_of_data_file.hdf5',
    catalog_tag_name='com_cam',
)

or

data = dict(
    LSST_Obs_u=24.5,
    LSST_Obs_g=24.5,
    LSST_Obs_r=24.5,
    LSST_Obs_i=24.5,
    LSST_Obs_z=24.5,
    LSST_Obs_y=24.5,
    LSST_Obs_u_err=0.5,
    LSST_Obs_g_err=0.5,
    LSST_Obs_r_err=0.5,
    LSST_Obs_i_err=0.5,
    LSST_Obs_z_err=0.5,
    LSST_Obs_y_err=0.5,
)
client = RZRailClient()
new_dataset = client.load.dataset(
    name='my_com_cam_dataset',
    data=data,
    catalog_tag_name='com_cam',
)

From client CLI

pz-rail-service-client load dataset

Load CatalogTags from RailEnv

pz-rail-service-client load dataset [OPTIONS]

Options

--name <name>: Name for a particular DB row

--path <path>: Name for a particular DB row

--catalog-tag-name <catalog_tag_name>: Name of associated catalog tag

-o, --output <output>

Output format. Summary table if not specified.

Options:: yaml | json

From python on server side

async rail_pz_service.db.cache.Cache.load_dataset_from_file(self, session, name, path, catalog_tag_name, data=None)

Import a data file to the archive area and add a Dataset row

Parameters:

session (async_scoped_session) – DB session manager
name (str) – Name for new Dataset
path (Path) – Path to input file. Note that it will be copied to DB area
catalog_tag_name (str) – Name of CatalogTag that described contents of file
data (dict | None)

Returns:

Newly created Dataset

Return type:

Dataset

Raises:

RAILIntegrityError – Rows already exist in database
RAILFileNotFoundError – Input file not found
RAILBadDatasetError – Input file failed validation checks

Example

from structlog import get_logger
logger = get_logger(__name__)
cache  = pz_rail_service.db.Cache.shared_cache(logger)
new_dataset = await cache.load_dataset_from_file(
    session,
    name='my_com_cam_dataset',
    path='local_version_of_file.hdf5',
    catalog_tag_name='com_cam',
)

From server CLI

pz-rail-service-admin load dataset

Load CatalogTags from RailEnv

pz-rail-service-admin load dataset [OPTIONS]

Options

--name <name>: Name for a particular DB row

--path <path>: Name for a particular DB row

--data <data>: Dict of magnitudes

--catalog-tag-name <catalog_tag_name>: Name of associated catalog tag

-o, --output <output>

Output format. Summary table if not specified.

Options:: yaml | json