scaffold.hydra.custom

This module helps you managing hydra configs and running hydra sessions with configs. Having an own runner for hydra session as a replacement for the @hydra.main annotation is necessary since the original annotation performs parsing of argparse parameters and running the sessions at the same time. To use e.g. click instead of argparse both aspects had to be disentangled.

Attributes

logger

Functions

get_cfg_from_config_fp(→ omegaconf.DictConfig)

Helper function that returns a hydra config object.

init_configstore(→ hydra.core.config_store.ConfigStore)

Method to initialise the configstore. To be called if the entrypoint gets executed as the config file should then

resolve_and_split_path(→ Tuple[pathlib.Path, str])

Check if the file exists and split the path into the directory and base name without .yaml.

resolve_rel_config_path(→ str)

Returns absolute path between relative file and folder

run_component_with_config_fp(→ Dict)

Generic entry point for an entrypoint component based on a config file path.

run_with_hydra(→ Any)

Run a function in a hydra session using the specified config.

Module Contents

scaffold.hydra.custom.get_cfg_from_config_fp(config_fp: str, config_class: dataclasses.dataclass | None = None, overrides: List[str] = None, add_schema_funcs: List[Callable] | None = None) → omegaconf.DictConfig

Helper function that returns a hydra config object. Can be used for tests or debugging and the show config command.

Before composing the config provided in config_fp, all functions in add_schema_funcs will be called.

Important

This does not work when Hydra is already initialized

Parameters:

  • config_fp (str) – absolute filepath to the component config (bundle).

  • config_class (Optional["dataclass"]) – Dataclass schema for the config.

  • overrides (List[str] = None) – Values in the config that should be overridden.

  • add_schema_funcs (Optional[List[Callable]] = None) – Functions that should be called after the config store is initialised. These functions can be used to add schemas to the config store so they can be checked against the given config file. Since these schemas can vary between components, every component implements these separately.

Returns:

The dataclass representing the config object

Raises:

FileNotFoundError – If specified config file does not exist.

scaffold.hydra.custom.init_configstore(config_name: str, config_schema: dataclasses.dataclass) → hydra.core.config_store.ConfigStore

Method to initialise the configstore. To be called if the entrypoint gets executed as the config file should then adhere to this schema.

Parameters:

  • config_name – Note that this name MUST be equal to the config file (without .yaml) name, otherwise Hydra doesn’t merge the config yaml with this config store.

  • config_schema – Schema dataclass definition for the config.

Returns: None

Raises:

ValueError – If config_name contains the file extension.

scaffold.hydra.custom.resolve_and_split_path(config_path: str) → Tuple[pathlib.Path, str]

Check if the file exists and split the path into the directory and base name without .yaml.

Parameters:

config_path – Path to the hydra config yaml.

Returns:

The absolute directory name and the file path without .yaml or .yml

Raises:

FileNotFoundError if file does not exist –

scaffold.hydra.custom.resolve_rel_config_path(a_file: str, config_path: str) → str

Returns absolute path between relative file and folder

Parameters:

  • a_file – Reference file path e.g. __file__

  • config_path – Relative path from e.g. __file__ to config

Returns: str with absolute path to config

scaffold.hydra.custom.run_component_with_config_fp(run_component_func: Callable, config_fp: str, config_bundle_class: dataclasses.dataclass | None = None, overrides: List[str] = None, add_schema_funcs: List[Callable] | None = None) → Dict

Generic entry point for an entrypoint component based on a config file path. This function should standardize how to instantiate the config store, the hydra environment and finally run the given function with the config in that environment.

Before running the component, all functions in add_schema_funcs will be called which should add all needed schemas to the hydra config store.

Important

This call will try to instantiate a new Hydra environment, so calling this function inside an environment where a global Hydra instance already exists will raise an Error. If you want to call an entrypoint component inside an existing Hydra state (i.e. in another project that also used hydra), please try to load the needed configs for that component manually.

Parameters:

  • run_component_func (Callable) – Component function that takes a config instance.

  • config_fp (str) – absolute filepath to the component config (bundle).

  • config_bundle_class (Optional["dataclass"]) – Dataclass schema for the config.

  • overrides (List[str] = None) – Values in the config that should be overridden.

  • add_schema_funcs (Optional[List[Callable]] = None) – Functions that should be called after the config store is initialised. These functions can be used to add schemas to the config store so they can be checked against the given config file. Since these schemas can vary between components, every component implements these separately.

Returns:

Return value of the run_component_func

scaffold.hydra.custom.run_with_hydra(task_function: hydra.types.TaskFunction, config_dir: str, config_name: str, overrides: List[str] = None) → Any

Run a function in a hydra session using the specified config.

Parameters:
Returns:

Return value from task_function

scaffold.hydra.custom.logger