scaffold.hydra.custom ===================== .. py:module:: scaffold.hydra.custom .. autoapi-nested-parse:: This module helps you managing hydra configs and running hydra sessions with configs. Having an own runner for hydra session as a replacement for the @hydra.main annotation is necessary since the original annotation performs parsing of argparse parameters and running the sessions at the same time. To use e.g. click instead of argparse both aspects had to be disentangled. Attributes ---------- .. autoapisummary:: scaffold.hydra.custom.logger Functions --------- .. autoapisummary:: scaffold.hydra.custom.get_cfg_from_config_fp scaffold.hydra.custom.init_configstore scaffold.hydra.custom.resolve_and_split_path scaffold.hydra.custom.resolve_rel_config_path scaffold.hydra.custom.run_component_with_config_fp scaffold.hydra.custom.run_with_hydra Module Contents --------------- .. py:function:: get_cfg_from_config_fp(config_fp: str, config_class: Optional[dataclasses.dataclass] = None, overrides: List[str] = None, add_schema_funcs: Optional[List[Callable]] = None) -> omegaconf.DictConfig Helper function that returns a hydra config object. Can be used for tests or debugging and the show config command. Before composing the config provided in `config_fp`, all functions in `add_schema_funcs` will be called. .. important:: This does not work when Hydra is already initialized :param config_fp: absolute filepath to the component config (bundle). :type config_fp: str :param config_class: Dataclass schema for the config. :type config_class: Optional["dataclass"] :param overrides: Values in the config that should be overridden. :type overrides: List[str] = None :param add_schema_funcs: Functions that should be called after the config store is initialised. These functions can be used to add schemas to the config store so they can be checked against the given config file. Since these schemas can vary between components, every component implements these separately. :type add_schema_funcs: Optional[List[Callable]] = None :returns: The dataclass representing the config object :raises FileNotFoundError: If specified config file does not exist. .. py:function:: init_configstore(config_name: str, config_schema: dataclasses.dataclass) -> hydra.core.config_store.ConfigStore Method to initialise the configstore. To be called if the entrypoint gets executed as the config file should then adhere to this schema. :param config_name: Note that this name MUST be equal to the config file (without .yaml) name, otherwise Hydra doesn't merge the config yaml with this config store. :param config_schema: Schema dataclass definition for the config. Returns: None :raises ValueError: If `config_name` contains the file extension. .. py:function:: resolve_and_split_path(config_path: str) -> Tuple[pathlib.Path, str] Check if the file exists and split the path into the directory and base name without .yaml. :param config_path: Path to the hydra config yaml. :returns: The absolute directory name and the file path without .yaml or .yml :raises FileNotFoundError if file does not exist: .. py:function:: resolve_rel_config_path(a_file: str, config_path: str) -> str Returns absolute path between relative file and folder :param a_file: Reference file path e.g. __file__ :param config_path: Relative path from e.g. __file__ to config Returns: str with absolute path to config .. py:function:: run_component_with_config_fp(run_component_func: Callable, config_fp: str, config_bundle_class: Optional[dataclasses.dataclass] = None, overrides: List[str] = None, add_schema_funcs: Optional[List[Callable]] = None) -> Dict Generic entry point for an entrypoint component based on a config file path. This function should standardize how to instantiate the config store, the hydra environment and finally run the given function with the config in that environment. Before running the component, all functions in `add_schema_funcs` will be called which should add all needed schemas to the hydra config store. .. important:: This call will try to instantiate a new Hydra environment, so calling this function inside an environment where a global Hydra instance already exists will raise an Error. If you want to call an entrypoint component inside an existing Hydra state (i.e. in another project that also used hydra), please try to load the needed configs for that component manually. :param run_component_func: Component function that takes a config instance. :type run_component_func: Callable :param config_fp: absolute filepath to the component config (bundle). :type config_fp: str :param config_bundle_class: Dataclass schema for the config. :type config_bundle_class: Optional["dataclass"] :param overrides: Values in the config that should be overridden. :type overrides: List[str] = None :param add_schema_funcs: Functions that should be called after the config store is initialised. These functions can be used to add schemas to the config store so they can be checked against the given config file. Since these schemas can vary between components, every component implements these separately. :type add_schema_funcs: Optional[List[Callable]] = None :returns: Return value of the run_component_func .. py:function:: run_with_hydra(task_function: hydra.types.TaskFunction, config_dir: str, config_name: str, overrides: List[str] = None) -> Any Run a function in a hydra session using the specified config. :param task_function: Method that should be executed. :param config_path: Absolute path to config. :param overrides: List of config parameter overrides with syntax https://hydra.cc/docs/next/advanced/override_grammar/basic/" :returns: Return value from task_function .. py:data:: logger