Flyte Launcher Reference

The Scaffold Flyte Launcher is a Hydra launcher plugin that collapses the build → register → execute cycle of a Flyte workflow into a single command. This page documents what it does and how all its options work.

How invocation works

When you run python workflow.py from the __main__ block, the HydraConf(mode=RunMode.MULTIRUN) entry in the store switches Hydra to multirun mode. Multirun mode is what activates a Hydra launcher plugin — without it, the launcher is never called and the workflow runs locally via main().

if __name__ == "__main__":
    workflow_store(HydraConf(mode=RunMode.MULTIRUN))
    workflow_store.add_to_hydra_store(overwrite_ok=True)
    launcher_store.add_to_hydra_store(overwrite_ok=True)
    main()

What happens when execution_environment=remote:

  1. Hydra renders the full config and calls the launcher’s launch() method.

  2. The launcher identifies the main @workflow in the script (the one not used as a sub-workflow).

  3. It builds the Docker image(s) for the tasks (unless build_images=False).

  4. It serializes and registers the workflow and all tasks with the Flyte backend.

  5. If run=True (default), it immediately executes the registered workflow.

When execution_environment=local, the launcher runs the workflow as a standard Python call — the same as running without MULTIRUN. This is useful for testing the full launcher code path without requiring a Flyte backend.

Configuration reference

The launcher is configured via FlyteLauncherConf in launcher_conf.py. See Deployment Configuration for the full setup. The key fields:

Field

Default

Description

execution_environment

remote

remote or local. Use local to test the launcher code path without Flyte.

endpoint

localhost:30081

Flyte admin endpoint. Forward with kubectl port-forward svc/flyteadmin 30081:81 -n flyte.

build_images

True

Build and push Docker images before registration. Set False to reuse the last pushed image.

fast_serialization

False

Inject local source code into an existing container instead of rebuilding. Fast iteration, but the container must already exist with all dependencies installed.

run

True

Execute the workflow immediately after registration. Set False to register only.

workflow.version

auto (git hash)

Version string for the registered workflow. By default derived from the git branch and commit hash.

workflow.cron_schedule

None

Cron expression to schedule the workflow (e.g. "0 5 * * *" for 5am daily). See Advanced Patterns for details.

notifications

[]

List of FlyteNotificationConf objects. See Integrations.

Reusing a registered workflow version

To execute an already-registered workflow without rebuilding images:

python workflow.py hydra.launcher.build_images=false hydra.launcher.workflow.version=<version>

The version string is logged at the end of every registration and is also visible in the Flyte UI.

Projects and domains

Flyte organizes workflows by project (a logical grouping, e.g. one ML use case) and domain (environment: development, staging, production).

The launcher registers to development domain by default. The conventional mapping is:

  • development — feature branch runs from a developer’s machine

  • staging — registered by CI/CD on merge to main

  • production — registered by CI/CD on a tagged release

The workflow version string automatically encodes the git branch name and commit hash, making it easy to identify exactly what code a given registered version contains.

Override domain and project via workflow config fields:

FlyteWorkflowConf(project="my-project", domain=FlyteDomainEnum.staging, ...)

Monitoring executions

The Flyte UI shows registered workflows, execution history, task outputs and logs.

Alternatively, use kubectl to inspect the underlying pods:

kubectl get pods -n default-development
kubectl logs -n default-development <pod-name> --follow

FlyteRemote — programmatic execution

To trigger an already-registered workflow from Python (e.g. from a backend service or a notebook), use FlyteRemoteHelper:

execute_registered_workflows.py
from scaffold.flyte.flyte_utils import FlyteRemoteHelper

# when registering a workflow, flyte launcher will log the launchplan names at the end. Can also be found in UI
launchplan_name = "hydra_workflow.scaffold.main_hydra_flyte_extra_inputs.pipeline_production_0"

# using overrides is bit more convenient than changing the cfg argument, especially if configs become huge
overrides = {"strip_string": False}
string_to_be_processed = " please clean Me UP"

# execute the workflow that was registered by flyte launcher before
FlyteRemoteHelper(domain="production", admin_endpoint="flyteadmin.flyte.svc.cluster.local:81").execute_flyte_launchplan(
    launchplan_name, {"overrides": overrides, "data_string": string_to_be_processed}
)

The launchplan name is logged at the end of every registration and can also be found in the Flyte UI by clicking Launch Workflow.