Integrations

Secrets

Tasks that need API keys or other credentials at runtime use Flyte’s secrets mechanism. The general pattern is:

Deploy a Kubernetes secret containing the credential (usually via a generator)
Request the secret in the @runtime_task decorator
Retrieve it inside the task body

WandB example:

# Generate and deploy the secret (one-time, per cluster)
mx g FlyteWandBSecret
devspace deploy --deployments=wandb_access

Request the secret in the task decorator:

from flytekit import Secret
import scaffold.wandb.helper

@runtime_task(
    requests=Resources(...),
    secret_requests=[
        Secret(key=scaffold.wandb.helper.WANDB_KEY, group=scaffold.wandb.helper.WANDB_SECRET)
    ],
)
def train_task(cfg: DictConfig, runtime_cfg: DictConfig, ...) -> ...:
    ...

For local execution and CI/CD, set the API key via environment variables (add to .zshrc):

export WANDB_USERNAME=<your GitHub handle>
export WANDB_API_KEY=<your WandB api key>

If calling wandb.init() directly (rather than via WandBCallback), call the helper first to configure the environment from the secret:

scaffold.wandb.helper.wandb_environment_setup(username=<your github handle>)
wandb.init(...)

Notifications

The launcher can send email or Slack notifications on specific workflow execution phases. Configure them in launcher_conf.py:

from hydra_plugins.flyte_launcher_plugin._flyte_launcher import (
    FlyteNotificationConf, FlyteNotificationEnum, FlyteWorkflowExecutionPhaseEnum,
)

launcher_store(
    FlyteLauncherConf(
        ...,
        notifications=[
            FlyteNotificationConf(
                type=FlyteNotificationEnum.email,
                phases=[
                    FlyteWorkflowExecutionPhaseEnum.SUCCEEDED,
                    FlyteWorkflowExecutionPhaseEnum.FAILED,
                ],
                recipients=["team@example.com"],
            ),
            FlyteNotificationConf(
                type=FlyteNotificationEnum.slack,
                phases=[FlyteWorkflowExecutionPhaseEnum.FAILED],
                recipients=["alerts@slack.example.com"],
            ),
        ],
    ),
    name="flyte",
    group="hydra/launcher",
)

For more details, see the Flyte notifications documentation.