diff --git a/content/docs/13.terraform/data-sources/namespace.md b/content/docs/13.terraform/data-sources/namespace.md
index c85cc222979..5bac675db07 100644
--- a/content/docs/13.terraform/data-sources/namespace.md
+++ b/content/docs/13.terraform/data-sources/namespace.md
@@ -4,7 +4,7 @@ title: kestra_namespace
editLink: false
description: |-
Use this data source to access information about an existing Kestra Namespace.
- -> This resource is only available on the Enterprise Edition https://kestra.io/enterprise
+ -> Some attributes are only available on the Enterprise Edition https://kestra.io/enterprise
---
# kestra_namespace (Data Source)
@@ -12,7 +12,7 @@ description: |-
Use this data source to access information about an existing Kestra Namespace.
::alert{type="info"}
-This resource is only available on the [Enterprise Edition](https://kestra.io/enterprise)
+Some attributes are only available on the [Enterprise Edition](https://kestra.io/enterprise)
::
## Example Usage
@@ -28,25 +28,26 @@ data "kestra_namespace" "example" {
### Required
-- `namespace_id` (String) The namespace.
+- `namespace_id` (String) The namespace id.
### Read-Only
- `allowed_namespaces` (List of Object) The allowed namespaces. (see [below for nested schema](#nestedatt--allowed_namespaces))
-- `description` (String) The namespace friendly description.
-- `id` (String) The ID of this resource.
-- `outputs_in_internal_storage` (Boolean) Whether outputs are stored in internal storage.
-- `plugin_defaults` (String) The namespace plugin defaults.
-- `secret_configuration` (Map of String) The secret configuration.
-- `secret_isolation` (List of Object) Secret isolation configuration (same shape as storage_isolation). (see [below for nested schema](#nestedatt--secret_isolation))
-- `secret_read_only` (Boolean) Whether secrets are read-only in this namespace.
-- `secret_type` (String) The secret type.
-- `storage_configuration` (Map of String) The storage configuration.
-- `storage_isolation` (List of Object) Storage isolation configuration. (see [below for nested schema](#nestedatt--storage_isolation))
-- `storage_type` (String) The storage type.
+- `concurrency` (List of Object) The concurrency limit applied to the executions of every flow of the namespace and its descendants: `limit` and `behavior`. (see [below for nested schema](#nestedatt--concurrency))
+- `default_worker_selector` (List of Object) The default routing applied to every task of the namespace that does not define its own: `tags`, `match` and `fallback`. (see [below for nested schema](#nestedatt--default_worker_selector))
+- `description` (String) The namespace description.
+- `id` (String) The namespace id.
+- `outputs_in_internal_storage` (Boolean) Whether the task outputs are stored in the internal storage.
+- `quotas` (List of Object) The quotas evaluated before an execution starts: `duration`, `limit` and `behavior`. (see [below for nested schema](#nestedatt--quotas))
+- `secret_configuration` (Dynamic) The namespace secret configuration.
+- `secret_isolation` (List of Object) Secret isolation configuration: `enabled` and `denied_services`. (see [below for nested schema](#nestedatt--secret_isolation))
+- `secret_read_only` (Boolean) Whether the namespace secret manager is read only.
+- `secret_type` (String) The namespace secret type.
+- `storage_configuration` (Map of String) The namespace storage configuration.
+- `storage_isolation` (List of Object) Storage isolation configuration: `enabled` and `denied_services`. (see [below for nested schema](#nestedatt--storage_isolation))
+- `storage_type` (String) The namespace storage type.
- `tenant_id` (String) The tenant id.
-- `variables` (String) The namespace variables.
-- `worker_group` (List of Object) The worker group. (see [below for nested schema](#nestedatt--worker_group))
+- `variables` (String) The namespace variables, as YAML.
### Nested Schema for `allowed_namespaces`
@@ -56,28 +57,48 @@ Read-Only:
- `namespace` (String)
-
-### Nested Schema for `secret_isolation`
+
+### Nested Schema for `concurrency`
Read-Only:
-- `denied_services` (List of String)
-- `enabled` (Boolean)
+- `behavior` (String)
+- `limit` (Number)
-
-### Nested Schema for `storage_isolation`
+
+### Nested Schema for `default_worker_selector`
+
+Read-Only:
+
+- `fallback` (String)
+- `match` (String)
+- `tags` (Set of String)
+
+
+
+### Nested Schema for `quotas`
Read-Only:
-- `denied_services` (List of String)
+- `behavior` (String)
+- `duration` (String)
+- `limit` (Number)
+
+
+
+### Nested Schema for `secret_isolation`
+
+Read-Only:
+
+- `denied_services` (Set of String)
- `enabled` (Boolean)
-
-### Nested Schema for `worker_group`
+
+### Nested Schema for `storage_isolation`
Read-Only:
-- `fallback` (String)
-- `key` (String)
+- `denied_services` (Set of String)
+- `enabled` (Boolean)
diff --git a/content/docs/13.terraform/data-sources/policy.md b/content/docs/13.terraform/data-sources/policy.md
new file mode 100644
index 00000000000..b72d97ce1b8
--- /dev/null
+++ b/content/docs/13.terraform/data-sources/policy.md
@@ -0,0 +1,43 @@
+---
+# generated by https://github.com/hashicorp/terraform-plugin-docs
+title: kestra_policy
+editLink: false
+description: |-
+ Reads a Kestra governance Policy (EE) at the INSTANCE, TENANT or NAMESPACE scope.
+---
+
+# kestra_policy (Data Source)
+
+Reads a Kestra governance Policy (EE) at the `INSTANCE`, `TENANT` or `NAMESPACE` scope.
+
+## Example Usage
+
+```hcl
+data "kestra_policy" "tenant" {
+ scope = "TENANT"
+ policy_id = "deny-shell-commands"
+}
+
+data "kestra_policy" "namespace" {
+ scope = "NAMESPACE"
+ namespace = "company.team"
+ policy_id = "require-owner-label"
+}
+```
+
+
+## Schema
+
+### Required
+
+- `policy_id` (String) The policy id.
+- `scope` (String) The policy scope: `INSTANCE`, `TENANT` or `NAMESPACE`.
+
+### Optional
+
+- `namespace` (String) The namespace the policy is attached to. Required for the `NAMESPACE` scope.
+- `tenant_id` (String) The tenant id, for `TENANT` and `NAMESPACE` scopes. Defaults to the provider tenant when omitted. Must not be set for the `INSTANCE` scope.
+
+### Read-Only
+
+- `content` (String) The policy YAML source.
diff --git a/content/docs/13.terraform/data-sources/role.md b/content/docs/13.terraform/data-sources/role.md
index e5b9239de3b..1fec1f1dcbb 100644
--- a/content/docs/13.terraform/data-sources/role.md
+++ b/content/docs/13.terraform/data-sources/role.md
@@ -41,13 +41,13 @@ data "kestra_role" "by_name" {
- `description` (String) The role description.
- `id` (String) The ID of this resource.
-- `permissions` (Set of Object) The role permissions. (see [below for nested schema](#nestedatt--permissions))
+- `resources` (Set of Object) The role resource permissions. (see [below for nested schema](#nestedatt--resources))
- `tenant_id` (String) The tenant id.
-
-### Nested Schema for `permissions`
+
+### Nested Schema for `resources`
Read-Only:
-- `permissions` (List of String)
+- `actions` (List of String)
- `type` (String)
diff --git a/content/docs/13.terraform/data-sources/tenant.md b/content/docs/13.terraform/data-sources/tenant.md
index 906bbc1f80c..4a50268d4e5 100644
--- a/content/docs/13.terraform/data-sources/tenant.md
+++ b/content/docs/13.terraform/data-sources/tenant.md
@@ -4,7 +4,7 @@ title: kestra_tenant
editLink: false
description: |-
Use this data source to access information about an existing Kestra Tenant.
- -> This resource is only available on the Enterprise Edition https://kestra.io/enterprise
+ -> This data source is only available on the Enterprise Edition https://kestra.io/enterprise
---
# kestra_tenant (Data Source)
@@ -12,7 +12,7 @@ description: |-
Use this data source to access information about an existing Kestra Tenant.
::alert{type="info"}
-This resource is only available on the [Enterprise Edition](https://kestra.io/enterprise)
+This data source is only available on the [Enterprise Edition](https://kestra.io/enterprise)
::
## Example Usage
@@ -32,41 +32,63 @@ data "kestra_tenant" "example" {
### Read-Only
-- `id` (String) The ID of this resource.
+- `concurrency` (List of Object) The concurrency limit applied to the executions of every flow of the tenant: `limit` and `behavior`. (see [below for nested schema](#nestedatt--concurrency))
+- `default_worker_selector` (List of Object) The default routing applied to every task of the tenant that does not define its own: `tags`, `match` and `fallback`. (see [below for nested schema](#nestedatt--default_worker_selector))
+- `id` (String) The tenant id.
- `name` (String) The tenant name.
- `outputs_in_internal_storage` (Boolean) Whether outputs are stored in internal storage.
-- `require_existing_namespace` (Boolean) Whether the tenant requires existing namespaces.
+- `quotas` (List of Object) The quotas evaluated before an execution starts: `duration`, `limit` and `behavior`. (see [below for nested schema](#nestedatt--quotas))
+- `require_existing_namespace` (Boolean) Whether tenant requires an existing namespace.
- `secret_configuration` (Map of String) The secret configuration.
-- `secret_isolation` (List of Object) Secret isolation configuration (same shape as storage_isolation). (see [below for nested schema](#nestedatt--secret_isolation))
+- `secret_isolation` (List of Object) Secret isolation configuration: `enabled` and `denied_services`. (see [below for nested schema](#nestedatt--secret_isolation))
- `secret_read_only` (Boolean) Whether secrets are read-only in this tenant.
- `secret_type` (String) The secret type.
- `storage_configuration` (Map of String) The storage configuration.
-- `storage_isolation` (List of Object) Storage isolation configuration. (see [below for nested schema](#nestedatt--storage_isolation))
+- `storage_isolation` (List of Object) Storage isolation configuration: `enabled` and `denied_services`. (see [below for nested schema](#nestedatt--storage_isolation))
- `storage_type` (String) The storage type.
-- `worker_group` (List of Object) The worker group. (see [below for nested schema](#nestedatt--worker_group))
-
-### Nested Schema for `secret_isolation`
+
+### Nested Schema for `concurrency`
Read-Only:
-- `denied_services` (List of String)
-- `enabled` (Boolean)
+- `behavior` (String)
+- `limit` (Number)
-
-### Nested Schema for `storage_isolation`
+
+### Nested Schema for `default_worker_selector`
+
+Read-Only:
+
+- `fallback` (String)
+- `match` (String)
+- `tags` (Set of String)
+
+
+
+### Nested Schema for `quotas`
Read-Only:
-- `denied_services` (List of String)
+- `behavior` (String)
+- `duration` (String)
+- `limit` (Number)
+
+
+
+### Nested Schema for `secret_isolation`
+
+Read-Only:
+
+- `denied_services` (Set of String)
- `enabled` (Boolean)
-
-### Nested Schema for `worker_group`
+
+### Nested Schema for `storage_isolation`
Read-Only:
-- `fallback` (String)
-- `key` (String)
+- `denied_services` (Set of String)
+- `enabled` (Boolean)
diff --git a/content/docs/13.terraform/data-sources/worker_group.md b/content/docs/13.terraform/data-sources/worker_group.md
index f9f6da43350..106b3d68bdb 100644
--- a/content/docs/13.terraform/data-sources/worker_group.md
+++ b/content/docs/13.terraform/data-sources/worker_group.md
@@ -4,7 +4,7 @@ title: kestra_worker_group
editLink: false
description: |-
Use this data source to access information about an existing Kestra Worker Group.
- -> This resource is only available on the Enterprise Edition https://kestra.io/enterprise
+ -> This data source is only available on the Enterprise Edition https://kestra.io/enterprise
---
# kestra_worker_group (Data Source)
@@ -12,20 +12,36 @@ description: |-
Use this data source to access information about an existing Kestra Worker Group.
::alert{type="info"}
-This resource is only available on the [Enterprise Edition](https://kestra.io/enterprise)
+This data source is only available on the [Enterprise Edition](https://kestra.io/enterprise)
::
+## Example Usage
+```hcl
+data "kestra_worker_group" "example" {
+ group_id = "gpu-workers"
+}
+```
## Schema
### Required
-- `id` (String) The worker group id.
-- `key` (String) The worker group key.
+- `group_id` (String) The worker group identifier.
### Read-Only
-- `allowed_tenants` (String) The list of tenants allowed to use the worker group.
- `description` (String) The worker group description.
+- `id` (String) The worker group id.
+- `name` (String) The worker group display name.
+- `subscriptions` (List of Object) The Worker Queue subscriptions of the worker group: `worker_queue_id` (`default` for the global default queue), `reserved_percent` (`-1` means no reservation), and `mode` (`STRICT` or `ELASTIC`). (see [below for nested schema](#nestedatt--subscriptions))
+
+
+### Nested Schema for `subscriptions`
+
+Read-Only:
+
+- `mode` (String)
+- `reserved_percent` (Number)
+- `worker_queue_id` (String)
diff --git a/content/docs/13.terraform/data-sources/worker_queue.md b/content/docs/13.terraform/data-sources/worker_queue.md
new file mode 100644
index 00000000000..4288d53492d
--- /dev/null
+++ b/content/docs/13.terraform/data-sources/worker_queue.md
@@ -0,0 +1,39 @@
+---
+# generated by https://github.com/hashicorp/terraform-plugin-docs
+title: kestra_worker_queue
+editLink: false
+description: |-
+ Use this data source to access information about an existing Kestra Worker Queue.
+ -> This data source is only available on the Enterprise Edition https://kestra.io/enterprise
+---
+
+# kestra_worker_queue (Data Source)
+
+Use this data source to access information about an existing Kestra Worker Queue.
+
+::alert{type="info"}
+This data source is only available on the [Enterprise Edition](https://kestra.io/enterprise)
+::
+
+## Example Usage
+
+```hcl
+data "kestra_worker_queue" "example" {
+ queue_id = "gpu-queue"
+}
+```
+
+
+## Schema
+
+### Required
+
+- `queue_id` (String) The Worker Queue identifier.
+
+### Read-Only
+
+- `allowed_tenants` (Set of String) The tenants allowed to use the Worker Queue. Empty means unrestricted.
+- `description` (String) The Worker Queue description.
+- `id` (String) The Worker Queue id.
+- `name` (String) The Worker Queue human-readable name.
+- `tags` (Set of String) The canonical tag set of the Worker Queue.
diff --git a/content/docs/13.terraform/guides/namespace-migration.md b/content/docs/13.terraform/guides/namespace-migration.md
new file mode 100644
index 00000000000..ace73210a9f
--- /dev/null
+++ b/content/docs/13.terraform/guides/namespace-migration.md
@@ -0,0 +1,121 @@
+# Namespace Migration Guide
+
+Kestra 2.0 removed two settings from the namespace and tenant APIs. This provider line drops them from `kestra_namespace` and `kestra_tenant` to match:
+
+| Removed | Replaced by |
+| --- | --- |
+| `kestra_namespace.plugin_defaults` | a `kestra_policy` with `Add` rules, created for you by the instance upgrade |
+| `kestra_namespace.worker_group` | `kestra_namespace.default_worker_selector` |
+| `kestra_tenant.worker_group` | `kestra_tenant.default_worker_selector` |
+
+## Why This Is Not Just A Cleanup
+
+The 2.0 API ignores fields it does not know rather than rejecting them. A configuration that still sets `plugin_defaults` against a 2.0 instance therefore **applies successfully and changes nothing** — the provider sends the field, the API drops it, and because the field never comes back in the response no drift is ever reported. The setting is silently dead.
+
+Removing the attributes turns that silence into a plan-time error, which is the point of this change: once you upgrade, `plugin_defaults` and `worker_group` fail with `Unsupported argument` instead of pretending to work.
+
+## What Changed
+
+1. **`plugin_defaults` is gone from `kestra_namespace`.** Plugin defaults are now expressed as governance Policies. The 2.0 instance upgrade migrates them for you (see below) — you do not re-author them, but you do have to adopt the result into Terraform.
+2. **`worker_group` became `default_worker_selector`** on both `kestra_namespace` and `kestra_tenant`. Routing is now a tag set matched against Worker Queues rather than a reference to a single worker group. See the [Worker Group Migration Guide](worker-group-migration.md) for the block-level before/after and for the Worker Queue resources it depends on.
+3. **The `kestra_namespace` schema version moved to 2.** The provider's state upgrader drops both attributes from your `.tfstate` on the next `terraform plan`; no `terraform state` surgery is required.
+4. **The data sources followed**: `data.kestra_namespace` no longer exposes `plugin_defaults`, and both `data.kestra_namespace` and `data.kestra_tenant` expose `default_worker_selector` instead of `worker_group`.
+
+## Migration Steps
+
+### 1. Back up your state
+
+```bash
+terraform state pull > backup.tfstate.json
+```
+
+### 2. Upgrade the instance, then the provider
+
+The instance upgrade to 2.0 runs a data migration that converts each namespace's stored `pluginDefaults` into a NAMESPACE-scope Policy with the id `plugin-defaults`, named *Migrated plugin defaults*. Every `{type, forced, values}` entry becomes one `io.kestra.plugin.ee.rules.Add` mutate rule on `PLUGIN`, matching the plugin type by prefix (`STARTS_WITH`, reproducing the old matching) and carrying the old `forced` flag as `override`.
+
+Nothing is lost, but the resulting policy is created by the server and is **not** in your Terraform state.
+
+### 3. Update your `.tf` files
+
+Delete every `plugin_defaults` argument, and replace every `worker_group` block with a `default_worker_selector` one:
+
+```hcl
+# Before
+resource "kestra_namespace" "team" {
+ namespace_id = "company.team"
+ plugin_defaults = < backup.tfstate.json
+```
+
+### 2. Upgrade the provider
+
+The provider's built-in state upgrader rewrites the worker group identifier from `key` to `group_id` in your `.tfstate` file when you next run `terraform plan`, so the provider can read state written by the previous implementation. No `terraform state` surgery or re-import is required: entries whose worker group no longer exists on the instance are reconciled by the refresh, not by hand.
+
+### 3. Update your `.tf` files
+
+The state is migrated automatically, but your `.tf` source files must be updated manually.
+
+Resources:
+
+```hcl
+# Before
+resource "kestra_worker_group" "gpu" {
+ key = "gpu-workers"
+ name = "GPU Workers"
+}
+
+# After
+resource "kestra_worker_group" "gpu" {
+ group_id = "gpu-workers"
+ name = "GPU Workers"
+}
+```
+
+Data sources:
+
+```hcl
+# Before
+data "kestra_worker_group" "gpu" {
+ id = "gpu-workers"
+}
+
+# After
+data "kestra_worker_group" "gpu" {
+ group_id = "gpu-workers"
+}
+```
+
+References to the renamed attribute must be updated too.
+
+The `worker_group` blocks of the `kestra_namespace` and `kestra_tenant` resources are a special case: they are gone, replaced by `default_worker_selector`. A namespace or tenant no longer points at one worker group by key — it declares the tags its tasks route with, and the Worker Queue whose tag set matches picks the work up:
+
+```hcl
+# Before
+worker_group {
+ key = kestra_worker_group.gpu.key
+ fallback = "WAIT"
+}
+
+# After
+default_worker_selector {
+ tags = kestra_worker_queue.gpu.tags
+ match = "ALL"
+ fallback = "WAIT"
+}
+```
+
+There is no automatic translation for this one: a worker group key is not a tag set, so the state upgrader drops the old block and the refresh repopulates `default_worker_selector` from the instance. `match` accepts `ALL` (default — the queue tags must be a superset of yours) or `ANY`, and `fallback` gained `IGNORE` alongside `FAIL`, `WAIT` and `CANCEL`. See the [Namespace Migration Guide](namespace-migration.md) for the rest of the namespace changes.
+
+### 4. Verify
+
+```bash
+terraform plan
+```
+
+`terraform plan` reports your worker groups as being **created**: the upgrade of the instance to Kestra 2.0 removed the ones that predate it, so the refresh no longer finds them and drops them from the state. The next `terraform apply` recreates them from your configuration, with the same `group_id`. This is expected — check that the plan creates the worker groups you expect and touches nothing else, then apply.
+
+The state upgrade is not optional even though the worker groups themselves are recreated: without it this provider cannot read state written by the previous implementation at all, and every command fails with `Unable to Upgrade Resource State`. If you see that error, the provider in use is not the one that ships this guide — upgrade the provider, and restore your backup if the state was edited by hand in the meantime.
+
+## Notes
+
+- Because the worker groups are recreated rather than migrated, they come back with exactly what your configuration declares — including no Worker Queue subscription at all unless you add `subscriptions` blocks. The ids are preserved, so any worker started with that group id stays valid.
+- Routing from a namespace or tenant no longer names a worker group at all: `default_worker_selector` names tags, which a Worker Queue matches, which a worker group subscribes to. Declare the `kestra_worker_queue` before the namespaces and tenants that select its tags.
+- Worker groups and Worker Queues are instance-level resources managed by a SuperAdmin: they are not tenant-scoped, and the provider's `tenant_id` does not apply to them.
+- Both are only available on the [Enterprise Edition](https://kestra.io/enterprise), and Worker Queues additionally require Kestra 2.0 or later.
diff --git a/content/docs/13.terraform/resources/flow.md b/content/docs/13.terraform/resources/flow.md
index 51108705f42..7680de705f3 100644
--- a/content/docs/13.terraform/resources/flow.md
+++ b/content/docs/13.terraform/resources/flow.md
@@ -20,7 +20,7 @@ resource "kestra_flow" "example" {
id: my-flow
namespace: company.team
inputs:
- - name: my-value
+ - id: my-value
type: STRING
variables:
@@ -28,12 +28,12 @@ variables:
tasks:
- id: t2
- type: io.kestra.core.tasks.log.Log
+ type: io.kestra.plugin.core.log.Log
message: first {{task.id}}
level: TRACE
pluginDefaults:
- - type: io.kestra.core.tasks.log.Log
+ - type: io.kestra.plugin.core.log.Log
values:
message: third {{flow.id}}
EOT
diff --git a/content/docs/13.terraform/resources/kv.md b/content/docs/13.terraform/resources/kv.md
index d8c68641f35..bbd42bd9c22 100644
--- a/content/docs/13.terraform/resources/kv.md
+++ b/content/docs/13.terraform/resources/kv.md
@@ -3,12 +3,12 @@
title: kestra_kv
editLink: false
description: |-
- Manages a Kestra Namespace File.
+ Manages a Kestra Key-Value pair.
---
# kestra_kv (Resource)
-Manages a Kestra Namespace File.
+Manages a Kestra Key-Value pair.
diff --git a/content/docs/13.terraform/resources/namespace.md b/content/docs/13.terraform/resources/namespace.md
index 31728860750..8030d78af77 100644
--- a/content/docs/13.terraform/resources/namespace.md
+++ b/content/docs/13.terraform/resources/namespace.md
@@ -18,22 +18,38 @@ This resource is only available on the [Enterprise Edition](https://kestra.io/en
## Example Usage
```hcl
+resource "kestra_worker_queue" "gpu" {
+ queue_id = "gpu"
+ tags = ["gpu", "linux"]
+}
+
resource "kestra_namespace" "example" {
- namespace_id = "company.team"
- description = "Friendly description"
- variables = <
+### Nested Schema for `concurrency`
+
+Required:
+
+- `behavior` (String) What happens to an execution once the limit is reached.
+- `limit` (Number) The maximum number of concurrent executions.
+
+
+
+### Nested Schema for `default_worker_selector`
+
+Required:
+
+- `tags` (Set of String) The tags used to route to a matching Worker Queue (each tag is an RFC 1123 label). Required: the API rejects `match` and `fallback` without a non-empty tag set.
+
+Optional:
+
+- `fallback` (String) The strategy when no worker is available: `FAIL` (default), `WAIT`, `CANCEL` or `IGNORE`.
+- `match` (String) How the tags are matched against a Worker Queue tag set: `ALL` (default, the queue tags must be a superset) or `ANY` (they must intersect).
+
+
+
+### Nested Schema for `quotas`
+
+Required:
+
+- `behavior` (String) What happens to an execution once the quota is exhausted.
+- `duration` (String) The sliding window the quota is counted over, as an ISO-8601 duration (for example `PT1H`).
+- `limit` (Number) The maximum number of executions allowed inside the window.
+
+
### Nested Schema for `secret_isolation`
@@ -90,18 +139,6 @@ Optional:
- `denied_services` (Set of String) Set of denied services.
- `enabled` (Boolean) Whether isolation is enabled.
-
-
-### Nested Schema for `worker_group`
-
-Required:
-
-- `key` (String) The worker group key.
-
-Optional:
-
-- `fallback` (String) The fallback strategy.
-
## Import
Import is supported using the following syntax:
diff --git a/content/docs/13.terraform/resources/policy.md b/content/docs/13.terraform/resources/policy.md
new file mode 100644
index 00000000000..9e3a3e7399b
--- /dev/null
+++ b/content/docs/13.terraform/resources/policy.md
@@ -0,0 +1,106 @@
+---
+# generated by https://github.com/hashicorp/terraform-plugin-docs
+title: kestra_policy
+editLink: false
+description: |-
+ Manages a Kestra governance Policy (EE) at the INSTANCE, TENANT or NAMESPACE scope. A policy bundles mutate and validate rules applied to flows and plugins; its YAML source is persisted by the API and round-tripped verbatim. STATIC policies are declared in the Kestra configuration and cannot be managed through the API.
+---
+
+# kestra_policy (Resource)
+
+Manages a Kestra governance Policy (EE) at the `INSTANCE`, `TENANT` or `NAMESPACE` scope. A policy bundles mutate and validate rules applied to flows and plugins; its YAML source is persisted by the API and round-tripped verbatim. `STATIC` policies are declared in the Kestra configuration and cannot be managed through the API.
+
+## Example Usage
+
+```hcl
+# A tenant-scope policy narrowed to a namespace subtree
+resource "kestra_policy" "tenant" {
+ scope = "TENANT"
+ policy_id = "deny-shell-commands"
+
+ content = <
+## Schema
+
+### Required
+
+- `content` (String) The policy YAML source: `id`, optional `displayName`, `description`, `enforcement` (defaults to `ACTIVE`) and `target`, and the non-empty `rules` list mixing mutate rules (`io.kestra.plugin.ee.rules.Add`, `Delete`) and validate rules (`Deny`, `Require`, `Restrict`). The scope, tenant and namespace are carried by the resource attributes, never by the content. Diffs are compared semantically, so a change that only reindents, reorders keys or edits comments produces no plan and the source persisted by the API keeps its previous formatting; change a value to push a reformatted source.
+- `policy_id` (String) The policy id — a lowercase RFC 1123 label, unique per (scope, tenant, namespace). Must match the `id` of the YAML content.
+- `scope` (String) The policy scope: `INSTANCE` (deployment-wide, super-admin only), `TENANT` or `NAMESPACE`.
+
+### Optional
+
+- `namespace` (String) The namespace the policy is attached to. Required for the `NAMESPACE` scope, must not be set otherwise.
+- `tenant_id` (String) The tenant id, for `TENANT` and `NAMESPACE` scopes. Defaults to the provider tenant when omitted; the value is captured at create time, so changing the provider tenant later does not retarget existing policies. Must not be set for the `INSTANCE` scope.
+
+## Import
+
+Import is supported using the following syntax:
+
+The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
+
+```shell
+terraform import kestra_policy.instance INSTANCE/{{policy_id}}
+terraform import kestra_policy.tenant TENANT/{{tenant_id}}/{{policy_id}}
+terraform import kestra_policy.namespace NAMESPACE/{{tenant_id}}/{{namespace}}/{{policy_id}}
+```
diff --git a/content/docs/13.terraform/resources/role.md b/content/docs/13.terraform/resources/role.md
index 1b86b8c29ff..c19508c76c4 100644
--- a/content/docs/13.terraform/resources/role.md
+++ b/content/docs/13.terraform/resources/role.md
@@ -22,14 +22,14 @@ resource "kestra_role" "example" {
name = "Friendly name"
description = "Friendly description"
- permissions {
- type = "FLOW"
- permissions = ["READ", "UPDATE"]
+ resources {
+ type = "FLOW"
+ actions = ["VIEW", "LIST", "UPDATE", "EXECUTE"]
}
- permissions {
- type = "EXECUTION"
- permissions = ["READ", "UPDATE"]
+ resources {
+ type = "EXECUTION"
+ actions = ["VIEW", "LIST", "ACCESS_LOGS"]
}
}
```
@@ -46,20 +46,20 @@ resource "kestra_role" "example" {
- `description` (String) The role description.
- `is_default` (Boolean) The role is the default one at user creation. Only one role can be default. Latest create/update to true will be keep as default. Defaults to `false`.
- `namespace` (String) The linked namespace.
-- `permissions` (Block Set) The role permissions. (see [below for nested schema](#nestedblock--permissions))
+- `resources` (Block Set) The role resource permissions. (see [below for nested schema](#nestedblock--resources))
### Read-Only
- `id` (String) The ID of this resource.
- `tenant_id` (String) The tenant id.
-
-### Nested Schema for `permissions`
+
+### Nested Schema for `resources`
Required:
-- `permissions` (List of String) The permissions for this type.
-- `type` (String) The type of permission.
+- `actions` (List of String) The allowed actions for this resource type (e.g., VIEW, LIST, CREATE, UPDATE, DELETE).
+- `type` (String) The resource type (e.g., FLOW, EXECUTION, NAMESPACE).
## Import
diff --git a/content/docs/13.terraform/resources/template.md b/content/docs/13.terraform/resources/template.md
index f5c96f2af79..4bab0348e6a 100644
--- a/content/docs/13.terraform/resources/template.md
+++ b/content/docs/13.terraform/resources/template.md
@@ -19,7 +19,7 @@ resource "kestra_template" "example" {
content = <
-### Nested Schema for `secret_isolation`
+
+### Nested Schema for `concurrency`
-Optional:
+Required:
-- `denied_services` (List of String) List of denied services for secret isolation.
-- `enabled` (Boolean) Enable secret isolation.
+- `behavior` (String) What happens to an execution once the limit is reached.
+- `limit` (Number) The maximum number of concurrent executions.
-
-### Nested Schema for `storage_isolation`
+
+### Nested Schema for `default_worker_selector`
+
+Required:
+
+- `tags` (Set of String) The tags used to route to a matching Worker Queue (each tag is an RFC 1123 label). The API rejects `match` and `fallback` without a non-empty tag set.
Optional:
-- `denied_services` (List of String) List of denied services for isolation.
-- `enabled` (Boolean) Enable storage isolation.
+- `fallback` (String) The strategy when no worker is available: `FAIL` (default), `WAIT`, `CANCEL` or `IGNORE`.
+- `match` (String) How the tags are matched against a Worker Queue tag set: `ALL` (default, the queue tags must be a superset) or `ANY` (they must intersect).
-
-### Nested Schema for `worker_group`
+
+### Nested Schema for `quotas`
Required:
-- `fallback` (String) The fallback strategy.
-- `key` (String) The worker group key.
+- `behavior` (String) What happens to an execution once the quota is exhausted.
+- `duration` (String) The sliding window the quota is counted over, as an ISO-8601 duration (for example `PT1H`).
+- `limit` (Number) The maximum number of executions allowed inside the window.
+
+
+
+### Nested Schema for `secret_isolation`
+
+Optional:
+
+- `denied_services` (Set of String) Set of denied services.
+- `enabled` (Boolean) Whether isolation is enabled.
+
+
+
+### Nested Schema for `storage_isolation`
+
+Optional:
+
+- `denied_services` (Set of String) Set of denied services.
+- `enabled` (Boolean) Whether isolation is enabled.
## Import
diff --git a/content/docs/13.terraform/resources/worker_group.md b/content/docs/13.terraform/resources/worker_group.md
index cc644cd299e..71a41a4aa39 100644
--- a/content/docs/13.terraform/resources/worker_group.md
+++ b/content/docs/13.terraform/resources/worker_group.md
@@ -15,20 +15,68 @@ Manages a Kestra Worker Group.
This resource is only available on the [Enterprise Edition](https://kestra.io/enterprise)
::
+## Example Usage
+```hcl
+resource "kestra_worker_queue" "gpu" {
+ queue_id = "gpu-queue"
+ tags = ["gpu"]
+}
+
+resource "kestra_worker_group" "example" {
+ group_id = "gpu-workers"
+ name = "GPU Workers"
+ description = "Worker group dedicated to GPU workloads"
+
+ # Subscribe to the global default queue with no reservation
+ subscriptions {
+ worker_queue_id = "default"
+ }
+
+ # Reserve 50% of each worker's slots for the GPU queue
+ subscriptions {
+ worker_queue_id = kestra_worker_queue.gpu.queue_id
+ reserved_percent = 50
+ mode = "ELASTIC"
+ }
+}
+```
## Schema
### Required
-- `key` (String) The worker group key.
+- `group_id` (String) The worker group identifier (RFC 1123 label: lowercase alphanumerics and hyphens, must start and end with an alphanumeric, max 64 chars). Used in URLs and on the worker auth path; immutable.
### Optional
-- `allowed_tenants` (List of String) The list of tenants allowed to use the worker group.
- `description` (String) The worker group description.
+- `name` (String) The worker group display name. Defaults to the `group_id` when omitted.
+- `subscriptions` (Block List) The Worker Queue subscriptions of the worker group. Subscriptions absent from the list are dropped on update; the underlying Worker Queue is preserved. (see [below for nested schema](#nestedblock--subscriptions))
### Read-Only
-- `id` (String) The ID of this resource.
+- `id` (String) The worker group id.
+
+
+### Nested Schema for `subscriptions`
+
+Required:
+
+- `worker_queue_id` (String) The id of the Worker Queue to subscribe to. Use the reserved `default` sentinel for the global default queue.
+
+Optional:
+
+- `mode` (String) The reservation interaction mode: `STRICT` (default, reserved slots are exclusive) or `ELASTIC` (idle reserved slots may be lent to other elastic subscriptions).
+- `reserved_percent` (Number) The reserved percentage of each worker's slots guaranteed to the Worker Queue: `-1` (no reservation, default) or a value in `[1, 100]`. The sum of reserved percentages across subscriptions must not exceed 100.
+
+## Import
+
+Import is supported using the following syntax:
+
+The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
+
+```shell
+terraform import kestra_worker_group.example {{group_id}}
+```
diff --git a/content/docs/13.terraform/resources/worker_queue.md b/content/docs/13.terraform/resources/worker_queue.md
new file mode 100644
index 00000000000..4f7db5e6fcc
--- /dev/null
+++ b/content/docs/13.terraform/resources/worker_queue.md
@@ -0,0 +1,58 @@
+---
+# generated by https://github.com/hashicorp/terraform-plugin-docs
+title: kestra_worker_queue
+editLink: false
+description: |-
+ Manages a Kestra Worker Queue. Worker Queues route tasks to Worker Groups through their tag set; Worker Groups subscribe to them via the kestra_worker_group resource.
+ -> This resource is only available on the Enterprise Edition https://kestra.io/enterprise
+---
+
+# kestra_worker_queue (Resource)
+
+Manages a Kestra Worker Queue. Worker Queues route tasks to Worker Groups through their tag set; Worker Groups subscribe to them via the `kestra_worker_group` resource.
+
+::alert{type="info"}
+This resource is only available on the [Enterprise Edition](https://kestra.io/enterprise)
+::
+
+## Example Usage
+
+```hcl
+resource "kestra_worker_queue" "example" {
+ queue_id = "gpu-queue"
+ name = "GPU Queue"
+ description = "Routes GPU workloads to dedicated workers"
+ tags = ["gpu", "high-memory"]
+
+ # Optional: restrict the queue to specific tenants
+ allowed_tenants = ["production"]
+}
+```
+
+
+## Schema
+
+### Required
+
+- `queue_id` (String) The Worker Queue identifier (RFC 1123 label: lowercase alphanumerics and hyphens, must start and end with an alphanumeric, max 64 chars). Used as the routing identity; immutable. The `default` and `system` ids are reserved.
+- `tags` (Set of String) The canonical tag set of the Worker Queue (each tag is an RFC 1123 label). Must not be empty, and must be unique across Worker Queues together with `allowed_tenants`.
+
+### Optional
+
+- `allowed_tenants` (Set of String) The tenants allowed to use the Worker Queue. Omit for an unrestricted queue.
+- `description` (String) The Worker Queue description.
+- `name` (String) The Worker Queue human-readable name.
+
+### Read-Only
+
+- `id` (String) The Worker Queue id.
+
+## Import
+
+Import is supported using the following syntax:
+
+The [`terraform import` command](https://developer.hashicorp.com/terraform/cli/commands/import) can be used, for example:
+
+```shell
+terraform import kestra_worker_queue.example {{queue_id}}
+```
diff --git a/public/llms.txt b/public/llms.txt
index 55bc2f68a55..2f15093dce5 100644
--- a/public/llms.txt
+++ b/public/llms.txt
@@ -3,6 +3,8 @@
> Kestra is an open-source workflow orchestration platform. Workflows are defined declaratively in YAML as **flows**. Each flow has an `id`, a `namespace`, a list of `tasks`, and optionally `inputs`, `outputs`, `triggers`, and `variables`. Tasks reference plugins via a fully-qualified `type` such as `io.kestra.plugin.core.log.Log`. Dynamic values use `{{ ... }}` Pebble expressions referencing `inputs`, `outputs`, `trigger`, `vars`, `execution`, and other context variables. Kestra is API-first: every action available in the UI is also available via REST API.
>
> **For agents and LLMs:** Append `.md` to any `kestra.io/docs/*` URL to retrieve that page as plain Markdown, or set `Accept: text/markdown` on the request. For a full content snapshot of all pages, see [/llms-full.txt](/llms-full.txt). For the complete documentation index, see [/llms.txt](/llms.txt).
+>
+> **Versioned docs:** Documentation for previous releases is available at versioned paths — for example, `kestra.io/docs/1.3/quickstart` for the 1.3 release. Append `.md` to any versioned URL to retrieve it as plain Markdown.
## Tools & Capability Discovery
@@ -12,7 +14,7 @@
Understanding these five concepts is enough to read and write most Kestra flows. Everything else in this file builds on them.
-- **Flow** — the unit of work. A YAML file with `id`, `namespace`, `tasks`, and optional `inputs`, `outputs`, `triggers`, `variables`, `errors`, `labels`, `pluginDefaults`.
+- **Flow** — the unit of work. A YAML file with `id`, `namespace`, `tasks`, and optional `inputs`, `outputs`, `triggers`, `variables`, `errors`, `labels`.
- **Task** — a single step in a flow. Every task has an `id`, a `type` (plugin reference), and type-specific properties. Tasks can be runnable (do work) or flowable (control execution logic).
- **Plugin** — the integration and logic layer. All task types live under `io.kestra.plugin.*`. Browse at [kestra.io/plugins](https://kestra.io/plugins).
- **Expression** — `{{ ... }}` syntax powered by Pebble. Used in any dynamic property to reference inputs, prior task outputs, trigger data, secrets, and environment variables.
@@ -29,6 +31,7 @@ This section covers two distinct things: (1) guidance for AI agents interacting
- [AI Agents](https://kestra.io/docs/ai-tools/ai-agents.md): Build autonomous orchestration patterns where agents decide which tasks to run based on runtime context
- [AI Workflows](https://kestra.io/docs/ai-tools/ai-workflows.md): Patterns for building AI-native workflows — LLM calls, tool use, and multi-step inference pipelines — using Kestra tasks
- [RAG Workflows](https://kestra.io/docs/ai-tools/ai-rag-workflows.md): Retrieval-augmented generation patterns — indexing, chunking, embedding, and querying — orchestrated as Kestra flows
+- [MCP Server](https://kestra.io/docs/ai-tools/mcp-server.md): Expose Kestra flows as MCP tools for AI agents — configure MCP servers, connect Claude Desktop/Code/Cursor, and understand OSS vs EE auth options
## Authoring flows
@@ -36,15 +39,16 @@ Use this section when writing, editing, or understanding the structure of a Kest
- [Flow definition](https://kestra.io/docs/workflow-components/flow.md): All top-level flow properties — `id`, `namespace`, `tasks`, `inputs`, `outputs`, `triggers`, `variables`, `errors`, `labels`, `pluginDefaults`, `disabled`, `concurrency`
- [Tasks](https://kestra.io/docs/workflow-components/tasks.md): Runnable tasks (do work on a worker) vs flowable tasks (control execution logic), and how task properties work
-- [Flowable tasks](https://kestra.io/docs/workflow-components/tasks/flowable-tasks.md): Control flow primitives — `Sequential`, `Parallel`, `ForEach`, `ForEachItem`, `Switch`, `If`, `DAG`, `LoopUntil`, `Subflow`, `AllowFailure`, `Pause`, `WorkingDirectory`
+- [Flowable tasks](https://kestra.io/docs/workflow-components/tasks/flowable-tasks.md): Control flow primitives — `Sequential`, `Parallel`, `Loop`, `Switch`, `If`, `DAG`, `LoopUntil`, `Subflow`, `AllowFailure`, `Pause`, `WorkingDirectory`
- [Inputs](https://kestra.io/docs/workflow-components/inputs.md): Typed runtime parameters (`STRING`, `INT`, `BOOLEAN`, `FILE`, `JSON`, `ARRAY`, `ENUM`, `DATETIME`, etc.) with optional defaults and validation
- [Outputs](https://kestra.io/docs/workflow-components/outputs.md): Reference task outputs with `{{ outputs.task_id.attribute }}`, dynamic task outputs with `{{ outputs.task_id[taskrun.value].attribute }}`, and sibling outputs inside loops
-- [Triggers](https://kestra.io/docs/workflow-components/triggers.md): Start flows automatically — Schedule (cron), Flow (react to another flow's completion), Webhook, Polling, and Realtime triggers
+- [Triggers](https://kestra.io/docs/workflow-components/triggers.md): Start flows automatically — Schedule (cron), Flow (react to another flow's completion), Webhook, Polling, Realtime, and MCP Tool triggers
+- [MCP Tool Trigger](https://kestra.io/docs/workflow-components/triggers/mcp-tool-trigger.md): Register a flow as a named MCP tool — `toolName`, `title`, `toolDescription`, `mcpServer`, and `annotations` properties; flow inputs/outputs auto-mapped to JSON schema
- [Variables](https://kestra.io/docs/workflow-components/variables.md): Flow-level named values referenced as `{{ vars.name }}`; useful for values reused across multiple tasks
- [Subflows](https://kestra.io/docs/workflow-components/subflows.md): Call another flow as a task, pass inputs, wait for completion, and consume its outputs
- [Errors](https://kestra.io/docs/workflow-components/errors.md): `errors` block for flow-level error handling tasks; `AllowFailure` for marking individual tasks as non-fatal
- [Retries](https://kestra.io/docs/workflow-components/retries.md): Retry policies — constant, exponential, random — with `maxAttempt` and `maxDuration`
-- [Plugin Defaults](https://kestra.io/docs/workflow-components/plugin-defaults.md): Set default property values for a plugin type across all tasks in a flow or namespace, avoiding repetition
+- [Policies](https://kestra.io/docs/enterprise/governance/policies.md): Inject, validate, and enforce configuration rules across namespaces and tenants — Enterprise Edition replacement for the removed pluginDefaults feature
- [Concurrency](https://kestra.io/docs/workflow-components/concurrency.md): Limit simultaneous executions of a flow with `concurrency.limit` and `behavior` (queue or cancel)
## Expressions and templating
@@ -62,7 +66,7 @@ Use this section when you need to write or debug a `{{ ... }}` expression — to
Use this section to find the right `type` for a task, understand its properties, or run code in a specific language or environment.
- [Plugin marketplace](https://kestra.io/plugins): Search all available task, trigger, condition, and storage plugins with full property documentation — the definitive reference for any plugin type
-- [Core plugins](https://kestra.io/plugins/core): Built-in tasks — `Log`, `Return`, `OutputValues`, `ForEach`, `ForEachItem`, `Http.Request`, `Http.Download`, `Subflow`, and more
+- [Core plugins](https://kestra.io/plugins/core): Built-in tasks — `Log`, `Return`, `OutputValues`, `Loop`, `Http.Request`, `Http.Download`, `Subflow`, and more
- [Script tasks](https://kestra.io/docs/scripts.md): Run Python, Shell, Node.js, R, Julia, Ruby, Perl, and PowerShell inline or from files inside a flow
- [Task runners](https://kestra.io/docs/task-runners.md): Execute script tasks on remote compute — Docker, Kubernetes, AWS Batch, AWS EC2, Azure Batch, Azure VM, Google Batch, or Google Cloud Run — instead of the local worker
@@ -82,7 +86,7 @@ Use this section when deciding *how* to structure a flow or *which* pattern to u
- [Best practices overview](https://kestra.io/docs/best-practices.md): Entry point for all design guidance
- [Flow design](https://kestra.io/docs/best-practices/flows.md): How to structure, split, and reuse flows — when to use subflows vs tasks, and how to avoid oversized executions
-- [ForEach vs ForEachItem](https://kestra.io/docs/best-practices/foreach-and-foreachitem.md): When to use each loop primitive, how to access sibling outputs inside loops with `outputs.task_id[taskrun.value]`, and common mistakes
+- [Loop task best practices](https://kestra.io/docs/best-practices/loop.md): How to use the Loop task, how to access sibling outputs inside loops with `outputs.task_id[taskrun.value]`, and common mistakes
- [Outputs patterns](https://kestra.io/docs/best-practices/outputs.md): Clean patterns for producing and consuming task outputs across tasks and flows
- [Credentials vs Secrets vs KV Store](https://kestra.io/docs/best-practices/credentials-vs-secrets-vs-kv-store.md): Decision guide for choosing the right storage backend for each type of value
- [Secrets management](https://kestra.io/docs/best-practices/secrets-management.md): How to handle credentials and sensitive values safely in flows
@@ -111,7 +115,7 @@ Use this section when interacting with Kestra programmatically — deploying flo
Enterprise and Cloud features require a Kestra EE license or a Kestra Cloud account. OSS deployments do not have access to the features in this section. Use this section when setting up authentication, governance, multi-tenancy, or scale-oriented features.
- [Enterprise overview](https://kestra.io/docs/enterprise.md): Full feature set, setup guidance, and migration path from OSS
-- [OSS vs Enterprise](https://kestra.io/docs/oss-vs-paid.md): Side-by-side feature comparison across Open Source, Enterprise, and Cloud editions
+- [OSS vs Enterprise](https://kestra.io/docs/oss-vs-paid.md): Side-by-side feature comparison across Open Source, Enterprise, and Cloud editions — covers auth, governance (including Policies), scalability, and when to upgrade
**Authentication and access control**
- [RBAC](https://kestra.io/docs/enterprise/auth/rbac.md): Role-based access control — define roles with namespace-scoped permissions and assign them to users or service accounts
@@ -125,10 +129,13 @@ Enterprise and Cloud features require a Kestra EE license or a Kestra Cloud acco
- [Secrets Manager](https://kestra.io/docs/enterprise/governance/secrets-manager.md): Connect Kestra to an external secrets backend — AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, or HashiCorp Vault
- [Namespace management](https://kestra.io/docs/enterprise/governance/namespace-management.md): Govern allowed plugins, allowed task types, and inherited defaults at the namespace level
- [Worker isolation](https://kestra.io/docs/enterprise/governance/worker-isolation.md): Restrict which workers can execute tasks from a given namespace
+- [Promote](https://kestra.io/docs/enterprise/governance/promote.md): Copy flows between Kestra instances from the UI — diff review, optional confirmation gate, drift detection, and promotion history without a Git pipeline
**Scalability**
-- [Worker groups](https://kestra.io/docs/enterprise/scalability/worker-group.md): Route tasks to labeled worker pools — use `workerGroup.key` on any task to target a specific pool
+- [Worker groups](https://kestra.io/docs/enterprise/scalability/worker-group.md): Route tasks to labeled worker pools using `workerSelector.tags` on any task; fallback behavior when no matching worker is available is configurable per task
- [Apps](https://kestra.io/docs/enterprise/scalability/apps.md): Build and publish lightweight form-based UIs on top of flows for non-technical users
+- [Flow Quotas](https://kestra.io/docs/workflow-components/quotas.md): Cap how many executions a flow can create within a time window — use `CANCEL` or `FAIL` behavior when the limit is exceeded
+- [Reusable Inputs](https://kestra.io/docs/workflow-components/reusable-inputs.md): Define a named input group once at the namespace level and reference it across multiple flows via `type: REUSABLE_INPUTS`; children resolve as `{{ inputs.. }}`
**Instance management**
- [Maintenance mode](https://kestra.io/docs/enterprise/instance/maintenance-mode.md): Pause execution scheduling without stopping the server
@@ -150,15 +157,17 @@ Use this section when building custom plugins, managing flows in Git, deploying
- [Plugin developer guide](https://kestra.io/docs/plugin-developer-guide.md): Build, test, document, and publish custom Kestra plugins in Java
- [Version control and CI/CD](https://kestra.io/docs/version-control-cicd.md): GitOps patterns — Git push/pull for flows, and automated deployment pipelines using GitHub Actions, GitLab CI, and others
-- [No Code editor](https://kestra.io/docs/ui/flows.md): Build and configure flows visually without writing YAML — form-based editing that generates schema-validated YAML in real time
+- [Promote (EE)](https://kestra.io/docs/version-control-cicd/promote.md): Move flows between environments directly from the Kestra UI with a diff review and confirmation gate — no Git or pipeline required
+- [VS Code extension](https://kestra.io/docs/version-control-cicd/vscode.md): Edit flows in VS Code with instance-aware validation, Pebble autocompletion, topology preview, and run-from-editor with live log streaming
+- [No Code editor](https://kestra.io/docs/ui/flows.md): Build flows visually using the No Code canvas — drag-and-drop editing that stays in three-way sync with the YAML editor and AI Copilot in real time
## Reference
Use this section for definitions, architecture questions, or understanding Kestra's internal design.
- [Glossary](https://kestra.io/docs/glossary.md): Definitions of Kestra-specific and general orchestration terms
-- [Architecture](https://kestra.io/docs/architecture.md): How Kestra's components fit together — Webserver, Scheduler, Executor, Worker, Queue, and storage layers
-- [Why Kestra](https://kestra.io/docs/why-kestra.md): How Kestra compares to other orchestration tools and where it is best suited
+- [Architecture](https://kestra.io/docs/architecture.md): How Kestra's components fit together — six server roles (Executor, Worker Controller, Worker, Scheduler, Webserver, Indexer), pluggable queue backends, and storage layers
+- [Why Kestra](https://kestra.io/docs/why-kestra.md): Positioning and market context — why Kestra is built for data, AI, and infrastructure workflows across the whole organization, not just a single team
## Getting started
@@ -179,14 +188,16 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Docker-in-Docker Behind a Proxy: Kestra on Kubernetes](https://kestra.io/docs/administrator-guide/dind-behind-proxy.md)
- [High Availability in Kestra: Scale Workers and Webservers](https://kestra.io/docs/administrator-guide/high-availability.md)
- [JVM CPU Limits for Kestra on Kubernetes](https://kestra.io/docs/administrator-guide/jvm-cpu-limits.md)
+- [External Log Data Store in Kestra](https://kestra.io/docs/administrator-guide/log-data-store.md)
- [MITM Proxy: Inspect Kestra's Outbound HTTPS Traffic](https://kestra.io/docs/administrator-guide/mitm-proxy-configuration.md)
- [Kestra Monitoring: Prometheus, Alerts, and Health Checks](https://kestra.io/docs/administrator-guide/monitoring.md)
- [OpenTelemetry for Kestra: Traces, Metrics, and Logs](https://kestra.io/docs/administrator-guide/open-telemetry.md)
- [Prometheus Metrics for Kestra: Reference and /prometheus Endpoint](https://kestra.io/docs/administrator-guide/prometheus-metrics.md)
-- [Purge Executions, Logs, and Files in Kestra](https://kestra.io/docs/administrator-guide/purge.md)
+- [Purge Executions, Logs, and Files in Kestra](https://kestra.io/docs/administrator-guide/purge.md): Scheduled purge patterns for executions, logs, KV pairs, namespace files, and orphaned execution storage; includes the isolated-worker-group cleanup pattern using PurgeStorage
- [Software and Hardware Requirements to Run Kestra](https://kestra.io/docs/administrator-guide/requirements.md)
- [Security Hardening for Kestra: Network and Process Isolation](https://kestra.io/docs/administrator-guide/security-hardening.md)
- [Server Heartbeats and Job Recovery in Kestra](https://kestra.io/docs/administrator-guide/server-lifecycle.md)
+- [Service Instance Metrics in Kestra – Expose via the Webserver](https://kestra.io/docs/administrator-guide/service-instance-metrics.md)
- [SSL/TLS Configuration: Enable HTTPS for Kestra](https://kestra.io/docs/administrator-guide/ssl-configuration.md)
- [Troubleshoot Kestra: Kubernetes, Docker, and Startup Issues](https://kestra.io/docs/administrator-guide/troubleshooting.md)
- [Upgrade Kestra: Rolling Updates, Migrations, and Rollback](https://kestra.io/docs/administrator-guide/upgrades.md)
@@ -199,6 +210,7 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [AI Copilot in Kestra – Generate and Edit Flows](https://kestra.io/docs/ai-tools/ai-copilot.md)
- [RAG Workflows in Kestra – Retrieval-Augmented Generation](https://kestra.io/docs/ai-tools/ai-rag-workflows.md)
- [AI Workflows in Kestra: Orchestrate with Any LLM](https://kestra.io/docs/ai-tools/ai-workflows.md)
+- [MCP Server in Kestra – Expose Flows as AI Tools](https://kestra.io/docs/ai-tools/mcp-server.md)
- [API Reference: Enterprise and Open Source Editions](https://kestra.io/docs/api-reference.md)
- [Cloud & Enterprise API Reference for Kestra](https://kestra.io/docs/api-reference/enterprise.md)
- [SDK Language Clients for the Kestra API](https://kestra.io/docs/api-reference/kestra-sdk.md)
@@ -219,7 +231,7 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Expressions with Namespace Files in Kestra](https://kestra.io/docs/best-practices/expressions-with-namespace-files.md)
- [Choose the Right Fetch Pattern in Kestra](https://kestra.io/docs/best-practices/fetch-patterns.md)
- [Flow Best Practices: Performance and Reliability](https://kestra.io/docs/best-practices/flows.md)
-- [ForEach vs ForEachItem in Kestra: When to Use Each](https://kestra.io/docs/best-practices/foreach-and-foreachitem.md)
+- [Loop Task Best Practices in Kestra](https://kestra.io/docs/best-practices/loop.md)
- [Dev to Production in Kestra: Promote Flows Safely](https://kestra.io/docs/best-practices/from-dev-to-prod.md)
- [Version Control with Git in Kestra](https://kestra.io/docs/best-practices/git.md)
- [Managing Environments in Kestra: Dev, Prod & Tenants](https://kestra.io/docs/best-practices/manage-environments.md)
@@ -262,7 +274,7 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Credentials in Kestra: Authenticate External Systems](https://kestra.io/docs/enterprise/auth/credentials.md)
- [Invitations in Kestra Enterprise: Onboard Users](https://kestra.io/docs/enterprise/auth/invitations.md)
- [RBAC in Kestra Enterprise: Roles and Permissions](https://kestra.io/docs/enterprise/auth/rbac.md)
-- [RBAC Permissions Reference for Kestra Enterprise](https://kestra.io/docs/enterprise/auth/rbac/permissions-reference.md)
+- [RBAC Resources and Actions Mapped to API Endpoints](https://kestra.io/docs/enterprise/auth/rbac/permissions-reference.md)
- [SCIM Directory Sync in Kestra Enterprise](https://kestra.io/docs/enterprise/auth/scim.md)
- [authentik SCIM Provisioning in Kestra](https://kestra.io/docs/enterprise/auth/scim/authentik.md)
- [Keycloak SCIM Provisioning in Kestra](https://kestra.io/docs/enterprise/auth/scim/keycloak.md)
@@ -283,16 +295,21 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Audit Logs in Kestra: Governance and Compliance](https://kestra.io/docs/enterprise/governance/audit-logs.md)
- [Custom Blueprints in Kestra Enterprise: Templates](https://kestra.io/docs/enterprise/governance/custom-blueprints.md)
- [Log Shipper in Kestra Enterprise: Centralize Logs](https://kestra.io/docs/enterprise/governance/logshipper.md)
+- [Policies in Kestra Enterprise: Governance Rules](https://kestra.io/docs/enterprise/governance/policies.md)
- [Namespace Management in Kestra Enterprise: Isolation](https://kestra.io/docs/enterprise/governance/namespace-management.md)
- [External Secrets Manager in Kestra: AWS, Azure, GCP](https://kestra.io/docs/enterprise/governance/secrets-manager.md)
- [Multi-Tenancy in Kestra: Configure Tenants](https://kestra.io/docs/enterprise/governance/tenants.md)
- [Unit Tests in Kestra Enterprise: Validate Flows](https://kestra.io/docs/enterprise/governance/unit-tests.md)
- [Worker Isolation in Kestra Enterprise: Separation](https://kestra.io/docs/enterprise/governance/worker-isolation.md)
+- [Cases in Kestra: Incident Management for Executions](https://kestra.io/docs/enterprise/governance/cases.md)
+- [Promote Flows Across Environments in Kestra](https://kestra.io/docs/enterprise/governance/promote.md)
- [Instance Management in Kestra Enterprise: Health](https://kestra.io/docs/enterprise/instance.md)
- [Announcements in Kestra Enterprise: In-App Banners](https://kestra.io/docs/enterprise/instance/announcements.md)
- [Kill Switch in Kestra Enterprise: Stop Executions](https://kestra.io/docs/enterprise/instance/kill-switch.md)
- [Maintenance Mode in Kestra Enterprise: Safe Upgrades](https://kestra.io/docs/enterprise/instance/maintenance-mode.md)
- [Versioned Plugins in Kestra Enterprise: Multi-Version](https://kestra.io/docs/enterprise/instance/versioned-plugins.md)
+- [Instance Owner Console in Kestra Enterprise](https://kestra.io/docs/enterprise/instance/instance-owner.md)
+- [Kestra Cloud: Getting Started](https://kestra.io/docs/enterprise/cloud-getting-started.md): Set up your Kestra Cloud instance, run your first flow, add secrets, invite your team, and connect to production.
- [Enterprise Edition in Kestra: Architecture and Setup](https://kestra.io/docs/enterprise/overview.md)
- [Enterprise Features in Kestra: High-Availability](https://kestra.io/docs/enterprise/overview/enterprise-edition.md)
- [Migrate from OSS to Kestra Enterprise Edition](https://kestra.io/docs/enterprise/overview/migrate-from-oss.md)
@@ -300,7 +317,6 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Install Kestra Enterprise from Standalone JAR](https://kestra.io/docs/enterprise/overview/standalone-server-installation.md)
- [Scale Kestra Enterprise: Worker Groups and Apps](https://kestra.io/docs/enterprise/scalability.md)
- [Apps in Kestra Enterprise: Frontends for Flows](https://kestra.io/docs/enterprise/scalability/apps.md)
-- [Task Runners in Kestra Enterprise: Offload Compute](https://kestra.io/docs/enterprise/scalability/task-runners.md)
- [Worker Groups in Kestra Enterprise: Target Workers](https://kestra.io/docs/enterprise/scalability/worker-group.md)
- [Expressions in Kestra: Pebble Syntax and Variables](https://kestra.io/docs/expressions.md)
- [Kestra Expression Context: Inputs, Outputs & Variables](https://kestra.io/docs/expressions/context.md)
@@ -324,12 +340,14 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Configure Alerts in Kestra](https://kestra.io/docs/how-to-guides/alerting.md)
- [Audit Machines and Tool Versions with Ansible in Kestra](https://kestra.io/docs/how-to-guides/ansible.md)
- [Detect Ansible Config Drift with Kestra](https://kestra.io/docs/how-to-guides/ansible-config-drift.md)
-- [Control Ansible Playbook Outputs to Protect Sensitive Data](https://kestra.io/docs/how-to-guides/ansible-explicit-outputs.md)
+- [Declare Explicit Outputs in Ansible Playbooks with Kestra](https://kestra.io/docs/how-to-guides/ansible-explicit-outputs.md)
- [Extend Kestra with the API](https://kestra.io/docs/how-to-guides/api.md)
- [Deploy Kestra with ArgoCD](https://kestra.io/docs/how-to-guides/argocd.md)
- [Use Azure Managed Workload Identity with Kestra](https://kestra.io/docs/how-to-guides/azure-workload-id.md)
+- [Automate Clever Cloud with Kestra](https://kestra.io/docs/how-to-guides/clever-cloud.md)
- [Add Conditional Branching in Kestra](https://kestra.io/docs/how-to-guides/conditional-branching.md)
- [Build a Custom Plugin for Kestra](https://kestra.io/docs/how-to-guides/custom-plugin.md)
+- [Build a KPI Success Ratio Chart in Kestra Dashboards](https://kestra.io/docs/how-to-guides/dashboard-kpi-chart.md)
- [Use Dataform in Kestra](https://kestra.io/docs/how-to-guides/dataform.md)
- [Manage dbt Projects with Kestra's Code Editor](https://kestra.io/docs/how-to-guides/dbt.md)
- [Use Debezium Tasks and Triggers in Kestra](https://kestra.io/docs/how-to-guides/debezium.md)
@@ -337,20 +355,18 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Build Dynamic Flows in Kestra](https://kestra.io/docs/how-to-guides/dynamic-flows.md)
- [Create a Dynamic Dropdown for Inputs](https://kestra.io/docs/how-to-guides/dynamic-inputs.md)
- [Build ETL Pipelines in Kestra](https://kestra.io/docs/how-to-guides/etl-pipelines.md)
+- [Generate and Pseudonymize Test Data in Kestra](https://kestra.io/docs/how-to-guides/generate-and-pseudonymize-data.md)
- [Validate and Deploy Flows with GitHub Actions](https://kestra.io/docs/how-to-guides/github-actions.md)
- [Back Up GitHub Repos with Kestra Playground](https://kestra.io/docs/how-to-guides/github-repo-backup.md)
-- [Run Go Inside Your Flows](https://kestra.io/docs/how-to-guides/golang.md)
- [Configure a Google Service Account in Kestra](https://kestra.io/docs/how-to-guides/google-credentials.md)
- [Connect Google Sheets to Kestra](https://kestra.io/docs/how-to-guides/google-sheets.md)
- [Make HTTP Requests Inside Your Flows](https://kestra.io/docs/how-to-guides/http-request.md)
- [Automate IAM Management with Tasks](https://kestra.io/docs/how-to-guides/iam-automation.md)
- [Prevent Duplicate Executions with Correlation IDs](https://kestra.io/docs/how-to-guides/idempotency.md)
-- [Pass Inputs via an API Call](https://kestra.io/docs/how-to-guides/inputs-api.md)
-- [Validate Inputs with the Enum Data Type](https://kestra.io/docs/how-to-guides/inputs-enum.md)
- [Run JavaScript Inside Your Flows](https://kestra.io/docs/how-to-guides/javascript.md)
- [Connect Web Apps to Kestra via Webhooks](https://kestra.io/docs/how-to-guides/js-webhook.md)
- [Work with JSON in Kestra](https://kestra.io/docs/how-to-guides/json.md)
-- [Run Julia Inside Your Flows](https://kestra.io/docs/how-to-guides/julia.md)
+
- [Set Up Secrets from a Helm Chart](https://kestra.io/docs/how-to-guides/kubernetes-secrets.md)
- [Synchronize Local Flows in Kestra](https://kestra.io/docs/how-to-guides/local-flow-sync.md)
- [Long-Running Tasks on Kubernetes in Kestra](https://kestra.io/docs/how-to-guides/long-running-intensive-tasks.md)
@@ -366,7 +382,6 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Deploy to OpenShift with Kestra](https://kestra.io/docs/how-to-guides/openshift.md)
- [Parallel vs. Sequential Tasks in Kestra](https://kestra.io/docs/how-to-guides/parallel-vs-sequential.md)
- [Pause and Resume Flows in Kestra](https://kestra.io/docs/how-to-guides/pause-resume.md)
-- [Run Perl Inside Your Flows](https://kestra.io/docs/how-to-guides/perl.md)
- [Populate Your Instance with Sample Data](https://kestra.io/docs/how-to-guides/populate-demo-data.md)
- [Run PowerShell Inside Your Flows](https://kestra.io/docs/how-to-guides/powershell.md)
- [Trigger a Flow on a Prometheus Alert](https://kestra.io/docs/how-to-guides/prometheus-alert-trigger.md)
@@ -375,18 +390,13 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Run Python Inside Your Flows](https://kestra.io/docs/how-to-guides/python.md)
- [Manage Python Dependencies in Kestra](https://kestra.io/docs/how-to-guides/python-dependencies.md)
- [Manage Python Dependencies with uv in Kestra](https://kestra.io/docs/how-to-guides/python-uv.md)
-- [Run R Inside Your Flows](https://kestra.io/docs/how-to-guides/r.md)
- [Realtime Triggers in Kestra: Kafka, SQS, Pub/Sub](https://kestra.io/docs/how-to-guides/realtime-triggers.md)
-- [Revision History and Rollback in Kestra](https://kestra.io/docs/how-to-guides/rollback-and-revision-history.md)
-- [Run Ruby Inside Your Flows](https://kestra.io/docs/how-to-guides/ruby.md)
-- [Run Rust Inside Your Flows](https://kestra.io/docs/how-to-guides/rust.md)
- [Build SecOps Workflows with Kestra](https://kestra.io/docs/how-to-guides/secops-with-kestra.md)
- [Configure Secrets in Kestra](https://kestra.io/docs/how-to-guides/secrets.md)
- [Install Only Selected Plugins in Kestra OSS](https://kestra.io/docs/how-to-guides/selected-plugin-installation.md)
- [Trigger Kestra Flows from ServiceNow](https://kestra.io/docs/how-to-guides/servicenow-trigger.md)
- [Trigger Kestra Flows from Splunk Alerts](https://kestra.io/docs/how-to-guides/splunk-alert-trigger.md)
- [Run Shell Scripts Inside Your Flows](https://kestra.io/docs/how-to-guides/shell.md)
-- [Migrate from Shipyard to Kestra](https://kestra.io/docs/how-to-guides/shipyard-migration.md)
- [Slack Events API with Kestra: Trigger Flows](https://kestra.io/docs/how-to-guides/slack-webhook.md)
- [Use SQLMesh to Run dbt Projects](https://kestra.io/docs/how-to-guides/sqlmesh.md)
- [Subflow Retries, Restarts, and Replays in Kestra](https://kestra.io/docs/how-to-guides/subflow-executions.md)
@@ -510,6 +520,22 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Enterprise License Upgrade in Kestra 1.3.0](https://kestra.io/docs/migration-guide/v1.3.0/ee-license-upgrade.md)
- [File-Listing Plugins Default to 25 Results in 1.3.0](https://kestra.io/docs/migration-guide/v1.3.0/file-listing-default-limit.md)
- [LTS Migration: Kestra 1.0 to 1.3 Upgrade Guide](https://kestra.io/docs/migration-guide/v1.3.0/lts-migration.md)
+- [2.0.0](https://kestra.io/docs/migration-guide/v2.0.0.md)
+- [Check.condition Renamed to when](https://kestra.io/docs/migration-guide/v2.0.0/checks-condition-renamed-when.md)
+- [ForEach and ForEachItem Replaced by Loop](https://kestra.io/docs/migration-guide/v2.0.0/foreach-loop.md)
+- [ION Output Files Are Now Binary](https://kestra.io/docs/migration-guide/v2.0.0/ion-binary-format.md)
+- [json() Function Removed](https://kestra.io/docs/migration-guide/v2.0.0/json-function-removed.md)
+- [local.Delete recursive Default Changed to false](https://kestra.io/docs/migration-guide/v2.0.0/local-delete-recursive-default.md)
+- [Database Migrations in Kestra 2.0.0](https://kestra.io/docs/migration-guide/v2.0.0/database-migrations.md)
+- [pluginDefaults Removed](https://kestra.io/docs/migration-guide/v2.0.0/plugin-defaults-removed.md)
+- [pluginDefaults.forced Removed from Flows](https://kestra.io/docs/migration-guide/v2.0.0/plugin-defaults-forced-removed.md)
+- [RBAC Permissions Replaced by Resource and Action Model](https://kestra.io/docs/migration-guide/v2.0.0/rbac-action-model.md)
+- [Execution API Response Shape Changed](https://kestra.io/docs/migration-guide/v2.0.0/execution-api-response.md)
+- [Trigger Conditions Redesign](https://kestra.io/docs/migration-guide/v2.0.0/trigger-conditions-redesign.md)
+- [Helm Chart gRPC Worker-Controller Changes in Kestra 2.0.0](https://kestra.io/docs/migration-guide/v2.0.0/helm-grpc-worker-controller.md)
+- [Management Endpoint Hardening in Kestra 2.0.0](https://kestra.io/docs/migration-guide/v2.0.0/management-endpoint-hardening.md)
+- [Execution Data in Internal Storage Removed](https://kestra.io/docs/migration-guide/v2.0.0/execution-data-internal-storage.md)
+- [Super Admin Renamed to Instance Owner](https://kestra.io/docs/migration-guide/v2.0.0/superadmin-renamed-instance-owner.md)
- [Open-Source vs. Enterprise Edition of Kestra](https://kestra.io/docs/oss-vs-paid.md)
- [Performance in Kestra: Benchmarks and Tuning](https://kestra.io/docs/performance.md)
- [Benchmarks: Orchestration Throughput & Latency](https://kestra.io/docs/performance/benchmark.md)
@@ -519,6 +545,7 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Develop a Kestra Condition Plugin](https://kestra.io/docs/plugin-developer-guide/condition.md)
- [Plugin Contribution Guidelines for Kestra](https://kestra.io/docs/plugin-developer-guide/contribution-guidelines.md)
- [Document Your Kestra Plugin with Annotations](https://kestra.io/docs/plugin-developer-guide/document.md)
+- [Develop a Kestra File Renderer Plugin](https://kestra.io/docs/plugin-developer-guide/file-renderer.md)
- [Gradle Configuration for Kestra Plugins](https://kestra.io/docs/plugin-developer-guide/gradle.md)
- [Build and Publish a Kestra Plugin](https://kestra.io/docs/plugin-developer-guide/publish.md)
- [Set Up for Kestra Plugin Development](https://kestra.io/docs/plugin-developer-guide/setup.md)
@@ -527,6 +554,7 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Add Unit Tests for Kestra Plugins](https://kestra.io/docs/plugin-developer-guide/unit-tests.md)
- [Quickstart Guide: Run Your First Kestra Workflow](https://kestra.io/docs/quickstart.md)
- [Releases & LTS Policy in Kestra: Cadence and Support](https://kestra.io/docs/releases.md)
+- [Kestra Changelog](https://kestra.io/docs/releases/changelog.md)
- [Run Scripts in Kestra: Multi-Language Tasks & Runners](https://kestra.io/docs/scripts.md)
- [Bind Mount Scripts into Kestra – Run Local Code](https://kestra.io/docs/scripts/bind-mount.md)
- [Commands vs Script Tasks in Kestra](https://kestra.io/docs/scripts/commands-vs-scripts.md)
@@ -550,8 +578,10 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Azure Batch Task Runner: Run Tasks on Azure Containers](https://kestra.io/docs/task-runners/types/azure-batch-task-runner.md)
- [Azure Virtual Machine Task Runner – Run Tasks Natively on Azure VMs](https://kestra.io/docs/task-runners/types/azure-virtualmachine-task-runner.md)
- [Docker Task Runner – Run Tasks in Containers](https://kestra.io/docs/task-runners/types/docker-task-runner.md)
+- [Huawei Cloud CCI Task Runner: Serverless Container Execution](https://kestra.io/docs/task-runners/types/huawei-cci-task-runner.md)
- [Google Batch Task Runner: Run Tasks on Cloud VMs](https://kestra.io/docs/task-runners/types/google-batch-task-runner.md)
- [Google Cloud Run Task Runner: Serverless Task Execution](https://kestra.io/docs/task-runners/types/google-cloudrun-task-runner.md)
+- [Google Compute Engine Task Runner: Run Tasks on Compute Engine VMs](https://kestra.io/docs/task-runners/types/google-computeengine-task-runner.md)
- [Kubernetes Task Runner – Run Tasks as K8s Pods](https://kestra.io/docs/task-runners/types/kubernetes-task-runner.md)
- [Process Task Runner – Run Tasks as Local Processes](https://kestra.io/docs/task-runners/types/process-task-runner.md)
- [Kestra Terraform Provider: Manage Resources as IaC](https://kestra.io/docs/terraform.md)
@@ -623,9 +653,9 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [GitHub Actions for Kestra – CI/CD Workflow Examples](https://kestra.io/docs/version-control-cicd/cicd/github-action.md)
- [GitLab CI for Kestra: Automate Flow Validation](https://kestra.io/docs/version-control-cicd/cicd/gitlab.md)
- [CI/CD Helpers in Kestra: Local Dev and Read-Only Flows](https://kestra.io/docs/version-control-cicd/cicd/helpers.md)
-- [Kubernetes Operator in Kestra: GitOps for Flows](https://kestra.io/docs/version-control-cicd/cicd/kubernetes-operator.md)
- [Terraform for Kestra – Manage Resources as Code](https://kestra.io/docs/version-control-cicd/cicd/terraform.md)
- [Version Control with Git: Sync, Push, and Clone Flows](https://kestra.io/docs/version-control-cicd/git.md)
+- [Kestra VS Code Extension: Edit, Validate, and Run Flows](https://kestra.io/docs/version-control-cicd/vscode.md)
- [Why Kestra: Simpler, More Powerful Orchestration](https://kestra.io/docs/why-kestra.md)
- [Workflow Components in Kestra: Complete Reference](https://kestra.io/docs/workflow-components.md)
- [afterExecution Tasks in Kestra – Post-Run Actions](https://kestra.io/docs/workflow-components/afterexecution.md)
@@ -641,8 +671,9 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Labels in Kestra – Tag Flows and Executions](https://kestra.io/docs/workflow-components/labels.md)
- [Namespaces in Kestra – Organize and Secure Workflows](https://kestra.io/docs/workflow-components/namespace.md)
- [Workflow Outputs in Kestra: Share Data Between Tasks](https://kestra.io/docs/workflow-components/outputs.md)
-- [Plugin Defaults in Kestra – Set Task-Level Defaults](https://kestra.io/docs/workflow-components/plugin-defaults.md)
- [Plugins in Kestra: Tasks, Triggers, Integrations](https://kestra.io/docs/workflow-components/plugins.md)
+- [Flow Quotas in Kestra: Rate-Limit Execution Creation](https://kestra.io/docs/workflow-components/quotas.md)
+- [Reusable Inputs in Kestra Enterprise](https://kestra.io/docs/workflow-components/reusable-inputs.md)
- [Task Retries in Kestra – Handle Transient Failures](https://kestra.io/docs/workflow-components/retries.md)
- [Workflow SLAs in Kestra – Assert Duration Targets](https://kestra.io/docs/workflow-components/sla.md)
- [Execution States in Kestra: Full Lifecycle Guide](https://kestra.io/docs/workflow-components/states.md)
@@ -659,4 +690,5 @@ Every page in the Kestra documentation. Use this section to enumerate all availa
- [Realtime Trigger in Kestra – Millisecond Eventing](https://kestra.io/docs/workflow-components/triggers/realtime-trigger.md)
- [Schedule Trigger in Kestra – Cron-Based Scheduling](https://kestra.io/docs/workflow-components/triggers/schedule-trigger.md)
- [Webhook Trigger in Kestra – Start Flows via HTTP](https://kestra.io/docs/workflow-components/triggers/webhook-trigger.md)
+- [MCP Tool Trigger in Kestra – Expose Flows as AI Tools](https://kestra.io/docs/workflow-components/triggers/mcp-tool-trigger.md)
- [Variables in Kestra – Reuse Values Across Flows](https://kestra.io/docs/workflow-components/variables.md)
diff --git a/src/components/content/HomePageButtons.astro b/src/components/content/HomePageButtons.astro
index d1ea05beba0..adab09374e2 100644
--- a/src/components/content/HomePageButtons.astro
+++ b/src/components/content/HomePageButtons.astro
@@ -2,7 +2,7 @@
import Button from "~/components/common/Button.vue"
interface Props {
- buttons: Array<{ label: string; href: string }>
+ buttons: Array<{ label: string; href: string; type?: string }>
}
const { buttons } = Astro.props
@@ -13,7 +13,7 @@ const { buttons } = Astro.props
buttons.map((button, index) => (
@@ -85,7 +86,7 @@
button {
display: inline-flex;
align-items: center;
- width: 137px;
+ width: auto;
height: 36px;
padding: 8px 16px;
border-radius: 44px;
@@ -99,6 +100,22 @@
img {
border-radius: 6px;
}
+
+ .ask-kbd {
+ align-items: center;
+ justify-content: center;
+ width: 18px;
+ height: 18px;
+ border-radius: 3px;
+ font-style: normal;
+ font-size: 11px;
+ font-weight: 500;
+ line-height: 1;
+ margin-left: 4px;
+ border: 1px solid rgba(0, 0, 0, 0.25);
+ background: rgba(0, 0, 0, 0.06);
+ color: $black;
+ }
}
span.online {
font-weight: normal;
diff --git a/src/components/layout/Header.vue b/src/components/layout/Header.vue
index 62da6eb7a05..2d875cbe635 100644
--- a/src/components/layout/Header.vue
+++ b/src/components/layout/Header.vue
@@ -759,6 +759,29 @@
let collapse: Collapse | undefined = undefined
+ function isEditable(target: EventTarget | null): boolean {
+ const el = target instanceof HTMLElement ? target : null
+ if (!el) return false
+ return (
+ el.tagName === "INPUT" ||
+ el.tagName === "TEXTAREA" ||
+ el.tagName === "SELECT" ||
+ el.isContentEditable
+ )
+ }
+
+ function handleShortcut(e: KeyboardEvent) {
+ if (e.repeat || e.metaKey || e.ctrlKey || e.altKey) return
+ if (isEditable(e.target)) return
+ if (e.key.toLowerCase() === "a") {
+ e.preventDefault()
+ const modal = document.getElementById("search-ai-modal")
+ if (modal && window.$bootstrap) {
+ window.$bootstrap.Modal.getOrCreateInstance(modal).show()
+ }
+ }
+ }
+
function getCollapseInstance(): Collapse | undefined {
if (!collapse) {
const BootstrapCollapse = window.$bootstrap?.Collapse
@@ -795,6 +818,7 @@
useEventListener(window, "resize", syncMobileState)
useEventListener(window, "scroll", syncScrollState, { passive: true })
+ useEventListener(window, "keydown", handleShortcut)
document.documentElement.style.setProperty(
"--top-bar-height",
@@ -1316,7 +1340,7 @@
padding: 0.25rem;
}
- @include media-breakpoint-between(xl, xxl) {
+@include media-breakpoint-between(xl, xxl) {
.btn:not(.icon-button) {
padding-inline: 0.5rem;
font-size: $font-size-sm;
diff --git a/src/contents/blogs/2023-02-23-techniques-kafka-streams-developer/index.md b/src/contents/blogs/2023-02-23-techniques-kafka-streams-developer/index.md
index a0275f0bf66..3e8ff35b701 100644
--- a/src/contents/blogs/2023-02-23-techniques-kafka-streams-developer/index.md
+++ b/src/contents/blogs/2023-02-23-techniques-kafka-streams-developer/index.md
@@ -119,7 +119,7 @@ Et voilà! We have detection of dead consumers using just the Kafka API. 🎉
## Beware of State Store `all()`
-We use a [GlobalKTable](https://kafka.apache.org/31/documentation/streams/developer-guide/dsl-api.html#streams_concepts_globalktable) to detect [flow triggers](../../docs/05.workflow-components/07.triggers/02.flow-trigger/index.md). For all the flows on the cluster, we test all the flow's [conditions](../../docs/05.workflow-components/07.triggers/index.mdx#conditions) to find matching flows. For this, we are using an API to fetch all flows from a `GlobalKTable` using `store.all()` that returns all the flows from RocksDB (internal database from Kafka Stream).
+We use a [GlobalKTable](https://kafka.apache.org/31/documentation/streams/developer-guide/dsl-api.html#streams_concepts_globalktable) to detect [flow triggers](../../docs/05.workflow-components/07.triggers/02.flow-trigger/index.md). For all the flows on the cluster, we test all the flow's [conditions](../../docs/05.workflow-components/07.triggers/index.mdx) to find matching flows. For this, we are using an API to fetch all flows from a `GlobalKTable` using `store.all()` that returns all the flows from RocksDB (internal database from Kafka Stream).
Our first assumption was that `all()` returns an object (Flow in our case), as the API return Object, but we discovered that the `all()` method will:
- Fetch all the data from RocksDB
diff --git a/src/contents/blogs/2023-12-07-dremio-kestra-integration/index.md b/src/contents/blogs/2023-12-07-dremio-kestra-integration/index.md
index dc171566d81..2e0d26328fe 100644
--- a/src/contents/blogs/2023-12-07-dremio-kestra-integration/index.md
+++ b/src/contents/blogs/2023-12-07-dremio-kestra-integration/index.md
@@ -166,7 +166,7 @@ This flow clones a Git repository with [dbt code](https://github.com/dbt-labs/ja
## How to Extend this Workflow
-This use case is deliberately simple to demonstrate the basic usage of these plugins. However, you can easily extend it to accommodate more complex data processing requirements. For example, if you need to automate business-critical operations **for each row from a large dataset**, add a `ForEachItem` task. [That task](../../docs/05.workflow-components/01.tasks/00.flowable-tasks/index.md#foreachitem) allows you to iterate over a list of items fetched from your data lakehouse and reliably execute downstream tasks in parallel for each row.
+This use case is deliberately simple to demonstrate the basic usage of these plugins. However, you can easily extend it to accommodate more complex data processing requirements. For example, if you need to automate business-critical operations **for each row from a large dataset**, add a `ForEachItem` task. [That task](../../docs/05.workflow-components/01.tasks/00.flowable-tasks/index.md) allows you to iterate over a list of items fetched from your data lakehouse and reliably execute downstream tasks in parallel for each row.
---
diff --git a/src/contents/blogs/2024-02-29-release-0-15/index.md b/src/contents/blogs/2024-02-29-release-0-15/index.md
index f6798e0e0d7..dd740803ec5 100644
--- a/src/contents/blogs/2024-02-29-release-0-15/index.md
+++ b/src/contents/blogs/2024-02-29-release-0-15/index.md
@@ -297,8 +297,6 @@ We've also revamped the RBAC system, with improved handling of Superadmin access
The new **Service Accounts UI page** allows you to create and manage [service accounts](../../docs/07.enterprise/03.auth/service-accounts/index.md), and we've introduced API tokens valid for a specific period of time, allowing you to grant programmatic access to Kestra for Users and Service Accounts. This feature is particularly useful for CI/CD with GitHub Actions and Terraform, as well as for using the API token in API calls.
-
-
---
## Other Enhancements
diff --git a/src/contents/blogs/2024-04-04-top-10-cool-features-i-love-about-kestra/index.md b/src/contents/blogs/2024-04-04-top-10-cool-features-i-love-about-kestra/index.md
index 5012d2e6332..81553a89c76 100644
--- a/src/contents/blogs/2024-04-04-top-10-cool-features-i-love-about-kestra/index.md
+++ b/src/contents/blogs/2024-04-04-top-10-cool-features-i-love-about-kestra/index.md
@@ -102,7 +102,7 @@ Here is an image of adding secret via the UI in EE edition:
This is yet another powerful feature from the developer productivity perspective. Generally, you develop a pipeline related to some technology, and it is extremely likely that you use multiple tasks that correspond to the same technology. For example, in a flow that queries Redshift, it is very likely that you connect to Redshift to create the table in one task and then insert data into it in another task, and then query it for some purpose. In this case, you would just end up duplicating the Redshift connection information in all these tasks. This hampers the developer's productivity and leads to configuration duplication.
-In order to avoid this duplication, Kestra provides [plugin defaults](../../docs/05.workflow-components/09.plugin-defaults/index.md). Mention the plugin defaults once in the flow, and it gets referenced in all the tasks of the corresponding type.
+In order to avoid this duplication, Kestra provides [plugin defaults](../../docs/migration-guide/v2.0.0/plugin-defaults-removed/index.md). Mention the plugin defaults once in the flow, and it gets referenced in all the tasks of the corresponding type.
You can even set the plugin defaults globally or on a namespace level to ensure that all flows using, e.g., the AWS plugin leverage the same credentials.
diff --git a/src/contents/blogs/2024-06-04-release-0-17/index.md b/src/contents/blogs/2024-06-04-release-0-17/index.md
index 6b1ca6fba42..698b807c27a 100644
--- a/src/contents/blogs/2024-06-04-release-0-17/index.md
+++ b/src/contents/blogs/2024-06-04-release-0-17/index.md
@@ -487,11 +487,7 @@ Here is how it looks like on the Cluster Dashboard page:
We've added a new feature that allows you to explicitly declare which namespaces are allowed to trigger flows and other resources for any given namespace.
-When you navigate to any Namespace and go to the `Edit` tab, you can explicitly configure which namespaces are allowed to access it. By default, all namespaces are allowed.
-
-
-
-However, you can restrict that access if you want only specific namespaces (or no namespace at all) to trigger its corresponding resources.
+When you navigate to any Namespace and go to the `Edit` tab, you can explicitly configure which namespaces are allowed to access it. By default, all namespaces are allowed. However, you can restrict that access if you want only specific namespaces (or no namespace at all) to trigger its corresponding resources.
:::alert{type="info"}
Check the [Allowed Namespaces](../../docs/07.enterprise/02.governance/07.namespace-management/index.md#allowed-namespaces) documentation for more details.
diff --git a/src/contents/blogs/2024-10-15-deploying-kestra-in-clever-cloud/index.md b/src/contents/blogs/2024-10-15-deploying-kestra-in-clever-cloud/index.md
index ddcfcc9249e..891b469a318 100644
--- a/src/contents/blogs/2024-10-15-deploying-kestra-in-clever-cloud/index.md
+++ b/src/contents/blogs/2024-10-15-deploying-kestra-in-clever-cloud/index.md
@@ -109,7 +109,7 @@ kestra:
2. The application will be available publicly, so we set a user and password. I strongly recommend you to do the same.
3. We configure Kestra to use the MinIO storage by using the environment variables injected from the add-on. The MinIO storage works with all S3 compatible storage including Cellar.
4. We set the URL of Kestra to the URL of the application, by default, this will be `${APP_ID}.cleverapps.io` where `APP_ID` is an environment variable injected by Clever Cloud with the identifier of your application.
-5. As the Docker engine is not accessible from the Kestra container, we configure globally the `Process` [task runner](../../docs/task-runners/index.mdx) for all plugins using [Plugins Default](../../docs/05.workflow-components/09.plugin-defaults/index.md).
+5. As the Docker engine is not accessible from the Kestra container, we configure globally the `Process` [task runner](../../docs/task-runners/index.mdx) for all plugins using a static [Policy](../../docs/enterprise/governance/policies/index.md).
diff --git a/src/contents/blogs/airflow-3-vs-airflow-2/index.md b/src/contents/blogs/airflow-3-vs-airflow-2/index.md
index 52cfd1d5522..74f706f716d 100644
--- a/src/contents/blogs/airflow-3-vs-airflow-2/index.md
+++ b/src/contents/blogs/airflow-3-vs-airflow-2/index.md
@@ -69,7 +69,7 @@ The Airflow team provides Ruff linter rules to identify breaking changes automat
Practically speaking, the Airflow 3 upgrade makes sense for a narrow subset of teams: those who are Python-native, have significant existing investment, and aren't hitting architectural walls.
-But even teams that fit this profile should pause to consider where their orchestration needs are heading. If you're building [AI agent workflows](../../docs/ai-tools/ai-workflows/index.md), for example, a YAML-based declarative DSL often works better than Python code. Many teams are integrating AI capabilities into their data stacks, and the orchestrator you choose today will either enable or constrain that roadmap.
+But even teams that fit this profile should pause to consider where their orchestration needs are heading. If you're building [AI agent workflows](../../docs/ai-tools/06.ai-workflows/index.md), for example, a YAML-based declarative DSL often works better than Python code. Many teams are integrating AI capabilities into their data stacks, and the orchestrator you choose today will either enable or constrain that roadmap.
## Who should reconsider
diff --git a/src/contents/blogs/airflow-to-kestra-migration-with-ai/index.md b/src/contents/blogs/airflow-to-kestra-migration-with-ai/index.md
index 8d62ec3de25..773638d6e5b 100644
--- a/src/contents/blogs/airflow-to-kestra-migration-with-ai/index.md
+++ b/src/contents/blogs/airflow-to-kestra-migration-with-ai/index.md
@@ -46,7 +46,7 @@ The migration uses three tools working together:
2. **`kestra-flow` agent skill**, which gives Claude Code live knowledge of Kestra's flow schema so it never generates invalid YAML
3. **`kestra-ops` agent skill**, which gives Claude Code the ability to operate `kestractl` for deployment, validation, and namespace file management
-Install both skills by following the instructions at [kestra.io/docs/ai-tools/agent-skills](../../docs/ai-tools/agent-skills/index.md). Once installed, Claude Code automatically invokes the right skill based on what you ask.
+Install both skills by following the instructions at [kestra.io/docs/ai-tools/agent-skills](../../docs/ai-tools/04.agent-skills/index.md). Once installed, Claude Code automatically invokes the right skill based on what you ask.
You also need `kestractl` installed and pointed at a running Kestra instance:
@@ -592,7 +592,7 @@ docker run --pull=always --rm -it -p 8080:8080 --user=root \
Then:
-1. Install Kestra agent skills: [kestra.io/docs/ai-tools/agent-skills](../../docs/ai-tools/agent-skills/index.md)
+1. Install Kestra agent skills: [kestra.io/docs/ai-tools/agent-skills](../../docs/ai-tools/04.agent-skills/index.md)
2. Install `kestractl`: [kestra.io/docs/kestra-cli/kestractl](../../docs/kestra-cli/kestractl/index.md)
3. Pick a simple DAG to start (a 3-5 task sequential pipeline works well) and run it through the migration workflow described above
diff --git a/src/contents/blogs/enterprise-airflow-alternatives/index.md b/src/contents/blogs/enterprise-airflow-alternatives/index.md
index 9d16438c1d5..9f271b089dc 100644
--- a/src/contents/blogs/enterprise-airflow-alternatives/index.md
+++ b/src/contents/blogs/enterprise-airflow-alternatives/index.md
@@ -139,7 +139,7 @@ If your team is Python-native and you want minimal disruption, Astronomer or Pre
But here's the question most comparisons don't ask: what happens when your orchestration needs expand beyond data pipelines?
-The reality of 2026 is that workloads are converging. [AI workflows](../../docs/ai-tools/ai-workflows/index.md) don't respect the boundaries between data engineering, ML ops, and application development. The same platform that orchestrates your [ETL pipelines](../../docs/15.how-to-guides/etl-pipelines/index.md) might need to coordinate model training, API calls, and business process automation. If you choose a Python-only orchestrator today, you're betting that Python will remain the center of gravity for every workflow your organization needs to run. That's a big bet.
+The reality of 2026 is that workloads are converging. [AI workflows](../../docs/ai-tools/06.ai-workflows/index.md) don't respect the boundaries between data engineering, ML ops, and application development. The same platform that orchestrates your [ETL pipelines](../../docs/15.how-to-guides/etl-pipelines/index.md) might need to coordinate model training, API calls, and business process automation. If you choose a Python-only orchestrator today, you're betting that Python will remain the center of gravity for every workflow your organization needs to run. That's a big bet.
This is where Kestra's language-agnostic approach pays off. You can start with [data pipelines](../../docs/use-cases/01.data-pipelines/index.md), migrate off Airflow without rewriting everything, and then expand to orchestrate workloads across teams and languages as your needs evolve. No silos, second orchestrator for non-Python work, or rearchitecting when requirements change.
diff --git a/src/contents/blogs/infra-automation/index.md b/src/contents/blogs/infra-automation/index.md
index a56c0a51881..0076cefda4c 100644
--- a/src/contents/blogs/infra-automation/index.md
+++ b/src/contents/blogs/infra-automation/index.md
@@ -48,7 +48,7 @@ And it’s why we shipped production-grade controls that matter specifically whe
- [Credentials](https://kestra.io/docs/enterprise/auth/credentials): reusable server-to-server auth configured once and referenced everywhere via `credential()`, so tokens aren’t scattered across flows and rotations don’t become a scavenger hunt.
-- [Plugin Defaults UI](https://kestra.io/docs/workflow-components/plugin-defaults): manage shared plugin configuration at the namespace level through a guided UI while keeping it versionable.
+- [Policies](https://kestra.io/docs/enterprise/governance/policies): enforce configuration rules and shared plugin settings at the namespace or tenant level, with governance controls and an audit trail (Enterprise Edition).
And because infrastructure automation is always part “process” (approvals, self-service, controlled access), Kestra’s Enterprise “Apps” matter deeply here: **Apps let you build a UI in front of flows**, forms for data entry, approval buttons, and controlled output views, while the flow remains the backend.
diff --git a/src/contents/blogs/introducing-ai-agents/index.md b/src/contents/blogs/introducing-ai-agents/index.md
index 4d26bb0e8b5..325debf63a8 100644
--- a/src/contents/blogs/introducing-ai-agents/index.md
+++ b/src/contents/blogs/introducing-ai-agents/index.md
@@ -255,7 +255,7 @@ With Kestra, you get the best of both worlds: traditional task-based orchestrati
## Next Steps
-Ready to get started? Check out our [AI Agent documentation](../../docs/ai-tools/ai-agents/index.md) for detailed examples and configuration options.
+Ready to get started? Check out our [AI Agent documentation](../../docs/ai-tools/05.ai-agents/index.md) for detailed examples and configuration options.
If you like the project, give us a [GitHub star](https://github.com/kestra-io/kestra).
diff --git a/src/contents/blogs/orchestrate-ai-agents-kestra/index.md b/src/contents/blogs/orchestrate-ai-agents-kestra/index.md
index 7f58192a93f..56308bd9a63 100644
--- a/src/contents/blogs/orchestrate-ai-agents-kestra/index.md
+++ b/src/contents/blogs/orchestrate-ai-agents-kestra/index.md
@@ -195,7 +195,7 @@ A quick reference, useful whether you're a student deciding what to build next o
Browse the [AI blueprints](https://kestra.io/blueprints?tags=AI) to find a use case close to yours, set your API key (Gemini, OpenAI, Anthropic Claude, Mistral, Bedrock, Vertex AI, and Ollama are all supported), and run it. Then add a `Pause` before a sensitive step and watch the execution wait for your approval. The blueprints cover a range of patterns from simple single-agent tasks to multi-agent pipelines, so it's worth scanning a few to see what's possible before you start building.
-- Docs: [AI Agents in Kestra](../../docs/ai-tools/ai-agents/index.md)
+- Docs: [AI Agents in Kestra](../../docs/ai-tools/05.ai-agents/index.md)
- Concept: [agentic orchestration](/resources/ai/agentic-orchestration)
If you're working through the LLM Zoomcamp agents module, this is the bridge from "my agent works" to "my agent runs in production."
diff --git a/src/contents/blogs/orchestrate-rag-pipeline-kestra/index.md b/src/contents/blogs/orchestrate-rag-pipeline-kestra/index.md
index 636f5e8e630..d1f4a706a21 100644
--- a/src/contents/blogs/orchestrate-rag-pipeline-kestra/index.md
+++ b/src/contents/blogs/orchestrate-rag-pipeline-kestra/index.md
@@ -175,5 +175,5 @@ If you're working through the LLM Zoomcamp orchestration module, this is the sam
Once the pipeline is running reliably, the natural next step is giving it more autonomy.
-- Docs: [RAG workflows in Kestra](../../docs/ai-tools/ai-rag-workflows/index.md)
+- Docs: [RAG workflows in Kestra](../../docs/ai-tools/07.ai-rag-workflows/index.md)
- Reference: [the RAG plugin](/plugins/plugin-ai/rag) and [vector databases explained](/resources/ai/vector-database)
diff --git a/src/contents/blogs/release-1-2/index.md b/src/contents/blogs/release-1-2/index.md
index c5804754e81..d388e22e441 100644
--- a/src/contents/blogs/release-1-2/index.md
+++ b/src/contents/blogs/release-1-2/index.md
@@ -360,7 +360,7 @@ id: vm_provisioning
namespace: company.team
checks:
- - condition: "{{ kv('VMs') | length < 2 }}"
+ - when: "{{ kv('VMs') | length < 2 }}"
message: "You have provisioned too many VMs"
style: ERROR
behavior: BLOCK_EXECUTION
diff --git a/src/contents/docs/01.quickstart/index.md b/src/contents/docs/01.quickstart/index.md
index 84f39f8a0ce..0daaaddfaaf 100644
--- a/src/contents/docs/01.quickstart/index.md
+++ b/src/contents/docs/01.quickstart/index.md
@@ -11,7 +11,7 @@ Launch Kestra locally, create a simple flow, and run your first execution in a f
## Watch the quickstart video
-
+
## Prerequisites
@@ -30,7 +30,8 @@ docker run --pull=always --rm -it -p 8080:8080 --user=root \
-v kestra_db:/app/data \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
- kestra/kestra:latest server local
+ -e KESTRA_PLUGINS_AUTO_INSTALL_ENABLED=true \
+ kestra/kestra:latest-slim server local
```
If you re-run the command and Docker reports `You have to remove (or rename) that container to be able to reuse that name.`, remove the old container with `docker rm -f kestra` or pick a different `--name`.
@@ -40,6 +41,11 @@ If you re-run the command and Docker reports `You have to remove (or rename) tha
- stores local files in the `kestra_data` Docker volume
- persists the H2 database in the `kestra_db` Docker volume
- mounts `/tmp` and the Docker socket so script and container tasks can run locally
+- uses the lightweight `slim` image and installs plugins automatically on demand
+:::
+
+:::alert{type="info"}
+The `kestra/kestra:latest-slim` image ships without any plugins to keep the download small. The `KESTRA_PLUGINS_AUTO_INSTALL_ENABLED=true` environment variable makes Kestra install any plugin automatically the first time a flow needs it, so you don't need to pre-install anything. If you prefer an image with all plugins bundled, use `kestra/kestra:latest` instead.
:::
The container is ready when the logs show `Main server is running at http://...:8080`.
@@ -48,9 +54,7 @@ The container is ready when the logs show `Main server is running at http://...:
Open `http://localhost:8080` in your browser. You will see the Kestra UI when the container is running. From here, create your user, create a flow with the AI Copilot, and complete the deep-dive tutorial to begin building your first flows.
-
-
-
+
:::alert{type="info"}
The above command starts Kestra with an embedded H2 database. Storage files are stored on the `kestra_data` Docker volume, and the H2 database is persisted on the `kestra_db` Docker volume. For production-ready persistence with a PostgreSQL database and more configurability, follow the [Docker Compose installation](../02.installation/03.docker-compose/index.md).
@@ -82,5 +86,5 @@ You've taken the product tour, executed your first flow, and explored Kestra. Ne
- [Contribute to Kestra](../04.contribute-to-kestra/index.mdx) – whether a developer or not, we value outside contribution of all kinds: Plugins, Features, Documentation, Feature Requests, and Bug Reports. Get involved!
:::alert{type="info"}
-**Using an AI coding agent?** Add the [Kestra MCP server](../ai-tools/kestra-mcp-resources/index.md) to Claude Code, Cursor, or any MCP-compatible tool for live access to plugin docs, blueprints, and Kestra documentation while you build.
+**Prefer to work from your terminal or editor?** Claude Code, Cursor, Codex, and other coding agents can write and deploy flows for you using the Kestra MCP server, Agent Skills, and `kestractl`. See [AI tools](../ai-tools/index.mdx) to compare that with the in-product AI Copilot.
:::
diff --git a/src/contents/docs/02.installation/02.docker/index.md b/src/contents/docs/02.installation/02.docker/index.md
index 8ac0ed0aad4..9fd32d63381 100644
--- a/src/contents/docs/02.installation/02.docker/index.md
+++ b/src/contents/docs/02.installation/02.docker/index.md
@@ -23,11 +23,16 @@ docker run --pull=always --rm -it -p 8080:8080 --user=root \
-v kestra_db:/app/data \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
- kestra/kestra:latest server local
+ -e KESTRA_PLUGINS_AUTO_INSTALL_ENABLED=true \
+ kestra/kestra:latest-slim server local
```
Open http://localhost:8080 in your browser to launch the UI and start building your first flows.
+:::alert{type="info"}
+The `kestra/kestra:latest-slim` image ships without any plugins to keep the download small. With `KESTRA_PLUGINS_AUTO_INSTALL_ENABLED=true`, Kestra installs any plugin automatically the first time a flow needs it, so you don't need to pre-install anything. If you prefer an image with all plugins bundled, use `kestra/kestra:latest` instead.
+:::
+
:::alert{type="info"}
The above command starts Kestra with an embedded H2 database. Storage files are stored on the `kestra_data` Docker volume, and the H2 database is persisted on the `kestra_db` Docker volume. For production-ready persistence with a PostgreSQL database and more configurability, follow the [Docker Compose installation](../03.docker-compose/index.md).
:::
@@ -148,15 +153,19 @@ The official Kestra Docker images are available on [DockerHub](https://hub.docke
Two image variants are available:
- `kestra/kestra:*`
-- `kestra/kestra:*-no-plugins`
+- `kestra/kestra:*-slim`
Both variants are based on the [`eclipse-temurin:21-jre`](https://hub.docker.com/_/eclipse-temurin) Docker image.
-The `kestra/kestra:*` images include all Kestra [plugins](/plugins) in their **latest versions**. The `kestra/kestra:*-no-plugins` images do not contain any plugins. Use the `kestra/kestra:*` version to access all available plugins.
+The `kestra/kestra:*` images include all Kestra [plugins](/plugins) in their **latest versions**. The `kestra/kestra:*-slim` images do not bundle any plugins, which keeps them much smaller to download. When plugin auto-install is enabled (`KESTRA_PLUGINS_AUTO_INSTALL_ENABLED=true`), Kestra downloads and installs any missing plugin automatically the first time a flow uses it — so the `-slim` image stays fully usable.
+
+:::alert{type="info"}
+The `*-slim` images were previously published under the `*-no-plugins` suffix. The `-no-plugins` tags are deprecated aliases of `-slim` and will be removed in a future release.
+:::
## Docker image tags
-The following tags are available for each Docker image (append `-no-plugins` to any image to exclude all but Kestra core plugins):
+The following tags are available for each Docker image (append `-slim` to any image to exclude all but Kestra core plugins):
- `latest`: The most recent stable release (rolling tag). Intended for trying new features; not an LTS. Support ends when the next stable release (~ 2 months) becomes available.
- `latest-lts`: The current Long-Term Support (rolling tag). Tracks the active LTS line (updates roughly every 6 months to the new LTS) and receives fixes for ~1 year.
@@ -164,7 +173,7 @@ The following tags are available for each Docker image (append `-no-plugins` to
- `v`: Immutable tag for an exact version (e.g., `v1.0.1`). Never changes; **best for locked-down production.**
- `develop`: Nightly/continuous build from the `develop` branch. Unstable and not recommended for production, only for testing.
-The **default Kestra image** `kestra/kestra:latest` already includes **all plugins**. To use a lightweight version of Kestra without plugins, add the suffix `*-no-plugins`.
+The **default Kestra image** `kestra/kestra:latest` already includes **all plugins**. To use a lightweight version of Kestra without bundled plugins, add the suffix `-slim`. These images ship without plugins on purpose — they are a fraction of the size of the full image, and combined with plugin auto-install (`KESTRA_PLUGINS_AUTO_INSTALL_ENABLED=true`), missing plugins are fetched on demand when a flow needs them.
### Recommended images for production
@@ -173,25 +182,25 @@ For production deployments, choose one of the following:
**Latest stable version** for staying most up to date while also stable (make note that this is a rolling tag that changes quite frequently):
- `kestra/kestra:latest` — latest stable with all plugins
-- `kestra/kestra:latest-no-plugins` — latest stable without plugins
+- `kestra/kestra:latest-slim` — latest stable without plugins
**Pinned versions** for maximum stability:
- `kestra/kestra:v` — all plugins included
-- `kestra/kestra:v-no-plugins` — no bundled plugins, only core to Kestra
+- `kestra/kestra:v-slim` — no bundled plugins, only core to Kestra
**LTS rolling tag** if you want automatic updates within the LTS line:
- `kestra/kestra:latest-lts`
-- `kestra/kestra:latest-lts-no-plugins`
+- `kestra/kestra:latest-lts-slim`
### Recommended images for development
For development or testing new features:
- `kestra/kestra:latest` — latest stable with all plugins
-- `kestra/kestra:latest-no-plugins` — latest stable without plugins
-- `kestra/kestra:develop` / `kestra/kestra:develop-no-plugins` — daily builds with unreleased features, unstable
+- `kestra/kestra:latest-slim` — latest stable without plugins
+- `kestra/kestra:develop` / `kestra/kestra:develop-slim` — daily builds with unreleased features, unstable
## Build a custom Docker image
@@ -215,12 +224,12 @@ RUN mkdir -p /app/plugins && \
### Add plugins to a Docker image
-By default, the base Docker image `kestra/kestra:latest` contains all plugins (unless you use the `kestra/kestra:latest-no-plugins` version). You can add specific plugins to the base image and build a custom image.
+By default, the base Docker image `kestra/kestra:latest` contains all plugins (unless you use the `kestra/kestra:latest-slim` version). You can add specific plugins to the base image and build a custom image.
The following `Dockerfile` creates an image from the base image and adds the `plugin-aws`, `storage-gcs` and `plugin-gcp` binaries using the command `kestra plugins install`:
```dockerfile
-ARG IMAGE_TAG=latest-no-plugins
+ARG IMAGE_TAG=latest-slim
FROM kestra/kestra:$IMAGE_TAG
RUN /app/kestra plugins install \
diff --git a/src/contents/docs/02.installation/03.docker-compose/index.md b/src/contents/docs/02.installation/03.docker-compose/index.md
index b748523203b..da02d36856c 100644
--- a/src/contents/docs/02.installation/03.docker-compose/index.md
+++ b/src/contents/docs/02.installation/03.docker-compose/index.md
@@ -90,7 +90,9 @@ Next, update the `kestra` service in the `docker-compose.yml` file to mount this
- $PWD/application.yaml:/etc/config/application.yaml
ports:
- "8080:8080"
- - "8081:8081"
+ # The management port (8081) is not exposed by default. It is unauthenticated
+ # and intended for internal use only. Only expose it within a trusted network.
+ # - "8081:8081"
depends_on:
postgres:
condition: service_started
@@ -233,7 +235,9 @@ services:
url: http://localhost:8080/
ports:
- "8080:8080"
- - "8081:8081"
+ # The management port (8081) is not exposed by default. It is unauthenticated
+ # and intended for internal use only. Only expose it within a trusted network.
+ # - "8081:8081"
depends_on:
postgres:
condition: service_started
@@ -289,11 +293,9 @@ services:
scripts:
docker:
volume-enabled: true
- defaults: # Example demonstrating global pluginDefaults
- - type: io.kestra.plugin.airbyte.connections.Sync
- url: http://host.docker.internal:8000/
- username: airbyte
- password: password
+ # Global plugin defaults removed in Kestra 2.0.
+ # Use kestra.policies for installation-wide defaults.
+ # See: https://kestra.io/docs/enterprise/governance/policies
url: http://localhost:8080/
variables:
env-vars-prefix: "" # To avoid requiring KESTRA_ prefix on env vars
@@ -347,7 +349,7 @@ Server components can run independently from each other. Each of them communicat
For more details on Kestra server commands, check out the [Server CLI documentation](../../kestra-cli/kestra-server/index.md).
-Here is an example Docker Compose configuration file running Kestra services with replicas on the Postgres database backend.
+Here is an example Docker Compose configuration file running Kestra services with replicas on the Postgres database backend. Each service maps a unique host port range to its internal management port (`8081`) so you can monitor individual replicas. The management port is unauthenticated by default — only expose these mappings within a trusted internal network.
```yaml
volumes:
diff --git a/src/contents/docs/02.installation/03.kubernetes/index.md b/src/contents/docs/02.installation/03.kubernetes/index.md
index 87a64703211..4c51c3e47f7 100644
--- a/src/contents/docs/02.installation/03.kubernetes/index.md
+++ b/src/contents/docs/02.installation/03.kubernetes/index.md
@@ -117,7 +117,7 @@ The `kestra` chart does not include PostgreSQL or object storage. Configure thes
## Access the Kestra UI
-To list all pods run:
+To list all pods, run:
```bash
kubectl get pods -n default -l app.kubernetes.io/name=kestra
@@ -214,14 +214,17 @@ Omit the `tls` block if TLS is terminated upstream (e.g., at a load balancer). T
## Scaling Kestra on Kubernetes
-For production deployments, run each Kestra component in its own pod.
-
-Example `values.yaml`:
+For production deployments, run each Kestra component in its own pod with a dedicated controller. Workers connect to the controller over gRPC on port 50051, so ensure any cluster network policies allow that traffic between pods before applying this configuration.
```yaml
deployments:
+ standalone:
+ enabled: false
webserver:
enabled: true
+ extraArgs:
+ - --no-controller # optional; see note below
+ - --no-indexer # disable embedded indexer when running a dedicated indexer pod
executor:
enabled: true
indexer:
@@ -230,10 +233,30 @@ deployments:
enabled: true
worker:
enabled: true
- standalone:
- enabled: false
+ controller:
+ enabled: true
+
+configurations:
+ application:
+ kestra:
+ worker:
+ controllers:
+ type: STATIC
+ static:
+ endpoints:
+ - host: my-kestra-controller # -controller Service; replace my-kestra with your Helm release name
+ port: 50051
```
+`--no-controller` disables the embedded controller that the webserver starts by default. It is optional — Kestra supports multiple simultaneous controllers, so the embedded one is harmless if left running. Disable it to recover resources when the dedicated `controller` deployment handles all controller duties.
+
+`--no-indexer` disables the indexer embedded in the webserver. Without it, enabling a dedicated `indexer` pod results in two indexers running simultaneously. The example above uses the dedicated pattern. The embedded pattern skips the `--no-indexer` flag and disables the separate pod instead:
+
+| Pattern | Webserver `extraArgs` | `indexer.enabled` |
+|---|---|---|
+| Dedicated indexer pod | `--no-indexer` | `true` |
+| Embedded indexer | — | `false` |
+
Apply changes:
```bash
@@ -440,6 +463,7 @@ dind:
- SETGID
args:
- '--log-level=fatal'
+ - '--group=1000'
```
### Troubleshooting DinD
@@ -470,14 +494,23 @@ dind:
enabled: false
```
-Use the Kubernetes task runner as the default method for running [script tasks](../../16.scripts/index.mdx):
+Use the Kubernetes task runner as the default method for running [script tasks](../../16.scripts/index.mdx). In Enterprise Edition, apply it across a namespace with a [Policy](../../07.enterprise/02.governance/policies/index.md):
```yaml
-pluginDefaults:
- - type: io.kestra.plugin.scripts
- forced: true
- values:
- taskRunner:
- type: io.kestra.plugin.ee.kubernetes.runner.Kubernetes
- # ... your Kubernetes runner configuration
+kestra:
+ policies:
+ - id: k8s-task-runner
+ description: "Use Kubernetes runner for all script tasks."
+ rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ override: true
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.scripts
+ values:
+ taskRunner:
+ type: io.kestra.plugin.ee.kubernetes.runner.Kubernetes
+ # ... your Kubernetes runner configuration
```
diff --git a/src/contents/docs/02.installation/11.digitalocean-droplet/index.md b/src/contents/docs/02.installation/11.digitalocean-droplet/index.md
index 1b7387f7f25..5ce1eb2df9b 100644
--- a/src/contents/docs/02.installation/11.digitalocean-droplet/index.md
+++ b/src/contents/docs/02.installation/11.digitalocean-droplet/index.md
@@ -203,3 +203,5 @@ This setup provides a simple starting point for running Kestra in production on
Reach out via [Slack](/slack) if you encounter any issues or if you have any questions regarding deploying Kestra to production.
Make sure to also check the [CI/CD guide](../../version-control-cicd/cicd/index.md) to automate your workflow deployments based on changes in Git.
+
+To manage DigitalOcean resources (Droplets, Kubernetes clusters, managed databases, load balancers, volumes, domains, and firewalls) directly from Kestra flows, see the [DigitalOcean plugin](/plugins/plugin-digitalocean).
diff --git a/src/contents/docs/02.installation/12.standalone-server/index.md b/src/contents/docs/02.installation/12.standalone-server/index.md
index 830fe9950a0..d2a72abbeaf 100644
--- a/src/contents/docs/02.installation/12.standalone-server/index.md
+++ b/src/contents/docs/02.installation/12.standalone-server/index.md
@@ -25,7 +25,7 @@ For example, to launch Kestra:
For more information on database configuration, check out the [Runtime and Storage configuration guide](../../configuration/02.runtime-and-storage/index.md)
:::alert{type="warning"}
-Running the jar version comes without any [plugins](/plugins). You need to install them manually with the `kestra plugins install directory_with_plugins/` command. Alternatively, point to a directory with the plugins in the configuration file or an environment variable `KESTRA_PLUGINS_PATH` (e.g., `KESTRA_PLUGINS_PATH=/Users/anna/dev/plugins`).
+Running the jar version comes without any [plugins](/plugins) — the JAR is kept small on purpose. When running `server local`, Kestra can install missing plugins automatically the first time a flow needs them (plugin auto-install, enabled with the `KESTRA_PLUGINS_AUTO_INSTALL_ENABLED=true` environment variable or `kestra.plugins.auto-install.enabled: true` in the configuration). Alternatively, install plugins manually with the `kestra plugins install directory_with_plugins/` command, or point to a directory with the plugins in the configuration file or an environment variable `KESTRA_PLUGINS_PATH` (e.g., `KESTRA_PLUGINS_PATH=/Users/anna/dev/plugins`).
:::
## Configuration
diff --git a/src/contents/docs/02.installation/index.mdx b/src/contents/docs/02.installation/index.mdx
index 77e27a99c02..461292a3272 100644
--- a/src/contents/docs/02.installation/index.mdx
+++ b/src/contents/docs/02.installation/index.mdx
@@ -25,7 +25,7 @@ Not sure where to start?
You can deploy Kestra from a laptop or on-prem server to a distributed cluster in a public cloud. Some plugins such as the [Script plugin](../16.scripts/index.mdx) require Docker-in-Docker (DinD), which is not supported in some environments like AWS Fargate. For production, use Kubernetes or a virtual machine.
-If looking for a fully-managed orchestration platform without the overhead of infrastructure maintenance, Kestra Cloud is in early access; [sign up here](/cloud).
+If you prefer a fully managed platform without the overhead of infrastructure maintenance, [Kestra Cloud](/cloud) is available now. See the [Cloud getting started guide](../07.enterprise/07.cloud-getting-started/index.md) to set up your first instance.
## Choose how to install Kestra for your environment
diff --git a/src/contents/docs/03.tutorial/01.fundamentals/create-button-2-0.png b/src/contents/docs/03.tutorial/01.fundamentals/create-button-2-0.png
new file mode 100644
index 00000000000..b06db979b08
Binary files /dev/null and b/src/contents/docs/03.tutorial/01.fundamentals/create-button-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/01.fundamentals/create_button.png b/src/contents/docs/03.tutorial/01.fundamentals/create_button.png
deleted file mode 100644
index 4ee2f9ac1d9..00000000000
Binary files a/src/contents/docs/03.tutorial/01.fundamentals/create_button.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/01.fundamentals/execution-overview-2-0.png b/src/contents/docs/03.tutorial/01.fundamentals/execution-overview-2-0.png
new file mode 100644
index 00000000000..97c8e6d11b4
Binary files /dev/null and b/src/contents/docs/03.tutorial/01.fundamentals/execution-overview-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/01.fundamentals/gantt-view-2-0.png b/src/contents/docs/03.tutorial/01.fundamentals/gantt-view-2-0.png
new file mode 100644
index 00000000000..6bad5d30a11
Binary files /dev/null and b/src/contents/docs/03.tutorial/01.fundamentals/gantt-view-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/01.fundamentals/gantt-view.png b/src/contents/docs/03.tutorial/01.fundamentals/gantt-view.png
deleted file mode 100644
index 18d15f811ee..00000000000
Binary files a/src/contents/docs/03.tutorial/01.fundamentals/gantt-view.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/01.fundamentals/index.md b/src/contents/docs/03.tutorial/01.fundamentals/index.md
index b5d95eb446a..bded7d39dfe 100644
--- a/src/contents/docs/03.tutorial/01.fundamentals/index.md
+++ b/src/contents/docs/03.tutorial/01.fundamentals/index.md
@@ -15,7 +15,7 @@ If you haven't already, follow the [Quickstart Guide](../../01.quickstart/index.
## Build your first Hello World Flow
-
+
## Flows
@@ -155,7 +155,7 @@ If you want to comment or uncomment out part of your code, use `CTRL + /` on Win
To this point, we have shown some flows to run and get familiar with. Now, let's create a flow to use throughout the rest of the tutorial. Open the **Flows** view and click **+ Create**:
-
+
Paste the following code into the Flow editor:
@@ -169,16 +169,18 @@ tasks:
uri: https://dummyjson.com/products
```
-Then, hit the **Save** button.
+Then, hit the **Save** button, or you can also save as a draft in case you'd like to keep editing.
-
+
-This flow has a single task that fetches data from the [dummyjson](https://dummyjson.com/) API via an [HTTP Request task](/plugins/core/http/io.kestra.plugin.core.http.request). Run it to see the output.
-
-
+This flow has a single task that fetches data from the [dummyjson](https://dummyjson.com/) API via an [HTTP Request task](/plugins/core/http/io.kestra.plugin.core.http.request). Run it with the **Execute** button to see the output.
After execution, you’ll be directed to the Gantt view to see the stages of your flow’s progress. In this simple example, we see the API request successfully execute.
-
+
+
+Toggle over to the **Execution Overview** for a visualization of your flow tasks and explore the other post-execution tabs for even more detailed information about your run.
+
+
While fetching data is a great first step, it is just that, a first step. In the next sections, you'll explore the other critical components of Kestra flows: Inputs, Outputs, Triggers, and more.
diff --git a/src/contents/docs/03.tutorial/01.fundamentals/new_execution.png b/src/contents/docs/03.tutorial/01.fundamentals/new_execution.png
deleted file mode 100644
index ffefc3f1aa2..00000000000
Binary files a/src/contents/docs/03.tutorial/01.fundamentals/new_execution.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/01.fundamentals/save-button-2-0.png b/src/contents/docs/03.tutorial/01.fundamentals/save-button-2-0.png
new file mode 100644
index 00000000000..c590068a104
Binary files /dev/null and b/src/contents/docs/03.tutorial/01.fundamentals/save-button-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/01.fundamentals/save_button.png b/src/contents/docs/03.tutorial/01.fundamentals/save_button.png
deleted file mode 100644
index 78921e8eece..00000000000
Binary files a/src/contents/docs/03.tutorial/01.fundamentals/save_button.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/02.inputs/index.md b/src/contents/docs/03.tutorial/02.inputs/index.md
index 26647d5be00..a6a3d5cd70a 100644
--- a/src/contents/docs/03.tutorial/02.inputs/index.md
+++ b/src/contents/docs/03.tutorial/02.inputs/index.md
@@ -11,7 +11,7 @@ Instead of hardcoding values in your flow, use inputs to make your workflows mor
## Make Flows dynamic with Inputs
-
+
## Defining inputs
@@ -43,7 +43,7 @@ Hit the `Backspace` or `Delete` key while building your expression? Use the keyb
Try running the above flow with different values for the `user` input. You can do this by clicking on the **Execute** button and then typing a new string value in the prompt for whatever name you'd like.
-
+
:::alert{type="info"}
The plural form `defaults` is used instead of `default` for two reasons. First, `default` is a reserved keyword in Java, so it cannot be used. Second, this property allows you to set default values for a JSON object which can be an array that simultaneously defines multiple default values.
diff --git a/src/contents/docs/03.tutorial/02.inputs/inputs-2-0.png b/src/contents/docs/03.tutorial/02.inputs/inputs-2-0.png
new file mode 100644
index 00000000000..3e3d6352ffa
Binary files /dev/null and b/src/contents/docs/03.tutorial/02.inputs/inputs-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/02.inputs/inputs.png b/src/contents/docs/03.tutorial/02.inputs/inputs.png
deleted file mode 100644
index d79df4ae346..00000000000
Binary files a/src/contents/docs/03.tutorial/02.inputs/inputs.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/03.outputs/eval-expression-2-0.png b/src/contents/docs/03.tutorial/03.outputs/eval-expression-2-0.png
new file mode 100644
index 00000000000..78affdbc04b
Binary files /dev/null and b/src/contents/docs/03.tutorial/03.outputs/eval-expression-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/03.outputs/eval_expressions.png b/src/contents/docs/03.tutorial/03.outputs/eval_expressions.png
deleted file mode 100644
index e284622385a..00000000000
Binary files a/src/contents/docs/03.tutorial/03.outputs/eval_expressions.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/03.outputs/file-outputs-2-0.png b/src/contents/docs/03.tutorial/03.outputs/file-outputs-2-0.png
new file mode 100644
index 00000000000..6da1cc40925
Binary files /dev/null and b/src/contents/docs/03.tutorial/03.outputs/file-outputs-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/03.outputs/index.md b/src/contents/docs/03.tutorial/03.outputs/index.md
index 12dc9bfdf8c..7d8ba213486 100644
--- a/src/contents/docs/03.tutorial/03.outputs/index.md
+++ b/src/contents/docs/03.tutorial/03.outputs/index.md
@@ -15,14 +15,14 @@ Tasks and flows can generate outputs that are passed to downstream processes. To
This storage layer helps avoid connector sprawl. For example, the PostgreSQL plugin can extract data and load it into internal storage. Other tasks can then load that data into Snowflake, BigQuery, or Redshift — or process it with another plugin — without direct point-to-point connections. Let's check out Outputs in practice.
-
+
## How to retrieve outputs
-To see which outputs have been generated during a flow execution, go to the **Outputs** tab on the Execution page:
+To see which outputs have been generated during a flow execution, go to the **Input/Output** tab on the Execution page:
-
+
Outputs are useful for troubleshooting and auditing. Additionally, you can use outputs to:
- share **downloadable artifacts** with business stakeholders (e.g., a table generated by a SQL query or a CSV file generated by a Python script)
@@ -96,7 +96,7 @@ tasks:
This flow processes data using Polars and stores the result as a CSV file.
-
+
:::alert{type="info"}
To avoid package dependency conflicts, the Python task runs in an **independent Docker container**. You can optionally provide a **custom Docker image** from a private container registry or use a public Python image from DockerHub and install any custom package dependencies using the `beforeCommands` argument. The `beforeCommands` argument allows you to install any custom package dependencies — here, we install [Polars](https://www.pola.rs/). Use as many commands as needed to prepare the containerized environment for script execution.
@@ -108,7 +108,7 @@ When referencing the output from the previous task, this flow uses `jq` [languag
You can test `{{ outputs.task_id.body | jq('.products') | first }}` and any other output parsing expression using the built-in expressions evaluator on the **Outputs** page:
-
+
## Passing data between tasks
@@ -156,9 +156,9 @@ tasks:
store: true
```
-This example flow passes data between tasks using Outputs. The `inputFiles` argument of the `io.kestra.plugin.jdbc.duckdb.Query` task allows you to pass files from internal storage to the task. The `store: true` property ensures that the result of the SQL query is stored in the internal storage and can be previewed and downloaded from the Outputs tab.
+This example flow passes data between tasks using Outputs. The `inputFiles` argument of the `io.kestra.plugin.jdbc.duckdb.Query` task allows you to pass files from internal storage to the task. The `store: true` property ensures that the result of the SQL query is stored in the internal storage and can be previewed and downloaded from the Input/Output tab.
-
+
This flow extracts data from an API, processes it in a Python script, executes a SQL query, and generates a downloadable artifact.
diff --git a/src/contents/docs/03.tutorial/03.outputs/output-2-0.png b/src/contents/docs/03.tutorial/03.outputs/output-2-0.png
new file mode 100644
index 00000000000..34f48defde3
Binary files /dev/null and b/src/contents/docs/03.tutorial/03.outputs/output-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/03.outputs/output.png b/src/contents/docs/03.tutorial/03.outputs/output.png
deleted file mode 100644
index 915da413621..00000000000
Binary files a/src/contents/docs/03.tutorial/03.outputs/output.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/03.outputs/preview-2-0.png b/src/contents/docs/03.tutorial/03.outputs/preview-2-0.png
new file mode 100644
index 00000000000..df18a12e0bc
Binary files /dev/null and b/src/contents/docs/03.tutorial/03.outputs/preview-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/03.outputs/preview.png b/src/contents/docs/03.tutorial/03.outputs/preview.png
deleted file mode 100644
index 1a464534e45..00000000000
Binary files a/src/contents/docs/03.tutorial/03.outputs/preview.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/03.outputs/tutorial-outputs-python.png b/src/contents/docs/03.tutorial/03.outputs/tutorial-outputs-python.png
deleted file mode 100644
index 6a875d41453..00000000000
Binary files a/src/contents/docs/03.tutorial/03.outputs/tutorial-outputs-python.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/04.triggers/flow-triggers-2-0.png b/src/contents/docs/03.tutorial/04.triggers/flow-triggers-2-0.png
new file mode 100644
index 00000000000..cc51955ee14
Binary files /dev/null and b/src/contents/docs/03.tutorial/04.triggers/flow-triggers-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/04.triggers/flow-triggers.png b/src/contents/docs/03.tutorial/04.triggers/flow-triggers.png
deleted file mode 100644
index 5f21fa41339..00000000000
Binary files a/src/contents/docs/03.tutorial/04.triggers/flow-triggers.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/04.triggers/index.md b/src/contents/docs/03.tutorial/04.triggers/index.md
index 2f9cde576e7..582c4781f32 100644
--- a/src/contents/docs/03.tutorial/04.triggers/index.md
+++ b/src/contents/docs/03.tutorial/04.triggers/index.md
@@ -13,7 +13,7 @@ Triggers automatically start your flow based on events or a schedule.
A trigger can be a scheduled date, the arrival of a new file, a new message in a queue, the completion of another flow's execution and much more.
-
+
## Defining triggers
@@ -22,7 +22,7 @@ Like `inputs` and `tasks`, use the `triggers` keyword in the flow to define a li
To get started, take a look at the flow below. The `schedule_trigger` defines a `cron` expression to run every day at 10 AM. The [Schedule trigger](../../05.workflow-components/07.triggers/01.schedule-trigger/index.md) is great for nightly jobs and other static schedule-oriented workflows. However, Kestra does not limit you to schedule-based orchestration.
-The workflow below also includes a `flow_trigger` that automatically starts the `getting_started` flow whenever the `first_flow` defined in the conditions finishes executing. In other words, a flow can be triggered by time-based schedules, by events, or by several triggers at once. For example, you can react to a change in a [Google Sheet](/plugins/plugin-googleworkspace/google-sheets/io.kestra.plugin.googleworkspace.sheets.sheetmodifiedtrigger), a new file in an [S3 bucket](/plugins/plugin-aws/aws-s3/io.kestra.plugin.aws.s3.trigger), a [PostgreSQL database](/plugins/plugin-jdbc-postgres/io.kestra.plugin.jdbc.postgresql.trigger) query result, or even when an [email is received](/plugins/plugin-email/io.kestra.plugin.email.realtimetrigger) in real time.
+The workflow below also includes a `flow_trigger` that automatically starts the `getting_started` flow whenever the `first_flow` listed in `dependsOn` finishes executing. In other words, a flow can be triggered by time-based schedules, by events, or by several triggers at once. For example, you can react to a change in a [Google Sheet](/plugins/plugin-googleworkspace/google-sheets/io.kestra.plugin.googleworkspace.sheets.sheetmodifiedtrigger), a new file in an [S3 bucket](/plugins/plugin-aws/aws-s3/io.kestra.plugin.aws.s3.trigger), a [PostgreSQL database](/plugins/plugin-jdbc-postgres/io.kestra.plugin.jdbc.postgresql.trigger) query result, or even when an [email is received](/plugins/plugin-email/io.kestra.plugin.email.realtimetrigger) in real time.
```yaml
id: getting_started
@@ -40,10 +40,9 @@ triggers:
- id: flow_trigger
type: io.kestra.plugin.core.trigger.Flow
- conditions:
- - type: io.kestra.plugin.core.condition.ExecutionFlow
+ dependsOn:
+ - flowId: first_flow
namespace: company.team
- flowId: first_flow
```
:::alert{type="info"}
@@ -109,6 +108,6 @@ tasks:
With a trigger added to a flow, you can now see the trigger's details in the flow's **Triggers** tab.
-
+
To learn more about Triggers, check out the full [Triggers documentation](../../05.workflow-components/07.triggers/index.mdx). Next up, we'll check out Flowable tasks – ways to loop, condition, and parallelize tasks.
diff --git a/src/contents/docs/03.tutorial/05.flowable/for-each-author.png b/src/contents/docs/03.tutorial/05.flowable/for-each-author.png
deleted file mode 100644
index 2ee4c8197e6..00000000000
Binary files a/src/contents/docs/03.tutorial/05.flowable/for-each-author.png and /dev/null differ
diff --git a/src/contents/docs/03.tutorial/05.flowable/index.md b/src/contents/docs/03.tutorial/05.flowable/index.md
index b0a69aaac59..d86062e6984 100644
--- a/src/contents/docs/03.tutorial/05.flowable/index.md
+++ b/src/contents/docs/03.tutorial/05.flowable/index.md
@@ -9,7 +9,7 @@ icon: /src/contents/docs/icons/tutorial.svg
Run tasks or subflows in parallel, create loops, and conditional branching.
-
+
The example flow from earlier in this tutorial extracts data from an API, processes it in a Python script, executes a SQL query, and generates a downloadable artifact on a predefined schedule. Many real-world use cases require branching, looping, or running several tasks simultaneously. Kestra handles these requirements with Flowable tasks.
@@ -20,17 +20,12 @@ For example, you can use the [If task](/plugins/core/flow/io.kestra.plugin.core.
The example below redesigns the flow to use a `SELECT` input for product category rather than a `STRING` URI, while still calling [dummyjson](https://dummyjson.com). An API request is made based on the selected category — `beauty` or `notebooks` (one does not exist).
-The `check_products` If task has a `condition` of `"{{ json(outputs.api.body).products | length > 0 }}"` (i.e., checking whether the API body is not empty and contains at least one product). The log message then depends on whether the actual product category exists or not. The `then` property defines the action for a true condition, and the `else` property defines the action for a false result.
+The `check_products` If task has a `condition` of `"{{ fromJson(outputs.api.body).products | length > 0 }}"` (i.e., checking whether the API body is not empty and contains at least one product). The log message then depends on whether the actual product category exists or not. The `then` property defines the action for a true condition, and the `else` property defines the action for a false result.
```yaml
id: getting_started
namespace: company.team
-triggers:
- - id: every_monday_at_10_am
- type: io.kestra.plugin.core.trigger.Schedule
- cron: 0 10 * * 1
-
inputs:
- id: category
type: SELECT
@@ -46,12 +41,12 @@ tasks:
- id: check_products
type: io.kestra.plugin.core.flow.If
- condition: "{{ json(outputs.api.body).products | length > 0 }}"
+ condition: "{{ fromJson(outputs.api.body).products | length > 0 }}"
then:
- id: log_status
type: io.kestra.plugin.core.log.Log
- message: "Found {{ json(outputs.api.body).products | length }} products for category {{ inputs.category }}"
- - id: transform
+ message: "Found {{ fromJson(outputs.api.body).products | length }} products for category {{ inputs.category }}"
+ - id: python
type: io.kestra.plugin.scripts.python.Script
containerImage: python:slim
dependencies:
@@ -65,10 +60,10 @@ tasks:
df.glimpse()
# Keep a simple view for this category
df.select(["title", "brand", "price"]).write_csv("products.csv")
- - id: sql_query
+ - id: sqlQuery
type: io.kestra.plugin.jdbc.duckdb.Query
inputFiles:
- in.csv: "{{ outputs.transform.outputFiles['products.csv'] }}"
+ in.csv: "{{ outputs.python.outputFiles['products.csv'] }}"
sql: |
SELECT brand, round(avg(price), 2) AS avg_price, count(*) AS cnt
FROM read_csv_auto('{{ workingDir }}/in.csv', header=True)
@@ -79,6 +74,11 @@ tasks:
- id: when_false
type: io.kestra.plugin.core.log.Log
message: "No products found for category {{ inputs.category }}."
+
+triggers:
+ - id: every_monday_at_10_am
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: 0 10 * * 1
```
Execute the flow twice, once with `beauty` and once with `notebooks` to examine the results.
@@ -87,31 +87,31 @@ Execute the flow twice, once with `beauty` and once with `notebooks` to examine
A common orchestration pattern is operating on a set of values. Kestra offers several approaches depending on your use case. The standalone examples below demonstrate each type.
-### ForEach
+### Loop
-The **ForEach** flowable task executes a group of tasks for each value in the list. There are many ways to implement ForEach for complex looping operations, possibly incorporating conditional flowable tasks or subtasks. See more examples in the [ForEach documentation](/plugins/core/flow/io.kestra.plugin.core.flow.foreach).
+The `Loop` flowable task iterates over a list of values and runs child tasks for each item. Each iteration runs as an isolated sub-execution. Access the current value with `{{ item.value }}` and the zero-based index with `{{ item.index }}`.
-As an introduction to the feature, the below example demonstrates using ForEach to make an API call to [OpenLibrary](https://openlibrary.org/dev/docs/api/search) to get a list of associated titles for each author in the list. The values are defined as a JSON string or an array, i.e., a list of string values `["value1", "value2"]` or a list of key-value pairs `[{"key": "value1"}, {"key": "value2"}]`.
-
-You can access the current iteration value using the variable `{{ taskrun.value }}`:
+Values can be a static list, a JSON array string, a map, or an ION file URI. The example below makes an API call for each author in the list:
```yaml
-id: for_loop_example
+id: loop_example
namespace: tutorial
tasks:
- - id: for_each
- type: io.kestra.plugin.core.flow.ForEach
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
values: ["pynchon", "dostoyevsky", "hedayat"]
tasks:
- id: api
type: io.kestra.plugin.core.http.Request
- uri: "https://openlibrary.org/search.json?author={{ taskrun.value }}&sort=new"
+ uri: "https://openlibrary.org/search.json?author={{ item.value }}&sort=new"
```
-After execution, the Gantt view shows separate runs for each of the three listed authors in the task.
+After execution, the Gantt view shows the main Loop task with the sub-executions for each loop through the values list for each author. Click into **Iterations** to view each loop individually.
+
+
-
+See the [Loop documentation](../../05.workflow-components/01.tasks/00.flowable-tasks/index.md#loop) for output collection, nested loops, error handling, and map-reduce patterns.
### LoopUntil
@@ -147,11 +147,11 @@ This flow checks an HTTP endpoint every 30 seconds and stops either when it retu
A common orchestration requirement is executing independent processes **in parallel**. For example, you can process data for each partition in parallel. This can significantly speed up the processing time.
-The flow below uses the `ForEach` flowable task to execute a list of `tasks` in parallel.
+The flow below uses the `Loop` flowable task with `concurrencyLimit: 0` to process all partitions simultaneously.
-1. The `concurrencyLimit` property with value `0` makes the list of `tasks` to execute in parallel.
+1. The `concurrencyLimit` property set to `0` removes the cap on parallel iterations.
2. The `values` property defines the list of items to iterate over.
-3. The `tasks` property defines the list of tasks to execute for each item in the list. You can access the iteration value using the `{{ taskrun.value }}` variable.
+3. The `tasks` property defines the child tasks for each iteration. Access the iteration value with `{{ item.value }}`.
```yaml
id: python_partitions
@@ -160,7 +160,7 @@ namespace: company.team
description: Process partitions in parallel
tasks:
- - id: get_partitions
+ - id: getPartitions
type: io.kestra.plugin.scripts.python.Script
taskRunner:
type: io.kestra.plugin.scripts.runner.docker.Docker
@@ -170,10 +170,10 @@ tasks:
partitions = [f"file_{nr}.parquet" for nr in range(1, 10)]
Kestra.outputs({'partitions': partitions})
- - id: process_partitions
- type: io.kestra.plugin.core.flow.ForEach
+ - id: processPartitions
+ type: io.kestra.plugin.core.flow.Loop
concurrencyLimit: 0
- values: '{{ outputs.get_partitions.vars.partitions }}'
+ values: '{{ outputs.getPartitions.vars.partitions }}'
tasks:
- id: partition
type: io.kestra.plugin.scripts.python.Script
@@ -186,7 +186,7 @@ tasks:
import time
from kestra import Kestra
- filename = '{{ taskrun.value }}'
+ filename = '{{ item.value }}'
print(f"Reading and processing partition {filename}")
nr_rows = random.randint(1, 1000)
processing_time = random.randint(1, 20)
diff --git a/src/contents/docs/03.tutorial/05.flowable/loop-authors-2-0.png b/src/contents/docs/03.tutorial/05.flowable/loop-authors-2-0.png
new file mode 100644
index 00000000000..6a8df9d0f70
Binary files /dev/null and b/src/contents/docs/03.tutorial/05.flowable/loop-authors-2-0.png differ
diff --git a/src/contents/docs/03.tutorial/06.errors/index.md b/src/contents/docs/03.tutorial/06.errors/index.md
index cf75e03db5f..6f96482f704 100644
--- a/src/contents/docs/03.tutorial/06.errors/index.md
+++ b/src/contents/docs/03.tutorial/06.errors/index.md
@@ -11,7 +11,7 @@ Handle errors with automatic retries and notifications.
Failure is inevitable. Kestra offers automatic retries and error handling to help you build resilient workflows.
-
+
## Handle errors with retries and alerts
@@ -56,11 +56,6 @@ Taking our flow from earlier stages, we can add a Slack alert on an execution er
id: getting_started_category_check
namespace: company.team
-triggers:
- - id: every_monday_at_10_am
- type: io.kestra.plugin.core.trigger.Schedule
- cron: 0 10 * * 1
-
inputs:
- id: category
type: SELECT
@@ -76,12 +71,12 @@ tasks:
- id: check_products
type: io.kestra.plugin.core.flow.If
- condition: "{{ json(outputs.api.body).products | length > 0 }}"
+ condition: "{{ fromJson(outputs.api.body).products | length > 0 }}"
then:
- id: log_status
type: io.kestra.plugin.core.log.Log
- message: "Found {{ json(outputs.api.body).products | length }} products for category {{ inputs.category }}"
- - id: transform
+ message: "Found {{ fromJson(outputs.api.body).products | length }} products for category {{ inputs.category }}"
+ - id: python
type: io.kestra.plugin.scripts.python.Script
containerImage: python:slim
dependencies:
@@ -94,10 +89,10 @@ tasks:
df = pl.from_dicts(data)
df.glimpse()
df.select(["title", "brand", "price", "rating"]).write_csv("products.csv")
- - id: sql_query
+ - id: sqlQuery
type: io.kestra.plugin.jdbc.duckdb.Queries
inputFiles:
- in.csv: "{{ outputs.transform.outputFiles['products.csv'] }}"
+ in.csv: "{{ outputs.python.outputFiles['products.csv'] }}"
sql: |
SELECT brand, round(avg(price), 2) AS avg_price, count(*) AS cnt
FROM read_csv_auto('{{ workingDir }}/in.csv', header=True)
@@ -114,6 +109,11 @@ errors:
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
url: "{{ secret('SLACK_WEBHOOK') }}"
messageText: "Failure alert for flow {{ flow.namespace }}.{{ flow.id }} with ID {{ execution.id }}"
+
+triggers:
+ - id: every_monday_at_10_am
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: 0 10 * * 1
```
Now if there is an error, say our API endpoint is unreachable, we'll get a Slack alert notifying a team to investigate. For more, check the [error handling](../../05.workflow-components/11.errors/index.md) page.
@@ -141,14 +141,9 @@ tasks:
triggers:
- id: listen
type: io.kestra.plugin.core.trigger.Flow
- conditions:
- - type: io.kestra.plugin.core.condition.ExecutionStatus
- in:
- - FAILED
- - WARNING
- - type: io.kestra.plugin.core.condition.ExecutionNamespace
- namespace: company.team
- prefix: true
+ dependsOn:
+ - states: [FAILED, WARNING]
+ when: "{{ namespace | startsWith('company.team') }}"
```
Adding this flow ensures you receive a Slack alert for any flow failure in the `company.team` namespace.
@@ -242,11 +237,11 @@ tasks:
- id: check_products
type: io.kestra.plugin.core.flow.If
- condition: "{{ json(outputs.api.body).products | length > 0 }}"
+ condition: "{{ fromJson(outputs.api.body).products | length > 0 }}"
then:
- id: log_status
type: io.kestra.plugin.core.log.Log
- message: "Found {{ json(outputs.api.body).products | length }} products for category {{ inputs.category }}"
+ message: "Found {{ fromJson(outputs.api.body).products | length }} products for category {{ inputs.category }}"
- id: python
type: io.kestra.plugin.scripts.python.Script
containerImage: python:slim
diff --git a/src/contents/docs/03.tutorial/index.mdx b/src/contents/docs/03.tutorial/index.mdx
index f9351a7069b..7a4af4a7394 100644
--- a/src/contents/docs/03.tutorial/index.mdx
+++ b/src/contents/docs/03.tutorial/index.mdx
@@ -14,21 +14,24 @@ Use this tutorial to learn Kestra’s core concepts and build flows step by step
**Before you start:** Complete the [Quickstart](../01.quickstart/index.md) so you have Kestra running locally. This tutorial assumes you can open the Kestra UI at `http://localhost:8080`.
:::
-:::alert{type="info"}
-**Using an AI coding agent?** Add the [Kestra MCP server](../ai-tools/kestra-mcp-resources/index.md) to Claude Code, Cursor, or any MCP-compatible tool. It gives your agent live access to plugin docs, blueprints, and Kestra documentation — useful throughout this tutorial and beyond.
-:::
-
-
+
## What you can do with Kestra
You can use Kestra to:
-- Run workflows on demand, event-driven, or on a schedule.
-- Interact with any system or language through plugins and tasks.
-- Orchestrate microservices, batch jobs, scripts, SQL queries, data syncs, dbt or Spark jobs, and other processes.
+- Run workflows on a schedule, on demand, or triggered by events such as webhooks, file arrivals, or messages on a queue.
+- Execute scripts in any language — Python, Node.js, R, Go, Bash — inside isolated containers.
+- Query databases, call APIs, read and write files in cloud storage, and send notifications.
+- Orchestrate microservices, batch jobs, data pipelines, dbt or Spark jobs, and ETL processes.
+- Provision and manage infrastructure — run Terraform plans, deploy to Kubernetes, and automate cloud resource lifecycle.
+- Build AI pipelines — call LLM APIs, run model training and evaluation jobs, and orchestrate multi-step agent workflows.
+- Pass data between tasks and flows, branch on conditions, run steps in parallel, and loop over dynamic item lists.
+- Handle failures with automatic retries, fallback tasks, and alerts.
+
+If you prefer to generate and deploy flows from an AI coding agent, see [AI tools](../ai-tools/index.mdx) to connect Claude Code, Cursor, or similar tools to Kestra’s MCP server.
This tutorial is a sequence of six modules. Work through them in order — each one builds on the flow you created in the previous module:
diff --git a/src/contents/docs/04.contribute-to-kestra/04.docs-contributor-guide/child-card.png b/src/contents/docs/04.contribute-to-kestra/04.docs-contributor-guide/child-card.png
index ddb25a44aab..f233b2705d2 100644
Binary files a/src/contents/docs/04.contribute-to-kestra/04.docs-contributor-guide/child-card.png and b/src/contents/docs/04.contribute-to-kestra/04.docs-contributor-guide/child-card.png differ
diff --git a/src/contents/docs/04.contribute-to-kestra/04.docs-contributor-guide/index.mdx b/src/contents/docs/04.contribute-to-kestra/04.docs-contributor-guide/index.mdx
index d65a2f6c30a..9485a29485f 100644
--- a/src/contents/docs/04.contribute-to-kestra/04.docs-contributor-guide/index.mdx
+++ b/src/contents/docs/04.contribute-to-kestra/04.docs-contributor-guide/index.mdx
@@ -18,7 +18,7 @@ To contribute to the documentation, fork the [docs repository](https://github.co
## Build the documentation locally
The following dependencies are required to build Kestra docs locally:
-- Node 14+ and npm
+- Node 26+ and npm
- An IDE (such as VS Code, IntelliJ, etc.)
To start contributing:
@@ -59,22 +59,24 @@ The index file's markdown looks like this:
```markdown
---
-title: Getting Started
+title: "Install Kestra: Docker, Kubernetes, VM, and JAR"
+sidebarTitle: Installation Guide
+icon: /src/contents/docs/icons/installation.svg
---
-Follow the [Quickstart Guide](../../01.quickstart/index.md) to install Kestra and start building your first workflows.
+import ChildCard from "~/components/docs/ChildCard.astro"
```
-And the page displays the following with all the sub topics of "Getting Started" listed with their card and icon:
+And the page displays the following with all child topics listed as cards:
-
+
-Note that when writing a standalone documentation page, the first sentence appears in the ChildCard view to introduce the topic. In the above example for [Quickstart Guide](../../01.quickstart/index.md) this sentence is visible:
+Note that when writing a standalone documentation page, the first sentence appears in the ChildCard view to introduce the topic. In the above example for the [Kubernetes installation page](../../02.installation/index.mdx) this sentence is visible:
```markdown
-Start Kestra in a Docker container and create your first flow.
+Deploy Kestra on Kubernetes using the official Helm chart, scalable for production with PostgreSQL and object storage.
```
Ideally, keep this first sentence as clear and concise as possible to not clutter the view on the card.
@@ -227,6 +229,10 @@ editions: ["OSS", "EE"]
### Writing style
+For the complete editorial reference — page types, capitalization rules, procedures structure, anti-patterns, and a full review checklist — see the [Kestra Docs Style Guide](../../docs-style-guide.md).
+
+The most common conventions to keep in mind:
+
#### Headings
Use sentence case for all body headings — capitalise only the first word and proper nouns. Use Title Case for `title` and `sidebarTitle` in front matter.
@@ -271,7 +277,7 @@ Here is where the collapsed text goes.
:::
```
-Here is a full example using a flow and subflow with a ForEach task:
+Here is a full example using a flow and subflow with a Loop task:
:::collapse{title="Full Flow Example"}
@@ -282,49 +288,44 @@ id: subflow
namespace: company.team
inputs:
- - id: items
+ - id: item
type: STRING
tasks:
- - id: for_each_item
+ - id: process
type: io.kestra.plugin.scripts.shell.Commands
taskRunner:
type: io.kestra.plugin.core.runner.Process
commands:
- - cat "{{ inputs.items }}"
+ - echo "Processing {{ inputs.item }}"
- - id: read
+ - id: log
type: io.kestra.plugin.core.log.Log
- message: "{{ read(inputs.items) }}"
+ message: "Processed: {{ inputs.item }}"
```
-Below is a Flow that uses the `ForEachItem` task to iterate over a list of items and run the `subflow` for a batch of 10 items at a time:
+Below is a flow that uses the `Loop` task to iterate over a list of items and run the `subflow` for each one:
```yaml
id: each_parent
namespace: company.team
tasks:
- - id: extract
- type: io.kestra.plugin.jdbc.duckdb.Query
- sql: |
- INSTALL httpfs;
- LOAD httpfs;
- SELECT *
- FROM read_csv_auto('https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv', header=True);
- store: true
-
- - id: each
- type: io.kestra.plugin.core.flow.ForEachItem
- items: "{{ outputs.extract.uri }}"
- batch:
- rows: 10
- namespace: company.team
- flowId: subflow
- wait: true
- transmitFailed: true
- inputs:
- items: "{{ taskrun.items }}"
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values:
+ - orders_jan.csv
+ - orders_feb.csv
+ - orders_mar.csv
+ tasks:
+ - id: process_file
+ type: io.kestra.plugin.core.flow.Subflow
+ namespace: company.team
+ flowId: subflow
+ wait: true
+ transmitFailed: true
+ inputs:
+ item: "{{ item.value }}"
```
:::
@@ -386,7 +387,7 @@ Kestra Plugins each have their own documentation page on the website in [Plugins
For example, if you want to contribute to the [OpenAI Plugin](/plugins/plugin-openai), you can find the documentation in the [OpenAI Plugin Repository](https://github.com/kestra-io/plugin-openai).
:::alert{type="info"}
-All plugin repos are searchable from the central [Kestra GitHub](https://github.com/kestra-io). The name of the repository is in the URL of the plugin documentation page. For example, the OpenAI repo is called `plugin-openai` which is in the URL path `https://kestra.io/plugins/plugin-openai/io.kestra.plugin.openai.chatcompletion`. Simply searching the tool's name should suffice, but this always works just in case.
+All plugin repos are searchable from the central [Kestra GitHub](https://github.com/kestra-io). The name of the repository is in the URL of the plugin documentation page. For example, the OpenAI repo is called `plugin-openai`, which is in the URL path `https://kestra.io/plugins/plugin-openai/io.kestra.plugin.openai.chatcompletion`. Searching the tool's name should suffice, but this always works as a fallback.
:::
To contribute to a plugin's documentation, fork the repository. Once cloned, contributions are welcome to four key components of plugin task documentation: title, description, examples, and properties.
diff --git a/src/contents/docs/05.workflow-components/01.flow/index.md b/src/contents/docs/05.workflow-components/01.flow/index.md
index b84352211ab..495b41af1bd 100644
--- a/src/contents/docs/05.workflow-components/01.flow/index.md
+++ b/src/contents/docs/05.workflow-components/01.flow/index.md
@@ -7,19 +7,17 @@ icon: /src/contents/docs/icons/flow.svg
docId: flows
---
-Flow is a container for tasks and their orchestration logic.
-
-A Flow is the fundamental unit of orchestration in Kestra. It defines a set of tasks, their execution order, inputs, outputs, and orchestration logic.
+A flow is a container for tasks and their orchestration logic.
-
+
## Components of a flow
-A flow organizes `tasks`, their `inputs` and `outputs`, error handling, and orchestration logic. It specifies **what** tasks run, **when** they run, and **how** they interact (sequentially, in parallel, or conditionally).
+A flow organizes tasks, their inputs and outputs, error handling, and orchestration logic. It specifies **what** tasks run, **when** they run, and **how** they interact (sequentially, in parallel, or conditionally).
-You can define a flow declaratively using a [YAML](https://en.wikipedia.org/wiki/YAML) file. Alternatively, you can also build flows using the [No Code editor](../../09.ui/01.flows/index.md) instead of writing your own YAML.
+You can define a flow declaratively in YAML or build it using the [No Code editor](../../09.ui/01.flows/index.md).
A flow must have:
@@ -29,25 +27,23 @@ A flow must have:
Optionally, a flow can also have:
-- [inputs](../05.inputs/index.md)
-- [outputs](../06.outputs/index.md)
-- [variables](../04.variables/index.md)
-- [triggers](../07.triggers/index.mdx)
-- [labels](../08.labels/index.md)
-- [pluginDefaults](../09.plugin-defaults/index.md)
-- [errors](../11.errors/index.md)
-- [finally](../19.finally/index.md)
-- [retries](../12.retries/index.md)
-- [sla](../18.sla/index.md)
-- [concurrency](../14.concurrency/index.md)
-- [descriptions](../15.descriptions/index.md)
-- [disabled](../16.disabled/index.md)
-- [revision](../../06.concepts/03.revision/index.md)
-- [checks](../07.checks/index.md)
+- [inputs](../05.inputs/index.md) — typed parameters passed at execution time
+- [outputs](../06.outputs/index.md) — values or files a flow produces for downstream use
+- [variables](../04.variables/index.md) — reusable key/value pairs scoped to the flow
+- [triggers](../07.triggers/index.mdx) — schedule or event conditions that start executions automatically
+- [labels](../08.labels/index.md) — key/value metadata for filtering and grouping executions
+- [errors](../11.errors/index.md) — tasks that run when a flow or task fails
+- [finally](../19.finally/index.md) — tasks that always run at the end, regardless of execution outcome
+- [retries](../12.retries/index.md) — automatic retry policy on task failure
+- [sla](../18.sla/index.md) — time-based constraints that fail or alert when exceeded
+- [concurrency](../14.concurrency/index.md) — limits on how many executions of this flow can run simultaneously
+- [descriptions](../15.descriptions/index.md) — Markdown documentation attached to flows and tasks
+- [disabled](../16.disabled/index.md) — prevent a flow from executing without deleting it
+- [checks](../07.checks/index.md) — assertions that must pass before an execution is created
## Flow sample
-Below is a sample flow definition. It uses tasks available in Kestra core for testing purposes, such as the `Return` or `Log` tasks, and demonstrates how to use `labels`, `inputs`, `variables`, `triggers`, and various `descriptions`.
+The example below uses several of the optional components listed above — refer to each component's documentation for full configuration details.
```yaml
id: hello-world
@@ -63,151 +59,67 @@ inputs:
- id: my-value
type: STRING
defaults: "default value"
- description: This input is has a default value.
+ description: This input has a default value.
variables:
first: "1"
- second: "{{vars.first}} > 2"
+ second: "{{ vars.first }} > 2"
tasks:
+ - id: hello
+ type: io.kestra.plugin.core.log.Log
+ description: "Log the input value passed at execution time."
+ message: "Hello, {{ inputs.['my-value'] }}!"
+
- id: date
type: io.kestra.plugin.core.debug.Return
- description: "Some tasks **documentation** in *Markdown*"
- format: "A log line content with a contextual date variable {{taskrun.startDate}}"
-
-pluginDefaults:
- - type: io.kestra.plugin.core.log.Log
- values:
- level: ERROR
-```
-
-### Plugin defaults
-
-Use `pluginDefaults` to avoid repeating common configurations across multiple tasks of the same type. This is a list of default task properties that will be applied to each task of a certain type inside your flow. Refer to the [Plugin Defaults documentation](../09.plugin-defaults/index.md) for more details.
-
-### Variables
-
-Flow-level variables define key/value pairs that tasks can access using `{{ vars.key }}`. Refer to the [flow variables documentation](../04.variables/index.md) for more details.
-
-### List of tasks
-
-The most important part of a flow is the list of tasks that will be run sequentially when the flow is executed.
-
-## Disable a flow
-
-By default, all flows are active and will execute whether or not a trigger has been set.
-
-You can [disable a flow](../16.disabled/index.md) to temporarily prevent it from running. This is useful for pausing scheduled executions, troubleshooting, or testing.
-
-
-
-## Task
-
-A task is a single action in a flow. A task can have properties, use flow inputs and other task's outputs, perform an action, and produce an [output](#outputs).
-
-There are two kinds of tasks in Kestra:
-
-- Runnable Tasks – Perform actual work (API calls, database queries, computations). Executed by workers.
-
-- Flowable Tasks – Control orchestration (branching, looping, parallelization). Executed by the executor, not suitable for heavy computation.
-
-### Runnable Task
-
-[Runnable Tasks](../01.tasks/01.runnable-tasks/index.md) handle computational work in the flow. For example, these include file system operations, API calls, database queries, etc. These tasks can be compute-intensive and are handled by workers.
-
-By default, Kestra only includes a few Runnable Tasks. However, many of them are available as [plugins](/plugins), and if you use our default Docker image, plenty of them are already included.
-
-### Flowable Task
-
-[Flowable Tasks](../01.tasks/00.flowable-tasks/index.md) only handle flow logic (branching, grouping, parallel processing, etc.) and start new tasks. For example, the [Switch task](/plugins/core/flow/io.kestra.plugin.core.flow.switch) decides the next task to run based on some inputs.
-
-A Flowable Task is handled by an executor and can be called very often. Because of that, these tasks cannot include intensive computations, unlike Runnable Tasks. Most of the common Flowable Tasks are available in the default Kestra installation.
+ description: "Return the current date as a task output."
+ format: "{{ taskrun.startDate }}"
-## Labels
-
-Labels are key-value pairs that you can add to flows. Labels are used to **organize** flows and can be used to **filter executions** of any given flow from the UI.
-
-## Inputs
-
-Inputs are strongly typed parameters provided at execution time. Can be required or optional, with default values and validation rules.
-
-Inputs of type `FILE` are uploaded to Kestra's [internal storage](../../08.architecture/data-components/index.md#internal-storage) and made available for all tasks.
-
-Flow inputs can be seen in the **Overview** tab of the **Execution** page.
-
-## Outputs
-
-Outputs are results produced by tasks or flows. Outputs can be reused in later tasks or downloaded if stored in internal storage.
-
-Some outputs are of a special type and are stored in Kestra's internal storage. Kestra automatically makes these outputs available for all tasks.
-
-You can view:
-
-- task outputs in the **Outputs** tab of the **Execution** page
-- flow outputs in the **Overview** tab of the **Execution** page
-
-If an output is a file from the internal storage, it will be available to download.
+outputs:
+ - id: execution_date
+ type: STRING
+ value: "{{ outputs.date.value }}"
-For more details on both task and flow outputs, see the [Outputs](../06.outputs/index.md) page.
+triggers:
+ - id: daily
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 9 * * *"
+```
## Revision
Every change to a flow creates a new revision. Kestra automatically manages revisions, similar to version control, and you can view them in the **Revisions** tab.
-You can access old revisions inside the **Revisions** tab of the **Flows** page.
-
-## Triggers
-
-[Triggers](../07.triggers) are a way to start a flow from external events. For example, a trigger might initiate a flow at a scheduled time or based on external events (webhooks, file creation, message in a broker, etc.).
+Use **Save as draft** to stage changes without affecting running executions. Draft revisions are not executed — any trigger or manual run falls back to the last published revision until you publish the draft. See [Draft revisions](../../06.concepts/03.revision/index.md#draft-revisions) for details.
## Flow variable expressions
-Flows have a number of variable expressions giving you information about them dynamically, a few examples include:
-
-| Parameter | Description |
-|-------------------------------|-----------------------------------------------------------------------------------------------------------------------------------|
-| `{{ flow.id }}` | The identifier of the flow. |
-| `{{ flow.namespace }}` | The name of the flow namespace. |
-| `{{ flow.tenantId }}` | The identifier of the tenant (EE only). |
-| `{{ flow.revision }}` | The revision of the flow. |
-
-## Listeners (deprecated)
-
-Listeners are special tasks that can listen to the current flow and launch tasks *outside the flow*, meaning launch tasks that are not part of the flow.
-
-The results of listeners do not change the execution status of the flow. Listeners are mainly used to send notifications or handle special behavior outside the primary flow.
-
-:::alert{type="warning"}
-These features are retained for backward compatibility and will be removed in future versions. Use alternative patterns (e.g., triggers, reusable tasks) instead.
-:::
-
-## Templates (deprecated)
-
-Templates are lists of tasks that can be shared between flows. You can define a template and call it from other flows. Templates allow you to share a list of tasks and keep them updated without changing all flows that use them.
+| Parameter | Description |
+|---|---|
+| `{{ flow.id }}` | The identifier of the flow. |
+| `{{ flow.namespace }}` | The name of the flow namespace. |
+| `{{ flow.tenantId }}` | The identifier of the tenant (EE and Cloud only). |
+| `{{ flow.revision }}` | The revision of the flow. |
## FAQ
### Where does Kestra store flows?
-Flows are stored in a serialized format directly **in the Kestra backend database**.
+Flows are stored in a serialized format directly in the Kestra backend database.
-The easiest way to add new flows is to add them directly from the Kestra UI. You can also use [`kestractl flows deploy`](../../kestra-cli/kestractl/index.md) to push flows from the command line, or use the Git Sync pattern or CI/CD integration to add flows automatically after a pull request is merged to a given Git branch.
+The easiest way to add new flows is from the Kestra UI. You can also use [`kestractl flows deploy`](../../kestra-cli/kestractl/index.md) to push flows from the command line, or use the Git Sync pattern or CI/CD integration to deploy flows automatically after a pull request is merged. On Kestra Enterprise, [Promote](../../version-control-cicd/06.promote/index.md) lets you move a flow between environments (dev, staging, production) directly from the UI without a pipeline.
-To see how flows are represented in a file structure, you can leverage the `_flows` directory in the [Namespace Files](../../06.concepts/02.namespace-files/index.md) editor.
+To see how flows are represented in a file structure, use the `_flows` directory in the [Namespace Files](../../06.concepts/02.namespace-files/index.md) editor.
### How to load flows at server startup?
-To pre-load flows from a directory when Kestra starts (so they’re available immediately), use the `-f` or `--flow-path` flag on the server command:
+To pre-load flows from a directory when Kestra starts, use the `-f` or `--flow-path` flag:
```bash
./kestra server standalone --flow-path /path/to/flows
```
-Point this to a directory of YAML flow definitions; Kestra will load them at startup and place them in the namespaces declared in each file.
-
-For more information about the Kestra server CLI, check the [Server CLI Reference](../../kestra-cli/kestra-server/index.md) section.
-
-### Can I sync a local flows directory to be continuously loaded into Kestra?
+### Can I sync a local flows directory into Kestra?
Yes. See [Synchronize Local Flows](../../15.how-to-guides/local-flow-sync/index.md) for syncing a local directory, or [Sync Flows from a Git Repository](../../15.how-to-guides/syncflows/index.md) for Git-based workflows.
-
diff --git a/src/contents/docs/05.workflow-components/01.tasks/00.flowable-tasks/index.md b/src/contents/docs/05.workflow-components/01.tasks/00.flowable-tasks/index.md
index 1b9500672ac..1fc037eeafc 100644
--- a/src/contents/docs/05.workflow-components/01.tasks/00.flowable-tasks/index.md
+++ b/src/contents/docs/05.workflow-components/01.tasks/00.flowable-tasks/index.md
@@ -6,17 +6,15 @@ sidebarTitle: Flowable Tasks
icon: /src/contents/docs/icons/flow.svg
---
-Control your orchestration logic.
+Flowable tasks control orchestration logic — branching, looping, and parallelizing work — without performing heavy computation themselves.
-## Control orchestration with flowable tasks
-
-Flowable tasks control orchestration logic — running tasks or subflows in parallel, creating loops, and handling conditional branching. They do not run heavy operations; those are handled by workers.
-
-Flowable tasks use [expressions](../../../expressions/index.mdx) from the execution context to determine which tasks run next. For example, you can use the outputs of a previous task in a `Switch` task to decide which task to run next.
+Flowable tasks use [expressions](../../../expressions/index.mdx) from the execution context to determine which tasks run next.
### Sequential
-This task runs tasks sequentially and is typically used to group them.
+`Sequential` runs child tasks one after another, with optional `errors` and `finally` hooks. It is useful for grouping related steps into a named block, particularly when mixing sequential and parallel constructs.
+
+Tasks inside a `Sequential` block can reference sibling task outputs using `{{ outputs.sibling_id.value }}`.
```yaml
id: sequential
@@ -26,28 +24,26 @@ tasks:
- id: sequential
type: io.kestra.plugin.core.flow.Sequential
tasks:
- - id: 1st
+ - id: first_task
type: io.kestra.plugin.core.debug.Return
format: "{{ task.id }} > {{ taskrun.startDate }}"
- - id: 2nd
+ - id: second_task
type: io.kestra.plugin.core.debug.Return
- format: "{{ task.id }} > {{ taskrun.id }}"
+ format: "{{ task.id }} > {{ outputs.first_task.value }}"
- id: last
type: io.kestra.plugin.core.debug.Return
format: "{{ task.id }} > {{ taskrun.startDate }}"
```
-:::alert{type="info"}
-You can access the output of a sibling task using the syntax `{{ outputs.sibling.value }}`.
-:::
-
-For more details on capabilities, check out the [Sequential Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.sequential).
+For more details, check out the [Sequential Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.sequential).
### Parallel
-This task runs tasks in parallel, making it convenient to process many tasks simultaneously.
+`Parallel` starts all child tasks concurrently, reducing total elapsed time when tasks are independent. Because branches run simultaneously, you cannot access the output of a sibling task from within the same `Parallel` block — outputs are only available to tasks that run after the `Parallel` task completes.
+
+Use the `concurrent` property to cap how many branches run at once (`0` = no limit, the default).
```yaml
id: parallel
@@ -56,12 +52,13 @@ namespace: company.team
tasks:
- id: parallel
type: io.kestra.plugin.core.flow.Parallel
+ concurrent: 2
tasks:
- - id: 1st
+ - id: branch_1
type: io.kestra.plugin.core.debug.Return
format: "{{ task.id }} > {{ taskrun.startDate }}"
- - id: 2nd
+ - id: branch_2
type: io.kestra.plugin.core.debug.Return
format: "{{ task.id }} > {{ taskrun.id }}"
@@ -70,17 +67,36 @@ tasks:
format: "{{ task.id }} > {{ taskrun.startDate }}"
```
-:::alert{type="warning"}
-You cannot access the output of a sibling task as tasks will be run in parallel.
-:::
+Nest `Sequential` inside `Parallel` branches to run multi-step sequences concurrently:
+
+ - id: parallel
+ type: io.kestra.plugin.core.flow.Parallel
+ tasks:
+ - id: sequence1
+ type: io.kestra.plugin.core.flow.Sequential
+ tasks:
+ - id: task1
+ type: io.kestra.plugin.core.log.Log
+ message: "step 1a"
+ - id: task2
+ type: io.kestra.plugin.core.log.Log
+ message: "step 1b"
+ - id: sequence2
+ type: io.kestra.plugin.core.flow.Sequential
+ tasks:
+ - id: task3
+ type: io.kestra.plugin.core.log.Log
+ message: "step 2a"
+ - id: task4
+ type: io.kestra.plugin.core.log.Log
+ message: "step 2b"
+```
For more task details, refer to the [Parallel Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.parallel).
### Switch
-This task conditionally runs tasks based on the value of a contextual variable.
-
-In the following example, an input is used to decide which task to run next.
+`Switch` routes execution to a matching case based on the value of an expression. In the following example, an input determines which branch runs.
```yaml
id: switch
@@ -88,7 +104,7 @@ namespace: company.team
inputs:
- id: param
- type: BOOLEAN
+ type: BOOL
tasks:
- id: decision
@@ -109,12 +125,7 @@ For more plugin details, refer to the [Switch Task documentation](/plugins/core/
### If
-This task processes a set of tasks conditionally depending on a condition.
-
-The condition must evaluate to a boolean. Values such as `0`, `-0`, `null`, and `''` evaluate to `false`; all other values evaluate to `true`.
-The `else` branch is optional.
-
-In the following example, an input is used to decide which task to run next.
+`If` runs one branch of tasks when a condition is true and an optional `else` branch when it is false. The condition must evaluate to a boolean — `0`, `-0`, `null`, and `''` evaluate to `false`; all other values evaluate to `true`.
```yaml
id: if_condition
@@ -122,7 +133,7 @@ namespace: company.team
inputs:
- id: param
- type: BOOLEAN
+ type: BOOL
tasks:
- id: if
@@ -140,139 +151,328 @@ tasks:
For more details, check out the [If Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.if).
-### ForEach
+### Loop
-This task executes a group of tasks for each value in the list.
+The `Loop` task iterates over a set of values and runs child tasks for each item in an isolated sub-execution with its own context.
-In the following example, the variable is static, but it could also be generated from a previous task output, starting any number of subtasks.
+`values` accepts a list, a JSON array string, a map, or an ION file URI. When `values` is a URI, Kestra performs one iteration per line of the file.
```yaml
-id: foreach_example
+id: loop-basic
namespace: company.team
tasks:
- - id: for_each
- type: io.kestra.plugin.core.flow.ForEach
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
values: ["value 1", "value 2", "value 3"]
tasks:
- - id: before_if
- type: io.kestra.plugin.core.debug.Return
- format: "Before if {{ taskrun.value }}"
- - id: if
- type: io.kestra.plugin.core.flow.If
- condition: '{{ taskrun.value == "value 2" }}'
- then:
- - id: after_if
- type: io.kestra.plugin.core.debug.Return
- format: "After if {{ parent.taskrun.value }}"
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "index={{ item.index }} value={{ item.value }}"
```
-In this execution, you can access:
+Inside each iteration, use the `item` variable to access the iteration context:
+
+| Expression | Description |
+|---|---|
+| `{{ item.index }}` | Zero-based iteration index |
+| `{{ item.value }}` | Current iteration value |
+| `{{ item.key }}` | Current map key when `values` is a map; not set for list or URI values |
+| `{{ item.parent.index }}` | Index of the nearest enclosing loop (nested loops only) |
+| `{{ item.parent.value }}` | Value of the nearest enclosing loop (nested loops only) |
+| `{{ item.parents[n].value }}` | Value of the nth ancestor loop, counting from innermost |
+
+For more details on `item`, see [loop iteration context](../../../expressions/01.context/index.mdx#loop-iteration-context) in the expressions reference.
-- The iteration value i.e., the index of a loop (the loop index starts at 0) using the syntax `{{ taskrun.iteration }}`
-- The output of a sibling task using the syntax `{{ outputs.sibling[taskrun.value].value }}`
+#### Iterating over objects
-This example shows how to run tasks in parallel for each value in the list. All child tasks of the parallel task run in parallel. However, due to the `concurrencyLimit` property set to 2, only two parallel task groups run at any given time.
+When `values` contains a list of objects, each `item.value` is a JSON string. Use `fromJson(item.value).field` to read fields — `item.value.field` does not work.
```yaml
id: parallel_tasks_example
namespace: company.team
tasks:
- - id: for_each
- type: io.kestra.plugin.core.flow.ForEach
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values:
+ - { id: 101, email: "a@example.com" }
+ - { id: 102, email: "b@example.com" }
+ fetchType: AUTO
+ outputs:
+ - id: user_id
+ type: INT
+ value: "{{ fromJson(item.value).id }}"
+ - id: email
+ type: STRING
+ value: "{{ fromJson(item.value).email }}"
+ tasks:
+ - id: log_user
+ type: io.kestra.plugin.core.log.Log
+ message: "User {{ fromJson(item.value).id }} -> {{ fromJson(item.value).email }}"
+```
+
+#### Concurrent execution
+
+By default (`concurrencyLimit: 1`), iterations run one at a time in order. Set `concurrencyLimit` to a higher value to run multiple iterations simultaneously, or `0` for no limit.
+
+```yaml
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
values: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
- concurrencyLimit: 2
+ concurrencyLimit: 0
tasks:
- id: parallel
type: io.kestra.plugin.core.flow.Parallel
tasks:
- - id: log
- type: io.kestra.plugin.core.log.Log
- message: Processing {{ parent.taskrun.value }}
- - id: shell
- type: io.kestra.plugin.scripts.shell.Commands
- commands:
- - sleep {{ parent.taskrun.value }}
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Processing {{ item.value }}"
+ - id: shell
+ type: io.kestra.plugin.scripts.shell.Commands
+ commands:
+ - "echo done {{ item.value }}"
```
-For more information on handling outputs generated from `ForEach`, check out the [dedicated loop how-to guide](../../../15.how-to-guides/loop/index.md) and the [Best Practices for ForEach and ForEachItem](../../../14.best-practices/11.foreach-and-foreachitem/index.md) guide, including how to access [sibling task outputs correctly](../../../14.best-practices/11.foreach-and-foreachitem/index.md#example-use-sibling-outputs-correctly-inside-foreach) inside the loop.
+#### Failure propagation
-For processing items, or forwarding processing to a subflow, [ForEachItem](#foreachitem) is better suited.
+By default (`transmitFailed: true`), a failed iteration causes the Loop task itself to fail. Set `transmitFailed: false` to let the loop continue even when individual iterations fail.
-:::alert{type="info"}
-For more details, refer to the [ForEach Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.foreach).
-:::
+```yaml
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["ok", "fail", "ok"]
+ transmitFailed: false
+ tasks:
+ - id: maybe_fail
+ type: io.kestra.plugin.core.flow.If
+ condition: '{{ item.value == "fail" }}'
+ then:
+ - id: do_fail
+ type: io.kestra.plugin.core.execution.Fail
+ else:
+ - id: success
+ type: io.kestra.plugin.core.log.Log
+ message: "OK: {{ item.value }}"
+```
-### ForEachItem
+#### Error handling per iteration
-This task iterates over a list of items and runs a subflow for each item, or for each batch of items.
+Use `errors:` to run tasks when an iteration fails, and `finally:` to run a block once after all iterations complete regardless of outcome. `errors:` runs within the failing iteration regardless of `transmitFailed` — with `transmitFailed: true` (default), the loop stops after the failing iteration completes; with `transmitFailed: false`, the loop continues to subsequent iterations. `finally:` always runs regardless.
```yaml
- - id: each
- type: io.kestra.plugin.core.flow.ForEachItem
- items: "{{ inputs.file }}" # could be also an output variable {{ outputs.extract.uri }}
- inputs:
- file: "{{ taskrun.items }}" # items of the batch
- batch:
- rows: 4
- namespace: company.team
- flowId: subflow
- revision: 1 # optional (default: latest)
- wait: true # wait for the subflow execution
- transmitFailed: true # fail the task run if the subflow execution fails
- labels: # optional labels to pass to the subflow to be executed
- key: value
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values:
+ - ok
+ - boom
+ - ok
+ transmitFailed: false
+ tasks:
+ - id: maybe_fail
+ type: io.kestra.plugin.scripts.shell.Commands
+ commands:
+ - |
+ if [ "{{ item.value }}" = "boom" ]; then
+ echo "failing on {{ item.value }}" >&2
+ exit 1
+ fi
+ echo "ok {{ item.value }}"
+ errors:
+ - id: handle_error
+ type: io.kestra.plugin.core.log.Log
+ message: "Iteration {{ item.index }} ({{ item.value }}) failed"
+ finally:
+ - id: cleanup
+ type: io.kestra.plugin.core.log.Log
+ message: "Loop completed (with or without failures)"
```
-This executes the subflow `company.team.subflow` for each batch of items.
-To pass the batch of items to a subflow, you can use inputs. The example above uses an input of `FILE` type called `file` that takes the URI of an internal storage file containing the batch of items.
+#### Nested loops
-The next example shows you how to access the outputs from each subflow executed. The ForEachItem automatically merges the URIs of the outputs from each subflow into a single file. The URI of this file is available through the `subflowOutputs` output.
+Loops can be nested to any depth. Because `item` is bound to the loop execution rather than individual task runs, flowable tasks nested inside a loop can access `item` directly without a `parent.` prefix.
+
+`item.parents[0]` is the immediate parent loop (same as `item.parent`), `item.parents[1]` is the next outer loop, and so on.
```yaml
-id: for_each_item
+tasks:
+ - id: outer
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["bucket1", "bucket2"]
+ tasks:
+ - id: middle
+ type: io.kestra.plugin.core.flow.Loop
+ values: [2025, 2026]
+ tasks:
+ - id: inner
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["Jan", "Feb", "Mar"]
+ tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "bucket={{ item.parents[1].value }} year={{ item.parent.value }} month={{ item.value }}"
+```
+
+#### Loop outputs
+
+By default, task outputs produced inside a loop are not accessible to tasks that run after the loop. Use the `outputs` property on the Loop task to explicitly declare which values to expose.
+
+```yaml
+id: loop-outputs
namespace: company.team
tasks:
- - id: generate
- type: io.kestra.plugin.scripts.shell.Script
- script: |
- for i in $(seq 1 10); do echo "$i" >> data; done
- outputFiles:
- - data
-
- - id: for_each_item
- type: io.kestra.plugin.core.flow.ForEachItem
- items: "{{ outputs.generate.outputFiles.data }}"
- batch:
- rows: 4
- wait: true
- flowId: my_subflow
- namespace: company.team
- inputs:
- value: "{{ taskrun.items }}"
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["a", "b", "c"]
+ fetchType: AUTO
+ outputs:
+ - id: result
+ type: STRING
+ value: "{{ outputs.process.value }}"
+ tasks:
+ - id: process
+ type: io.kestra.plugin.core.debug.Return
+ format: "processed {{ item.value }}"
- - id: for_each_outputs
+ - id: summary
type: io.kestra.plugin.core.log.Log
- message: "{{ outputs.forEachItem_merge.subflowOutputs }}" # Log the URI of the file containing the URIs of the outputs from each subflow
+ message: "Loop ran {{ outputs.loop.iterationCount }} iterations"
```
-:::alert{type="info"}
-For more details, refer to the [ForEachItem Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.foreachitem).
-:::
+The loop also exposes monitoring outputs regardless of whether `outputs` is declared:
+
+| Output | Description |
+|---|---|
+| `iterationCount` | Total number of iterations |
+| `runningIterations` | Iterations still in progress |
+| `terminatedIterations` | Iterations that have finished |
+
+The `fetchType` property controls how iteration outputs are collected: `FETCH` returns them inline in the execution context (suitable for small iteration counts), `STORE` writes them to internal storage and exposes a URI (preferred for large iteration counts), and `AUTO` (the default) chooses based on whether `values` is a URI.
+
+#### Processing large files
+
+When `values` is a list of URIs from a [`Split`](/plugins/core/storage/io.kestra.plugin.core.storage.split) task, each iteration receives one chunk URI as `item.value`. Combine `Split`, `Loop`, and `Concat` to implement a map-reduce pattern: split a large file into chunks, process each chunk in parallel, then merge the per-chunk outputs into a single result.
-#### `ForEach` vs `ForEachItem`
+Passing `values: "{{ outputs.split.uris }}"` where `outputs.split.uris` is a **list** is different from passing a single file URI. When `values` is a list, each `item.value` is one element of that list. When `values` is a single URI string, Kestra iterates line-by-line through the file.
-Both `ForEach` and `ForEachItem` are similar, but there are specific use cases that suit one over the other:
-- `ForEach` generates a lot of [Task Runs](../02.taskruns/index.md) which can impact performance.
-- `ForEachItem` generates separate executions using [Subflows](../../10.subflows/index.md) for the group of tasks. This scales better for larger datasets.
+```yaml
+id: map-reduce
+namespace: company.team
-Read more about performance optimization in our [best practices guides](../../../14.best-practices/0.flows/index.md#tasks-in-the-same-execution).
+tasks:
+ - id: download
+ type: io.kestra.plugin.core.http.Download
+ uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
+
+ - id: to_ion
+ type: io.kestra.plugin.serdes.csv.CsvToIon
+ from: "{{ outputs.download.uri }}"
+
+ - id: split
+ type: io.kestra.plugin.core.storage.Split
+ from: "{{ outputs.to_ion.uri }}"
+ rows: 25
+
+ - id: per_chunk
+ type: io.kestra.plugin.core.flow.Loop
+ values: "{{ outputs.split.uris }}"
+ concurrencyLimit: 4
+ fetchType: FETCH
+ outputs:
+ - id: data
+ type: STRING
+ value: "{{ outputs.aggregate.uri }}"
+ tasks:
+ - id: aggregate
+ type: io.kestra.plugin.transform.Aggregate
+ from: "{{ item.value }}"
+ outputType: STORE
+ groupBy: [customer_email]
+ aggregates:
+ orders:
+ expr: count()
+ type: INT
+ revenue:
+ expr: sum(todecimal(total))
+ type: DECIMAL
+
+ - id: concat
+ type: io.kestra.plugin.core.storage.Concat
+ files: "{{ loopOutputs(outputs.per_chunk.outputs, 'data') }}"
+ extension: .ion
+
+ - id: reduce
+ type: io.kestra.plugin.transform.Aggregate
+ from: "{{ outputs.concat.uri }}"
+ outputType: STORE
+ groupBy: [customer_email]
+ aggregates:
+ orders:
+ expr: sum(orders)
+ type: INT
+ revenue:
+ expr: sum(revenue)
+ type: DECIMAL
+```
+
+Use `fetchType: FETCH` to collect per-iteration output URIs inline, then pass them to `Concat` via `loopOutputs(outputs.per_chunk.outputs, 'data')`.
+
+#### Accessing loop outputs in a script task
+
+The following example runs a Python task inside a loop to compute a value, then reads the collected results in a subsequent Python task using the monitoring output and the Kestra Python SDK.
+
+```yaml
+id: loop-python-outputs
+namespace: company.team
+
+tasks:
+ - id: process_items
+ type: io.kestra.plugin.core.flow.Loop
+ values: [1, 2, 3, 4, 5]
+ outputs:
+ - id: squared
+ type: INT
+ value: "{{ outputs.compute.vars.result }}"
+ tasks:
+ - id: compute
+ type: io.kestra.plugin.scripts.python.Script
+ dependencies:
+ - kestra
+ script: |
+ from kestra import Kestra
+ n = {{ item.value }}
+ Kestra.outputs({"result": n * n})
+
+ - id: analyze
+ type: io.kestra.plugin.scripts.python.Script
+ dependencies:
+ - kestra
+ script: |
+ from kestra import Kestra
+
+ iteration_count = {{ outputs.process_items.iterationCount }}
+
+ # outputs.process_items.outputs is a list of iteration results:
+ # [{"item": {"value": "1", "iteration": 1}, "outputs": {"squared": 1}}, ...]
+ all_outputs = {{ outputs.process_items.outputs | toJson }}
+
+ squared_values = [iteration["outputs"]["squared"] for iteration in all_outputs]
+
+ print(f"Processed {iteration_count} items")
+ print(f"Squared values: {squared_values}")
+ print(f"Sum of squares: {sum(squared_values)}")
+
+ Kestra.outputs({"total": sum(squared_values)})
+```
+
+`outputs.process_items.iterationCount` is always available after the loop finishes. `outputs.process_items.outputs` is a list of iteration results — each entry contains an `item` object (with `value`, `iteration`, and `key`) and an `outputs` map of the declared output values. To access the first iteration's output in an expression, use `outputs.process_items.outputs[0].outputs.squared`. To extract one output across all iterations as a list, use the `loopOutputs()` function: `{{ loopOutputs(outputs.process_items.outputs, 'squared') }}`.
+
+For more details, see the [Loop task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.loop).
-
-
-
### LoopUntil
@@ -285,6 +485,9 @@ Key properties:
- `condition` — expression evaluated after each iteration; has access to the child task outputs from the most recent run (e.g. `{{ outputs.checkStatus.code }}`).
- `tasks` — the list of child tasks to run before re-evaluating the condition.
- `checkFrequency` — optional guardrails that define `interval`, `maxIterations`, and/or `maxDuration` between repeats. (See the [LoopUntil migration note](../../../11.migration-guide/v0.23.0/loop-until-defaults/index.md) for default values.)
+- `failOnMaxReached` — if `true`, the task fails when `maxIterations` or `maxDuration` is reached without the condition becoming true. Default: `false` (the task succeeds when limits are reached).
+
+After the loop completes, `outputs..iterationCount` holds the total number of iterations (1-based). Use this in downstream tasks to report how many attempts were needed.
Example: poll an API until it returns HTTP 200, checking every 30 seconds and stopping after 50 attempts if it never succeeds.
@@ -310,9 +513,7 @@ For more details, refer to the [LoopUntil Task documentation](/plugins/core/flow
### AllowFailure
-This task allows child tasks to fail.
-
-If any child task fails:
+`AllowFailure` lets child tasks fail without failing the overall execution. If any child task fails:
- The `AllowFailure` task is marked with status `WARNING`.
- All child tasks inside `AllowFailure` stop immediately.
- The execution continues for all other tasks.
@@ -343,15 +544,11 @@ tasks:
format: "{{ task.id }} > {{ taskrun.startDate }}"
```
-:::alert{type="info"}
For more details, refer to the [AllowFailure Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.allowfailure).
-:::
### Fail
-This task fails the flow; it can be used with or without conditions.
-
-Without conditions, it can be used, for example, to fail on some switch value.
+`Fail` explicitly fails the execution, optionally guarded by a `condition` expression. Without a condition, it is useful inside a `Switch` branch to reject invalid cases.
```yaml
id: fail_on_switch
@@ -384,7 +581,7 @@ tasks:
message: default
```
-With conditions, it can be used, for example, to validate inputs.
+With a condition, it can validate inputs before any work begins.
```yaml
id: fail_on_condition
@@ -411,37 +608,50 @@ For more information, refer to the [Fail Task documentation](/plugins/core/execu
### Subflow
-This task triggers another flow. This enables you to decouple the first flow from the second and monitor each flow individually.
+`Subflow` starts a child execution of another flow, letting you decompose complex workflows, share reusable logic across namespaces, and monitor each execution independently.
+
+Required properties are `namespace` and `flowId`. Pass values to the subflow via `inputs` — those inputs must be declared in the subflow's definition.
+
+By default (`wait: true`), the parent execution waits for the subflow to finish before continuing. Set `wait: false` to fire-and-forget; the parent moves on immediately without tracking the child's result.
+
+When `wait: true`, the parent captures the subflow's final state and outputs:
-You can pass flow outputs as inputs to the triggered subflow (those must be declared in the subflow).
+- `{{ outputs.subflow_task.executionId }}` — the child execution ID
+- `{{ outputs.subflow_task.state }}` — the child's final state (`SUCCESS`, `FAILED`, etc.)
+- `{{ outputs.subflow_task.outputs.some_key }}` — a value from the subflow's declared outputs
+
+`transmitFailed: true` (the default when `wait: true`) causes the parent to fail if the subflow fails. Set it to `false` to continue the parent regardless of the child's outcome.
```yaml
-id: subflow_example
+id: parent_flow
namespace: company.team
-inputs:
- - id: my_file
- type: FILE
-
tasks:
- - id: subflow
+ - id: call_subflow
type: io.kestra.plugin.core.flow.Subflow
namespace: company.team
flowId: my_subflow
inputs:
- file: "{{ inputs.my_file }}"
- store: 12
+ user: "{{ inputs.username }}"
+ wait: true
+ transmitFailed: true
+
+ - id: use_output
+ type: io.kestra.plugin.core.log.Log
+ message: "Subflow returned: {{ outputs.call_subflow.outputs.result }}"
```
+Use `revision` to pin the subflow to a specific version. Use `inheritLabels: true` to forward the parent's execution labels to the child. Use `scheduleDate` to defer the child execution to a future time instead of starting it immediately.
+
For more details, refer to the [Subflow Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.subflow).
### WorkingDirectory
-By default, Kestra launches each task in a new working directory, possibly on different workers if multiple ones exist.
-
-The example below runs all tasks nested under the `WorkingDirectory` task sequentially in the same directory, allowing downstream tasks to reuse output files from previous ones. To share a working directory, all tasks nested under the `WorkingDirectory` task are launched on the same worker.
+`WorkingDirectory` runs all nested tasks sequentially in the same directory on the same worker, so downstream tasks can read files written by earlier ones. It is useful for compute-intensive file system operations.
-This task can be particularly useful for compute-intensive file system operations.
+:::alert{type="warning"}
+Only runnable tasks are accepted as children of a `WorkingDirectory`. Nesting a flowable task such as `Parallel` or `Loop` inside a `WorkingDirectory` is rejected at save time.
+:::
```yaml
id: working_dir_flow
@@ -512,110 +722,104 @@ tasks:
- cat dir1/file1.txt
```
-:::alert{type="info"}
-[WorkingDirectory Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.workingdirectory)
-:::
+For more details, refer to the [WorkingDirectory Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.workingdirectory).
### Pause
-Kestra flows run until all tasks complete, but sometimes you need to:
-- Add a manual validation before continuing the execution
-- Wait for some duration before continuing the execution
-
-For this, you can use the Pause task.
+`Pause` halts the execution until it is manually resumed or a timeout expires. Tasks declared after the `Pause` in the flow run once the execution resumes.
-In the following example, the `validation` task pauses until it is manually resumed, while the `wait` task pauses for 5 minutes.
+To resume manually, open the **Gantt** tab on the execution, click the Pause task, select **Change status**, and choose **Mark as RUNNING**. You can also resume via the API: `POST /api/v1/executions/{executionId}/resume`.
```yaml
-id: pause
+id: pause_for_approval
namespace: company.team
tasks:
- - id: validation
+ - id: before
+ type: io.kestra.plugin.core.log.Log
+ message: "Waiting for manual approval"
+
+ - id: approval
type: io.kestra.plugin.core.flow.Pause
- tasks:
- - id: ok
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.core.runner.Process
- commands:
- - 'echo "started after manual validation"'
+ - id: after
+ type: io.kestra.plugin.core.log.Log
+ message: "Approved — continuing execution"
+```
+
+Use `pauseDuration` to resume automatically after a fixed interval (ISO 8601 duration format). The `behavior` property controls what happens when that duration expires without a manual resume: `RESUME` continues (the default), `WARN` continues with a warning, `CANCEL` cancels the execution, or `FAIL` fails the task.
+
+```yaml
+tasks:
- id: wait
type: io.kestra.plugin.core.flow.Pause
- delay: PT5M
- tasks:
- - id: waited
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.core.runner.Process
- commands:
- - 'echo "start after 5 minutes"'
+ pauseDuration: PT5M
+ behavior: WARN
```
-:::alert{type="info"}
-A Pause task without delay waits indefinitely until the task state is changed to **Running**.
-For this: go to the **Gantt** tab of the **Execution** page, click on the task, select **Change status** on the contextual menu, and select **Mark as RUNNING** on the form. This makes the task run until its end. For more details, refer to the [Pause Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.pause).
-:::
+Use `onResume` to collect structured input from the person approving the pause. Downstream tasks access those values via `{{ outputs..onResume. }}`.
+
+```yaml
+tasks:
+ - id: wait_for_approval
+ type: io.kestra.plugin.core.flow.Pause
+ onResume:
+ - id: approved
+ description: Approve or reject
+ type: BOOL
+ defaults: true
+ - id: reason
+ description: Reason for decision
+ type: STRING
+
+ - id: log_decision
+ type: io.kestra.plugin.core.log.Log
+ message: "Decision: {{ outputs.wait_for_approval.onResume.approved }} — {{ outputs.wait_for_approval.onResume.reason }}"
+```
+
+For more details, refer to the [Pause Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.pause).
### DAG
-This task allows defining dependencies between tasks by creating a directed acyclic graph (DAG). Instead of an explicit DAG structure, this task defines dependencies for each task using the `dependsOn` property. This way, you can set dependencies more implicitly for each task, and Kestra figures out the overall flow structure.
+`DAG` lets you declare tasks and their `dependsOn` links; Kestra derives execution order and runs tasks in parallel as their dependencies are satisfied. Use it when your dependency graph cannot be expressed as a flat sequence or a single `Parallel` block — for example, when task C depends on both A and B, but A and B are independent.
+
+Tasks with no `dependsOn` start immediately. The `concurrent` property caps how many tasks run at once (`0` = no limit, the default).
+
+Note: UI no-code forms are not available for DAG tasks — configure them in YAML or the code editor.
```yaml
-id: dag
+id: dag_flow
namespace: company.team
+
tasks:
- id: dag
- description: "my task"
type: io.kestra.plugin.core.flow.Dag
tasks:
- task:
id: task1
type: io.kestra.plugin.core.log.Log
- message: I'm the task 1
+ message: task 1
- task:
id: task2
type: io.kestra.plugin.core.log.Log
- message: I'm the task 2
+ message: task 2 (depends on task1)
dependsOn:
- task1
- task:
id: task3
type: io.kestra.plugin.core.log.Log
- message: I'm the task 3
+ message: task 3 (depends on task1)
dependsOn:
- task1
- task:
id: task4
type: io.kestra.plugin.core.log.Log
- message: I'm the task 4
+ message: task 4 (depends on task2 and task3)
dependsOn:
- task2
- - task:
- id: task5
- type: io.kestra.plugin.core.log.Log
- message: I'm the task 5
- dependsOn:
- - task4
- task3
```
-For more details, refer to the [Dag Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.dag).
-
-### Template (deprecated)
-
-Templates are lists of tasks that can be shared between flows. You can define a template and call it from other flows, allowing them to share a list of tasks and keep these tasks updated without changing your flow.
-
-The following example uses the Template task to use a template.
-
-```yaml
-id: template
-namespace: company.team
+In this example, `task2` and `task3` run in parallel after `task1` completes, and `task4` starts once both finish.
-tasks:
- - id: template
- type: io.kestra.plugin.core.flow.Template
- namespace: company.team
- templateId: template
-```
+For more details, refer to the [DAG Task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.dag).
diff --git a/src/contents/docs/05.workflow-components/01.tasks/01.runnable-tasks/index.md b/src/contents/docs/05.workflow-components/01.tasks/01.runnable-tasks/index.md
index a7b3df417da..56eb189521a 100644
--- a/src/contents/docs/05.workflow-components/01.tasks/01.runnable-tasks/index.md
+++ b/src/contents/docs/05.workflow-components/01.tasks/01.runnable-tasks/index.md
@@ -6,17 +6,11 @@ sidebarTitle: Runnable Tasks
icon: /src/contents/docs/icons/flow.svg
---
-Data processing tasks handled by the workers.
+Runnable tasks perform the actual work in a flow — file system operations, API calls, database queries, and scripts — executed by distributed workers.
-## Execute work with runnable tasks
+Each task requires an `id` and a `type`, defined by its fully qualified Java class name. Refer to each task’s plugin documentation for a full list of available properties.
-Runnable tasks handle data processing, such as file system operations, API calls, and database queries. They can be compute-intensive and are executed by workers.
-
-Each task requires an identifier (`id`) and a type, defined by its Java Fully Qualified Class Name (FQCN).
-
-Tasks include properties specific to their type. Refer to each task’s documentation for a full list of available properties.
-
-Most tasks are runnable, except for [Flowable tasks](../00.flowable-tasks/index.md), which control orchestration logic.
+Most tasks are runnable. The exception is [Flowable tasks](../00.flowable-tasks/index.md), which control orchestration logic and run on the executor rather than workers.
By default, Kestra includes only a few runnable tasks. Many more are available as [plugins](/plugins), and the default Docker image comes preloaded with several of them.
diff --git a/src/contents/docs/05.workflow-components/01.tasks/02.taskruns/index.md b/src/contents/docs/05.workflow-components/01.tasks/02.taskruns/index.md
index ce6dbb36caa..ff2a650f081 100644
--- a/src/contents/docs/05.workflow-components/01.tasks/02.taskruns/index.md
+++ b/src/contents/docs/05.workflow-components/01.tasks/02.taskruns/index.md
@@ -9,15 +9,6 @@ docId: taskruns
A task run is a single execution of an individual task within an [Execution](../../03.execution/index.md), where an execution represents a run of the entire flow. One execution can therefore contain multiple task runs.
-## Understand task runs
-
-Each task run includes associated data such as:
-
-- Execution ID
-- State
-- Start Date
-- End Date
-
## Attempts
A task run can include one or more attempts. Most have only a single attempt, but you can configure [retries](../../12.retries/index.md) if needed.
@@ -40,9 +31,7 @@ Similar to executions, task runs can exist in different states.
| `KILLING` | A kill command was issued and the system is terminating the task run. |
| `KILLED` | An execution or task run was killed (upon request), and no more tasks will run. |
-:::alert{type="info"}
-For a detailed overview of how each task run transition through different states, see the [States](../../17.states/index.md#how-are-task-run-states-different-from-execution-states) page.
-:::
+For a detailed overview of how task runs transition through states, see the [States](../../17.states/index.md#how-are-task-run-states-different-from-execution-states) page.
## Expression
@@ -70,123 +59,33 @@ The logs show the following:
}
```
-## Task run values
+## Loop iteration context
-Some [Flowable tasks](../00.flowable-tasks/index.md), such as [ForEach](../00.flowable-tasks/index.md) and [ForEachItem](../00.flowable-tasks/index.md#foreachitem), group tasks together. You can use `{{ taskrun.value }}` to access the value of a specific task run.
-
-In the example below, `foreach` iterates twice over the values `[1, 2]`:
+Inside a [Loop](../00.flowable-tasks/index.md#loop) task, each iteration runs as an isolated sub-execution. Use `{{ item.value }}` and `{{ item.index }}` to access the current iteration value and zero-based index from any task inside that sub-execution, including tasks nested inside `If`, `Parallel`, or other flowable tasks.
```yaml
id: loop
namespace: company.team
tasks:
- - id: foreach
- type: io.kestra.plugin.core.flow.ForEach
- values: [1, 2]
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: [1, 2, 3]
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
- message:
- - "{{ taskrun }}"
- - "{{ taskrun.value }}"
- - "{{ taskrun.id }}"
- - "{{ taskrun.startDate }}"
- - "{{ taskrun.attemptsCount }}"
- - "{{ taskrun.parentId }}"
- - "{{ taskrun.iteration }}"
+ message: |
+ value={{ item.value }}
+ index={{ item.index }}
+ taskrun.id={{ taskrun.id }}
+ taskrun.startDate={{ taskrun.startDate }}
+ taskrun.attemptsCount={{ taskrun.attemptsCount }}
+ taskrun.parentId={{ taskrun.parentId }}
```
-This produces two separate log entries, one with `1` and the other with `2`.
-
-### Parent task run values
-You can also use the `{{ parent.taskrun.value }}` expression to access a task run value from a parent task within nested flowable child tasks:
-
-```yaml
-id: loop
-namespace: company.team
-
-tasks:
- - id: foreach
- type: io.kestra.plugin.core.flow.ForEach
- values: [1, 2]
- tasks:
- - id: log
- type: io.kestra.plugin.core.log.Log
- message: "{{ taskrun.value }}"
- - id: if
- type: io.kestra.plugin.core.flow.If
- condition: "{{ true }}"
- then:
- - id: log_parent
- type: io.kestra.plugin.core.log.Log
- message: "{{ parent.taskrun.value }}"
-```
-
-This iterates through the `log` and `if` tasks twice as there are two items in `values` property. The `log_parent` task logs the parent task run value as `1` and then `2`.
-
-### Parent vs. parents in nested Flowable tasks
-
-With nested [Flowable tasks](../00.flowable-tasks/index.md), only the immediate parent is available through `taskrun.value`. To access a parent task higher up the tree, you can use the `parent` and the `parents` expressions.
-
-The following flow shows a more complex example with nested flowable parent tasks:
-
-```yaml
-id: each_switch
-namespace: company.team
-
-tasks:
- - id: simple
- type: io.kestra.plugin.core.log.Log
- message:
- - "{{ task.id }}"
- - "{{ taskrun.startDate }}"
-
- - id: hierarchy_1
- type: io.kestra.plugin.core.flow.ForEach
- values: ["caseA", "caseB"]
- tasks:
- - id: hierarchy_2
- type: io.kestra.plugin.core.flow.Switch
- value: "{{ taskrun.value }}"
- cases:
- caseA:
- - id: hierarchy_2_a
- type: io.kestra.plugin.core.debug.Return
- format: "{{ task.id }}"
- caseB:
- - id: hierarchy_2_b_first
- type: io.kestra.plugin.core.debug.Return
- format: "{{ task.id }}"
-
- - id: hierarchy_2_b_second
- type: io.kestra.plugin.core.flow.ForEach
- values: ["case1", "case2"]
- tasks:
- - id: switch
- type: io.kestra.plugin.core.flow.Switch
- value: "{{ taskrun.value }}"
- cases:
- case1:
- - id: switch_1
- type: io.kestra.plugin.core.log.Log
- message:
- - "{{ parents[0].taskrun.value }}"
- - "{{ parents[1].taskrun.value }}"
- case2:
- - id: switch_2
- type: io.kestra.plugin.core.log.Log
- message:
- - "{{ parents[0].taskrun.value }}"
- - "{{ parents[1].taskrun.value }}"
- - id: simple_again
- type: io.kestra.plugin.core.log.Log
- message:
- - "{{ task.id }}"
- - "{{ taskrun.startDate }}"
-```
+For nested loops, `{{ item.parent.value }}` accesses the immediate enclosing loop's value, and `{{ item.parents[n].value }}` accesses deeper ancestors (`[0]` = immediate parent, `[1]` = grandparent, and so on).
-The `parent` variable gives direct access to the first parent, while the `parents[INDEX]` gives you access to the parent higher up the tree.
+See [Loop iteration context](../../../expressions/01.context/index.mdx#loop-iteration-context) in the expressions reference for the full `item` variable table.
:::collapse{title="Task Run JSON Object Example"}
```json
@@ -204,7 +103,7 @@ The `parent` variable gives direct access to the first parent, while the `parent
{
"name": "length",
"tags": {
- "format": "{{task.id}} > {{taskrun.value}} ⬅ {{taskrun.startDate}}"
+ "format": "{{task.id}} > {{item.value}} ⬅ {{taskrun.startDate}}"
},
"value": 45.0,
"type": "counter"
@@ -212,7 +111,7 @@ The `parent` variable gives direct access to the first parent, while the `parent
{
"name": "duration",
"tags": {
- "format": "{{task.id}} > {{taskrun.value}} ⬅ {{taskrun.startDate}}"
+ "format": "{{task.id}} > {{item.value}} ⬅ {{taskrun.startDate}}"
},
"type": "timer",
"value": "PT0.007213673S"
diff --git a/src/contents/docs/05.workflow-components/01.tasks/index.mdx b/src/contents/docs/05.workflow-components/01.tasks/index.mdx
index 450b6ee704b..5f6a00ebcda 100644
--- a/src/contents/docs/05.workflow-components/01.tasks/index.mdx
+++ b/src/contents/docs/05.workflow-components/01.tasks/index.mdx
@@ -8,9 +8,7 @@ sidebarTitle: Tasks
import ChildCard from "~/components/docs/ChildCard.astro"
-Tasks are the steps within a flow.
-
-They represent discrete actions, capable of processing inputs and variables and producing outputs for downstream consumption by end users and other tasks.
+Tasks are the discrete steps within a flow that process inputs, run logic, and produce outputs.
-
-
-You can think of a namespace as a **folder for your flows**. Similar to folders on your file system, namespaces can be used to organize flows into logical categories. Similar to filesystems, namespaces can be indefinitely nested.
-
-If you're looking to completely isolate environments with their own resources on the same Kestra instance, you should consider [Tenants](../../07.enterprise/02.governance/tenants/index.md), part of the [Enterprise Edition](../../07.enterprise/index.mdx).
-
-## Hierarchical structure with nested namespaces
-
-Using the dot `.` symbol, you can add a hierarchical structure to your namespaces which allows you to logically separate environments, projects, teams, and departments. This way, your product, engineering, marketing, finance, and data teams can all use the same Kestra instance, all while keeping their flows organized and separated. Various stakeholders can have their own child namespaces that belong to a parent namespace grouping them by environment, project, or team.
-
-## Namespace name
-A namespace name can be built from alphanumerical characters, optionally separated by `.`. The hierarchy depth for namespaces is unlimited. Here are some examples of namespaces:
- `project_one`
- `company.project_two`
- `company.team.project_three`
-## Using namespaces to organize flows and files
+To fully isolate environments with their own resources, see [Tenants](../../07.enterprise/02.governance/tenants/index.md) (Enterprise Edition).
+
+
+
+
+
+## Assigning a namespace
-When you create a flow, you can assign a namespace to it:
+Every flow declares its namespace directly in the flow definition:
```yaml
id: hello_world
@@ -43,45 +35,21 @@ tasks:
```
:::alert{type="warning"}
-**Note:** Once you've saved your flow, you won't be able to change its namespace. You'll need to make a new flow in order to change the namespace.
+Once you save a flow, its namespace cannot be changed. Create a new flow to move it to a different namespace.
:::
-Below, the flow is assigned to the `company.team` namespace. This assignment of a namespace to a flow already provides a benefit of improved organization and filtering:
-
-
-
-Additionally, you can organize your code on a namespace-level using the embedded Code editor and [Namespace Files](../../06.concepts/02.namespace-files/index.md), with the option to [sync those files from Git](../../version-control-cicd/04.git/index.md):
-
-
-
-## Namespace tab
-
-In the **Namespaces** tab, you can see all the namespaces associated with the different flows in Kestra. You can also list and filter namespaces from the command line using [`kestractl namespaces list`](../../kestra-cli/kestractl/index.md).
-
-
-
-
-
-You can open the details about any namespace by clicking on the name or details button to the right of that namespace.
-
-
-
-When you select the details button for any namespace, the namespace overview page opens which details the executions of flows in that namespace.
-
-
-
-On the top of this page, you have different tabs:
+
-1. **Overview:** the default landing page of the Namespace. This page contains the dashboards and summary about the executions of different flows in this namespace.
+## Namespace Files
-2. **Executions:** View and manage all the executions details.
+Each namespace has an embedded code editor for managing scripts, configuration, and other files shared across flows in that namespace. You can [sync these files from Git](../../version-control-cicd/04.git/index.md) or manage them through the [Namespace Files](../../06.concepts/02.namespace-files/index.md) tab.
-3. **Flows:** View all flows in the namespace with execution details and statistics. Select the details button to navigate to a specific flow's page.
+
-4. **Dependencies:** View flow dependencies through subflows or flow triggers.
+## Namespace overview
-5. **KV Store:** Manage key-value pairs for this namespace. See [KV Store](../../06.concepts/05.kv-store/index.md) for details.
+Open any namespace to see its execution dashboards, flows, dependencies, and [KV Store](../../06.concepts/05.kv-store/index.md). You can also list and filter namespaces from the command line using [`kestractl namespaces list`](../../kestra-cli/kestractl/index.md).
-6. **Files:** To manage, view and modify all the Namespace files.
+Tabs marked with a lock icon — Edit, Secrets, Credentials, Assets, Variables, Policies, Reusable Inputs, Revisions, and Audit Logs — are available in Kestra EE. See the [Enterprise Edition documentation](../../07.enterprise/index.mdx) for details.
-The other tabs: Edit, Variables, Plugin Defaults, Secrets, and Audit Logs are only available for Kestra EE. More details about them can be found in our [Enterprise Edition documentation](../../07.enterprise/index.mdx).
+
diff --git a/src/contents/docs/05.workflow-components/02.namespace/namespace-files-2-0.png b/src/contents/docs/05.workflow-components/02.namespace/namespace-files-2-0.png
new file mode 100644
index 00000000000..9df9be6306b
Binary files /dev/null and b/src/contents/docs/05.workflow-components/02.namespace/namespace-files-2-0.png differ
diff --git a/src/contents/docs/05.workflow-components/02.namespace/namespace-flows-2-0.png b/src/contents/docs/05.workflow-components/02.namespace/namespace-flows-2-0.png
new file mode 100644
index 00000000000..c56c1a6af7f
Binary files /dev/null and b/src/contents/docs/05.workflow-components/02.namespace/namespace-flows-2-0.png differ
diff --git a/src/contents/docs/05.workflow-components/02.namespace/namespace-overview-2-0.png b/src/contents/docs/05.workflow-components/02.namespace/namespace-overview-2-0.png
new file mode 100644
index 00000000000..c6c42e2bc00
Binary files /dev/null and b/src/contents/docs/05.workflow-components/02.namespace/namespace-overview-2-0.png differ
diff --git a/src/contents/docs/05.workflow-components/02.plugins/index.md b/src/contents/docs/05.workflow-components/02.plugins/index.md
index 2b3a2919b08..e58cd6a6d5d 100644
--- a/src/contents/docs/05.workflow-components/02.plugins/index.md
+++ b/src/contents/docs/05.workflow-components/02.plugins/index.md
@@ -6,29 +6,28 @@ sidebarTitle: Plugins
icon: /src/contents/docs/icons/flow.svg
---
-Plugins power every task and trigger in Kestra. They wrap external systems, expose orchestration primitives, and let you extend the platform with custom code. Think of them as the "integrations" or “drivers” that let flows talk to databases, queues, SaaS APIs, file systems, and runtime environments.
+Plugins are the integrations that let flows connect to databases, APIs, file systems, queues, and runtime environments — every task and trigger in Kestra is provided by a plugin. Browse the full catalog at [kestra.io/plugins](/plugins).
## Plugin categories
-Most flows mix several categories:
+Plugins come in three categories:
+
- **Tasks** perform work (HTTP, JDBC, Python, Spark, Script, etc.).
- **Triggers** start executions ([Schedule](../07.triggers/01.schedule-trigger/index.md), [Webhook](../07.triggers/03.webhook-trigger/index.md), [Kafka](../07.triggers/05.realtime-trigger/index.md), Pub/Sub).
-- **Conditions** gate paths (`If`, `Switch`, expressions).
-
-Browse all available plugins at [kestra.io/plugins](/plugins).
+- **Conditions** control when triggers fire or which branch of a `Switch` runs.
:::alert{type="info"}
-The [Kestra MCP server](../../ai-tools/kestra-mcp-resources/index.md) gives AI coding agents like Claude Code and Cursor live access to plugin task schemas, properties, and version history. Add it to your agent for accurate plugin reference while writing flows.
+The [Kestra MCP server](../../ai-tools/02.kestra-mcp-resources/index.md) gives AI coding agents like Claude Code and Cursor live access to plugin task schemas, properties, and version history. Add it to your agent for accurate plugin reference while writing flows.
:::
-## Choosing versions (Enterprise)
+## Choosing versions
+
+Kestra can host multiple versions of the same plugin:
-Kestra can host multiple versions of the same plugin. You can:
-- Pin a version on an individual task/trigger (`version: "0.21.0"`).
-- Rely on the instance-wide `defaultVersion` (often `LATEST`) when you omit it.
-- In Enterprise, install and pin versions centrally under **Instance → Versioned Plugins** (see [Versioned Plugins](../../07.enterprise/05.instance/versioned-plugins/index.md)).
+- Pin a version on an individual task or trigger with `version: "1.0.0"`.
+- Omit `version` to use the instance-wide default (often `LATEST`).
+- In Enterprise Edition, install and manage versions centrally under **Instance → Versioned Plugins** (see [Versioned Plugins](../../07.enterprise/05.instance/versioned-plugins/index.md)).
-Example of pinning a task to a specific version:
```yaml
id: postgres_query
namespace: company.team
@@ -46,44 +45,42 @@ tasks:
## Common configuration patterns
-Plugins often share the same properties; use them wisely to keep executions fast and safe:
-
-- **Result handling (`fetchType` / `storeType`)** chooses how outputs are returned: `FETCH_ONE`, `FETCH`, `NONE`, or `STORE`. `STORE` writes results to internal storage and returns a URI instead of inlining the payload.
-- **Pagination limits** (`fetchSize`, `limit`, `maxResults`) prevent oversized responses when you expect big result sets.
-- [**Secrets**](../../06.concepts/04.secret/index.md): keep connection strings, tokens, and usernames in secrets (`{{ secret('KEY') }}`) so they don’t leak into flow revisions or logs.
-
### Handling outputs: fetch vs. store
-A quick rule set to avoid bloated execution context:
-- Use fetch-style outputs (`fetch`, `fetchType`, `store=false`) only for small payloads you need inline for control flow (e.g., a few rows feeding `Switch` or `ForEach`).
-- For large datasets, switch to store-style (`store=true`, `storeType: STORE`): the data is written to internal storage, and only a URI is kept in the execution context, preventing repeated serialization on every task state change.
-- `value` and `uri` are mutually exclusive: `store=false` exposes `value`; `store=true` exposes `uri`. Accessing the wrong one raises an execution error.
+The `fetchType` property controls how task outputs are returned. For large datasets, use `STORE` — it writes results to internal storage and returns only a URI, preventing large payloads from bloating the execution context.
-| Setting | Use when you need | Stored in execution context | Pebble access | Good for |
-|--------------------|-------------------|-----------------------------|--------------------------|-------------------|
-| `fetchType: FETCH_ONE` | A single small record | The value itself | `{{ outputs.task.value }}` | Lookups, routing |
-| `fetchType: FETCH` | A small list | The list values | `{{ outputs.task.value }}` | Branching logic |
-| `fetchType: NONE` | No result needed | Nothing | n/a | Fire-and-forget |
-| `storeType: STORE` or `store: true` | Large payloads/file-like results | Only a URI | `{{ outputs.task.uri }}` | Large exports, heavy queries |
+| Setting | Use when you need | Stored in context | Pebble access |
+|---|---|---|---|
+| `fetchType: FETCH_ONE` | A single small record | The value itself | `{{ outputs.task.value }}` |
+| `fetchType: FETCH` | A small list | The list values | `{{ outputs.task.value }}` |
+| `fetchType: NONE` | No result | Nothing | n/a |
+| `fetchType: STORE` | Large payloads | Only a URI | `{{ outputs.task.uri }}` |
+
+`value` and `uri` are mutually exclusive: `FETCH`/`FETCH_ONE` exposes `value`; `STORE` exposes `uri`. Accessing the wrong one raises an execution error.
:::alert{type="info"}
-Handling large outputs? Prefer `STORE`/`storeType` and see [Managing output data volume](../../14.best-practices/0.flows/index.md#managing-output-data-volume).
+For guidance on large outputs, see [Managing output data volume](../../14.best-practices/0.flows/index.md#managing-output-data-volume).
:::
-### Secrets in configuration properties
+### Secrets
-Some configuration properties such as "Database Password" are obvious secrets and should be protected, but consider using secrets for connection URLs, database names, user or service account names, and similar. Remember using these values in the flow code even once will appear in a [revision](../../06.concepts/03.revision/index.md).
+Use secrets for connection strings, URLs, usernames, tokens, and passwords. Any value written directly in a flow definition is stored in its [revision history](../../06.concepts/03.revision/index.md) and visible to anyone with access to the flow.
-Check out the how-to guide for [Secrets in Open Source](../../15.how-to-guides/secrets/index.md), or [Secrets Manager](../../07.enterprise/02.governance/secrets-manager/index.md) in Enterprise Edition.
+```yaml
+username: "{{ secret('POSTGRES_USERNAME') }}"
+password: "{{ secret('POSTGRES_PASSWORD') }}"
+```
+
+See [Secrets in Open Source](../../15.how-to-guides/secrets/index.md) or [Secrets Manager](../../07.enterprise/02.governance/secrets-manager/index.md) (Enterprise Edition).
## Installing plugins
-Installation paths vary by role:
-- **UI (Enterprise)**: install/upgrade/pin versions under **Instance → Versioned Plugins**.
-- **CLI/API**: automate installs; see [Selected Plugin Installation](../../15.how-to-guides/selected-plugin-installation/index.md).
+- **OSS**: the standard Kestra Docker image ships with plugins pre-installed. For a minimal build, use the `-no-plugins` image and install selectively via `kestra plugins install ` or by mounting plugin JARs to `/app/plugins/`.
+- **UI (Enterprise Edition)**: install, upgrade, and pin versions under **Instance → Versioned Plugins**.
+
+See [Selected Plugin Installation](../../15.how-to-guides/selected-plugin-installation/index.md) for full setup options including Docker Compose automation.
## Building or requesting plugins
-If you can’t find the integration you need, you can build or request it:
-- Build: follow the [Plugin Developer Guide](../../plugin-developer-guide/index.mdx) to scaffold, test, and publish.
-- Request: ask in the [Kestra Slack community](https://kestra.io/slack) or open an issue in the [Kestra repository](https://github.com/kestra-io/kestra/issues).
+- **Build**: follow the [Plugin Developer Guide](../../plugin-developer-guide/index.mdx) to scaffold, test, and publish.
+- **Request**: open an issue in the [Kestra repository](https://github.com/kestra-io/kestra/issues) or ask in the [Kestra Slack community](https://kestra.io/slack).
diff --git a/src/contents/docs/05.workflow-components/03.execution/executions-overview.png b/src/contents/docs/05.workflow-components/03.execution/executions-overview.png
new file mode 100644
index 00000000000..2fe977d9f4a
Binary files /dev/null and b/src/contents/docs/05.workflow-components/03.execution/executions-overview.png differ
diff --git a/src/contents/docs/05.workflow-components/03.execution/index.md b/src/contents/docs/05.workflow-components/03.execution/index.md
index 1e78650bf90..03c344aecbf 100644
--- a/src/contents/docs/05.workflow-components/03.execution/index.md
+++ b/src/contents/docs/05.workflow-components/03.execution/index.md
@@ -7,47 +7,23 @@ icon: /src/contents/docs/icons/flow.svg
docId: executions
---
-Execute flows and view the results.
-
An execution is a single run of a flow with a specific state.
-## Task run
-
-A task run is a single run of an individual task within an execution.
-
-Each task run has associated data such as:
-
-- Execution ID
-- State
-- Start Date
-- End Date
-
-Read more about task runs on the [dedicated docs page](../01.tasks/02.taskruns/index.md).
+Each execution contains one or more [task runs](../01.tasks/02.taskruns/index.md) — one per task in the flow. Task runs support [retries](../12.retries/index.md): if retries are configured, a failure generates new attempts until the `maxAttempts` or `maxDuration` threshold is reached.
-## Attempts
-
-Each task run can have one or more attempts. Most task runs have only one attempt, but you can configure [retries](../12.retries/index.md) for a task.
-If retries have been configured, a task failure will generate new attempts until the retry `maxAttempts` or `maxDuration` threshold is hit.
+
## Outputs
-Each task can generate output data that other tasks in the current flow execution can use.
-These outputs can be variables or files that are stored inside Kestra's internal storage.
-
-Outputs are described on each task’s documentation page and can be viewed in the **Outputs** tab of the **Execution** page.
-Read more on the [Outputs page](../../05.workflow-components/06.outputs/index.md).
+Each task can produce output data — variables or files stored in Kestra's internal storage — that downstream tasks in the same execution can reference. View outputs in the **Outputs** tab of the execution page. See the [Outputs page](../../05.workflow-components/06.outputs/index.md) for details.
## Metrics
-Each task can expose metrics that help you understand task internals. Metrics may include file size, number of returned rows, or query duration. You can view the available metrics for a task type on its documentation page.
-
-Metrics can be seen in the **Metrics** tab of the **Executions** page.
-
-Below is an example of a flow generating metrics:
+Tasks can expose metrics such as file size, row count, or query duration. View them in the **Metrics** tab of the execution page, or on the task's plugin documentation page.
```yaml
id: load_data_to_bigquery
@@ -60,7 +36,6 @@ tasks:
- id: load_bigquery
type: io.kestra.plugin.gcp.bigquery.Load
- description: Load data into BigQuery
autodetect: true
csvOptions:
fieldDelimiter: ","
@@ -69,85 +44,53 @@ tasks:
from: "{{ outputs.http_download.uri }}"
```
-You can see the list of generated metrics generated in the [BigQuery Load task documentation](/plugins/plugin-gcp/google-cloud-bigquery/io.kestra.plugin.gcp.bigquery.load#metrics).
-
-After executing the flow, view the BigQuery Load task metrics in the **Metrics** tab.
+## States
-
+Executions and task runs move through the following states:
-## State
-
-An execution or a task run can be in a particular state.
-
-There are multiple possible states:
-
-| State | Description |
-| - |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `CREATED` | The Execution or task run is waiting to be processed. This state usually means that the Execution is in a queue and is yet to be started. |
-| `RUNNING` | The Execution or task run is currently being processed. |
-| `PAUSED` | The Execution or task run has been paused. Used for manual validation or waiting for a specified duration before continuing the execution. |
-| `SUCCESS` | The Execution or task run has been completed successfully. |
-| `WARNING` | The Execution or task run exhibited unintended behavior, but the execution continued and was flagged with a warning. |
-| `FAILED` | The Execution or task run exhibited unintended behavior that caused the execution to fail. |
-| `KILLING` | A command was issued that asked for the Execution or task run to be killed. The system is in the process of killing the associated tasks. |
-| `KILLED` | An Execution or task run was killed (upon request), and no more tasks will run. |
-| `RESTARTED` | Transitional status equivalent to `CREATED`for a flow that was executed, failed, and then restarted. |
-| `CANCELLED` | An Execution or task run has been aborted because it has reached the defined [concurrency limit](../14.concurrency/index.md) or exceeded the [SLA](../18.sla/index.md) . The behavior was set to the `CANCEL`. |
-| `QUEUED` | An Execution or task run has been put on hold because it has reached its defined concurrency limit. The limit was set to the `QUEUE` behavior. |
-| `RETRYING` | The Execution or task run is currently being [retried](../12.retries/index.md). |
-| `RETRIED` | An Execution or task run exhibited unintended behavior, stopped, and created a new execution as defined by its [flow-level retry policy](../12.retries/index.md#flow-level-retries). The policy was set to the `CREATE_NEW_EXECUTION` behavior. |
-
-:::alert{type="info"}
-For a detailed overview of how each execution and task run transitions through states, see the [States](../17.states/index.md) page.
-:::
+| State | Description |
+| - | - |
+| `CREATED` | Waiting to be processed — queued but not yet started. |
+| `RUNNING` | Currently being processed. |
+| `PAUSED` | Paused for manual validation or a configured delay. |
+| `SUCCESS` | Completed successfully. |
+| `WARNING` | Completed with warnings — execution continued but was flagged. |
+| `FAILED` | Encountered errors that caused the execution to fail. |
+| `KILLING` | Kill command issued; system is terminating associated tasks. |
+| `KILLED` | Killed on request — no further tasks will run. |
+| `RESTARTED` | Transitional state equivalent to `CREATED` for a restarted failed execution. |
+| `CANCELLED` | Aborted due to a [concurrency limit](../14.concurrency/index.md) or [SLA](../18.sla/index.md) with `CANCEL` behavior. |
+| `QUEUED` | On hold due to a concurrency limit with `QUEUE` behavior. |
+| `RETRYING` | Currently being [retried](../12.retries/index.md). |
+| `RETRIED` | Stopped and created a new execution as defined by a [flow-level retry policy](../12.retries/index.md#flow-level-retries) with `CREATE_NEW_EXECUTION` behavior. |
+
+For a detailed overview of state transitions, see the [States](../17.states/index.md) page.
## Execution expressions
-You can use the following execution expressions in your flow.
-
| Parameter | Description |
| - | - |
-| `{{ execution.id }}` | The execution ID, a generated unique ID for each execution |
-| `{{ execution.startDate }}` | The start date of the current execution, can be formatted with `{{ execution.startDate | date("yyyy-MM-dd HH:mm:ss.SSSSSS") }}`. |
-| `{{ execution.originalId }}` | The original execution ID, this ID never changes, even in case of a replay and keeps the first execution ID. |
-
-
-## Execute a flow from the UI
+| `{{ execution.id }}` | Unique identifier generated for each execution. |
+| `{{ execution.startDate }}` | Start date of the current execution; can be formatted with `{{ execution.startDate \| date("yyyy-MM-dd HH:mm:ss.SSSSSS") }}`. |
+| `{{ execution.originalId }}` | The original execution ID — never changes across replays. |
-You can trigger a flow manually from the Kestra UI by clicking the **Execute** button on the flow page. This is useful when you want to test a flow or run it on demand.
+## Execute from the UI
-
+Click **Execute** on the flow page to trigger a run manually.
## Use automatic triggers
-You can add a **Schedule trigger** to automatically launch a flow execution at a regular time interval.
+Add a [Schedule trigger](../07.triggers/01.schedule-trigger/index.md) to launch executions on a time interval, or a [Flow trigger](../07.triggers/index.mdx) to launch an execution when another flow completes — useful for namespace-level error handling or event-driven patterns where flows are decoupled rather than explicitly calling each other as subflows.
-Alternatively, you can add a **Flow trigger** to automatically launch a flow execution when another flow execution is completed. This is helpful when you want to:
-
-- Implement a centralized namespace-level error handling strategy, e.g., to send a notification when any flow execution fails in a production namespace. Check the [Alerting & Monitoring](../../10.administrator-guide/03.monitoring/index.md) section for more details.
-- Decouple your flows by following an event-driven pattern where a flow is triggered by the completion of another flow (as opposed to the [subflow pattern]… where a parent flow explicitly calls child flows).
-
-You can also use the **Webhook trigger** to automatically launch a flow execution when a given HTTP request is received. You can leverage the `{{ trigger.body }}` variable to access the request body and the `{{ trigger.headers }}` variable to access the request headers in your flow.
-
-To launch a flow and send data to the flow's execution context from an external system using a webhook, you can send a POST request to the Kestra API using the following URL:
+Use a [Webhook trigger](../07.triggers/03.webhook-trigger/index.md) to launch an execution from an external HTTP request. Access the request body with `{{ trigger.body }}` and headers with `{{ trigger.headers }}`. See the [Webhooks how-to guide](../../15.how-to-guides/webhooks/index.md) for setup and real-world examples.
```bash
http://:/api/v1/main/executions/webhook///
```
-Below is an example:
-
-```bash
-http://localhost:8080/api/v1/main/executions/webhook/dev/hello-world/secretWebhookKey42
-```
-
-You can also pass inputs to the flow using the `inputs` query parameter.
-
-## Execute a flow via an API call
+## Execute via API
-You can trigger a flow execution by calling the [API](../../api-reference/index.mdx) directly. This is useful when you want to start a flow execution from another application or service.
-
-Use the following flow as an example:
+Trigger an execution by calling the [API](../../api-reference/index.mdx) directly. Given this flow:
```yaml
id: hello_world
@@ -162,62 +105,46 @@ tasks:
- id: hello
type: io.kestra.plugin.core.log.Log
message: "{{ inputs.greeting }}"
-
-triggers:
- - id: webhook
- type: io.kestra.plugin.core.trigger.Webhook
- key: test1234
```
-If Kestra runs locally, trigger a flow by calling `/api/v1/main/executions/{namespace}/{flowId}` endpoint. This example uses `curl` but you could use something else like [Postman](https://www.postman.com/) to test this too:
+Trigger it with `curl`:
```bash
-curl -X POST \
-http://localhost:8080/api/v1/main/executions/company.team/hello_world
+curl -X POST http://localhost:8080/api/v1/main/executions/company.team/hello_world
```
-The above command triggers an execution of the latest revision of the `hello_world` flow from the `company.team` namespace.
-
-### Execute a specific revision of a flow
-
-If you want to trigger an execution for a specific revision, you can use the `revision` query parameter:
+### Execute a specific revision
```bash
-curl -X POST \
-http://localhost:8080/api/v1/main/executions/company.team/hello_world?revision=2
+curl -X POST http://localhost:8080/api/v1/main/executions/company.team/hello_world?revision=2
```
-### Execute a flow with inputs
+### Execute with inputs
-You can also trigger a flow execution with inputs by adding the `inputs` as form data (the `-F` flag in the `curl` command):
+Pass inputs as form data:
```bash
-curl -X POST \
-http://localhost:8080/api/v1/main/executions/company.team/hello_world \
+curl -X POST http://localhost:8080/api/v1/main/executions/company.team/hello_world \
-F greeting="hey there"
```
-You can pass inputs of different types, such as `STRING`, `INT`, `FLOAT`, `DATETIME`, `FILE`, `BOOLEAN`, and more.
+For multiple input types:
```bash
curl -v "http://localhost:8080/api/v1/main/executions/company.team/kestra-inputs" \
-H "Transfer-Encoding:chunked" \
-H "Content-Type:multipart/form-data" \
- -F string="a string" \
- -F optional="an optional string" \
- -F int=1 \
- -F float=1.255 \
- -F boolean=true \
+ -F string="a string" \
+ -F int=1 \
+ -F float=1.255 \
+ -F boolean=true \
-F instant="2023-12-24T23:00:00.000Z" \
-F "files=@/tmp/128M.txt;filename=file"
```
+### Execute with FILE inputs
-### Execute a flow with FILE-type inputs
-
-You can also pass files as an input. All files must be sent as multipart form data named `files` with a header `filename=your_kestra_input_name` indicating the name of the input.
-
-Let's look at an example to make this clearer. Suppose you have a flow that takes a JSON file as input and reads the file's content:
+Pass files as multipart form data named `files`, with a `filename` header matching the input ID:
```yaml
id: large_json_payload
@@ -238,20 +165,38 @@ tasks:
- cat myfile.json
```
-Assuming you have a file `myfile.json` in the current working directory, you can invoke the flow using the following `curl` command:
-
```bash
-curl -X POST -F "files=@./myfile.json;filename=myCustomFileInput" 'http://localhost:8080/api/v1/main/executions/company.team/large_json_payload'
+curl -X POST -F "files=@./myfile.json;filename=myCustomFileInput" \
+ 'http://localhost:8080/api/v1/main/executions/company.team/large_json_payload'
```
:::alert{type="info"}
-We recommend this pattern if you need to pass large payloads to a flow. Passing a large payload directly in the request body (e.g., as `JSON`-type input or as a raw JSON webhook body) is not recommended for privacy, performance, and maintainability reasons. Such large payloads would be stored directly in your Kestra's database backend, cluttering valuable storage space and leading to potential performance or privacy issues. However, if you pass it as a JSON file using a `FILE`-type input, it will be stored in internal storage (such as S3, GCS, Azure Blob), making it more performant and cost-effective to store and retrieve.
+Prefer FILE-type inputs for large payloads. Files are stored in internal storage (S3, GCS, Azure Blob), whereas JSON-type inputs or raw webhook bodies are stored directly in the database.
:::
+### Get a URL to follow execution progress
-### Execute a flow via an API call in Python
+The executions endpoint returns a `url` field in its response, which links directly to the execution in the UI:
-You can also use the `requests` library in Python to make requests to the Kestra API. Here's an example:
+```bash
+curl -X POST http://localhost:8080/api/v1/main/executions/company.team/myflow
+```
+
+```json
+{
+ "id": "1ZiZQWCHj7bf9XLtgvAxyi",
+ "url": "http://localhost:8080/ui/executions/company.team/myflow/1ZiZQWCHj7bf9XLtgvAxyi"
+}
+```
+
+To receive a full URL rather than a path suffix, configure your instance URL in [Runtime and Storage configuration](../../configuration/02.runtime-and-storage/index.md):
+
+```yaml
+kestra:
+ url: http://localhost:8080
+```
+
+### Execute via API in Python
```python
import requests
@@ -273,120 +218,14 @@ with open("/tmp/128M.txt", 'rb') as fh:
)
```
-### Get URL to follow the Execution progress
-
-The executions endpoint also [returns a URL](https://github.com/kestra-io/kestra/issues/4256), allowing you to follow the execution progress from the UI. This is helpful for externally triggered, long-running executions that require users to monitor workflow progress. Below are the steps to use it:
-
-1) First, create a flow:
-
-```yaml
-id: myflow
-namespace: company.team
-
-tasks:
- - id: long_running_task
- type: io.kestra.plugin.scripts.shell.Commands
- commands:
- - sleep 90
- taskRunner:
- type: io.kestra.plugin.core.runner.Process
-```
-
-2) Execute the flow via an API call:
-
-```shell
-curl -X POST http://localhost:8080/api/v1/main/executions/company.team/myflow
-```
-
-You will see output similar to the following:
-
-```bash
-{
- "id": "1ZiZQWCHj7bf9XLtgvAxyi",
- "namespace": "company.team",
- "flowId": "myflow",
- "flowRevision": 1,
- "state": {
- "current": "CREATED",
- "histories": [
- {
- "state": "CREATED",
- "date": "2024-09-24T13:35:32.983335847Z"
- }
- ],
- "duration": "PT0.017447417S",
- "startDate": "2024-09-24T13:35:32.983335847Z"
- },
- "originalId": "1ZiZQWCHj7bf9XLtgvAxyi",
- "deleted": false,
- "metadata": {
- "attemptNumber": 1,
- "originalCreatedDate": "2024-09-24T13:35:32.983420055Z"
- },
- "url": "http://localhost:8080/ui/executions/company.team/myflow/1ZiZQWCHj7bf9XLtgvAxyi"
-}
-```
-
-You can click directly on that last URL to follow the execution progress from the UI, or you can return that URL from your application to the user who initiated the flow.
-
-Keep in mind that you need to configure the URL of your Kestra instance within your [Runtime and Storage configuration](../../configuration/02.runtime-and-storage/index.md) file to have a full URL rather than just the suffix `/ui/executions/company.team/myflow/uuid`. Here is how you can do it:
-
-```yaml
-kestra:
- url: http://localhost:8080
-```
-
-
## Webhook vs. API call
-When sending a POST request to the `/api/v1/main/executions/{namespace}/{flowId}` endpoint, you can send data to the flow's execution context using `inputs`. If you want to send arbitrary metadata to the flow's execution context based on some event happening in your application, you can leverage a Webhook trigger.
-
-You can adjust the previous `hello_world` example to use the webhook trigger instead of an API call:
-
-```yaml
-id: hello_world
-namespace: company.team
-
-inputs:
- - id: greeting
- type: STRING
- defaults: hey
-
-tasks:
- - id: hello
- type: io.kestra.plugin.core.log.Log
- message: "{{ trigger.body ?? inputs.greeting }}"
-
-triggers:
- - id: webhook
- type: io.kestra.plugin.core.trigger.Webhook
- key: test1234
-```
-
-You can now send a POST request to the `/api/v1/main/executions/webhook/{namespace}/{flowId}/{webhookKey}` endpoint to trigger an execution and pass any metadata to the flow using the request body. In this example, the webhook URL would be `http://localhost:8080/api/v1/main/executions/webhook/company.team/hello_world/test1234`.
-
-You can test the webhook trigger using a tool like Postman or cURL. Paste the webhook URL in the URL field and a [sample JSON payload](https://gist.github.com/anna-geller/df2532c0699e3ba4f572a88fbdf19a13) in the request body. Make sure to set:
-
-- the request method to POST
-- the request body type to raw JSON format
-
-Finally, click the **Send** button to trigger the flow execution. You should get a response with the execution ID and status code 200 OK.
-
-
-
:::alert{type="info"}
-⚡️ **When to use a webhook trigger vs. an API call?** To decide whether to use a webhook trigger or an API call to create an Execution, consider the following:
-
-- Use the **webhook trigger** when you want **to send arbitrary metadata** to the flow's execution context based on some event happening in your application.
-- Use the **webhook trigger** when you want to create new executions based on some **event** happening in an **external application**, such as a GitHub event (_e.g. a Pull Request is merged_) or a new record in a SaaS application, and you want to send the event metadata (header and body) to the flow to act on it.
-- Use an **API call** when you only need to pass **typed inputs** and do not need to send an arbitrary payload.
+- Use the **webhook trigger** when you want to pass arbitrary metadata from an external event (GitHub PR merged, new SaaS record, etc.) to the flow via `{{ trigger.body }}` and `{{ trigger.headers }}`.
+- Use an **API call** when you only need to pass typed inputs and don't need an arbitrary payload.
:::
----
-
-## Execute a flow via kestractl
-
-You can trigger and inspect executions from the command line using [kestractl](../../kestra-cli/kestractl/index.md).
+## Execute via kestractl
```bash
# Run a flow and wait for completion
@@ -396,138 +235,33 @@ kestractl executions run prod nightly-refresh --wait
kestractl executions run prod nightly-refresh --wait --output json
```
----
-
-## Execute a flow from Python
+See [kestractl](../../kestra-cli/kestractl/index.md) for the full command reference.
-You can also execute a flow using the [kestra pip package](https://github.com/kestra-io/libs). This is useful when you want to trigger a flow execution from a Python application without crafting the HTTP request manually, as shown earlier.
+## Execute from Python
-First, install the package:
+Use the [kestra pip package](https://github.com/kestra-io/libs) to trigger executions without crafting HTTP requests manually:
```bash
pip install kestra
```
-Then, you can trigger a flow execution by calling the `execute()` method. Below is an example for the same `hello_world` flow in the namespace `company.team` as above:
-
```python
from kestra import Flow
flow = Flow()
flow.execute('company.team', 'hello_world', {'greeting': 'hello from Python'})
```
-Now imagine that you have a flow that takes a FILE-type input and reads the file's content:
-
-```yaml
-id: myflow
-namespace: company.team
-
-inputs:
- - id: myfile
- type: FILE
-
-tasks:
- - id: print_data
- type: io.kestra.plugin.core.log.Log
- message: "file's content {{ read(inputs.myfile) }}"
-```
-
-Assuming you have a file called `example.txt` in the same directory as your Python script, you can pass a file as an input to the flow using the following Python code:
+To pass a FILE input:
```python
import os
from kestra import Flow
-os.environ["KESTRA_HOSTNAME"] = "http://host.docker.internal:8080" # Set this when executing inside Kestra
+os.environ["KESTRA_HOSTNAME"] = "http://host.docker.internal:8080"
flow = Flow()
with open('example.txt', 'rb') as fh:
flow.execute('company.team', 'myflow', {'files': ('myfile', fh, 'text/plain')})
```
-Keep in mind that `files` is a tuple with the following structure: `('input_id', file_object, 'content_type')`.
-
-## Execute with ForEachItem
-
-The `ForEachItem` task allows you to iterate over a list of items and run a subflow for each item, or for each batch containing multiple items. Use this to process large lists in parallel, e.g., millions of records from a database table or an API payload.
-
-The `ForEachItem` task is a **Flowable** task, which means that it can be used to define the flow logic and control the execution of the flow.
-
-Syntax:
-
-```yaml
-id: each_example
-namespace: company.team
-tasks:
- - id: each
- type: io.kestra.plugin.core.flow.ForEachItem
- items: "{{ inputs.file }}" # could be also an output variable {{ outputs.extract.uri }}
- inputs:
- file: "{{ taskrun.items }}" # batch items passed to the subflow
- batch:
- rows: 4
- bytes: "1024"
- partitions: 2
- namespace: company.team
- flowId: subflow
- revision: 1 # optional (default: latest)
- wait: true # wait for the subflow execution
- transmitFailed: true # fail the task run if the subflow execution fails
- labels: # optional labels to pass to the subflow to be executed
- key: value
-```
-
-:::collapse{title="Full Example"}
-
-Subflow:
-
-```yaml
-id: subflow
-namespace: company.team
-
-inputs:
- - id: items
- type: FILE
-
-tasks:
- - id: for_each_item
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.core.runner.Process
- commands:
- - cat "{{ inputs.items }}"
-
- - id: read
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(inputs.items) }}"
-```
-
-Below is a flow that uses the `ForEachItem` task to iterate over a list of items and run the `subflow` for a batch of 10 items at a time:
-
-```yaml
-id: each_parent
-namespace: company.team
-
-tasks:
- - id: extract
- type: io.kestra.plugin.jdbc.duckdb.Query
- sql: |
- INSTALL httpfs;
- LOAD httpfs;
- SELECT *
- FROM read_csv_auto('https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv', header=True);
- store: true
-
- - id: each
- type: io.kestra.plugin.core.flow.ForEachItem
- items: "{{ outputs.extract.uri }}"
- batch:
- rows: 10
- namespace: company.team
- flowId: subflow
- wait: true
- transmitFailed: true
- inputs:
- items: "{{ taskrun.items }}"
-```
-:::
+`files` takes a tuple: `('input_id', file_object, 'content_type')`.
diff --git a/src/contents/docs/05.workflow-components/04.variables/index.md b/src/contents/docs/05.workflow-components/04.variables/index.md
index 37ea6c61d44..baf8953c098 100644
--- a/src/contents/docs/05.workflow-components/04.variables/index.md
+++ b/src/contents/docs/05.workflow-components/04.variables/index.md
@@ -7,17 +7,15 @@ icon: /src/contents/docs/icons/flow.svg
docId: variables
---
-Variables are key-value pairs that let you reuse values across tasks.
-
-You can also store variables at the namespace level to reuse them across multiple flows in that namespace.
+Variables are key-value pairs that let you reuse values across tasks in a flow, or across multiple flows when stored at the namespace level.
-## How to configure variables
+## Declaring variables
-The example below shows how you can configure variables in your flow:
+Define variables under the `variables` key in a flow and reference them with `{{ vars.variable_name }}`:
```yaml
id: hello_world
@@ -33,27 +31,17 @@ tasks:
format: "{{ vars.myvar }} world {{ vars.numeric_variable }}"
```
-Use variables with the syntax `{{ vars.variable_name }}`.
-
-## How variables are rendered
+## Rendering
-You can use variables in any task property documented as **dynamic**.
-
-Dynamic variables are rendered by the Pebble templating engine, which processes expressions with filters and functions. More information on variable processing can be found under [Expressions](../../expressions/index.mdx).
+Variables are rendered by the [Pebble templating engine](../../expressions/index.mdx), which processes expressions with filters and functions. You can use variables in any task property marked as **dynamic**.
:::alert{type="info"}
-Variables are no longer rendered recursively. Learn more about this change — and how to adjust behavior — in the [migration guide](../../11.migration-guide/v0.14.0/recursive-rendering/index.md).
+Variables are no longer rendered recursively. See the [migration guide](../../11.migration-guide/v0.14.0/recursive-rendering/index.md) for details.
:::
## Dynamic variables
-
-
-
-
-If a variable contains an expression, wrap it with `render` when using it in a task.
-
-For example, the variable below displays the current time only when wrapped with `render`; otherwise, the log prints the expression as a string:
+If a variable contains an expression, wrap it with `render()` when using it in a task — otherwise the expression is treated as a literal string:
```yaml
id: dynamic_variable
@@ -68,55 +56,35 @@ tasks:
message: "{{ render(vars.time) }}"
```
-:::alert{type="info"}
-Wrap the variable expression with `render` every time you use it in a task.
+:::alert{type="warning"}
+Always wrap expression-valued variables with `render()` when referencing them in tasks.
:::
-## Set or modify execution variables
+## Set or modify variables at runtime
-The `SetVariables` and `UnsetVariables` tasks can modify or delete variables within the execution context. For example, take the following flow:
+The `SetVariables` task updates variables in the execution context. Later tasks see the new values immediately:
```yaml
id: variables_demo
namespace: company.team
variables:
- state: FAILED
- ansibleTicket: myticket
- nested:
- child: property
- unchanged: stay the same
+ status: pending
tasks:
- - id: request
- type: io.kestra.plugin.core.output.OutputValues
- values:
- ansibleTicket: new ticket value
- state: SUCCESS
-
- - id: updateVariables
+ - id: update
type: io.kestra.plugin.core.execution.SetVariables
- overwrite: true # true by default
variables:
- state: "{{ outputs.request.values.state }}"
- ansibleTicket: "{{ outputs.request.values.ansibleTicket }}"
- nested:
- child: new value
+ status: complete
- - id: confirmUpdate
+ - id: log
type: io.kestra.plugin.core.log.Log
- message: Hello "{{ vars }}"
+ message: "Status is now {{ vars.status }}"
```
-Initially, `state` is `FAILED` and `ansibleTicket` is `myticket`. Within the flow, the `updateVariables` task uses `io.kestra.plugin.core.execution.SetVariables` to modify `state` to `SUCCESS` and `ansibleTicket` to `new ticket value` per the `request` task, as well as change one of the nested variables, `nested.child` to `new value` (`nested.unchanged` is unmodified so it'll remain the same).
-
-After the flow runs, `state`, `ansibleTicket`, and `nested.child` have their new values, and `nested.unchanged` remains unchanged.
-
-
+## Unset variables
-## Delete or unset execution variables
-
-To unset variables, use `io.kestra.plugin.core.execution.UnsetVariables`. Building on the example above, add the following task:
+The `UnsetVariables` task deletes variables from the execution context. It supports dot notation for nested keys:
```yaml
- id: deleteVariables
@@ -124,164 +92,46 @@ To unset variables, use `io.kestra.plugin.core.execution.UnsetVariables`. Buildi
variables:
- state
- ansibleTicket
- - nested.child # remove only this key from the nested map
+ - nested.child
```
-After executing the flow, the only remaining variable is `nested.unchanged` with the value `stay the same`. In the unset task, `state`, `ansibleTicket`, and `nested.child` were deleted.
-
-
-
## FAQ
-### How do I escape a block in Pebble syntax to ensure that it won't be parsed?
+### How do I escape a Pebble expression so it is not evaluated?
-To ensure that a block of code won't be parsed by Pebble, you can use the `{% raw %}` and `{% endraw %}` [Pebble tags](../../expressions/02.syntax/index.mdx#raw). For example, the following returns the string `{{ myvar }}` instead of the value of `myvar`:
+Use the `{% raw %}` and `{% endraw %}` tags. The following returns the string `{{ myvar }}` literally:
```yaml
{% raw %}{{ myvar }}{% endraw %}
```
-### In which order are inputs and variables resolved?
+See [Pebble syntax](../../expressions/02.syntax/index.mdx#raw) for details.
-[Inputs](../05.inputs/index.md) are resolved first, before the execution starts. If a flow has an invalid input value, the execution will not be created.
+### In what order are inputs and variables resolved?
-Therefore, you can use inputs within variables, but you cannot use variables or Pebble expressions in most contexts (Check out [Dynamic Inputs](../05.inputs/index.md#dynamic-inputs) for more information) within inputs.
-
-[Expressions](../../expressions/index.mdx) are rendered recursively: if a variable references another variable, the inner one is resolved first.
-
-Triggers are handled similarly to inputs because they are known before the execution starts (they create the execution). This means you cannot use inputs (unless they have `defaults`) within triggers, but you can use trigger variables inside `variables`.
-
-#### Examples
-
-This flow uses inputs, trigger, and execution variables which are resolved before variables:
-
-```yaml
-id: upload_to_s3
-namespace: company.team
-
-inputs:
- - id: bucket
- type: STRING
- defaults: declarative-data-orchestration
-
-tasks:
- - id: get_zip_file
- type: io.kestra.plugin.core.http.Download
- uri: https://wri-dataportal-prod.s3.amazonaws.com/manual/global_power_plant_database_v_1_3.zip
-
- - id: unzip
- type: io.kestra.plugin.compress.ArchiveDecompress
- algorithm: ZIP
- from: "{{outputs.get_zip_file.uri}}"
-
- - id: csv_upload
- type: io.kestra.plugin.aws.s3.Upload
- from: "{{ outputs.unzip.files['global_power_plant_database.csv'] }}"
- bucket: "{{ inputs.bucket }}"
- key: "powerplant/{{ trigger.date ?? execution.startDate | date('yyyy_MM_dd__HH_mm_ss') }}.csv"
-
-triggers:
- - id: hourly
- type: io.kestra.plugin.core.trigger.Schedule
- cron: "@hourly"
-```
-
-This flow starts a task conditionally based on whether the input is provided or not:
-
-```yaml
-id: conditional_branching
-namespace: company.team
-
-inputs:
- - id: parameter
- type: STRING
- required: false
-
-tasks:
- - id: if
- type: io.kestra.plugin.core.flow.If
- condition: "{{inputs.parameter ?? false }}"
- then:
- - id: if_not_null
- type: io.kestra.plugin.core.log.Log
- message: Received input {{inputs.parameter}}
- else:
- - id: if_null
- type: io.kestra.plugin.core.log.Log
- message: No input provided
-```
-
-Below is an example that uses a trigger variable within a trigger itself (_that's allowed!_):
-
-```yaml
-id: backfill_past_mondays
-namespace: company.team
-
-tasks:
- - id: log_trigger_or_execution_date
- type: io.kestra.plugin.core.log.Log
- message: "{{ trigger.date ?? execution.startDate }}"
-
-triggers:
- - id: first_monday_of_the_month
- type: io.kestra.plugin.core.trigger.Schedule
- timezone: Europe/Berlin
- backfill:
- start: 2023-11-11T00:00:00Z
- cron: "0 11 * * MON" # at 11:00 every Monday
- conditions: # only first Monday of the month
- - type: io.kestra.plugin.core.condition.DayWeekInMonth
- date: "{{ trigger.date }}"
- dayOfWeek: "MONDAY"
- dayInMonth: "FIRST"
-```
+[Inputs](../05.inputs/index.md) are resolved first, before the execution starts — an invalid input value prevents the execution from being created. You can use inputs within variables, but not variables within inputs (see [Dynamic Inputs](../05.inputs/index.md#dynamic-inputs) for the exception).
+Triggers are resolved like inputs — before the execution starts — so you can reference trigger variables inside `variables`, but not inputs within triggers unless they have `defaults`.
### Can I transform variables with Pebble expressions?
-Yes. Kestra uses [Pebble templates](https://pebbletemplates.io/) along with the execution context to render **dynamic properties**. (such as filters, functions, and operators) to transform [inputs](../05.inputs/index.md) and [variables](../04.variables/index.md).
-
-The example below illustrates how to use variables and Pebble expressions to transform string values in dynamic task properties:
+Yes. Pebble filters and functions work in any dynamic property. For example, use a variable to store a date format and apply it with the `date` filter:
```yaml
-id: variables_demo
-namespace: company.team
-
variables:
DATE_FORMAT: "yyyy-MM-dd"
tasks:
- - id: seconds_of_day
- type: io.kestra.plugin.core.debug.Return
- format: '{{ 60 * 60 * 24 }}'
-
- - id: start_date
+ - id: formatted
type: io.kestra.plugin.core.debug.Return
format: "{{ execution.startDate | date(vars.DATE_FORMAT) }}"
-
- - id: curr_date_unix
- type: io.kestra.plugin.core.debug.Return
- format: "{{ now() | date(vars.DATE_FORMAT) | timestamp() }}"
-
- - id: next_date
- type: io.kestra.plugin.core.debug.Return
- format: "{{ now() | dateAdd(1, 'DAYS') | date(vars.DATE_FORMAT) }}"
-
- - id: next_date_unix
- type: io.kestra.plugin.core.debug.Return
- format: "{{ now() | dateAdd(1, 'DAYS') | date(vars.DATE_FORMAT) | timestamp() }}"
-
- - id: pass_downstream
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.core.runner.Process
- commands:
- - echo "{{ outputs.next_date_unix.value }}"
```
+See the [Expressions reference](../../expressions/index.mdx) for the full list of available filters and functions.
+
### Can I use nested variables?
-Yes. Depending on the task, you may need to wrap the root variable with `json()` to access specific keys. Below is an example using a list of maps as a variable:
+Yes. Use `json(item.value).key` to access fields on a nested object:
```yaml
id: vars
@@ -293,19 +143,17 @@ variables:
user: root
- fqn: server02.mydomain.io
user: guest
- - fqn: server03.mydomain.io
- user: rick
tasks:
- - id: parallel
- type: io.kestra.plugin.core.flow.ForEach
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
concurrencyLimit: 0
values: "{{ vars.servers }}"
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
message:
- - "{{ taskrun.value }}" # for each element, prints the full JSON object (e.g., {"fqn":"server01.mydomain.io","user":"root"})
- - "{{ json(taskrun.value).fqn }}" # prints the value for that key (e.g., server01.mydomain.io)
- - "{{ json(taskrun.value).user }}" # prints the value for that key (e.g., root)
+ - "{{ item.value }}"
+ - "{{ json(item.value).fqn }}"
+ - "{{ json(item.value).user }}"
```
diff --git a/src/contents/docs/05.workflow-components/04.variables/set-variables.png b/src/contents/docs/05.workflow-components/04.variables/set-variables.png
deleted file mode 100644
index 82ddfc50e1b..00000000000
Binary files a/src/contents/docs/05.workflow-components/04.variables/set-variables.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/04.variables/unset-variables.png b/src/contents/docs/05.workflow-components/04.variables/unset-variables.png
deleted file mode 100644
index 420cfa71e70..00000000000
Binary files a/src/contents/docs/05.workflow-components/04.variables/unset-variables.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/05.inputs/index.md b/src/contents/docs/05.workflow-components/05.inputs/index.md
index 2ac2e48a078..e9292990332 100644
--- a/src/contents/docs/05.workflow-components/05.inputs/index.md
+++ b/src/contents/docs/05.workflow-components/05.inputs/index.md
@@ -6,310 +6,180 @@ icon: /src/contents/docs/icons/flow.svg
sidebarTitle: Inputs
---
-Inputs are dynamic values passed to the flow at runtime.
+Inputs are typed, validated parameters passed to a flow at execution time.
-A flow can be parameterized with inputs, allowing multiple executions of the same flow with different values. Flow inputs are stored in the execution context and accessed with `{{ inputs.parameter_name }}`.
-
-You can use inputs to make your tasks more dynamic. For instance, you can use an input to dynamically define the path of a file that needs to be processed within a flow.
-
-You can inspect input values in the **Overview** tab of the **Execution** page and set a custom `displayName` for each input to make the interface more readable.
+Flow inputs are stored in the execution context and accessed with `{{ inputs.parameter_name }}`. All inputs are validated when the execution is created — invalid or missing required inputs prevent the execution from being created and it will not appear in the executions list.
## Declaring inputs
-You can declare as many inputs as necessary for any flow. Inputs can be **required** or **optional**. Each input is treated as **required** by default - set `required: false` to make an input optional.
-
-If an input is required, you must provide a value at runtime or set a `defaults` value; otherwise, the execution will not be created.
-
-All inputs are validated when the execution is created; invalid inputs prevent the execution from being created.
-
-:::alert{type="warning"}
-If an execution is **not created** due to invalid or missing inputs, it will not appear in the executions list.
-:::
-
-Below is an example flow using several inputs:
+Inputs are declared under the `inputs` key. Each input requires an `id` and a `type`. Inputs are required by default; set `required: false` to make one optional.
```yaml
-id: inputs
+id: inputs_demo
namespace: company.team
inputs:
- - id: string
+ - id: username
type: STRING
- defaults: "Hello World!"
- displayName: "A string input"
+ defaults: "alice"
+ description: The user to greet.
- - id: optional
- type: STRING
- required: false
- displayName: "An optional string"
-
- - id: int
+ - id: threshold
type: INT
- defaults: 100
- displayName: "An integer input"
-
-
- - id: list_of_int
- type: ARRAY
- itemType: INT
- defaults: [1, 2, 3]
- displayName: "A list of integers"
+ min: 1
+ max: 100
- - id: bool
- type: BOOL
- defaults: true
- displayName: "A boolean input displayed as a toggle."
-
- - id: float
- type: FLOAT
- defaults: 100.12
- displayName: "A float input"
-
- - id: dropdown
+ - id: environment
type: SELECT
- displayName: "A dropdown input"
- defaults: VALUE_1
values:
- - VALUE_1
- - VALUE_2
- - VALUE_3
-
- - id: dropdown_multi
- type: MULTISELECT
- values:
- - VALUE_1
- - VALUE_2
- - VALUE_3
- required: true
-
- - id: instant
- type: DATETIME
- defaults: "2013-08-09T14:19:00Z"
- displayName: "A datetime input"
+ - dev
+ - staging
+ - prod
+ defaults: dev
- - id: date
- type: DATE
- defaults: "2013-10-25"
- displayName: "A date input"
-
- - id: time
- type: TIME
- displayName: "A time input"
- defaults: "14:19:00"
-
- - id: duration
- type: DURATION
- defaults: "PT5M6S"
- displayName: "A duration input"
-
- - id: file
+ - id: config_file
type: FILE
- displayName: "Upload a file"
- defaults: nsfile:///hello.txt
- allowedFileExtensions: [".md", ".txt"]
+ allowedFileExtensions: [".json", ".yaml"]
- - id: json
- type: JSON
- displayName: "A JSON input"
- defaults: |
- [{"name": "kestra", "rating": "best in class"}]
-
- - id: uri
- type: URI
- defaults: "https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv"
- displayName: "A URI input"
-
- - id: secret
- type: SECRET
- displayName: "A secret input"
-
- - id: yaml
- type: YAML
- defaults:
- - user: john
- email: john@example.com
- - user: will
- email: will@example.com
- displayName: YAML
-
- - id: nested.string
+ - id: optional_note
type: STRING
- defaults: "Hello World!"
- displayName: "A nested string input"
-```
-
-:::alert{type="info"}
-The `FILE` type supports defaults via the universal file protocol. Use `nsfile:///` for namespace files or `file:///` for local files.
+ required: false
-Note: `file:///` works only for explicitly allowed paths. Bind‑mount the host directory into the Kestra container and include that path under `kestra.local-files.allowed-paths` in your configuration (e.g., `/scripts`). Otherwise, access to the path will be denied for security reasons.
-:::
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Hello {{ inputs.username }} — deploying to {{ inputs.environment }}"
+```
## Input types
-Inputs in Kestra are strongly typed and validated before starting the flow execution.
-
-Here is the list of supported data types:
-
-- `STRING`: Any string. Values are passed without parsing; for additional validation, use a regex `validator`.
-- `INT`: Must be a valid integer value (i.e., without any decimals).
-- `FLOAT`: Must be a valid float value (i.e., with decimals).
-- `SELECT`: Must be a valid string value from a predefined list of values. You can either pass those values directly using the `values` property or use the `expression` property to fetch the values dynamically from a KV store. Additionally, if `allowCustomValue` is set to true, the user can provide a custom value that is not in the predefined list.
+Inputs are strongly typed and validated before execution starts.
+
+| Type | Accepts | Constraints & extra properties |
+|---|---|---|
+| `STRING` | Any string | `validator` (regex) |
+| `EMAIL` | Valid email address | `validator` (regex) |
+| `INT` | Integer | `min`, `max` |
+| `FLOAT` | Float | `min`, `max` |
+| `BOOL` | `true` or `false` | — |
+| `DATETIME` | [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) datetime in UTC — e.g. `2042-04-02T04:20:42.000Z` | `after`, `before` |
+| `DATE` | ISO 8601 date without timezone — e.g. `2042-12-03` | `after`, `before` |
+| `TIME` | ISO 8601 time without timezone — e.g. `10:15:30` | `after`, `before` |
+| `DURATION` | ISO 8601 duration — e.g. `PT5M6S` | `min`, `max` |
+| `SELECT` | One value from a predefined list | `values`, `expression`, `allowCustomValue`, `autoSelectFirst` |
+| `MULTISELECT` | One or more values from a predefined list | Same as `SELECT` |
+| `FILE` | Uploaded file, `nsfile:///` (namespace file), or `file:///` (local allowed path) | `allowedFileExtensions`; stored in [internal storage](../../08.architecture/data-components/index.md#internal-storage); the only type that accepts a multipart file upload via the API — all other types require a plain string value |
+| `JSON` | Valid JSON string | `jsonSchema` (JSON Schema Draft 2020-12) |
+| `ION` | Ion-formatted text, parsed into a structured object (Map or List) accessible with dot notation — e.g. `{{ inputs.record.name }}`; use Ion syntax for `defaults`: `'{name:"Ada",score:21}'` | — |
+| `YAML` | Valid YAML string | — |
+| `URI` | Valid URI, kept as a string | — |
+| `SECRET` | Encrypted string, decrypted at runtime and masked in UI and logs | `validator` (regex); requires [encryption key](../../configuration/05.security-and-secrets/index.md) |
+| `ARRAY` | JSON array or YAML list | `itemType` (required) |
+| `FORM` | Groups child inputs as a multi-step wizard in the Execute modal | Cannot nest; no `defaults`/`prefill` on the FORM itself |
+| `REUSABLE_INPUTS` | References a namespace-level named input group (Enterprise Edition) | See [Reusable Inputs](../22.reusable-inputs/index.md) |
:::alert{type="info"}
-**Note:** Due to [YAML allowing Scalar content](https://yaml.org/spec/1.1/current.html#id864510) to be presented in several formats, the boolean “true” might also be written as “yes” and “false” as “no”. To avoid errors using Yes/No in the `SELECT` input type, wrap them in quotation marks to preserve string format: "Yes", "No".
+Due to [YAML's scalar formats](https://yaml.org/spec/1.1/current.html#id864510), `yes`/`no` may be parsed as booleans. Wrap them in quotes when using them as `SELECT` values: `"Yes"`, `"No"`.
:::
-- `MULTISELECT`: Must be one or more valid string values from a predefined list of values. You can either pass those values directly using the `values` property or use the `expression` property to fetch the values dynamically from a KV store. Additionally, if `allowCustomValue` is set to true, the user can provide a custom value that is not in the predefined list.
-- `BOOL`: Must be `true` or `false` passed as strings.
-- `DATETIME`: Must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) date and time with the timezone expressed in UTC format; pass input of type DATETIME in a string format following the pattern `2042-04-02T04:20:42.000Z`.
-- `DATE`: Must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) date without the timezone from a text string such as `2042-12-03`.
-- `TIME`: Must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) time without the timezone from a text string such as `10:15:30`.
-- `DURATION`: Must be a valid full [ISO 8601](https://en.wikipedia.org/wiki/ISO_8601) duration from a text string such as `PT5M6S`.
-- `FILE`: Either a file uploaded at execution time as `Content-Type: multipart/form-data` with `Content-Disposition: form-data; name=""; filename=""` (where `` is the input name and `` is the original filename of the file being uploaded), or a default file referenced via the universal file protocol using `nsfile:///path/to/file` (namespace file) or `file:///path/to/file` (local file from an allowed path). `FILE` type inputs also have the `allowedFileExtensions` property to control which types of files can be uploaded.
-- `JSON`: Must be a valid JSON string and will be converted to a typed form.
-- `YAML`: Must be a valid YAML string.
-- `URI`: Must be a valid URI and will be kept as a string.
-- `SECRET`: Encrypted string stored in the database. It is decrypted at runtime and can be used in all tasks. The value of a `SECRET` input is masked in the UI and in the execution context. Note that you need to set the [encryption key](../../configuration/05.security-and-secrets/index.md) in your [Kestra configuration](../../configuration/index.mdx) before using it.
-- `ARRAY`: Must be a valid JSON array or a YAML list. The `itemType` property is required to ensure validation of the type of the array items.
-- `FORM`: Groups related inputs under a shared `displayName` and `description`. When a flow contains at least one FORM input, the Execute modal renders a multi-step wizard — one step per FORM group plus any ungrouped inputs, then a recap. Children are referenced as `{{ inputs.. }}`. FORM inputs cannot be nested and do not support `defaults` or `prefill`.
-
-All `FILE` inputs are automatically uploaded to Kestra's [internal storage](../../08.architecture/data-components/index.md#internal-storage) and accessible to all tasks. After the upload, the input variable will contain a fully qualified URL of the form `kestra:///.../.../` that will be automatically managed by Kestra and can be used as-is within any task.
+:::alert{type="info"}
+`FILE` defaults use the universal file protocol: `nsfile:///` for namespace files, `file:///` for local files from an explicitly allowed path. Configure allowed paths under `kestra.local-files.allowed-paths` in your Kestra configuration.
+:::
## Input properties
-Below is the list of available properties for all inputs regardless of their types:
-
-- `id`: The input parameter identifier — this property is important as it's used to reference the input variables in your flow, e.g., `{{ inputs.user }}` references the input parameter named `user`.
-- `type`: The data type of the input parameter, as described in the previous section.
-- `required`: Whether the input is required. Defaults to `true` - set `required: false` to make an input optional. An input with `defaults` must stay required, since the default is always applied.
-- `defaults`: The default value that is used if no custom input value is provided at runtime; this value must be provided as a string and will be set to the desired data type specified using the `type` property.
-- `prefill`: Starts with an initial value that can be cleared or set to `null` when the input is not required. Like an editable default, it allows workflows to support optional inputs that start with a suggestion but can still be reset to `null` at runtime.
-- `dependsOn`: Makes the input dependent on other inputs that must be provided first.
-- `displayName`: Label shown in the UI instead of the `id`.
-- `description`: Markdown description for the input.
-- `expression`: Use a pebble expression as a value -- e.g., `expression: "{{ kv('SELECT_VALUES') }}"`.
-- `autoSelectFirst`: A boolean property to auto-select the first list value in the dropdown as a default value (only usable for `SELECT` and `MULTISELECT` input types). This way, you don't need to explicitly set any `defaults` for that property.
+| Property | Description |
+|---|---|
+| `id` | Identifier used to reference the input — e.g. `{{ inputs.user }}`. |
+| `type` | Data type, as listed above. |
+| `required` | Whether the input is required. Defaults to `true`. |
+| `defaults` | Default value applied when no value is provided at runtime. |
+| `prefill` | Initial value shown in the UI that can be cleared to `null`. Unlike `defaults`, a cleared prefill resolves to `null`. |
+| `displayName` | Label shown in the UI instead of the `id`. |
+| `description` | Markdown description displayed in the UI. |
+| `validator` | Regex pattern for `STRING` and `SECRET` types. |
+| `expression` | Pebble expression used to populate `SELECT` and `MULTISELECT` values dynamically — e.g. `{{ kv('MY_LIST') }}`. |
+| `dependsOn` | Makes this input conditional on other inputs being provided or matching a `condition`. |
+| `autoSelectFirst` | Auto-selects the first value in `SELECT`/`MULTISELECT` lists as the default. |
## Input validation
-Kestra validates the `type` of each input. In addition to the type validation, some input types can be configured with validation rules that are enforced at execution time.
-
-- `STRING`: A `validator` property allows the addition of a validation [regex](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/util/regex/Pattern.html).
-- `INT`: `min` and `max` define the allowed range.
-- `FLOAT`: `min` and `max` define the allowed range.
-- `DURATION`: `min` and `max` define the allowed range.
-- `DATE`: `after` and `before` properties help you ensure that the input value is within the allowed date range.
-- `TIME`: `after` and `before` properties help you ensure that the input value is within the allowed time range.
-- `DATETIME`: `after` and `before` properties help you ensure that the input value is within the allowed date and time range.
+### Type constraints
-### Example: use input validators in your flows
-
-To ensure that your input value is within a certain `integer` value range, you can use the `min` and `max` properties. Similarly, to ensure that your string input matches a regex pattern, you can provide a custom regex `validator`. The following flow demonstrates how this can be accomplished:
+`INT`, `FLOAT`, and `DURATION` accept `min` and `max`. `DATE`, `TIME`, and `DATETIME` accept `after` and `before`. `STRING` and `SECRET` accept a `validator` regex.
```yaml
-id: regex_input
-namespace: company.team
-
inputs:
- id: age
type: INT
- prefill: 42
- required: false
min: 18
max: 64
- - id: user
+ - id: username
type: STRING
- prefill: student
- required: false
- validator: ^student(\d+)?$
+ validator: ^[a-z0-9_]{3,20}$
- - id: float
- type: FLOAT
- defaults: 3.2
- min: 0.2
- max: 5.3
+ - id: start_date
+ type: DATE
+ after: "2024-01-01"
+ before: "2025-01-01"
+```
- - id: duration
- type: DURATION
- min: "PT5M6S"
- max: "PT12H58M46S"
+### JSON Schema validation
- - id: date
- type: DATE
- defaults: "2024-04-12"
- after: "2024-04-10"
- before: "2024-04-15"
+Use the `jsonSchema` property to validate a `JSON` input against a schema at execution time. An invalid payload rejects the execution before any task runs:
- - id: time
- type: TIME
- after: "11:01:01"
- before: "11:04:01"
+```yaml
+id: json_schema_validation
+namespace: company.team
- - id: datetime
- type: DATETIME
- defaults: "2024-04-13T14:17:00Z"
- after: "2024-04-10T14:19:00Z"
- before: "2024-04-15T14:19:00Z"
+inputs:
+ - id: payload
+ type: JSON
+ jsonSchema: |
+ {
+ "$schema": "https://json-schema.org/draft/2020-12/schema",
+ "type": "object",
+ "required": ["name"],
+ "properties": {
+ "name": { "type": "string" }
+ },
+ "additionalProperties": false
+ }
tasks:
- - id: validator
+ - id: log
type: io.kestra.plugin.core.log.Log
- message: User {{ inputs.user }}, age {{ inputs.age }}
+ message: "Hello, {{ inputs.payload.name }}!"
```
-The `age`, `float`, and `duration` input must be within a valid range between `min` and `max` values. Specifically for the `age` input, we specify that this input is by default set to 42, but it can be overwritten at runtime to a value between 18 and 64. If you attempt to execute the flow with the `age` input set to 17 or 65, the validation will fail and the execution won't start.
-
-Similarly, the Regex expression `^student(\d+)?$` is used to validate that the input argument `user` of type STRING follows the following pattern:
-- `^student`: This part of the regex asserts that the string must begin with the lowercase string value `student`.
-- `\d`: This part of the regex matches any digit (0-9).
-- `+`: This part of the regex asserts that there is one or more of the preceding token (i.e., one or more digits are allowed after the value `student`).
-- `()?`: The parentheses group the digits together, and the question mark makes the entire group optional — this means that the digits after the word `student` are optional.
-- `$`: This part of the regex asserts the end of the string. This ensures that the string doesn't contain any additional characters after the optional digits.
-
-With this pattern:
-- "student" is a match.
-- "student123" is a match.
-- "studentabc" is not a match because "abc" isn't a sequence of digits.
-- "student123abc" is not a match because no characters are allowed after `student` and the optional digits.
-
-Lastly, the `date`, `time`, and `datetime` inputs must be within a valid range between `after` and `before`. In the `date` example, the date provided must be between 10th April 2024 and 15th April 2024. Anything outside of this range will fail and the execution won't start.
-
-Try running this flow with various inputs or adjust the regex pattern to see how the input validation works.
-
## Nested inputs
-Using a `.` in an input `id` creates a nested input.
-
-Here's an example that includes 2 nested inputs:
+Use `.` in an input `id` to create a nested structure, accessible with the same dot notation in expressions:
```yaml
-id: nested_inputs
-namespace: company.team
-
inputs:
- - id: nested.string
+ - id: db.host
type: STRING
- required: false
+ defaults: localhost
- - id: nested.int
+ - id: db.port
type: INT
+ defaults: 5432
tasks:
- - id: log_inputs
+ - id: log
type: io.kestra.plugin.core.log.Log
- message: "{{ inputs.nested.string }} and {{ inputs.nested.int }}"
+ message: "Connecting to {{ inputs.db.host }}:{{ inputs.db.port }}"
```
-You can access the first input value using `{{ inputs.nested.string }}`. This provides type validation for nested inputs without resorting to raw JSON (JSON inputs are passed as strings).
-
## FORM inputs
-Use a `FORM` input to group related inputs under a shared label and description. When a flow has at least one FORM input, the Execute modal renders a multi-step wizard: one step per FORM group, a step for any ungrouped inputs, then a recap. Apps using `CreateExecutionForm` render the same wizard automatically.
+`FORM` groups related inputs under a shared label and renders a multi-step wizard in the Execute modal:
```yaml
id: provision_environment
@@ -319,7 +189,6 @@ inputs:
- id: requester
type: STRING
required: true
- description: Name or team submitting this request.
- id: environment
type: FORM
@@ -328,7 +197,6 @@ inputs:
inputs:
- id: region
type: SELECT
- required: true
defaults: eu-central-1
values:
- eu-central-1
@@ -337,560 +205,143 @@ inputs:
- id: instance_type
type: SELECT
- required: true
defaults: t3.medium
values:
- t3.medium
- t3.large
- t3.xlarge
- - id: notifications
- type: FORM
- displayName: Notifications
- description: Where to send status updates.
- inputs:
- - id: slack_channel
- type: STRING
- defaults: "#platform-ops"
-
- - id: notify_on_failure
- type: BOOL
- defaults: true
-
tasks:
- - id: log_request
+ - id: log
type: io.kestra.plugin.core.log.Log
message: |
- Requested by: {{ inputs.requester }}
+ Requester: {{ inputs.requester }}
Region: {{ inputs.environment.region }}
Instance: {{ inputs.environment.instance_type }}
- Slack: {{ inputs.notifications.slack_channel }}
-```
-
-Children are accessed via `{{ inputs.. }}`. In the example above, `region` inside the `environment` FORM is `{{ inputs.environment.region }}`.
-
-### dependsOn across FORM children
-
-To make one FORM child depend on another, use the full dotted path in `dependsOn`:
-
-```yaml
-inputs:
- - id: cloud
- type: FORM
- displayName: Cloud configuration
- inputs:
- - id: provider
- type: SELECT
- values: [AWS, GCP, Azure]
-
- - id: region
- type: SELECT
- dependsOn:
- inputs:
- - cloud.provider
- condition: "{{ inputs.cloud.provider == 'AWS' }}"
- values:
- - us-east-1
- - eu-west-1
```
-### Constraints
-
:::alert{type="warning"}
-- A FORM cannot contain another FORM — grouping is limited to one level.
-- A FORM cannot have `defaults` or `prefill` — those properties belong on the individual child inputs.
+A FORM cannot contain another FORM. `defaults` and `prefill` belong on individual child inputs, not on the FORM itself.
:::
-### API submission
-
-When triggering a flow with FORM inputs via the API, use flat dotted field names in the multipart form data. Kestra maps them to the nested execution context automatically.
+When triggering a flow with FORM inputs via the API, use flat dotted field names:
```bash
curl -X POST "http://localhost:8080/api/v1/main/executions/company.team/provision_environment" \
-H "Content-Type: multipart/form-data" \
-F "requester=platform-team" \
-F "environment.region=eu-central-1" \
- -F "environment.instance_type=t3.large" \
- -F "notifications.slack_channel=#ops" \
- -F "notifications.notify_on_failure=true"
+ -F "environment.instance_type=t3.large"
```
## Array inputs
-Array inputs are used to pass a list of values to a flow. The `itemType` property is required to ensure validation of the type of the array items.
-
-This is useful when you want the user triggering the workflow to provide multiple values of a specific type, for example, a list of integers, strings, booleans, datetimes, etc. You can provide the default values as a JSON array or as a YAML list — both are supported.
+`ARRAY` accepts a JSON array or YAML list. The `itemType` property is required:
```yaml
-id: array_demo
-namespace: company.team
-
inputs:
- - id: my_numbers_json_list
+ - id: ids
type: ARRAY
itemType: INT
defaults: [1, 2, 3]
-
- - id: my_numbers_yaml_list
- type: ARRAY
- itemType: INT
- defaults:
- - 1
- - 2
- - 3
-
-tasks:
- - id: print_status
- type: io.kestra.plugin.core.log.Log
- message: received inputs {{ inputs }}
```
-Below is how the array inputs are rendered in the UI when you create an execution:
-
-
+## Using inputs in a flow
-## Use an input value in a flow
-
-Inputs are available via `{{ inputs.name }}` or `{{ inputs['name'] }}`. If an input `id` contains characters like `-`, use the bracket form: `{{ inputs['name-example'] }}`.
-
-For example, if you declare the following inputs:
+Reference inputs with `{{ inputs.name }}` in any dynamic property. Use bracket notation for IDs containing hyphens or other special characters:
```yaml
inputs:
- - id: mystring
+ - id: message
type: STRING
- required: true
-
- - id: my-file
- type: FILE
-```
-You can use the value of the input `mystring` inside dynamic task properties with `{{ inputs.mystring }}` but `my-file` would have to use `{{ inputs['my-file'] }}` because of the hyphen (`-`).
-
-We can see a full example below where `inputFiles` property is set to `{{ inputs['my-file'] }}`:
-
-```yaml
-id: input_files
-namespace: company.team
-
-description: This flow shows how to pass files between inputs and tasks in Shell scripts.
-
-inputs:
- id: my-file
type: FILE
tasks:
- - id: rename
+ - id: use_inputs
type: io.kestra.plugin.scripts.shell.Commands
commands:
- - mv file.tmp output.tmp
+ - echo "{{ inputs.message }}"
inputFiles:
- file.tmp: "{{ inputs['my-file'] }}"
- outputFiles:
- - "*.tmp"
-```
-
-## Set input values at flow execution
-
-When executing a flow with inputs, you must provide all required inputs (unless a default is defined) for the execution to be created.
-
-Let's consider the following example that defines multiple inputs:
-
-```yaml
-id: kestra_inputs
-namespace: company.team
-
-inputs:
- - id: string
- type: STRING
- defaults: hello
-
- - id: optional
- type: STRING
- required: false
-
- - id: int
- type: INT
-
- - id: float
- type: FLOAT
-
- - id: instant
- type: DATETIME
-
- - id: file
- type: FILE
+ upload.tmp: "{{ inputs['my-file'] }}"
```
-Here, `string` and `optional` can be omitted because `string` has a default and `optional` is not required. All other inputs must be specified at runtime.
-
-### Set inputs from the web UI
+## Setting inputs at execution time
-When creating an execution from the web UI, you must set the inputs in the UI form. Kestra's UI generates a dedicated form based on your `inputs` definition. For example, datetime input properties have a date picker.
+Provide input values from the **UI** (Kestra generates a form based on your input definitions), the **API**, the **CLI** (`kestractl`), **Python** (`kestra` pip package), or any HTTP client. See [Execute a flow](../03.execution/index.md#execute-via-api) for full examples with `curl`, Python, and kestractl.
-The input form for the inputs above looks as follows:
+## Inputs vs. variables
-
-
-Once the inputs are set, you can trigger an execution of the flow.
-
-### Set inputs when executing the flow using the API
-
-To create an execution with these inputs using the API, we can use the `curl` command to make an API request:
-
-```bash
-curl -v "http://localhost:8080/api/v1/main/executions/example/kestra-inputs" \
- -H "Transfer-Encoding:chunked" \
- -H "Content-Type:multipart/form-data" \
- -F string="a string" \
- -F optional="an optional string" \
- -F int=1 \
- -F float=1.255 \
- -F instant="2023-12-24T23:00:00.000Z" \
- -F "files=@/tmp/128M.txt;filename=file"
-```
-
-Send files as `multipart/form-data` under the `files` field with `filename=""`, where `` is the input name.
-
-### Set inputs when executing the flow in Python
-
-To create an execution with these inputs in Python, you can use the following script:
-
-```python
-import io
-import requests
-from kestra import Flow
-
-flow = Flow()
-
-with open('/tmp/example.txt', 'rb') as fh:
- flow.execute('example',
- 'kestra-inputs',
- {'string': 'a string',
- 'optional': 'an optional string',
- 'int': 1,
- 'float': str(1.255),
- 'instant': '2020-01-14T23:00:00.000Z',
- 'files': ('file', fh, 'text/plain')})
-```
-
-:::alert{type="info"}
-Wrap floats with `str()`; otherwise, a bytes-like object error may occur when sending a file input.
-:::
-
-You can also use the `requests` library in Python to make requests to the Kestra API. Here's an example to execute a flow with multiple inputs:
-
-```python
-import io
-import requests
-from requests_toolbelt.multipart.encoder import MultipartEncoder
-
-with open("/tmp/128M.txt", 'rb') as fh:
- url = f"http://kestra:8080/api/v1/main/executions/io.kestra.docs/my-flow"
- mp_encoder = MultipartEncoder(fields={
- "string": "a string",
- "optional": "an optional string",
- "int": 1,
- "float": 1.255,
- "instant": "2020-01-14T23:00:00.000Z",
- "files": ("file", fh, "text/plain")
- })
- result = requests.post(
- url,
- data=mp_encoder,
- headers={"Content-Type": mp_encoder.content_type},
- )
-```
-
-### Set inputs when executing the flow in Java
-
-To create an execution with these inputs in Java (with [Apache Http Client 5](https://hc.apache.org/index.html)), you can use the following script:
-
-```java
-import org.apache.hc.client5.http.classic.methods.HttpPost;
-import org.apache.hc.client5.http.entity.mime.FileBody;
-import org.apache.hc.client5.http.entity.mime.MultipartEntityBuilder;
-import org.apache.hc.client5.http.entity.mime.StringBody;
-import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
-import org.apache.hc.client5.http.impl.classic.CloseableHttpResponse;
-import org.apache.hc.client5.http.impl.classic.HttpClientBuilder;
-import org.apache.hc.core5.http.ContentType;
-import org.apache.hc.core5.http.HttpEntity;
-
-import java.io.File;
-
-class Application {
- public static void main(String[] args) {
- HttpEntity multipartEntity = MultipartEntityBuilder.create()
- .addPart("string", new StringBody("test", ContentType.DEFAULT_TEXT))
- .addPart("int", new StringBody("1", ContentType.DEFAULT_TEXT))
- .addPart("files", new FileBody(new File("/tmp/test.csv"), ContentType.DEFAULT_TEXT, "file"))
- .build();
-
- try (CloseableHttpClient httpclient = HttpClientBuilder.create().build()) {
- HttpPost request = new HttpPost("http://kestra:8080/api/v1/main/executions/com.kestra.lde/inputs");
- request.setEntity(multipartEntity);
-
- CloseableHttpResponse response = httpclient.execute(request);
-
- System.out.println("Response " + response);
- } catch (Exception e) {
- throw new RuntimeException(e);
- }
- }
-}
-```
-
-## Difference between inputs and variables
-
-[Variables] are similar to constants. They behave like inputs during execution but cannot be overridden once the execution starts. Variables must be defined before execution, whereas inputs can be set at execution time.
-
-Variables are best suited for values that you don't want to change and are used in multiple places within the flow. For example, a URL you use for an API request that won't change is best as a variable whereas an email address that changes every time you execute your flow is best as an input.
+[Variables](../04.variables/index.md) are defined before execution and cannot be changed once it starts. Inputs are provided at execution time and can differ between runs. Use variables for fixed values reused across tasks; use inputs for values that change per execution.
## Dynamic inputs
-
-
-
-
-Inputs in Kestra are strongly typed. Currently, you cannot enforce strong types and simultaneously use dynamically rendered Pebble expressions.
-
-The example below demonstrates using an expression inside of an input. When you select execute, the expression is rendered.
+`SELECT` and `MULTISELECT` inputs support an `expression` property that populates the dropdown from a Pebble expression — a KV store lookup, an HTTP API call, or a subflow result:
```yaml
-id: test
-namespace: company.team
-
inputs:
- - id: date
- type: DATETIME
- defaults: "{{ now() }}"
-
-tasks:
- - id: print_date
- type: io.kestra.plugin.core.log.Log
- message: "hello on {{ inputs.date }}"
+ - id: environment
+ type: SELECT
+ expression: "{{ kv('ENVIRONMENTS') }}"
```
-### Dynamic Inputs with HTTP function
+See the [Dynamic inputs how-to guide](../../15.how-to-guides/dynamic-inputs/index.md) for HTTP function examples, subflow-populated dropdowns, and chaining dependent dropdowns.
-With the `http()` function, you can make `SELECT` and `MULTISELECT` inputs dynamic by fetching options from an external API. This proves valuable when your data used in dropdowns changes frequently or when you already have an API serving that data for existing applications.
+## Conditional inputs
-The example below demonstrates how to create a flow with two dynamic dropdowns: one for selecting a product category and another for selecting a product from that category. The first dropdown fetches the product categories from an external HTTP API. The second dropdown makes another HTTP call to dynamically retrieve products matching the selected category.
+Use `dependsOn` and `condition` to show inputs only when a previous input matches a value:
```yaml
-id: dynamic_dropdowns
-namespace: company.team
inputs:
- - id: category
- type: SELECT
- expression: "{{ http(uri = 'https://dummyjson.com/products/categories') | jq('.[].slug') }}"
- - id: product
- type: SELECT
+ - id: notify
+ type: BOOL
+ defaults: false
+
+ - id: slack_channel
+ type: STRING
dependsOn:
inputs:
- - category
- expression: "{{ http(uri = 'https://dummyjson.com/products/category/' + inputs.category) | jq('.products[].title') }}"
-tasks:
- - id: display_selection
- type: io.kestra.plugin.core.log.Log
- message: |
- You selected Category: {{ inputs.category }}
- And Product: {{ inputs.product }}
+ - notify
+ condition: "{{ inputs.notify == true }}"
```
----
+`slack_channel` only appears in the Execute modal when `notify` is `true`. See the [Dynamic inputs how-to guide](../../15.how-to-guides/dynamic-inputs/index.md) for full conditional provisioning examples.
-Dynamic inputs are useful for flows using authenticated API requests like the following:
+## Label/value pairs in SELECT and MULTISELECT
-```yaml
-id: approversFlow
-namespace: company.team
-
-inputs:
- - id: executionIdsToBeApproved
- type: MULTISELECT
- expression: >-
- {{
- http(
- uri = 'http://localhost:8080/api/v1/internal/executions/search?state=PAUSED',
- method = 'GET',
- contentType = 'application/json',
- headers={
- 'User-Agent': 'kestra',
- 'Connection': 'keep-alive',
- 'Authorization': 'Bearer ' ~ secret("bearerToken")
- }
- ) | jq('.results[] | "ExecutionId: \(.id), FlowId: \(.flowId), RequestedBy: \(.labels[] | select(.key == "system.username").value) InputParams: \( .inputs | to_entries | map("\(.key):\(.value)") | join(" ") )"') }}
-
-tasks:
- - id: hello
- type: io.kestra.plugin.core.log.Log
- message: Hello World! 🚀
-```
-
-:::alert{type="info"}
-When using `http()` inside an `expression` with secrets in headers (e.g., an authenticated API request), use named arguments and string concatenation ([Pebble Literals](https://pebbletemplates.io/wiki/guide/basic-usage/#literals)). The key to the syntax is to use string interpolation with `~`.
-:::
-
-### Dynamic inputs from a subflow
-
-For cases that require complex logic — running a script, calling a CLI command, or executing multi-step tasks — use the `subflow()` Pebble function in the `expression:` property. `subflow()` runs a flow synchronously at form render time and populates the dropdown from its outputs:
-
-```yaml
-inputs:
- - id: region
- type: SELECT
- expression: "{{ subflow(namespace='company.ops', id='fetch_regions').outputs.region_list }}"
-```
-
-See [Populate a dropdown from a subflow](../../15.how-to-guides/dynamic-inputs/index.md#populate-a-dropdown-from-a-subflow) for a full example and constraints.
-
-## Conditional inputs for interactive workflows
-
-You can set up inputs that depend on other inputs, letting further inputs be conditionally displayed based on user choices. This is useful for use cases such as approval workflows or dynamic resource provisioning.
-
-### How it works
-
-Create inputs that change based on other inputs using the `dependsOn` and `condition` properties. The example below shows different inputs appearing based on the selected resource type:
+Each entry in `values` can be a plain string or a `{label, value}` object. The UI shows `label`; `{{ inputs.x }}` resolves to `value`:
```yaml
-id: request_resources
-namespace: company.team
-
inputs:
- - id: resource_type
- displayName: Resource type
- type: SELECT
- values:
- - Access permissions
- - SaaS application
- - Development tool
- - Cloud VM
-
- - id: access_permissions
- displayName: Access permissions
- type: SELECT
- expression: "{{ kv('access_permissions') }}"
- dependsOn:
- inputs:
- - resource_type
- condition: "{{ inputs.resource_type == 'Access permissions' }}"
-
- - id: saas_applications
- displayName: SaaS spplication
- type: MULTISELECT
- expression: "{{ kv('saas_applications') }}"
- dependsOn:
- inputs:
- - resource_type
- condition: "{{ inputs.resource_type == 'SaaS application' }}"
-
- - id: cloud_provider
- displayName: Cloud provider
+ - id: aws_account
type: SELECT
+ displayName: AWS Account
values:
- - AWS
- - GCP
- - Azure
- dependsOn:
- inputs:
- - resource_type
- condition: "{{ inputs.resource_type == 'Cloud VM' }}"
-
- - id: cloud_vms
- displayName: Cloud VM
- type: SELECT
- expression: "{{ kv('cloud_vms')[inputs.cloud_provider] }}"
- dependsOn:
- inputs:
- - resource_type
- - cloud_provider
- condition: "{{ inputs.resource_type == 'Cloud VM' }}"
-```
-
-
-In this example:
-- The `resource_type` input controls which additional inputs (such as `access_permissions`, `saas_applications`, and `cloud_vms`) appear.
-- `dependsOn` links inputs; `condition` defines when to display the related input.
-
-Before running the flow, set up the key-value pairs for each input. Expand the example below to add all key-value pairs with a helper flow.
-
-:::collapse{title="Flow adding key-value pairs"}
-```yaml
-id: add_kv_pairs
-namespace: company.team
+ - label: "Production"
+ value: "123456789012"
+ - label: "Staging"
+ value: "987654321098"
tasks:
- - id: access_permissions
- type: io.kestra.plugin.core.kv.Set
- key: "{{ task.id }}"
- kvType: JSON
- value: |
- ["Admin", "Developer", "Editor", "Launcher", "Viewer"]
-
- - id: saas_applications
- type: io.kestra.plugin.core.kv.Set
- key: "{{ task.id }}"
- kvType: JSON
- value: |
- ["Slack", "Notion", "HubSpot", "GitHub", "Jira"]
-
- - id: development_tools
- type: io.kestra.plugin.core.kv.Set
- key: "{{ task.id }}"
- kvType: JSON
- value: |
- ["Cursor", "IntelliJ IDEA", "PyCharm Professional", "DataGrip"]
-
- - id: cloud_vms
- type: io.kestra.plugin.core.kv.Set
- key: "{{ task.id }}"
- kvType: JSON
- value: |
- {
- "AWS": ["t2.micro", "t2.small", "t2.medium", "t2.large"],
- "GCP": ["f1-micro", "g1-small", "n1-standard-1", "n1-standard-2"],
- "Azure": ["Standard_B1s", "Standard_B1ms", "Standard_B2s", "Standard_B2ms"]
- }
-
- - id: cloud_regions
- type: io.kestra.plugin.core.kv.Set
- key: "{{ task.id }}"
- kvType: JSON
- value: |
- {
- "AWS": ["us-east-1", "us-west-1", "us-west-2", "eu-west-1"],
- "GCP": ["us-central1", "us-east1", "us-west1", "europe-west1"],
- "Azure": ["eastus", "westus", "centralus", "northcentralus"]
- }
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Account ID: {{ inputs.aws_account }}"
```
-:::
-
-You can also [add these key-value pairs](../../06.concepts/05.kv-store/index.md) via the API or the UI.
-## Custom values in SELECT and MULTISELECT inputs
+`defaults`, `autoSelectFirst`, and validation all operate on the `value` field, not the `label`.
-If the predefined dropdown values do not fit a user’s needs, set `allowCustomValue` to `true` to allow custom entries. This lets you offer defaults while still accepting user-provided values.
+## Custom values in SELECT and MULTISELECT
-In the example below, `cloud_provider` lets users select a common provider (AWS, GCP, Azure) or enter a custom value (e.g., Oracle Cloud).
+Set `allowCustomValue: true` to let users enter a value outside the predefined list:
```yaml
-id: custom_values
-namespace: company.team
-
inputs:
- id: cloud_provider
- displayName: Cloud provider
type: SELECT
allowCustomValue: true
values:
- AWS
- GCP
- Azure
-
-tasks:
- - id: print_status
- type: io.kestra.plugin.core.log.Log
- message: Selected cloud provider {{ inputs.cloud_provider }}
```
diff --git a/src/contents/docs/05.workflow-components/06.outputs/encrypted-outputs.png b/src/contents/docs/05.workflow-components/06.outputs/encrypted-outputs.png
deleted file mode 100644
index d9268242f35..00000000000
Binary files a/src/contents/docs/05.workflow-components/06.outputs/encrypted-outputs.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/06.outputs/index.md b/src/contents/docs/05.workflow-components/06.outputs/index.md
index 494d341c1f4..f4775ac5aa9 100644
--- a/src/contents/docs/05.workflow-components/06.outputs/index.md
+++ b/src/contents/docs/05.workflow-components/06.outputs/index.md
@@ -12,23 +12,15 @@ Outputs let you pass data between tasks and flows.
-A workflow execution can generate **outputs**. Outputs are stored in the flow’s execution context and can be accessed by all downstream tasks and flows.
-
-Each task defines its own output attributes — see the task’s documentation for details.
-
-You can retrieve outputs from other tasks within all [dynamic properties](../01.tasks/index.mdx#dynamic-vs-static-task-properties).
+Outputs are stored in the flow's execution context and accessible by all downstream tasks and flows. Each task defines its own output attributes — see the task's plugin documentation for details, or inspect them in the **Input/Output** tab of the **Execution** page.
:::alert{type="warning"}
-**Do not use Outputs to fetch sensitive data (such as passwords, secrets, or API tokens).**
-
-Fetching Secrets from an external Secrets Manager via a task imposes a significant security risk. All data fetched via outputs is **stored in clear text in multiple places** (including the backend database, internal storage, logs, API requests).
-
-For secure handling of secrets, **exclusively** use [Secrets](../../06.concepts/04.secret/index.md). [Kestra EE](../../07.enterprise/02.governance/secrets-manager/index.md) and [Kestra Cloud](/cloud) offer reliable secrets management including native integrations with various [secrets managers](../../07.enterprise/02.governance/secrets-manager/index.md).
+Do not use outputs to fetch sensitive data such as passwords, secrets, or API tokens. All data fetched via outputs is stored in clear text in the backend database, internal storage, logs, and API responses. Use [Secrets](../../06.concepts/04.secret/index.md) instead. Enterprise Edition and Kestra Cloud offer native integrations with [external secrets managers](../../07.enterprise/02.governance/secrets-manager/index.md).
:::
## Using outputs
-Below is an example of how to use the output of the `produce_output` task in the `use_output` task. We use the [Return](/plugins/core/debug/io.kestra.plugin.core.debug.return) task that has one output attribute named `value`.
+Reference a previous task's output with `{{ outputs.. }}` in any dynamic property:
```yaml
id: task_outputs_example
@@ -44,50 +36,13 @@ tasks:
message: The previous task output is {{ outputs.produce_output.value }}
```
-In this example, the first task produces an output from the `format` property. This output attribute is then used in the second task `message` property.
-
-The expression `{{ outputs.produce_output.value }}` references the previous task output attribute.
-
-:::alert{type="info"}
-In the example above, the **Return** task produces an output attribute `value`. Every task produces different output attributes. You can look at each task outputs documentation or use the **Outputs** tab of the **Executions** page to find out about specific task output attributes.
-:::
-
-The **Outputs** tab shows the output for `produce_output` task. There is no output for `use_output` task as it only logs a message.
-
-
+File outputs are previewable and downloadable from the execution's **Input/Output** tab; any output can be inspected and debugged using the built-in expression evaluator on the same tab. See [Outputs](../../03.tutorial/03.outputs/index.md) in the tutorial if you're unfamiliar with either.
-In the next example, we can see a file is passed between an input and a task, where the task generates a new file as an output:
-
-```yaml
-id: bash_with_files
-namespace: company.team
-
-description: This flow shows how to pass files between inputs and tasks in Shell scripts.
-
-inputs:
- - id: file
- type: FILE
-
-tasks:
- - id: rename
- type: io.kestra.plugin.scripts.shell.Commands
- commands:
- - mv file.tmp output.tmp
- inputFiles:
- file.tmp: "{{ inputs.file }}"
- outputFiles:
- - "*.tmp"
-```
-
-:::alert{type="info"}
-Since 0.14, Outputs are no longer rendered recursively. You can read more about this change and how to change this behavior in the [0.14 Migration guide](../../11.migration-guide/v0.14.0/recursive-rendering/index.md).
-:::
+For loop iteration outputs, sibling task outputs, and the `loopOutputs()` function, see [Flowable tasks](../01.tasks/00.flowable-tasks/index.md#loop) and the [Loop how-to guide](../../15.how-to-guides/loop/index.md).
## Internal storage
-Each task can store data in Kestra's internal storage. If an output is stored in internal storage, it contains a URI pointing to the file location. This output attribute could be used by other tasks to access the stored data.
-
-The following example stores the query results in internal storage, then accesses it in the `write_to_csv` task:
+Tasks that produce large results write them to Kestra's internal storage and return a URI. Pass that URI to downstream tasks:
```yaml
id: output_sample
@@ -110,9 +65,7 @@ tasks:
## Flow outputs
-A flow can also produce strongly typed outputs. You can add them using the `outputs` attribute in the flow definition.
-
-Here is an example of a flow that produces an output:
+Flows can declare strongly typed outputs that are surfaced in the **Overview** tab and accessible to parent flows via the Subflow task:
```yaml
id: flow_outputs
@@ -129,17 +82,11 @@ outputs:
value: "{{ outputs.mytask.value }}"
```
-An Output can have one of the following types: `ARRAY`, `BOOLEAN`, `DATE`, `DATETIME`, `DURATION`, `EMAIL`, `ENUM`, `FILE`, `FLOAT`, `INT`, `JSON`, `MULTISELECT`, `SECRET`, `STRING`, `TIME`, `URI`, or `YAML`.
-
-Outputs are defined as a list of key-value pairs. The `id` is the name of the output attribute (must be unique within a flow), and the `value` is the value of the output. You can also add a `description` to the output.
-
-Flow outputs appear in the **Overview** tab of the **Executions** page.
-
-
+Supported output types: `ARRAY`, `BOOLEAN`, `DATE`, `DATETIME`, `DURATION`, `EMAIL`, `ENUM`, `FILE`, `FLOAT`, `INT`, `JSON`, `MULTISELECT`, `SECRET`, `STRING`, `TIME`, `URI`, `YAML`.
-### Pass data between flows using flow outputs
+### Pass data between flows
-Here is how you can access the flow output in a parent flow:
+Access a child flow's declared outputs in the parent via `{{ outputs..outputs. }}`:
```yaml
id: parent_flow
@@ -157,21 +104,11 @@ tasks:
message: "{{ outputs.subflow.outputs.final }}"
```
-In the example above, the `subflow` task produces an output attribute `final`. This output attribute is then used in the `log_subflow_output` task.
-
-:::alert{type="info"}
-Note how the `outputs` are set twice within the `"{{outputs.subflow.outputs.final}}"`:
-1. once to access outputs of the `subflow` task
-2. once to access the outputs of the subflow itself — specifically, the `final` output
-:::
-
-Here is what you will see in the **Outputs** tab of the **Executions** page in the parent flow:
-
-
+The double `outputs` is intentional: the first accesses the Subflow task's outputs, the second accesses the child flow's declared output `final`.
### Return outputs conditionally
-You can return different outputs based on conditions. For instance, if a given task is skipped, you may want to return a fallback value or return the output of another task. Here is an example of how you can achieve this:
+Use a ternary expression to return different outputs based on task state:
```yaml
id: conditionally_return_output
@@ -179,7 +116,7 @@ namespace: company.team
inputs:
- id: run_task
- type: BOOLEAN
+ type: BOOL
defaults: true
tasks:
@@ -198,237 +135,15 @@ outputs:
value: "{{ tasks.main.state != 'SKIPPED' ? outputs.main.value : outputs.fallback.value }}"
```
-Note how the Ternary Operator `{{ condition ? value_if_true : value_if_false }}` is used in the output expression `{{ tasks.main.state != 'SKIPPED' ? outputs.main.value : outputs.fallback.value }}` to return the output of the `main` task if it is not skipped, otherwise, it returns the output of the `fallback` task.
-
-## Dynamic variables (Each tasks)
-
-### Current taskrun value
-
-In dynamic flows (for example, with an **Each** loop), variables are passed to tasks dynamically. You can access the current taskrun value with `{{ taskrun.value }}` like this:
-
-```yaml
-id: taskrun_value_example
-namespace: company.team
-
-tasks:
- - id: each
- type: io.kestra.plugin.core.flow.ForEach
- values: ["alpha", "beta", "gamma"]
- tasks:
- - id: inner
- type: io.kestra.plugin.core.debug.Return
- format: "{{ task.id }} > {{ taskrun.value }} > {{ taskrun.startDate }}"
-```
-
-The **Outputs** tab contains the output for each of the inner task.
-
-
-
-### Loop over a list of JSON objects
-
-Within the loop, the `value` is always a JSON string, so the `{{ taskrun.value }}` is the current element as JSON string. To access properties, you need to wrap it in the `fromJson()` function to have a JSON object allowing to access each property easily.
-
-```yaml
-id: loop_sequentially_over_list
-namespace: company.team
-
-tasks:
- - id: each
- type: io.kestra.plugin.core.flow.ForEach
- values:
- - {"key": "my-key", "value": "my-value"}
- - {"key": "my-complex", "value": {"sub": 1, "bool": true}}
- tasks:
- - id: inner
- type: io.kestra.plugin.core.debug.Return
- format: "{{ fromJson(taskrun.value).key }} > {{ fromJson(taskrun.value).value }}"
-```
-
-
-### Specific outputs for dynamic tasks
-
-Dynamic tasks are tasks that run other tasks a certain number of times. A dynamic task runs multiple iterations of a set of sub-tasks.
-
-For example, **ForEach** produces other tasks dynamically depending on its `values` property.
-
-It is possible to reach each iteration output of dynamic tasks by using the following syntax:
-
-```yaml
-id: output_sample
-namespace: company.team
-
-tasks:
- - id: each
- type: io.kestra.plugin.core.flow.ForEach
- values: ["s1", "s2", "s3"]
- tasks:
- - id: sub
- type: io.kestra.plugin.core.debug.Return
- format: "{{ task.id }} > {{ taskrun.value }} > {{ taskrun.startDate }}"
-
- - id: use
- type: io.kestra.plugin.core.debug.Return
- format: "Previous task produced output: {{ outputs.sub.s1.value }}"
-```
-
-The `outputs.sub.s1.value` variable reaches the `value` of the `sub` task of the `s1` iteration.
-
-### Previous task lookup
-
-It is also possible to locate a specific dynamic task by its `value`:
-
-```yaml
-id: dynamic_looping
-namespace: company.team
-
-tasks:
- - id: each
- type: io.kestra.plugin.core.flow.ForEach
- values: ["alpha", "beta", "gamma"]
- tasks:
- - id: inner
- type: io.kestra.plugin.core.debug.Return
- format: "{{ taskrun.value }}"
-
- - id: end
- type: io.kestra.plugin.core.debug.Return
- format: "{{ task.id }} > {{ outputs.inner['alpha'].value }}"
-```
-
-It uses the format `outputs.TASKID[VALUE].ATTRIBUTE`. The special bracket `[]` in `[VALUE]` is called the subscript notation; it enables using special chars like space or '-' in task identifiers or output attributes.
-
-### Lookup in sibling tasks
-
-Sometimes it is useful to access outputs from other tasks in the same task tree, known as sibling tasks.
-
-If the task tree is static, for example when using the [Sequential](/plugins/core/flow/io.kestra.plugin.core.flow.sequential) task, you can use the `{{ outputs.task_id.value }}` notation where `task_id` is the identifier of the sibling task, as you would outside of the task tree.
-
-For example:
-
-```yaml
-id: sibling_tasks
-namespace: company.team
-
-tasks:
- - id: sequential
- type: io.kestra.plugin.core.flow.Sequential
- tasks:
- - id: first
- type: io.kestra.plugin.core.output.OutputValues
- values:
- data: "hello from task 1"
-
- - id: second
- type: io.kestra.plugin.core.output.OutputValues
- values:
- data: "{{ outputs.first.values.data }}"
-
- - id: log_siblings
- type: io.kestra.plugin.core.log.Log
- message: "{{ outputs.second.values.data }}"
-```
-
-If the task tree is dynamic, for example when using the [ForEach](/plugins/core/flow/io.kestra.plugin.core.flow.foreach) task, you need to use `{{ outputs.task_id[taskrun.value] }}` to access the current tree task. `taskrun.value` is a special variable that holds the current value of the ForEach task.
-
-For example:
-
-```yaml
-id: loop_with_sibling_tasks
-namespace: company.team
-
-tasks:
- - id: foreach
- type: io.kestra.plugin.core.flow.ForEach
- values: ["alpha", "beta", "gamma"]
- tasks:
- - id: first
- type: io.kestra.plugin.core.output.OutputValues
- values:
- data: "First value: {{ taskrun.value }}"
-
- - id: second
- type: io.kestra.plugin.core.output.OutputValues
- values:
- data: "{{ outputs.first[taskrun.value].values.data }}"
-
- - id: log_output_from_foreach
- type: io.kestra.plugin.core.log.Log
- message: "{{ outputs.second['alpha'].values.data }}"
-```
-
-You can also use the `currentEachOutput` function to access the current tree task. See [Function Reference](../../expressions/04.functions/index.mdx) for more details.
-
-:::alert{type="warning"}
-Accessing sibling task outputs is impossible on [Parallel](/plugins/core/flow/io.kestra.plugin.core.flow.parallel) as it runs tasks in parallel.
-:::
-
-For more examples and guidance on accessing sibling outputs inside `ForEach`, including how to read them both inside and outside the loop, see [Best Practices for ForEach and ForEachItem](../../14.best-practices/11.foreach-and-foreachitem/index.md#example-use-sibling-outputs-correctly-inside-foreach).
-
-## Outputs preview
-
-Kestra provides a preview option for output files stored in internal storage. The following flow demonstrates this feature:
-
-```yaml
-id: get_employees
-namespace: company.team
-
-tasks:
- - id: download
- type: io.kestra.plugin.core.http.Download
- uri: https://huggingface.co/datasets/kestra/datasets/raw/main/ion/employees.ion
-```
-
-On flow execution, the file is downloaded into the Kestra internal storage. When you go to the Outputs tab for this execution, the `uri` attribute of the `download` task contains the file location on Kestra's internal storage and has a Download and a Preview button.
-
-
-
-On clicking the Preview button, you can preview the contents of the file in a tabular format, making it extremely easy to check the contents of the file without downloading it.
-
-
-
-## Using debug expression
-
-You can evaluate the output further using the **Debug Expression** functionality in the **Outputs** tab. Consider the following flow:
-
-```yaml
-id: json_values
-namespace: company.team
-
-tasks:
-- id: sample_json
- type: io.kestra.plugin.core.debug.Return
- format: '{"data": [1, 2, 3]}'
-```
-
-When you run this flow, the **Outputs** tab will contain the output for the `sample_json` task, as shown below:
-
-
-
-You can select the task from the drop-down menu. Here, we select "sample_json" and select **Debug Expression**:
-
-
-
-You can now use Pebble expressions to evaluate and analyze the output data further.
-
-
-
-
-
-:::alert{type="info"}
-Note: This was previously called **Render expression**.
-:::
-
-## Encrypted outputs from script tasks
+## Encrypted outputs
:::badge{version=">=0.23" editions="EE,Cloud"}
:::
-For [script task Outputs](../../16.scripts/06.outputs-metrics/index.md) that have sensitive values, you can protect the information by using the `encryptedOutputs` syntax such as `::{"encryptedOutputs":{"encrypted":"my secret value"}}::`.
-
-In the following flow, the `encrypted` output is not shown in plain text in the Outputs UI.
+Script tasks can emit encrypted outputs that are masked in the UI using the `encryptedOutputs` syntax:
```yaml
-id: encryped_output
+id: encrypted_output
namespace: company.team
tasks:
@@ -443,6 +158,4 @@ tasks:
message: "{{ outputs.hello['vars']['encrypted'] }}"
```
-The `encrypted` output is displayed encoded:
-
-
+The `encrypted` output is stored and displayed encoded. See [Script outputs and metrics](../../16.scripts/06.outputs-metrics/index.md) for full details.
diff --git a/src/contents/docs/05.workflow-components/06.outputs/json_values.png b/src/contents/docs/05.workflow-components/06.outputs/json_values.png
deleted file mode 100644
index 02900cd3edf..00000000000
Binary files a/src/contents/docs/05.workflow-components/06.outputs/json_values.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/06.outputs/json_values_render_expression.png b/src/contents/docs/05.workflow-components/06.outputs/json_values_render_expression.png
deleted file mode 100644
index 184aa720aa8..00000000000
Binary files a/src/contents/docs/05.workflow-components/06.outputs/json_values_render_expression.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/06.outputs/preview.png b/src/contents/docs/05.workflow-components/06.outputs/preview.png
deleted file mode 100644
index fc23cdccafe..00000000000
Binary files a/src/contents/docs/05.workflow-components/06.outputs/preview.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/06.outputs/preview_button.png b/src/contents/docs/05.workflow-components/06.outputs/preview_button.png
deleted file mode 100644
index 4dfa85ee10f..00000000000
Binary files a/src/contents/docs/05.workflow-components/06.outputs/preview_button.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/06.outputs/subflow_output.png b/src/contents/docs/05.workflow-components/06.outputs/subflow_output.png
deleted file mode 100644
index 45a76ba2e4c..00000000000
Binary files a/src/contents/docs/05.workflow-components/06.outputs/subflow_output.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/06.outputs/subflow_output_parent.png b/src/contents/docs/05.workflow-components/06.outputs/subflow_output_parent.png
deleted file mode 100644
index c1476fc371a..00000000000
Binary files a/src/contents/docs/05.workflow-components/06.outputs/subflow_output_parent.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/06.outputs/task_outputs_example.png b/src/contents/docs/05.workflow-components/06.outputs/task_outputs_example.png
deleted file mode 100644
index 818fc213952..00000000000
Binary files a/src/contents/docs/05.workflow-components/06.outputs/task_outputs_example.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/06.outputs/taskrun_value_example.png b/src/contents/docs/05.workflow-components/06.outputs/taskrun_value_example.png
deleted file mode 100644
index 70af264c260..00000000000
Binary files a/src/contents/docs/05.workflow-components/06.outputs/taskrun_value_example.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/07.checks/checks-fail.png b/src/contents/docs/05.workflow-components/07.checks/checks-fail.png
index 554648e66d5..e58040612bf 100644
Binary files a/src/contents/docs/05.workflow-components/07.checks/checks-fail.png and b/src/contents/docs/05.workflow-components/07.checks/checks-fail.png differ
diff --git a/src/contents/docs/05.workflow-components/07.checks/index.md b/src/contents/docs/05.workflow-components/07.checks/index.md
index e5949695a61..1d7feec27d2 100644
--- a/src/contents/docs/05.workflow-components/07.checks/index.md
+++ b/src/contents/docs/05.workflow-components/07.checks/index.md
@@ -1,45 +1,34 @@
---
title: Checks in Kestra – Pre-Execution Validations
h1: Validate Inputs Before Any Task Runs with Checks
-description: Implement Checks in Kestra for pre-execution validation. Guard your workflows by enforcing conditions on inputs before any task begins execution.
+description: Use checks to enforce conditions on inputs before any task runs, blocking or failing executions that don't meet your criteria.
sidebarTitle: Checks
icon: /src/contents/docs/icons/flow.svg
version: ">= 1.2.0"
---
-Add pre-execution validations that can block or fail an execution before any tasks run.
-
-## Add checks to validate inputs before execution
-
-`checks` are flow-level assertions evaluated when validating inputs and before creating a new execution. Each check defines a boolean `condition` and a `message` shown when the condition is false. You can choose how Kestra reacts (block, fail, or still create the execution) and how the message is styled in the UI.
-
-Checks are useful to enforce business rules on inputs (e.g., allowed values, date windows, required flags) or to nudge users with warnings before they launch a run.
+Checks are flow-level assertions evaluated against inputs before an execution is created. Each check defines a boolean `when` expression and a `message` to display when the expression evaluates to false.
## Properties
-Each item in `checks` supports the following properties:
+| Property | Required | Default | Description |
+|---|---|---|---|
+| `when` | Yes | — | Pebble expression that must evaluate to a boolean. Can reference inputs, KV pairs, and other [expression](../../expressions/index.mdx) variables. |
+| `message` | Yes | — | Text displayed in the Execute modal when the condition is false. |
+| `style` | No | `INFO` | Visual style for the message: `ERROR`, `SUCCESS`, `WARNING`, or `INFO`. |
+| `behavior` | No | `BLOCK_EXECUTION` | How the flow reacts when the condition is false: `BLOCK_EXECUTION` (do not create), `FAIL_EXECUTION` (create in failed state), or `CREATE_EXECUTION` (create anyway). |
-- `condition` *(required)*: Pebble expression that must evaluate to a boolean `true`. For example, you can design checks against inputs, key-value pairs, or other [expression](../../expressions/index.mdx) accessible workflow components.
-- `message` *(required)*: Text displayed when the condition is false.
-- `style` *(optional, default `INFO`)*: Visual style for the message. One of `ERROR`, `SUCCESS`, `WARNING`, `INFO`.
-- `behavior` *(optional, default `BLOCK_EXECUTION`)*: How the flow should react when the condition is false. One of:
- - `BLOCK_EXECUTION`: Do not create the execution.
- - `FAIL_EXECUTION`: Create the execution immediately in a failed state.
- - `CREATE_EXECUTION`: Allow execution creation even if the check fails.
-
-When clicking **Execute**, with an `ERROR` message display set in the flow code, the modal will display the `message` as soon as an input is set that doesn't satisfy the check like below:
+When you click **Execute**, the modal displays the `message` as soon as an input fails a check:
----
-
-### Multiple checks
+## Multiple checks
If several checks fail, the most restrictive behavior wins in this priority order: `BLOCK_EXECUTION` → `FAIL_EXECUTION` → `CREATE_EXECUTION`. This lets you mix hard stops with softer warnings in the same flow.
-### Evaluation behavior
+## Evaluation behavior
-Keep these rules in mind when writing `condition` expressions:
+Keep these rules in mind when writing `when` expressions:
- **The condition must evaluate to a boolean `true`.** Only a real boolean passes — not the string `"true"`, `"yes"`, a number, or any other truthy value. Use comparisons and boolean operators (e.g. `{{ inputs.age >= 18 }}`) rather than returning a string.
- **An unevaluatable condition always blocks.** If the condition cannot be evaluated (for example, an undefined variable or a syntax error), the check fails safe: the execution is hard-blocked with `BLOCK_EXECUTION` and an `ERROR` style, regardless of the `behavior` and `style` you declared. Fix the expression and reference only variables that exist at validation time to restore your declared behavior.
@@ -60,7 +49,7 @@ inputs:
checks:
- message: "Sorry, this flow can only be executed with 'Kestra'"
- condition: "{{ (inputs.name | upper) == 'KESTRA' }}"
+ when: "{{ (inputs.name | upper) == 'KESTRA' }}"
style: ERROR
behavior: BLOCK_EXECUTION
@@ -93,13 +82,13 @@ inputs:
checks:
# Block risky prod runs outside the allowed window
- message: "Prod runs are only allowed between 06:00 and 22:00 UTC"
- condition: "{{ inputs.environment != 'prod' or (inputs.run_date | date('HH') | number >= 6 and inputs.run_date | date('HH') | number < 22) }}"
+ when: "{{ inputs.environment != 'prod' or (inputs.run_date | date('HH') | number >= 6 and inputs.run_date | date('HH') | number < 22) }}"
style: ERROR
behavior: BLOCK_EXECUTION
# Warn if the payload is not the approved source
- message: "Non-approved source detected. Use https://dummyjson.com when possible."
- condition: "{{ inputs.payload_url | startsWith('https://dummyjson.com') }}"
+ when: "{{ inputs.payload_url | startsWith('https://dummyjson.com') }}"
style: WARNING
behavior: CREATE_EXECUTION
diff --git a/src/contents/docs/05.workflow-components/07.triggers/01.schedule-trigger/index.md b/src/contents/docs/05.workflow-components/07.triggers/01.schedule-trigger/index.md
index b2ce0151149..52db0f91153 100644
--- a/src/contents/docs/05.workflow-components/07.triggers/01.schedule-trigger/index.md
+++ b/src/contents/docs/05.workflow-components/07.triggers/01.schedule-trigger/index.md
@@ -18,20 +18,19 @@ Kestra can trigger flows on a defined schedule. If you need to wait for another
Kestra can automatically handle [backfills](../../../06.concepts/08.backfill/index.md) to recover missed executions.
-Check the [Schedule task](/plugins/core/trigger/io.kestra.plugin.core.trigger.schedule) documentation for the list of the task properties and outputs.
+Check the [Schedule trigger](/plugins/core/trigger/io.kestra.plugin.core.trigger.schedule) documentation for the list of properties and outputs.
:::alert{type="warning"}
To avoid unexpected differences, keep your Kestra server and database timezones aligned. If this isn’t possible, account for timezone implications such as Daylight Saving Time or regional variations.
:::
-## Cron extension
+## Cron shortcuts
Kestra supports the following cron extensions instead of writing a cron expression:
- `@yearly` and `@annually` - runs yearly on 1st January at `00:00`
- `@monthly` - runs monthly on the 1st at `00:00`
- `@weekly` - runs weekly on Sunday at `00:00`
- `@daily` and `@midnight` - runs at `00:00` every day
-- `@midnight` - runs at `00:00` every day
- `@hourly` - runs every hour, on the hour
## Examples
@@ -52,11 +51,7 @@ triggers:
- id: schedule
type: io.kestra.plugin.core.trigger.Schedule
cron: "0 11 * * 1"
- conditions:
- - type: io.kestra.plugin.core.condition.DayWeekInMonth
- date: "{{ trigger.date }}"
- dayOfWeek: "MONDAY"
- dayInMonth: "FIRST"
+ when: "{{ isDayWeekInMonth(trigger.date, 'MONDAY', 'FIRST') }}"
```
A schedule that runs daily at midnight US Eastern time:
@@ -69,11 +64,7 @@ triggers:
timezone: America/New_York
```
-Schedule that runs on the last day of month:
-
-The Schedule trigger also supports `L` in the day-of-month field to represent the last day of the month.
-
-For example:
+Schedule that runs on the last day of every month. The `L` symbol in the day-of-month field represents the last day:
```yaml
triggers:
@@ -95,26 +86,15 @@ You can use this expression to make your **manual execution work**: `{{ trigger.
:::
-## Schedule conditions
-
-When a `cron` expression alone is not sufficient (e.g., only first Monday of the month, only weekends), you can refine schedules using `conditions`.
+## Refining schedules with `when`
-You **must** use the `{{ trigger.date }}` expression on the property `date` of the current schedule.
+When a `cron` expression alone is not sufficient (e.g., only first Monday of the month, only weekends), you can refine schedules using a `when` Pebble expression.
-This condition will be evaluated and `{{ trigger.previous }}` and `{{ trigger.next }}` will reflect the date **with** the conditions applied.
+You can use the `{{ trigger.date }}` expression to access the current schedule date within the `when` expression. The [date and calendar helper functions](../../../expressions/04.functions/06.dates/index.mdx) in the expressions reference cover all available date functions such as `isDayWeekInMonth()`, `dayOfWeek()`, `isWeekend()`, `isPublicHoliday()`, and `isLastWorkingDay()`.
-The list of core conditions that can be used are:
+The `when` expression is evaluated and `{{ trigger.previous }}` and `{{ trigger.next }}` reflect the date **with** the condition applied.
- - [DateTimeBetween](/plugins/core/condition/io.kestra.plugin.core.condition.datetimebetween)
- - [DayWeek](/plugins/core/condition/io.kestra.plugin.core.condition.dayweek)
- - [DayWeekInMonth](/plugins/core/condition/io.kestra.plugin.core.condition.dayweekinmonth)
- - [Not](/plugins/core/condition/io.kestra.plugin.core.condition.not)
- - [Or](/plugins/core/condition/io.kestra.plugin.core.condition.or)
- - [Weekend](/plugins/core/condition/io.kestra.plugin.core.condition.weekend)
- - [PublicHoliday](/plugins/core/condition/io.kestra.plugin.core.condition.publicholiday)
- - [TimeBetween](/plugins/core/condition/io.kestra.plugin.core.condition.timebetween)
-
-Here's an example using the `DayWeek` condition:
+Here's an example using a day-of-week check:
```yaml
id: conditions
@@ -129,9 +109,7 @@ triggers:
- id: schedule
type: io.kestra.plugin.core.trigger.Schedule
cron: "@hourly"
- conditions:
- - type: io.kestra.plugin.core.condition.DayWeek
- dayOfWeek: "THURSDAY"
+ when: "{{ dayOfWeek(trigger.date) == 'THURSDAY' }}"
```
## Recover missed schedules
@@ -173,29 +151,17 @@ In this example, the `recoverMissedSchedules` is set to `NONE`, which means that
Backfills are replays of missed schedule intervals between a defined start and end date.
-To backfill the missed executions, go to the `Triggers` tab on the flow's detail page and click on the `Backfill executions` button.
-
-
-
-:::alert{type="info"}
-Note: Ensure the backfill date range spans every missed schedule so the trigger can replay each execution.
-:::
-
-For more information on Backfill, check out the [dedicated documentation](../../../06.concepts/08.backfill/index.md).
+To backfill the missed executions, use **Backfill executions** on the flow's **Triggers** tab. Ensure the date range spans every missed schedule so the trigger can replay each execution. See the [Backfill documentation](../../../06.concepts/08.backfill/index.md) for details.
#### Disabling the trigger
-If you are unsure how to proceed, you can temporarily disable the trigger by setting `disabled: true` in the YAML or toggling it in the UI.
+To pause the schedule while you decide what to do next, set `disabled: true` in the YAML or use the **Enabled** toggle in the UI. See [Disabled](../../16.disabled/index.md) for details.
-This is useful if you are figuring out what to do before the next schedule is due to run.
+## Passing inputs to the Schedule trigger
-For more information on Disabled, check out the [dedicated documentation](../../16.disabled/index.md).
+Use the `inputs` property to set input values before execution:
-## Setting inputs inside of the schedule trigger
-
-You can easily pass inputs to the Schedule Trigger by using the `inputs` property and passing them as a key-value pair.
-
-In this example, the `user` input is set to "John Smith" inside of the `schedule` trigger:
+In this example, the `user` input is set to "John Smith" by the `schedule` trigger:
```yaml
id: myflow
@@ -221,9 +187,7 @@ triggers:
## Disable a schedule trigger after a specified execution state
-Schedule triggers have an optional property, `stopAfter`, that disables a trigger after a specified execution state has been reached: for example, `SUCCESS`, `FAILED`, `KILLED`, `SKIPPED`, etc. Refer to the [Schedule Trigger documentation](/plugins/core/trigger/io.kestra.plugin.core.trigger.schedule#properties_stopAfter-body) for more property details.
-
-For example, you may want to disable a trigger for a `FAILED` or `KILLED` flow to avoid multiple runs of that flow that is misconfigured and needs attention. The property is added to the trigger definition like below:
+The `stopAfter` property disables the trigger when the execution reaches one of the specified states — for example, `FAILED` or `KILLED` — preventing repeated runs of a broken flow until you manually re-enable it.
```yaml
id: myflow
@@ -271,8 +235,8 @@ triggers:
- id: stuck_schedules
type: io.kestra.plugin.kestra.triggers.ScheduleMonitor
auth:
- username: admin@kestra.io # pass your Kestra username as secret
- password: Admin1234 # pass your Kestra password as secret
+ username: "{{ secret('KESTRA_USERNAME') }}"
+ password: "{{ secret('KESTRA_PASSWORD') }}"
namespace: company.team
flowId: daily_sync
interval: PT1H # poll for stuck schedules every 1h
diff --git a/src/contents/docs/05.workflow-components/07.triggers/02.flow-trigger/index.md b/src/contents/docs/05.workflow-components/07.triggers/02.flow-trigger/index.md
index d39c244a2a2..8b94bab4db1 100644
--- a/src/contents/docs/05.workflow-components/07.triggers/02.flow-trigger/index.md
+++ b/src/contents/docs/05.workflow-components/07.triggers/02.flow-trigger/index.md
@@ -8,33 +8,25 @@ icon: /src/contents/docs/icons/flow.svg
Trigger one flow based on the execution of another flow.
-A Flow trigger runs a flow after another flow completes, enabling event-driven workflows and dependencies across teams.
-
```yaml
type: io.kestra.plugin.core.trigger.Flow
```
-Kestra can automatically start a flow as soon as another flow ends. This allows you to create dependencies between flows, even when those flows are owned by different teams.
+A Flow trigger runs a flow after another flow completes, enabling event-driven dependencies across teams and namespaces.
Check the [Flow trigger](/plugins/core/trigger/io.kestra.plugin.core.trigger.flow) documentation for the list of all properties.
-## Preconditions
-
-A Flow trigger requires preconditions to filter which upstream executions can trigger the flow, often within a defined time window.
-
-:::alert{type="info"}
-[Pebble expressions](../../../expressions/index.mdx) cannot be used in Flow Trigger (pre)conditions. You must declaratively define any condition variables.
-:::
+## Upstream flow dependencies
-### Filters
+The `dependsOn` property is a list of upstream flow entries that must all complete in matching states before the trigger fires.
-- `flows`: A list of preconditions to meet, in the form of upstream flows
+### Basic single upstream flow
-The example below shows a Flow trigger that runs when `flow_a` completes successfully.
+The example below triggers `flow_b` when `flow_a` from the `company.team` namespace completes successfully:
```yaml
id: flow_b
-namespace: kestra.sandbox
+namespace: company.team
tasks:
- id: hello
@@ -42,191 +34,380 @@ tasks:
message: "Hello World!"
triggers:
- - id: upstream_dependancy
+ - id: after_extract
type: io.kestra.plugin.core.trigger.Flow
- preconditions:
- id: flow_trigger
- flows:
- - namespace: kestra.sandbox
- flowId: flow_a
- states: [SUCCESS]
+ dependsOn:
+ - flowId: extract
+ namespace: company.team
+ states: [SUCCESS]
```
-:::alert{type="info"}
-It is [best practice](../../../14.best-practices/0.flows/index.md#flow-trigger-on-state-change) when using a flow trigger to use `preconditions.flows.states` rather than the `states` task property when defining state conditions for one specific flow.
-:::
+### Multiple upstream flows
+
+List multiple entries under `dependsOn`. All entries must be satisfied before the trigger fires:
+
+```yaml
+triggers:
+ - id: after_staging
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: stg_sales
+ namespace: company.team
+ - flowId: stg_marketing
+ namespace: company.team
+```
+
+### Entry properties
+
+| Property | Type | Description |
+|-------------|-----------------------|----------------------------------------------------------------------------------------------------------------------|
+| `flowId` | `String` | The ID of the upstream flow to match. Omit to match any flow (combine with `when` to narrow the scope). |
+| `namespace` | `String` | The namespace of the upstream flow. Exact match only — use `when` for prefix or pattern matching. |
+| `states` | `List` | States that satisfy this entry. Defaults to all terminal states and `PAUSED` when omitted. |
+| `labels` | `Map` | Key-value pairs that must all be present on the upstream execution's labels. |
+| `when` | `String` | A Pebble expression evaluated against the upstream execution. The entry is satisfied only when this evaluates to true.|
+
+### Satisfaction mode
-- `where`: filter executions based on fields like `FLOW_ID`, `NAMESPACE`, `STATE`, and `EXPRESSION`.
+The `mode` property controls how `dependsOn` entries are combined before the trigger fires:
-For example, the following Flow Trigger triggers on execution from flows in FAILED or WARNING states in namespaces starting with "company":
+| Value | Behavior | Extra property |
+|---|---|---|
+| `ALL` (default) | All entries must be satisfied | — |
+| `ANY` | Fires as soon as any one entry is satisfied | — |
+| `AT_LEAST` | Fires when at least N entries are satisfied | `minSatisfied` (integer ≥ 1) |
+
+Use `mode: ANY` to replace multiple separate Flow triggers with one:
```yaml
triggers:
- - id: alert_on_failure
+ - id: react_to_any_source
type: io.kestra.plugin.core.trigger.Flow
- states:
- - FAILED
- - WARNING
- preconditions:
- id: company_namespace
- where:
- - id: company
- filters:
- - field: NAMESPACE
- type: STARTS_WITH
- value: company
+ mode: ANY
+ dependsOn:
+ - flowId: ingest_salesforce
+ namespace: company.sources
+ states: [SUCCESS]
+ - flowId: ingest_hubspot
+ namespace: company.sources
+ states: [SUCCESS]
```
-### Time Window & SLA
+Use `mode: AT_LEAST` with `minSatisfied` for N-of-M logic — fire when 2 out of 3 upstream flows succeed:
-The `timeWindow` property lets you define how Kestra evaluates upstream flow executions over time. It supports several modes:
+```yaml
+triggers:
+ - id: partial_success
+ type: io.kestra.plugin.core.trigger.Flow
+ mode: AT_LEAST
+ minSatisfied: 2
+ dependsOn:
+ - flowId: ingest_salesforce
+ namespace: company.sources
+ states: [SUCCESS]
+ - flowId: ingest_hubspot
+ namespace: company.sources
+ states: [SUCCESS]
+ - flowId: ingest_zendesk
+ namespace: company.sources
+ states: [SUCCESS]
+ window:
+ deadline: "09:00:00"
+```
-- `DURATION_WINDOW`: This is the default type. It uses a start time (windowAdvance) and end time (window) that are moving forward to the next interval whenever the evaluation time reaches the end time, based on the defined duration window.
+### Prefix and pattern matching
-For example, with a 1-day window (`window: PT1D`, the default), SLA conditions are evaluated over a 24-hour period starting at midnight each day. If you set `windowAdvance: PT6H`, the window will start at 6 AM each day. If you set `windowAdvance: PT6H` and you also override `window: PT6H`, the window will start at 6 AM and last for 6 hours — as a result, Kestra will check the SLA conditions during the following time periods: `06:00` to `12:00`, `12:00` to `18:00`, `18:00` to `00:00`, and `00:00` to `06:00`, and so on.
+When `namespace` is set, Kestra matches it exactly. To match a range of namespaces or flows, omit `namespace` and use `when` with a Pebble expression:
-- `SLIDING_WINDOW`: This option also evaluates SLA conditions over a fixed time window, but it always goes backward from the current time. For example, a sliding window of 1 hour (window: PT1H) will evaluate executions for the past hour (so between now and one hour before now). It uses a default window of 1 day.
+```yaml
+triggers:
+ - id: alert_on_failure
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - states: [FAILED, WARNING]
+ when: "{{ namespace | startsWith('company') }}"
+```
-For example, the flow below evaluates every hour if the flow `flow_a` is in SUCCESS state. If so, it triggers the `flow_b` passing corresponding inputs (reading `flow_a` outputs).
+## Conditional guard with `when`
+
+Like all triggers, the Flow trigger supports a top-level `when` Pebble expression. It is evaluated before `dependsOn` — if it returns a falsy value, the trigger does not fire regardless of upstream state:
```yaml
-id: flow_b
-namespace: kestra.sandbox
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ when: "{{ labels.env == 'production' }}"
+ dependsOn:
+ - flowId: extract
+ namespace: company.team
+```
-inputs:
- - id: value_from_a
- type: STRING
+## Time window
-tasks:
- - id: hello
- type: io.kestra.plugin.core.log.Log
- message: "{{ inputs.value_from_a }}"
+The `window` property controls how long Kestra accumulates upstream executions before evaluating whether all `dependsOn` entries are satisfied.
+
+### Deadline
+All upstream flows must complete before a fixed time each day. The deadline is a `java.time.LocalTime` value in `HH:mm:ss` format. It is resolved in the server timezone unless a `timezone` is set:
+
+```yaml
triggers:
- - id: upstream_dep
+ - id: after_staging
type: io.kestra.plugin.core.trigger.Flow
- inputs:
- value_from_a: "{{ trigger.outputs.return_value }}"
- preconditions:
- id: test
- flows:
- - namespace: kestra.sandbox
- flowId: flow_a
- states: [SUCCESS]
- timeWindow:
- type: SLIDING_WINDOW
- window: PT1H
+ dependsOn:
+ - flowId: stg_sales
+ namespace: company.team
+ - flowId: stg_marketing
+ namespace: company.team
+ window:
+ deadline: "09:00:00"
```
-For reference, below is `flow_a`:
+### Daily time range
+
+Only executions that completed within a specific time range each day are counted:
```yaml
-id: flow_a
-namespace: kestra.sandbox
+triggers:
+ - id: after_staging
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: stg_sales
+ namespace: company.team
+ - flowId: stg_marketing
+ namespace: company.team
+ window:
+ from: "06:00:00"
+ to: "12:00:00"
+```
-tasks:
- - id: hello
- type: io.kestra.plugin.core.log.Log
- message: Hello World! 🚀
+### Fixed interval
-outputs:
- - id: return_value
- type: STRING
- value: "Flow A run succesfully"
+`every` defines the window size and `offset` shifts its start relative to midnight:
+
+```yaml
+triggers:
+ - id: after_staging
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: stg_sales
+ namespace: company.team
+ window:
+ every: PT1H
+ offset: PT30M
```
+### Lookback
-- `DAILY_TIME_DEADLINE`: This option enforces SLA conditions that must be met before a specific cutoff time each day. With the string property deadline, you can configure a daily cutoff for checking conditions. For example, deadline: `09:00:00.00Z` means that the defined SLA conditions should be met from midnight until 9 AM each day; otherwise, the flow will not be triggered.
+Count executions that completed within the past duration, relative to the current evaluation time:
-For the example, this trigger definition only triggers the flow if `flow_a` is in SUCCESS state before `9:00` AM every day.
+```yaml
+triggers:
+ - id: after_staging
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: stg_sales
+ namespace: company.team
+ window:
+ lookback: PT1H
+```
+
+### Timezone
+
+`deadline`, `from`, `to`, `every` and `offset` are anchored on daily or midnight boundaries, which are resolved in the server timezone by default. A flow therefore behaves differently depending on where it is deployed. Set `timezone` to a time-zone ID so the window follows the intended zone, including across daylight-saving transitions:
```yaml
triggers:
- - id: upstream_dep
+ - id: after_staging
type: io.kestra.plugin.core.trigger.Flow
- preconditions:
- id: should_be_success_by_nine
- flows:
- - namespace: kestra.sandbox
- flowId: flow_a
- states: [SUCCESS]
- timeWindow:
- type: DAILY_TIME_DEADLINE
- deadline: "09:00:00.00Z"
+ dependsOn:
+ - flowId: stg_sales
+ namespace: company.team
+ window:
+ deadline: "09:00:00"
+ timezone: Europe/Paris
```
-- `DAILY_TIME_WINDOW`: This option enforces SLA conditions that must be met within a specific daily time range. For example, a window from `startTime: "06:00:00"` to `endTime: "09:00:00"` evaluates executions within that interval each day. This option is particularly useful for declarative definition of freshness conditions when building data pipelines. For example, if you only need one successful execution within a given time range to guarantee that some data has been successfully refreshed in order for you to proceed with the next steps of your pipeline, this option can be more useful than a strict DAG-based approach. Usually, each failure in your flow would block the entire pipeline, whereas with this option, you can proceed with the next steps of the pipeline as soon as the data is successfully refreshed at least once within the given time range.
+`lookback` is relative to the evaluation time rather than to a daily boundary, so it is not affected by `timezone`.
+
+### Firing more than once in a window
+
+Once every `dependsOn` entry has been satisfied and an execution has been created, the stored results are reset. The trigger can fire again in the same window, though every dependency has to be satisfied again first. To create an execution as soon as any single upstream flow succeeds, use `mode: ANY` instead.
+
+## Scoped trigger outputs
+
+When a Flow trigger fires, upstream execution outputs are available under `trigger.outputs`. Outputs are scoped by flow ID to avoid key collisions when multiple upstream flows are involved:
+
+```
+trigger.outputs..
+```
+
+For example, to pass an output from an upstream flow named `extract`:
```yaml
triggers:
- - id: upstream_dep
+ - id: after_extract
type: io.kestra.plugin.core.trigger.Flow
inputs:
- value_from_a: "{{ trigger.outputs.return_value }}"
- preconditions:
- id: test
- flows:
- - namespace: kestra.sandbox
- flowId: flow_a
- states: [SUCCESS]
- timeWindow:
- type: DAILY_TIME_WINDOW
- startTime: "06:00:00"
- endTime: "12:00:00"
+ date: "{{ trigger.outputs.extract.date }}"
+ dependsOn:
+ - flowId: extract
+ namespace: company.team
+```
+
+:::alert{type="warning"}
+The output scoping format changed in Kestra 2.0. If you previously used `trigger.outputs.` (a flat map), update your expressions to the new `trigger.outputs..` format.
+:::
+
+## Label-based filtering
+
+Use the `labels` map on a `dependsOn` entry to restrict which upstream executions are counted. All specified labels must be present on the upstream execution:
+
+```yaml
+triggers:
+ - id: after_prod
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - namespace: company.team
+ labels:
+ env: production
+ states: [SUCCESS]
+```
+
+## Filtering with `when` expressions
+
+Use `when` on a `dependsOn` entry to apply arbitrary Pebble conditions against the upstream execution context.
+
+Filter on an output value:
+
+```yaml
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: extract
+ namespace: company.team
+ when: "{{ outputs.row_count > 0 }}"
```
+Filter on retry attempts:
+```yaml
+triggers:
+ - id: after_flaky
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: flaky_pipeline
+ namespace: company.team
+ states: [SUCCESS]
+ when: "{{ hasRetryAttempt == true }}"
+```
-## Example
+## Example: data pipeline with SLA deadline
-This example triggers the `silver_layer` flow once the `bronze_layer` flow finishes successfully by 9 AM. The deadline time string must include the timezone offset. This ensures that no new executions are triggered past the deadline. Here is the `silver_layer` flow:
+This example triggers the `silver_layer` flow once the `bronze_layer` flow finishes successfully by 9 AM:
```yaml
id: silver_layer
namespace: company.team
+
tasks:
- id: transform_data
type: io.kestra.plugin.core.log.Log
message: deduplication, cleaning, and minor aggregations
+
triggers:
- id: flow_trigger
type: io.kestra.plugin.core.trigger.Flow
- preconditions:
- id: bronze_layer
- timeWindow:
- type: DAILY_TIME_DEADLINE
- deadline: "09:00:00+01:00"
- flows:
- - namespace: company.team
- flowId: bronze_layer
- states: [SUCCESS]
+ dependsOn:
+ - flowId: bronze_layer
+ namespace: company.team
+ states: [SUCCESS]
+ window:
+ deadline: "09:00:00"
```
-## Example: Alerting
+## Example: alerting on failure
-This example creates a `System Flow` to send a Slack alert on any failure or warning state within the `company` namespace. This example uses the Slack webhook secret to notify the `#general` channel about the failed flow.
+This example creates a system flow that sends a Slack alert on any failure or warning state within the `company` namespace:
```yaml
id: alert
namespace: system
+
tasks:
- id: send_alert
type: io.kestra.plugin.slack.notifications.SlackExecution
- url: "{{secret('SLACK_WEBHOOK')}}" # format: https://hooks.slack.com/services/xzy/xyz/xyz
+ url: "{{secret('SLACK_WEBHOOK')}}"
channel: "#general"
executionId: "{{trigger.executionId}}"
+
triggers:
- id: alert_on_failure
type: io.kestra.plugin.core.trigger.Flow
- states:
- - FAILED
- - WARNING
- preconditions:
- id: company_namespace
- where:
- - id: company
- filters:
- - field: NAMESPACE
- type: STARTS_WITH
- value: company
+ dependsOn:
+ - states: [FAILED, WARNING]
+ when: "{{ namespace | startsWith('company') }}"
```
+
+## Example: mixed success and failure triggers
+
+You can define multiple Flow triggers on the same flow to react differently to upstream success vs. failure:
+
+```yaml
+triggers:
+ - id: on_completion
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: flow_a
+ namespace: company.team
+ states: [SUCCESS]
+ - id: on_failure
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: flow_a
+ namespace: company.team
+ states: [FAILED]
+```
+
+## Example: passing upstream outputs downstream
+
+Reference upstream outputs using the scoped path `trigger.outputs..`:
+
+```yaml
+id: flow_b
+namespace: company.team
+
+inputs:
+ - id: value_from_a
+ type: STRING
+
+tasks:
+ - id: hello
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ inputs.value_from_a }}"
+
+triggers:
+ - id: upstream_dep
+ type: io.kestra.plugin.core.trigger.Flow
+ inputs:
+ value_from_a: "{{ trigger.outputs.flow_a.return_value }}"
+ dependsOn:
+ - flowId: flow_a
+ namespace: company.team
+ states: [SUCCESS]
+```
+
+:::alert{type="info"}
+`dependsOn` condition IDs are derived from a stable hash of each entry's `namespace`, `flowId`, `when`, `states`, and `labels`. Reordering entries in the list does not reset accumulated window state.
+:::
+
+## Input rendering failures create FAILED executions
+
+If an `inputs` expression on a Flow trigger fails to render — for example, because an upstream output key does not exist — Kestra creates a `FAILED` execution instead of silently dropping the event. This makes failures visible in the UI and actionable via alerting.
+
+## Removed: `preconditions` and `conditions`
+
+The `preconditions` and `conditions` properties are removed in Kestra 2.0. Flows that still use them will fail to parse after upgrading. Migrate to `dependsOn`.
+
+See the [trigger conditions migration guide](../../../11.migration-guide/v2.0.0/trigger-conditions-redesign/index.md) for a complete before/after reference.
diff --git a/src/contents/docs/05.workflow-components/07.triggers/03.webhook-trigger/index.md b/src/contents/docs/05.workflow-components/07.triggers/03.webhook-trigger/index.md
index 66f729e865e..64c6ab110bd 100644
--- a/src/contents/docs/05.workflow-components/07.triggers/03.webhook-trigger/index.md
+++ b/src/contents/docs/05.workflow-components/07.triggers/03.webhook-trigger/index.md
@@ -8,81 +8,193 @@ icon: /src/contents/docs/icons/flow.svg
Trigger flows automatically in response to web-based events.
-A Webhook trigger generates a unique URL that lets external applications (such as GitHub, Amazon EventBridge, or any system that can send HTTP requests) automatically start new executions in Kestra.
+A Webhook trigger generates a unique URL that lets external applications (such as GitHub, Amazon EventBridge, or any system that can send HTTP requests) start new executions in Kestra. Each webhook URL requires a secret `key`. Store the key value in [Kestra Secrets](../../../07.enterprise/02.governance/secrets/index.md) and reference it from the trigger definition — never hardcode a key directly in the flow YAML. Kestra accepts `GET`, `POST`, and `PUT` requests on the webhook URL.
-Each webhook URL requires a secret `key` to secure it. This prevents unauthorized access and ensures only trusted systems can trigger your flow.
+
+
+
+
+## Example
```yaml
-type: io.kestra.plugin.core.trigger.Webhook
+id: trigger
+namespace: company.team
+
+tasks:
+ - id: hello
+ type: io.kestra.plugin.core.log.Log
+ message: "Hello World! 🚀"
+
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: "{{ secret('WEBHOOK_KEY') }}"
+```
+
+The `key` is embedded in the webhook URL: `/api/v1/main/executions/webhook/{namespace}/{flowId}/{key}`. To start the flow:
+
+```bash
+https://{kestra_domain}/api/v1/main/executions/webhook/{namespace}/{flowId}/{key}
```
-A Webhook trigger enables triggering a flow from a webhook URL.
-When you create the trigger, you must provide a `key`. This `key` is embedded in the webhook URL: `/api/v1/main/executions/webhook/{namespace}/{flowId}/{key}`.
+Replace `kestra_domain`, `namespace`, `flowId`, and `key` with your values. You can also copy the webhook URL from the **Triggers** tab.
+
+## Handling the request body
+
+By default, the webhook trigger reads the request body and makes it available as `trigger.body`. Use the `fetchType` property to change this behavior.
+
+```yaml
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: "{{ secret('WEBHOOK_KEY') }}"
+ fetchType: FETCH # FETCH (default) | STORE | NONE
+```
-For security, use a randomly generated string rather than something easy to guess. Kestra accepts `GET`, `POST`, and `PUT` requests on the webhook URL. Both the request body and headers are automatically available as variables inside your flow.
+| `fetchType` | Body handling |
+|---|---|
+| `FETCH` | Body is read into memory and exposed as `trigger.body`. Text, JSON, XML, YAML, CSV, and form-urlencoded bodies are decoded as strings or parsed objects. Binary bodies (any other content type) are base64-encoded. This is the default. |
+| `STORE` | Body is streamed directly to internal storage without being held in memory. The flow receives a `kestra://` URI on `trigger.uri`. Use this for large payloads or when you do not want body content written into the execution record. |
+| `NONE` | Body is read off the connection and dropped. The flow receives neither `trigger.body` nor `trigger.uri`. |
:::alert{type="info"}
-Starting in Kestra 0.24, [Basic Authentication is required](../../../11.migration-guide/v0.24.0/basic-authentication/index.md) for all instances. This change makes it so API requests require an `Authorization` header. Follow these [Basic Authentication Encoding Steps](../../../15.how-to-guides/synchronous-executions-api/index.md#basic-authentication) to configure requests correctly.
+File parts in a `multipart/form-data` request are always stored in internal storage, regardless of `fetchType`.
:::
-## Example
+### Binary and non-text bodies
+
+When a request body has a content type that is not text, JSON, XML, YAML, CSV, or form-urlencoded, `FETCH` base64-encodes it and stores the result in `trigger.body`. Pass `trigger.body` to a script task to decode and process the raw bytes.
```yaml
-id: trigger
+id: webhook_binary_body
namespace: company.team
tasks:
- - id: hello
+ - id: log_body
type: io.kestra.plugin.core.log.Log
- message: "Hello World! 🚀"
+ message: "base64={{ trigger.body }}"
triggers:
- id: webhook
type: io.kestra.plugin.core.trigger.Webhook
- key: 4wjtkzwVGBM9yKnjm3yv8r
+ key: "{{ secret('WEBHOOK_KEY') }}"
+ fetchType: FETCH
```
-After creating the trigger, include the key in the webhook URL to start the flow. For example:
+### Stream large bodies to internal storage
-```bash
-https://{kestra_domain}/api/v1/main/executions/webhook/{namespace}/{flowId}/4wjtkzwVGBM9yKnjm3yv8r
+Use `STORE` to stream the body directly to internal storage without loading it into memory. The flow receives `trigger.uri` instead of `trigger.body`.
+
+```yaml
+id: webhook_store_body
+namespace: company.team
+
+tasks:
+ - id: log_uri
+ type: io.kestra.plugin.core.log.Log
+ message: "uri={{ trigger.uri }}"
+
+ - id: measure_body
+ type: io.kestra.plugin.core.storage.Size
+ uri: "{{ trigger.uri }}"
+
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: "{{ secret('WEBHOOK_KEY') }}"
+ fetchType: STORE
```
-Make sure to replace `kestra_domain`, `namespace`, and `flowId`.
+:::alert{type="warning"}
+When `fetchType: STORE`, the body is never deserialized. `when` conditions that reference `trigger.body` will not work — filter on headers or query parameters instead.
+:::
-You can also copy the formed Webhook URL from the **Triggers** tab.
+### File uploads — multipart/form-data
-
-
-
+When a request arrives as `multipart/form-data`, file parts are stored in internal storage and exposed on `trigger.parts`. Text fields are exposed on `trigger.formFields`. This works regardless of `fetchType`.
-## Webhook response
+Each entry in `trigger.parts` has the following shape:
+
+```yaml
+trigger:
+ parts:
+ - name: photo
+ filename: result.jpg
+ contentType: image/jpeg
+ size: 20345
+ uri: kestra:///company/team/executions/5cVhZ…/webhook/0/result.jpg
+ formFields:
+ note:
+ - "looks good"
+```
-By default, a webhook trigger answers with JSON. When you need the caller to wait for a custom response (e.g., validation handshakes that require `text/plain`), enable `wait` and set the `responseContentType` to `text/plain`.
+Flow that receives a file upload and measures its size:
```yaml
+id: webhook_multipart
+namespace: company.team
+
+tasks:
+ - id: log_parts
+ type: io.kestra.plugin.core.log.Log
+ message: "parts={{ trigger.parts }} fields={{ trigger.formFields }}"
+
+ - id: measure_upload
+ type: io.kestra.plugin.core.storage.Size
+ uri: "{{ trigger.parts[0].uri }}"
+
+ - id: log_size
+ type: io.kestra.plugin.core.log.Log
+ message: "size={{ outputs.measure_upload.size }}"
+
triggers:
- id: webhook
type: io.kestra.plugin.core.trigger.Webhook
- key: your-secret-key
- wait: true
- returnOutputs: true
- responseContentType: text/plain # optional, defaults to application/json
+ key: "{{ secret('WEBHOOK_KEY') }}"
```
-Behavior:
-- `wait: true` keeps the HTTP connection open until the flow finishes or hits the trigger’s timeout.
-- `returnOutputs: true` returns the flow outputs as the HTTP response body (JSON by default). Override with `responseContentType` for plaintext or other formats.
+Stored bytes are scoped to the execution and purged with it. If no execution is created — for example, because a `when` condition vetoed the request — stored bytes are cleaned up automatically.
-## Webhook trigger testing
+## Filtering webhook executions with `when`
-If your flow uses trigger variables (such as `{{ trigger.body }})`, you can test it directly from the execution modal. Kestra generates a ready-to-use `cURL` command that lets you trigger the flow with a custom JSON payload.
+Use the `when` property to conditionally fire the trigger based on the request body or headers. The `when` value is a [Pebble expression](../../../expressions/index.mdx) evaluated against the incoming request. If the expression evaluates to a falsy value, Kestra ignores the request and no execution is created.
-
+```yaml
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: "{{ secret('WEBHOOK_KEY') }}"
+ when: "{{ trigger.body.hello == 'world' }}"
+```
-See the [Webhook trigger plugin documentation](/plugins/core/trigger/io.kestra.plugin.core.trigger.webhook) for a full list of properties and outputs.
+You can combine multiple criteria in a single expression using `and` / `or`:
+
+```yaml
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: "{{ secret('WEBHOOK_KEY') }}"
+ when: "{{ trigger.body.event == 'push' and trigger.headers['x-github-event'] == 'push' }}"
+```
+
+## Webhook response
+
+By default, the trigger responds immediately with JSON. When the caller needs to wait for the result — for example, a validation handshake that requires `text/plain` — enable `wait` and set `responseContentType`.
+
+```yaml
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: "{{ secret('WEBHOOK_KEY') }}"
+ wait: true
+ returnOutputs: true
+ responseContentType: text/plain # optional, defaults to application/json
+```
+
+- `wait: true` keeps the HTTP connection open until the flow finishes or the trigger's timeout is reached.
+- `returnOutputs: true` returns the flow outputs as the HTTP response body.
-### Return flow outputs in the webhook response
+## Return flow outputs in the webhook response
To send task outputs back to the caller in the HTTP response, configure the Webhook trigger to wait for the execution and return outputs. The flow must expose at least one `outputs` entry.
@@ -103,7 +215,7 @@ outputs:
triggers:
- id: webhook
type: io.kestra.plugin.core.trigger.Webhook
- key: 4wjtkzwVGBM9yKnjm3yv8r
+ key: "{{ secret('WEBHOOK_KEY') }}"
wait: true
returnOutputs: true
# optional: responseContentType: "text/plain"
@@ -112,3 +224,11 @@ triggers:
- Call the webhook URL with a query parameter (for example `?name=Alice`). The execution runs synchronously because `wait: true` is set.
- The HTTP response body contains the flow outputs (JSON by default). With the example above, the response includes `"greeting": "Hello Alice!"`.
- Set `responseContentType: "text/plain"` when you want the response body to be plain text (ensure the flow returns a single string output, such as from the `Return` task).
+
+## Test a webhook trigger
+
+To test a webhook trigger without an external tool, go to the flow's **Triggers** tab and click **Send a test event**. The modal lets you post a custom JSON payload and optional headers directly to the webhook URL:
+
+
+
+See the [Webhook trigger plugin documentation](/plugins/core/trigger/io.kestra.plugin.core.trigger.webhook) for a full list of properties and outputs.
diff --git a/src/contents/docs/05.workflow-components/07.triggers/03.webhook-trigger/webhook-trigger-test.png b/src/contents/docs/05.workflow-components/07.triggers/03.webhook-trigger/webhook-trigger-test.png
index 210314c0b11..e88bee92b1c 100644
Binary files a/src/contents/docs/05.workflow-components/07.triggers/03.webhook-trigger/webhook-trigger-test.png and b/src/contents/docs/05.workflow-components/07.triggers/03.webhook-trigger/webhook-trigger-test.png differ
diff --git a/src/contents/docs/05.workflow-components/07.triggers/04.polling-trigger/index.md b/src/contents/docs/05.workflow-components/07.triggers/04.polling-trigger/index.md
index 92c1625e35a..b59eefaa7f3 100644
--- a/src/contents/docs/05.workflow-components/07.triggers/04.polling-trigger/index.md
+++ b/src/contents/docs/05.workflow-components/07.triggers/04.polling-trigger/index.md
@@ -12,6 +12,8 @@ Polling triggers repeatedly check an external system at a fixed interval. When n
Kestra provides polling triggers for a wide variety of systems, including databases, message queues, cloud storage, and FTP servers.
+Polling triggers are not limited to external connectors. Some plugins also provide script-based polling triggers, allowing you to run code on an interval and emit only when a condition matches. The script plugins for [Python](../../../15.how-to-guides/python/index.md#run-python-code-as-a-polling-trigger), [Shell](../../../15.how-to-guides/shell/index.md#automate-shell-with-triggers), and [JavaScript](../../../15.how-to-guides/javascript/index.md#automate-javascript-with-triggers) each provide `ScriptTrigger` and `CommandsTrigger` variants for polling with code or commands.
+
The polling frequency is controlled by the `interval` property. When triggered, the flow has access to the polling results through the `trigger` variable, making the retrieved data immediately available for downstream tasks.
## Example
@@ -27,14 +29,14 @@ inputs:
type: STRING
tasks:
-- id: update
- type: io.kestra.plugin.jdbc.postgresql.Query
- url: "{{ inputs.db_url }}"
- sql: DELETE * FROM my_table
+ - id: update
+ type: io.kestra.plugin.jdbc.postgresql.Query
+ url: "{{ inputs.db_url }}"
+ sql: DELETE FROM my_table
-- id: log
- type: io.kestra.plugin.core.log.Log
- message: "{{ trigger.rows }}"
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ trigger.rows }}"
triggers:
- id: watch
@@ -44,11 +46,15 @@ triggers:
sql: "SELECT * FROM my_table"
```
-In [Enterprise Edition](../../../07.enterprise/01.overview/01.enterprise-edition/index.md), you can assign polling triggers to a specific [Worker Group](../../../07.enterprise/04.scalability/worker-group/index.md) using the `workerGroup.key` property. This allows you to control where the polling is executed.
+Like all triggers, polling triggers support a `when` Pebble expression. When set, the trigger only creates an execution if `when` evaluates to `true` after new data is detected — useful for filtering results before the flow runs.
+
+In [Enterprise Edition](../../../07.enterprise/01.overview/01.enterprise-edition/index.md), you can assign polling triggers to a specific [Worker Queue](../../../07.enterprise/04.scalability/worker-group/index.md) using `workerSelector.tags`. This controls which workers execute the polling.
+
+Browse the [plugin catalog](/plugins) and filter by **Trigger** to see all available polling triggers, including file detection, database, and message queue variants.
## Enterprise example
-In Enterprise Edition (Kestra 0.24+), the `Salesforce Trigger` enables flows to start automatically when new records are created in Salesforce. For example, the flow below sends a Slack notification whenever a new contact is added.
+In Enterprise Edition, the `Salesforce Trigger` enables flows to start automatically when new records are created in Salesforce. For example, the flow below sends a Slack notification whenever a new contact is added.
```yaml
id: salesforce_contact_trigger
diff --git a/src/contents/docs/05.workflow-components/07.triggers/05.realtime-trigger/index.md b/src/contents/docs/05.workflow-components/07.triggers/05.realtime-trigger/index.md
index f8b4aaa4655..1d3fee60d38 100644
--- a/src/contents/docs/05.workflow-components/07.triggers/05.realtime-trigger/index.md
+++ b/src/contents/docs/05.workflow-components/07.triggers/05.realtime-trigger/index.md
@@ -4,18 +4,17 @@ h1: React Instantly to Kafka, SQS, and MQTT Events
description: Achieve low-latency automation with Kestra's Realtime Triggers. React instantly to events from Kafka, SQS, MQTT, and other streaming systems.
sidebarTitle: Realtime Trigger
icon: /src/contents/docs/icons/flow.svg
-version: ">= 0.17.0"
---
Trigger workflows instantly as events occur, with millisecond latency.
-[Triggers](./index.md) in Kestra can listen to external events and start a workflow execution when the event occurs. Most triggers in Kestra **poll** external systems at regular intervals (e.g., every second) to detect new events. This is effective for batch-style data processing. However, business-critical workflows often demand immediate reactions — within milliseconds. **Realtime Triggers** address this need by listening directly for events and starting workflows as soon as they occur.
+Most Kestra [triggers](./index.md) poll external systems at a fixed interval, which works well for batch processing. Realtime Triggers listen directly for events and start a flow the moment one arrives — with millisecond latency.
-## What are Realtime Triggers
+## Supported event sources
Realtime Triggers continuously listen for events and launch a new workflow execution the moment an event occurs, such as:
@@ -102,9 +101,9 @@ kestra:
terminationGracePeriod: PT1M # default is PT5M
```
-::alert{type="info"}
+:::alert{type="info"}
Events are not lost during the failover window. They remain in the source system (Kafka topic, SQS queue, etc.) and will be consumed once the trigger listener is restarted on another worker.
-::
+:::
## Comparison with real-time data processing engines
diff --git a/src/contents/docs/05.workflow-components/07.triggers/06.mcp-tool-trigger/index.md b/src/contents/docs/05.workflow-components/07.triggers/06.mcp-tool-trigger/index.md
new file mode 100644
index 00000000000..79b2e86ba5b
--- /dev/null
+++ b/src/contents/docs/05.workflow-components/07.triggers/06.mcp-tool-trigger/index.md
@@ -0,0 +1,109 @@
+---
+title: MCP Tool Trigger in Kestra – Expose Flows as AI Tools
+h1: Expose Flows as MCP Tools with the McpToolTrigger
+description: Use the McpToolTrigger to expose Kestra flows as tools on an MCP server. AI agents can discover and invoke them automatically, with inputs and outputs mapped to a JSON schema.
+sidebarTitle: MCP Tool Trigger
+icon: /src/contents/docs/icons/flow.svg
+version: ">= 2.0.0"
+---
+
+Expose a flow as a named tool on a Kestra MCP server.
+
+The `McpToolTrigger` makes any flow discoverable and callable by MCP-compatible AI agents such as Claude Desktop, Claude Code, and Cursor. Flow inputs are automatically converted to a JSON schema tool spec so the AI agent knows exactly what parameters to pass. Each invocation creates a new flow execution tagged with `system.from:mcp` for observability.
+
+```yaml
+type: io.kestra.plugin.core.trigger.McpToolTrigger
+```
+
+:::alert{type="info"}
+Every tenant has a `default` MCP server provisioned on startup, so the trigger works without creating a server first. See [MCP Server](../../../ai-tools/03.mcp-server/index.md) to create additional servers and connect AI agent clients.
+:::
+
+## Example
+
+```yaml
+id: hello_world
+namespace: company.team
+
+inputs:
+ - id: user
+ type: STRING
+ defaults: John Doe
+ description: "The name of the user to greet."
+
+tasks:
+ - id: greet
+ type: io.kestra.plugin.core.output.OutputValues
+ values:
+ greeting: "Hello, {{ inputs.user }}!"
+
+outputs:
+ - id: greeting
+ type: STRING
+ value: "{{ outputs.greet.values.greeting }}"
+
+triggers:
+ - id: mcp
+ type: io.kestra.plugin.core.trigger.McpToolTrigger
+ toolName: hello_world
+ title: Hello World greeting tool
+ toolDescription: Returns a personalised greeting. Call this when the user asks for a greeting.
+ mcpServer: default
+```
+
+When deployed, an MCP client connected to the `default` server will discover a tool named `hello_world`. It will accept a `user` parameter (typed as `string` from the flow input) and return a `greeting` string in the tool response.
+
+
+
+## Properties
+
+| Property | Required | Default | Description |
+|---|---|---|---|
+| `toolName` | Yes | — | Tool identifier shown to the AI agent. Must contain only alphanumeric characters, hyphens, underscores, or dots, and must start and end with an alphanumeric character. Maximum 64 characters. |
+| `title` | Yes | — | Human-readable name shown to the AI agent. |
+| `toolDescription` | Yes | — | Description of the tool shown to the AI agent, used to decide when to invoke it. A well-written description significantly improves tool-selection accuracy. |
+| `mcpServer` | No | `"default"` | ID of the MCP server to register this tool on. Must match the `id` of an existing [MCP server](../../../ai-tools/03.mcp-server/index.md). |
+| `annotations.readOnly` | No | `false` | Hint that this tool does not modify its environment. |
+| `annotations.destructive` | No | `true` | Hint that this tool may perform destructive updates. Only meaningful when `readOnly` is `false`. |
+| `annotations.openWorld` | No | `true` | Hint that this tool may interact with entities outside its closed domain. |
+| `annotations.idempotent` | No | `false` | Hint that calling the tool repeatedly with the same arguments has no additional effect. Only meaningful when `readOnly` is `false`. |
+| `annotations.returnDirect` | No | `false` | When `true`, the AI agent forwards the raw tool output to the user without further interpretation or summarization. |
+
+Annotations are informational hints for MCP clients. They do not affect execution behavior.
+
+A flow can be registered on exactly one MCP server at a time via the `mcpServer` property. Multiple flows can share the same server, each appearing as a separate tool.
+
+### Writing effective tool descriptions
+
+The `toolDescription` is what the AI agent reads to decide whether to call your tool. Describe *when* and *why* to invoke the flow, not just what it does. For example:
+
+```yaml
+toolDescription: >
+ Returns a personalised greeting for a named user.
+ Call this tool whenever the user asks to be greeted or wants a welcome message.
+```
+
+## Common trigger properties
+
+`McpToolTrigger` supports all [common trigger properties](../index.mdx#trigger-common-properties), including `when` (a Pebble guard evaluated before invoking the flow) and `inputs` (values passed to flow inputs on each invocation).
+
+## Input and output mapping
+
+Flow inputs and outputs are automatically mapped to the MCP tool's input and output schema.
+
+- Each flow `input` becomes a tool parameter. The `description` field on the input is passed to the AI agent as the parameter description.
+- Flow `outputs` are returned in the tool response. Each output's `displayName` is used as the label in the response.
+
+To constrain the structure of a `JSON`-type input, use the `jsonSchema` property. See [JSON input validation](../../05.inputs/index.md#input-validation).
+
+## Observability
+
+Every execution created via MCP carries these [system labels](../../../06.concepts/system-labels/index.md):
+
+| Label | Value |
+|---|---|
+| `system.from` | `mcp` |
+| `system.mcpServerId` | The `id` of the MCP server that invoked the tool |
+| `system.mcpSessionId` | The session ID of the MCP client connection |
+
+Filter executions by `system.from: mcp` in the Executions view to see all MCP-triggered runs.
diff --git a/src/contents/docs/05.workflow-components/07.triggers/delete-triggers.png b/src/contents/docs/05.workflow-components/07.triggers/delete-triggers.png
deleted file mode 100644
index 9b4f0efde4b..00000000000
Binary files a/src/contents/docs/05.workflow-components/07.triggers/delete-triggers.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/07.triggers/index.mdx b/src/contents/docs/05.workflow-components/07.triggers/index.mdx
index d5787976403..f01bfc7af3f 100644
--- a/src/contents/docs/05.workflow-components/07.triggers/index.mdx
+++ b/src/contents/docs/05.workflow-components/07.triggers/index.mdx
@@ -9,7 +9,7 @@ sidebarTitle: Triggers
import ChildCard from "~/components/docs/ChildCard.astro"
-A trigger is a mechanism that automatically starts the execution of a flow.
+A trigger starts a flow automatically, either on a schedule or in response to an event.
-Triggers can be either scheduled or event-based, giving you flexibility in how you automate workflow execution.
-
## Trigger types
-Kestra supports both **scheduled** and **external** events.
-
-Kestra supports five core trigger types:
+Kestra supports six core trigger types:
-- [Schedule trigger](./01.schedule-trigger/index.md) allows you to execute your flow on a regular cadence e.g. using a CRON expression and custom scheduling conditions.
-- [Flow trigger](./02.flow-trigger/index.md) allows you to execute your flow when another flow finishes its execution (based on a configurable list of states).
-- [Webhook trigger](./03.webhook-trigger/index.md) allows you to execute your flow based on an HTTP request emitted by a webhook.
-- [Polling trigger](./04.polling-trigger/index.md) allows you to execute your flow by polling external systems for the presence of data.
-- [Realtime trigger](./05.realtime-trigger/index.md) allows you to execute your flow when events happen with millisecond latency.
+- [Schedule trigger](./01.schedule-trigger/index.md) — runs a flow on a cron schedule with optional calendar conditions.
+- [Flow trigger](./02.flow-trigger/index.md) — runs a flow when one or more upstream flows complete in a matching state.
+- [Webhook trigger](./03.webhook-trigger/index.md) — runs a flow in response to an inbound HTTP request.
+- [Polling trigger](./04.polling-trigger/index.md) — runs a flow when new data is detected in an external system.
+- [Realtime trigger](./05.realtime-trigger/index.md) — runs a flow with millisecond latency when an event arrives from a streaming source.
+- [MCP Tool trigger](./06.mcp-tool-trigger/index.md) — exposes a flow as a named tool on a Kestra MCP server, callable by AI agents such as Claude Desktop, Claude Code, and Cursor.
Many other triggers are available from the plugins, such as triggers based on file detection events, e.g. the [S3 trigger](/plugins/plugin-aws/aws-s3/io.kestra.plugin.aws.s3.trigger), or a new message arrival in a message queue, such as the [SQS](/plugins/plugin-aws/aws-sqs/io.kestra.plugin.aws.sqs.realtimetrigger) or [Kafka trigger](/plugins/plugin-kafka/io.kestra.plugin.kafka.trigger).
-### Trigger Common Properties
+### Trigger common properties
The following properties are common to all triggers:
| Field | Description |
| ----------------- | ---------------------------------------------------------------------------------------- |
-| `id` | The flow identifier, must be unique inside a flow. |
-| `type` | The Java FQDN of the trigger. |
+| `id` | The trigger identifier, must be unique within the flow. |
+| `type` | The fully qualified class name of the trigger. |
| `description` | The description of the trigger. |
| `disabled` | Set it to `true` to disable execution of the trigger. |
| `allowConcurrent` | Set it to `true` to allow multiple executions from this trigger to run at the same time. |
-| `workerGroup.key` | To execute this trigger on a specific Worker Group (EE). |
+| `workerSelector` | Route this trigger to a specific Worker Queue by tags (EE). See [Worker Groups](../../07.enterprise/04.scalability/worker-group/index.md). |
+| `when` | A Pebble expression that must evaluate to `true` for the trigger to fire. |
## Trigger variables
Triggers expose metadata through expressions. For example:
-– `{{ trigger.date }}` returns the current date for the [Schedule trigger](./01.schedule-trigger/index.md)
-– `{{ trigger.uri }}` returns the file or message for file detection or message arrival events
-– `{{ trigger.rows }}` provides query results for triggers like [PostgreSQL Query](/plugins/plugin-jdbc-postgres/io.kestra.plugin.jdbc.postgresql.trigger) trigger.
+- `{{ trigger.date }}` returns the current date for the [Schedule trigger](./01.schedule-trigger/index.md)
+- `{{ trigger.uri }}` returns the file or message for file detection or message arrival events
+- `{{ trigger.rows }}` provides query results for triggers like the [PostgreSQL Query](/plugins/plugin-jdbc-postgres/io.kestra.plugin.jdbc.postgresql.trigger) trigger
+- `{{ trigger._context.id }}` and `{{ trigger._context.type }}` return the ID and fully-qualified type of the trigger that started the execution — available for every trigger type
-This example will log the date when the trigger executes the flow:
+This example logs the date when the trigger executes the flow:
```yaml
id: variables
@@ -77,56 +76,108 @@ triggers:
```
:::alert{type="warning"}
-The **templated variables** above are only available when the execution is created **automatically** by the trigger. You'll get an error if you try to run a flow containing such variables **manually**.
+Trigger variables like `{{ trigger.date }}` are only available when the execution is created automatically by the trigger. Running the flow manually will produce an error.
+:::
-**You don't need an extra task to consume** the file or message from the event. Kestra downloads it automatically to **internal storage** and makes it available in your flow via the `{{ trigger.uri }}` variable. Check the documentation of a specific trigger and [Blueprints](/blueprints) with the **Trigger** tag for more details and examples.
+:::alert{type="info"}
+You don't need an extra task to consume a file or message from the event. Kestra automatically downloads it to internal storage and makes it available via `{{ trigger.uri }}`. See the plugin docs for a specific trigger and [Blueprints](/blueprints) tagged **Trigger** for examples.
:::
-Each trigger ID is limited to a single active execution at a time. If a scheduled execution is still running, the next one will be queued instead of started immediately. For instance, if an execution from a flow with a `Schedule` trigger with ID `hourly` is still in a `Running` state, another one will not be started. However, you can still trigger the same flow manually (from the UI or API), and the scheduled executions will not be affected.
+### Identifying the trigger in a multi-trigger flow
+
+When a flow has multiple triggers, use `{{ trigger._context.id }}` and `{{ trigger._context.type }}` to determine which trigger started the current execution. Unlike trigger-specific variables such as `trigger.date`, these two fields are available for every trigger type.
+
+The flow below branches based on which trigger started the execution:
```yaml
-id: hourlyFlow
+id: multi_trigger_flow
namespace: company.team
+
tasks:
- - id: important-task
- type: io.kestra.plugin.core.log.Log
- message: If this runs for longer than 1h, next Executions will be queued rather than being started immediately
+ - id: check_trigger
+ type: io.kestra.plugin.core.flow.If
+ condition: "{{ trigger._context.type == 'io.kestra.plugin.core.trigger.Schedule' }}"
+ then:
+ - id: scheduled_path
+ type: io.kestra.plugin.core.log.Log
+ message: "Started by schedule '{{ trigger._context.id }}' on {{ trigger.date }}"
+ else:
+ - id: webhook_path
+ type: io.kestra.plugin.core.log.Log
+ message: "Started by webhook '{{ trigger._context.id }}'"
+
triggers:
- - id: hourly
- type: io.kestra.plugin.core.trigger.Schedule
- cron: "@hourly"
+ - id: daily
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "@daily"
+
+ - id: api
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: my_secret_key
```
-## Conditions
+### Concurrency
-Conditions are criteria that determine when a trigger should create a new execution. Usually, they limit the scope of a trigger to a specific set of cases.
+By default, each trigger allows only one active execution at a time. If an execution is still running when the next trigger fires, the new execution is queued rather than started immediately. Manual executions from the UI or API are not subject to this limit — they always start regardless of trigger state.
-For example, you can restrict a Flow trigger to a specific namespace prefix or execution status, and you can restrict a Schedule trigger to a specific time of the week or month.
+To allow multiple concurrent executions from the same trigger, set `allowConcurrent: true` on the trigger.
-You can pass a list of conditions; in this case, all the conditions must match to enable the current action.
+## `when` and `dependsOn`
-Available conditions include:
+Kestra 2.0 replaces the old `conditions` list on triggers with two composable properties:
-- [HasRetryAttempt](/plugins/core/condition/io.kestra.plugin.core.condition.hasretryattempt)
-- [MultipleCondition](/plugins/core)
-- [Not](/plugins/core/condition/io.kestra.plugin.core.condition.not)
-- [Or](/plugins/core/condition/io.kestra.plugin.core.condition.or)
-- [ExecutionFlow](/plugins/core/condition/io.kestra.plugin.core.condition.executionflow)
-- [ExecutionNamespace](/plugins/core/condition/io.kestra.plugin.core.condition.executionnamespace)
-- [ExecutionLabels](/plugins/core/condition/io.kestra.plugin.core.condition.executionlabels)
-- [ExecutionStatus](/plugins/core/condition/io.kestra.plugin.core.condition.executionstatus)
-- [ExecutionOutputs](/plugins/core/condition/io.kestra.plugin.core.condition.executionoutputs)
-- [Expression](/plugins/core/condition/io.kestra.plugin.core.condition.expression)
+### `when` — Pebble expression (all triggers)
-You can also find datetime related conditions [on the Schedule trigger page](./01.schedule-trigger/index.md#schedule-conditions).
+Every trigger type supports a `when` property. It accepts a [Pebble expression](../../expressions/index.mdx) that is evaluated at trigger time. If the expression evaluates to a falsy value (`false`, `0`, empty string), the trigger does not fire.
-## Unlocking, enabling, and disabling triggers
+Use `when` to express time-based conditions on Schedule triggers, to filter Webhook payloads, or to add a global guard on any trigger type:
-Triggers do not always need to be enabled. Disable a trigger whenever you want to pause a flow during the development phase or other instances.
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 9 * * *"
+ when: "{{ not isWeekend(trigger.date) and not isPublicHoliday(trigger.date, 'FR') }}"
+```
+
+```yaml
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: 4wjtkzwVGBM9yKnjm3yv8r
+ when: "{{ trigger.body.hello == 'world' }}"
+```
+
+For calendar-based scheduling, helper functions like `isWeekend()`, `isPublicHoliday()`, `isDayWeekInMonth()`, and `dayOfWeek()` are available. See [date and calendar helpers](../../expressions/04.functions/06.dates/index.mdx) in the expressions reference.
+
+### `dependsOn` — upstream flow dependencies (Flow trigger only)
+
+The [Flow trigger](./02.flow-trigger/index.md) replaces both `conditions` and `preconditions` with a `dependsOn` list. Each entry declares one upstream flow that must complete in a matching state. All entries must be satisfied before the trigger fires.
+
+```yaml
+triggers:
+ - id: after_staging
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: stg_sales
+ namespace: company.team
+ - flowId: stg_marketing
+ namespace: company.team
+ window:
+ deadline: "09:00:00"
+```
+
+See the [Flow trigger documentation](./02.flow-trigger/index.md) for the full `dependsOn` and `window` property reference.
+
+:::alert{type="warning"}
+The `conditions` list is removed in Kestra 2.0. Flows that still use it will fail to parse after upgrading. See the [trigger conditions migration guide](../../11.migration-guide/v2.0.0/trigger-conditions-redesign/index.md) for before/after examples.
+:::
+
+## Unlocking, enabling, and disabling triggers
### Disabling a trigger in the source code
-If you want to temporarily disable a trigger, you could do so by setting the `disabled` property to `true`, as you can see in the example below:
+Set `disabled: true` on the trigger to stop it from firing without removing it:
```yaml
id: hello_world
@@ -147,30 +198,24 @@ triggers:
disabled: true
```
-However, this approach requires changing the source code. A simpler approach is to use the `Enabled` toggle in the UI.
+To avoid a source code change, use the **Enabled** toggle in the UI instead.
### Disabling a trigger from the UI
-You can disable or re-enable a trigger from the UI. Here is how you can do it:
-
-1. Go to the `Flows` page and click on the flow you want to disable the trigger for.
-2. Go to the `Triggers` tab and click on the `Enabled` toggle next to the trigger you want to disable. You can re-enable it by clicking the toggle again.
+To disable or re-enable a trigger:
-
+1. Open the flow and go to the **Triggers** tab.
+2. Toggle **Enabled** next to the trigger. Toggle again to re-enable.
-If your trigger is locked due to an execution in progress, you can unlock it by clicking the `Unlock trigger` button.
-
-
-
-The **Unlock trigger** functionality is useful for troubleshooting, e.g. if a process is stuck due to infrastructure issues. Keep in mind that manually unlocking triggers may result in multiple concurrent (potentially duplicated) executions — use it with caution.
+If your trigger is locked due to an execution in progress, click **Unlock trigger** in the Triggers tab. Manually unlocking a trigger may result in duplicate executions — use it with caution.
:::alert{type="info"}
-Only scheduled-based triggers (triggers handled by the Scheduler) will be visible in the UI. Triggers handled by the Executor and Webserver will not be displayed. This also applies when fetching triggers from the API.
+Only scheduler-based triggers are visible in the Triggers tab and API. Flow Triggers (handled by the Executor) and Webhook triggers (handled by the Webserver) do not appear there.
:::
-### Toggle, unlock, or delete triggers from the Administration page
+### Toggle, unlock, or delete triggers from Tenant → Triggers
-From **Administration → Triggers** you can bulk manage trigger state:
+From **Tenant → Triggers** you can bulk manage trigger state:
- **Toggle** — enable or disable one or more triggers without editing the flow code.
- **Unlock** — clear the “locked” state if a trigger is stuck waiting on a long-running execution (use carefully, as this may create duplicate executions).
@@ -180,50 +225,45 @@ From **Administration → Triggers** you can bulk manage trigger state:
Deleting a trigger is different from deleting a backfill: removing a backfill only cancels pending catch-up runs, while deleting a trigger resets the trigger entity itself. Use **Delete backfill** to stop scheduled replays and **Delete trigger** to rebuild the trigger state.
-
-
## Troubleshooting a trigger from the UI
If you misconfigured a trigger, and as a result, no Executions are created, take the following actions to troubleshoot.
-The example flow below illustrates this scenario. Note how the `sqs_trigger` trigger is misconfigured with invalid AWS credentials:
+The example flow below illustrates this scenario — the `http_poll` trigger points to a non-existent host:
```yaml
-id: bad_trigger_example
+id: misconfigured_trigger
namespace: company.team
tasks:
- - id: hello
- type: io.kestra.plugin.core.log.Log
- message: Hello World!
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ trigger.body }}"
triggers:
- - id: sqs_trigger
- type: io.kestra.plugin.aws.sqs.Trigger
- accessKeyId: "nonExistingKey"
- secretKeyId: "nonExistingSecret"
- region: "us-east-1"
- queueUrl: "https://sqs.us-east-1.amazonaws.com/123456789/testQueue"
- maxRecords: 10
+ - id: http_poll
+ type: io.kestra.plugin.core.http.Trigger
+ uri: https://not-a-real-host-xyz123.invalid/endpoint
+ interval: PT10S
```
-When you add that flow to Kestra, you'll see that no Executions are created. To troubleshoot this, you can go to the `Triggers` tab on the Flow's page and **expand the logs** of the trigger that is causing the issue. You'll see a detailed error message that will help you identify the problem:
+When you save the flow, no executions are created. To troubleshoot, go to the **Triggers** tab on the flow's page and click the trigger row to expand its logs. A detailed error message identifies the problem:
-## The `stopAfter` property
+## `stopAfter`
-Kestra 0.15 introduced a generic `stopAfter` property which is a list of states that will disable the trigger after the flow execution has reached one of the states in the list.
+The `stopAfter` property is a list of states that disable the trigger after the flow execution reaches one of those states.
This property is most useful with `Schedule` triggers and polling-based triggers such as HTTP, JDBC, or File Detection.
:::alert{type="info"}
-Note that we don't handle any automatic trigger reenabling logic. After a trigger has been disabled due to the `stopAfter` state condition, you can take some action based on it and manually reenable the trigger.
+Kestra does not automatically re-enable a trigger after it has been disabled by `stopAfter`. You must re-enable it manually once you are ready to resume.
:::
-### Pause the schedule trigger after a failed execution
+### Pause a schedule after failure
-The `stopAfter` property can be used to pause a schedule trigger after a failed execution. Here is an example of how to use it:
+This flow runs daily at 9 AM. If it fails, the schedule pauses until you re-enable it manually — preventing repeated failures on a broken pipeline:
```yaml
id: business_critical_flow
@@ -242,11 +282,9 @@ triggers:
- FAILED
```
-The above flow will be triggered every day at `9:00` AM, but if it fails, the schedule will be paused so that you can manually reenable the trigger once the issue is fixed. This is useful for business-critical flows that should not continue running the next scheduled executions if a previous execution has failed.
-
-### Disable the HTTP trigger after the first successful execution
+### Disable the HTTP trigger after first success
-The example below shows how to use the `stopAfter` property with the HTTP trigger condition. The use case is to poll an API endpoint and send a Slack alert if the price is below $110. If the condition is met, the trigger will be disabled so that you don't get alerted every 30 seconds about the same condition.
+This flow polls an API every 30 seconds and sends a Slack alert when the price drops below $110. `stopAfter: [SUCCESS]` disables the trigger after one alert fires, preventing repeated notifications for the same condition.
```yaml
id: http
@@ -268,24 +306,12 @@ triggers:
- SUCCESS
```
-Let's break down the above example:
-
-1. The HTTP trigger will poll the API endpoint every 30 seconds to check if the price of a product is below $110.
-2. If the condition is met, the Execution will be created
-3. Within that execution, the `slack` task will send a Slack message to notify about the price change
-4. After that execution finishes successfully, the `stopAfter` property condition is met — it will disable the trigger ensuring that you don't get alerted every 30 seconds about the same condition.
-
-## Locked triggers
-
-[Flow](./02.flow-trigger/index.md), [Schedule](./01.schedule-trigger/index.md), and [Polling triggers](./04.polling-trigger/index.md) have locks to avoid concurrent trigger evaluation and concurrent execution of a flow for a trigger.
-
-To see a list of triggers and inspect their current status, go to the **Administration -> Triggers** section in the Kestra UI. From here, you can unlock a trigger if it is locked. Note that doing so raises a risk of concurrent trigger evaluation or flow execution for this trigger if you unlock it manually.
-## Setting inputs inside of triggers
+## Passing inputs to triggers
-You can easily pass inputs to triggers by using the `inputs` property and passing them as a key-value pair.
+Use the `inputs` property on any trigger to set input values before execution:
-In this example, the `user` input is set to "John Smith" inside of the `schedule` trigger:
+In this example, the `user` input is set to "John Smith" by the `schedule` trigger:
```yaml
id: myflow
@@ -313,7 +339,7 @@ triggers:
By default, if a trigger fails, no execution is created; this is by design to avoid excessive executions on the instance. To troubleshoot, you must [investigate the trigger logs](#troubleshooting-a-trigger-from-the-ui). If you'd prefer an execution to be created on trigger failure, set the `failOnTriggerError` property to `true` in the trigger configuration. This will cause the flow to fail and produce an execution with its own logs.
-For example, take the following flow with a misconfigured trigger:
+Adding `failOnTriggerError: true` produces a `FAILED` execution with full logs instead of a silent no-op:
```yaml
id: bad_trigger_example
@@ -335,6 +361,6 @@ triggers:
failOnTriggerError: true
```
-With this configuration, the flow will produce an execution containing logs that describe the trigger failure. This execution can be used for both troubleshooting and notification, in addition to the trigger logs.
+The resulting execution is visible in the UI and can trigger notification flows, in addition to appearing in the trigger logs.
\ No newline at end of file
diff --git a/src/contents/docs/05.workflow-components/07.triggers/invalid_trigger_configuration.png b/src/contents/docs/05.workflow-components/07.triggers/invalid_trigger_configuration.png
index 7efe82839c8..3d8e7f74586 100644
Binary files a/src/contents/docs/05.workflow-components/07.triggers/invalid_trigger_configuration.png and b/src/contents/docs/05.workflow-components/07.triggers/invalid_trigger_configuration.png differ
diff --git a/src/contents/docs/05.workflow-components/07.triggers/trigger_unlock.png b/src/contents/docs/05.workflow-components/07.triggers/trigger_unlock.png
deleted file mode 100644
index 4ac6afc674e..00000000000
Binary files a/src/contents/docs/05.workflow-components/07.triggers/trigger_unlock.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/07.triggers/triggers-administration.png b/src/contents/docs/05.workflow-components/07.triggers/triggers-administration.png
index 995b1c313cb..eb1a4d94a27 100644
Binary files a/src/contents/docs/05.workflow-components/07.triggers/triggers-administration.png and b/src/contents/docs/05.workflow-components/07.triggers/triggers-administration.png differ
diff --git a/src/contents/docs/05.workflow-components/07.triggers/triggers_flow.png b/src/contents/docs/05.workflow-components/07.triggers/triggers_flow.png
deleted file mode 100644
index ef6eb09215a..00000000000
Binary files a/src/contents/docs/05.workflow-components/07.triggers/triggers_flow.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/08.labels/bulk-set-labels.png b/src/contents/docs/05.workflow-components/08.labels/bulk-set-labels.png
new file mode 100644
index 00000000000..bb7955a7089
Binary files /dev/null and b/src/contents/docs/05.workflow-components/08.labels/bulk-set-labels.png differ
diff --git a/src/contents/docs/05.workflow-components/08.labels/execution-overview-labels.png b/src/contents/docs/05.workflow-components/08.labels/execution-overview-labels.png
new file mode 100644
index 00000000000..a28d5778f8a
Binary files /dev/null and b/src/contents/docs/05.workflow-components/08.labels/execution-overview-labels.png differ
diff --git a/src/contents/docs/05.workflow-components/08.labels/executions-list-labels.png b/src/contents/docs/05.workflow-components/08.labels/executions-list-labels.png
new file mode 100644
index 00000000000..7e5814478bc
Binary files /dev/null and b/src/contents/docs/05.workflow-components/08.labels/executions-list-labels.png differ
diff --git a/src/contents/docs/05.workflow-components/08.labels/index.md b/src/contents/docs/05.workflow-components/08.labels/index.md
index b9a1640c6d0..e6d941e0dd4 100644
--- a/src/contents/docs/05.workflow-components/08.labels/index.md
+++ b/src/contents/docs/05.workflow-components/08.labels/index.md
@@ -6,19 +6,15 @@ sidebarTitle: Labels
icon: /src/contents/docs/icons/flow.svg
---
-Labels are key-value pairs in Kestra that let you organize [flows](../01.flow/index.md) and [executions](../03.execution/index.md) across multiple dimensions, without being restricted to a single hierarchy.
+Labels are key-value pairs attached to [flows](../01.flow/index.md) and [executions](../03.execution/index.md). Unlike namespaces, which form a fixed hierarchy, labels let you slice execution data across any dimension — team, project, environment, priority — and combine them freely.
-You can organize flows and executions by project, priority, maintainer, or any other relevant criteria. Unlike fixed categories, labels support flexible filtering, grouping, and discovery.
-
-Labels can be associated with both the flow definition and individual execution instances. This allows you to distinguish between different executions of the same flow.
+A label can be set on the flow definition, on individual execution instances, or both, making it possible to group across flows or distinguish between runs of the same flow.
-Labels help organize and filter flows and their executions based on your criteria. Adding a labels section to flows lets you sort and group executions, making them easier to discover and analyze.
-
-Here's a simple example of a flow with two labels defined:
+A flow with two labels:
```yaml
id: process_invoice_flow
@@ -34,11 +30,11 @@ tasks:
message: hello from a flow with labels
```
-Executing such a flow results in the execution inheriting both `team: finance` and `priority: HIGH` labels by default. However, you can also define additional labels at the time of execution launch.
+When you execute this flow, executions inherit both `team: finance` and `priority: HIGH` labels. You can also define additional labels at execution launch.
## Benefits of labels
-Labels provide a simple and effective way to organize and filter flows and their executions. Key benefits include:
+Labels let you organize and filter flows and their executions. Key benefits include:
- **Observability**: Track execution status, monitor errors, and rerun only a subset of executions.
@@ -46,7 +42,7 @@ Labels provide a simple and effective way to organize and filter flows and their
- **Organization**: Manage workflows at scale by grouping executions by team, project, maintainer, or environment.
-You can also build custom dashboards using labels, for example: `http://localhost:8080/ui/executions?filters[labels][EQUALS][team]=finance`.
+You can also build custom dashboards using labels to filter executions by any label value.
### Common scenarios
@@ -54,8 +50,7 @@ To group flows related to the same project across [Kestra namespaces](../02.name
When running the `process_invoice_flow`, you can add execution labels (e.g., `currency`) to capture attributes of the processed invoice. This allows you to filter executions by specific values, like `currency: USD`.
-You can also label executions related to a pre-production run. For example, using a `purpose: pre-prod` label. This enables you to safely
-delete only those executions associated with the pre-production phase.
+You can also label executions related to a pre-production run — for example, `purpose: pre-prod` — so you can safely delete only those executions when the pre-production phase is complete.
In multi-team environments, labels help you separate executions by team, for example `support: EMEA` and `support: APAC`, when the same flow handles data from different regions.
@@ -63,37 +58,35 @@ In multi-team environments, labels help you separate executions by team, for exa
When you execute a flow with labels, those labels are automatically applied to its executions.
-
+
-
+
-## Set execution labels when executing a flow from the UI
+## Set execution labels manually
-When executing flows manually from the UI, you can override and define new labels at the execution's start by expanding the **Advanced configuration** section:
+When executing a flow manually, expand **Advanced configuration** to override or define labels at start:
-You can set labels from the UI even after an execution completes. This helps with collaboration and troubleshooting.
+You can also set labels after an execution completes — useful for collaboration and troubleshooting.
For example, you can add a label to a failed execution to indicate its status, such as whether it has been acknowledged, is being investigated, or has been resolved.
-To set labels from the UI, go to the **Overview** tab of an **Execution** and click on the "Set labels" button. You can add multiple labels at once.
+Go to the **Overview** tab of an execution and click **Set labels** to add one or more labels.
-
+
-You can even set labels for multiple executions at once from the UI. This feature is helpful for bulk operations, such as acknowledging multiple failed executions at once after an outage.
+You can also set labels for multiple executions at once — useful for bulk operations such as acknowledging multiple failed executions after an outage.
-
+
## Set labels based on flow inputs and task outputs
-You have the ability to set execution labels from a dedicated [Labels task](/plugins/core/execution/io.kestra.plugin.core.execution.labels). This task provides a dynamic way to label your flows, helping with observability, debugging, and monitoring of failures.
+Use the [Labels task](/plugins/core/execution/io.kestra.plugin.core.execution.labels) to set execution labels based on flow inputs, task outputs, or other runtime data. There are two ways to set labels in this task:
-This task lets you set custom execution labels based on flow inputs, task outputs, or other dynamic workflow data. There are two ways to set labels in this task:
-
-1. **Using a Map (Key-Value Pairs)**: ideal when the `key` is static and the `value` is dynamic. The key is the label name, and the value is a dynamic label value that might be derived from the flow inputs or task outputs. In the example below, the task `update_labels` overrides the default label `song` with the output of the `get` task, and adds a new label called `artist`.
+1. **Using a map (key-value pairs)**: ideal when the key is static and the value is dynamic. In the example below, `update_labels` overrides the default label `song` with the output of the `get` task and adds a new label `artist`.
```yaml
id: labels_override
@@ -114,7 +107,7 @@ tasks:
artist: rick_astley # new label
```
-2. **Using a List of Key-Value Pairs**: particularly useful if both the `key` and the `value` are dynamic properties.
+2. **Using a list of key-value pairs**: use this form when both the key and value are dynamic.
```yaml
id: labels
@@ -148,7 +141,7 @@ tasks:
### Overriding flow labels at runtime
-You can set default labels at the flow level and override them at runtime. This approach is useful for overriding labels dynamically during execution, based on task results.
+You can set default labels at the flow level and override them during execution based on task results.
The example below shows how to override the default label `song` with the output of the `get` task:
@@ -173,3 +166,51 @@ tasks:
```
In this example, the default label `song` is overridden by the output of the `get` task.
+
+## Dynamic labels in trigger-started executions
+
+When a trigger starts an execution, the trigger's `labels` values accept Pebble expressions. This lets you embed runtime context — such as the current date or a trigger variable — directly in labels at the moment execution begins, without needing a separate `Labels` task.
+
+**Using a Pebble function:**
+
+```yaml
+id: scheduled_flow
+namespace: company.team
+
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "* * * * *"
+ labels:
+ year: "year-{{now(format='YYYY')}}"
+
+tasks:
+ - id: hello
+ type: io.kestra.plugin.core.log.Log
+ message: Hello World!
+```
+
+Each execution started by this trigger carries a `year` label with the current year when the trigger fires.
+
+**Using a trigger variable:**
+
+```yaml
+id: scheduled_flow
+namespace: company.team
+
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "* * * * *"
+ labels:
+ previous_run: "{{trigger.previous}}"
+
+tasks:
+ - id: hello
+ type: io.kestra.plugin.core.log.Log
+ message: Hello World!
+```
+
+`trigger.previous` holds the date of the previous scheduled run. Labelling executions with this value helps identify late or catch-up runs.
+
+Static values and expressions can be mixed in the same `labels` block. Available trigger variables differ by trigger type — see the [Schedule trigger](../07.triggers/01.schedule-trigger/index.md), [Realtime trigger](../07.triggers/05.realtime-trigger/index.md), and other trigger reference pages for the full list.
diff --git a/src/contents/docs/05.workflow-components/08.labels/labels1.png b/src/contents/docs/05.workflow-components/08.labels/labels1.png
deleted file mode 100644
index 8b8d6c6d318..00000000000
Binary files a/src/contents/docs/05.workflow-components/08.labels/labels1.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/08.labels/labels2.png b/src/contents/docs/05.workflow-components/08.labels/labels2.png
deleted file mode 100644
index 67550cd7f2a..00000000000
Binary files a/src/contents/docs/05.workflow-components/08.labels/labels2.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/08.labels/labels3.png b/src/contents/docs/05.workflow-components/08.labels/labels3.png
deleted file mode 100644
index 4bb024f0762..00000000000
Binary files a/src/contents/docs/05.workflow-components/08.labels/labels3.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/08.labels/labels4.png b/src/contents/docs/05.workflow-components/08.labels/labels4.png
deleted file mode 100644
index e1adc23308f..00000000000
Binary files a/src/contents/docs/05.workflow-components/08.labels/labels4.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/08.labels/set-labels-dialog.png b/src/contents/docs/05.workflow-components/08.labels/set-labels-dialog.png
new file mode 100644
index 00000000000..46a0815ae86
Binary files /dev/null and b/src/contents/docs/05.workflow-components/08.labels/set-labels-dialog.png differ
diff --git a/src/contents/docs/05.workflow-components/09.plugin-defaults/index.md b/src/contents/docs/05.workflow-components/09.plugin-defaults/index.md
deleted file mode 100644
index 86cad7b38d8..00000000000
--- a/src/contents/docs/05.workflow-components/09.plugin-defaults/index.md
+++ /dev/null
@@ -1,172 +0,0 @@
----
-title: Plugin Defaults in Kestra – Set Task-Level Defaults
-h1: Reduce Repetition by Setting Global Plugin Defaults
-description: Streamline Kestra flow configuration with Plugin Defaults. Set global or flow-level default values for task properties to reduce repetition and boilerplate.
-sidebarTitle: Plugin Defaults
-icon: /src/contents/docs/icons/flow.svg
-docId: plugin-defaults
----
-
-Plugin defaults are default values applied to every task of a given type within one or more flows.
-
-They work like default function arguments, helping you avoid repetition when tasks or plugins frequently use the same values.
-
-
-
-
-
-## Plugin Defaults on a flow-level
-
-You can define plugin defaults in the `pluginDefaults` section to avoid repeating properties across multiple tasks of the same type. For example:
-
-```yaml
-id: api_python_sql
-namespace: company.team
-
-tasks:
- - id: api
- type: io.kestra.plugin.core.http.Request
- uri: https://dummyjson.com/products
-
- - id: hello
- type: io.kestra.plugin.scripts.python.Script
- script: |
- print("Hello World!")
-
- - id: python
- type: io.kestra.plugin.scripts.python.Script
- beforeCommands:
- - pip install polars
- outputFiles:
- - "products.csv"
- script: |
- import polars as pl
- data = {{outputs.api.body | jq('.products') | first}}
- df = pl.from_dicts(data)
- df.glimpse()
- df.select(["brand", "price"]).write_csv("products.csv")
-
- - id: sql_query
- type: io.kestra.plugin.jdbc.duckdb.Query
- inputFiles:
- in.csv: "{{ outputs.python.outputFiles['products.csv'] }}"
- sql: |
- SELECT brand, round(avg(price), 2) as avg_price
- FROM read_csv_auto('{{workingDir}}/in.csv', header=True)
- GROUP BY brand
- ORDER BY avg_price DESC;
- store: true
-
-pluginDefaults:
- - type: io.kestra.plugin.scripts.python.Script
- values:
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- pullPolicy: ALWAYS # set it to NEVER to use a local image
- containerImage: python:slim
-```
-
-In this example, Docker and Python configurations are defined once in `pluginDefaults`, instead of being repeated in every task. This approach helps to streamline the configuration process and reduce the chances of errors caused by inconsistent settings across different tasks.
-
-:::alert{type="info"}
-If you move required attributes into `pluginDefaults`, the UI code editor may show warnings about missing arguments, because defaults are only resolved at runtime. As long as `pluginDefaults` contains the relevant arguments, you can save the flow and ignore the warning displayed in the editor.
-
-
-
-:::
-
-### `forced` attribute in `pluginDefaults`
-
-Setting `forced: true` in `pluginDefaults` ensures that default values override any properties defined directly in the task. By default, the value of the `forced` attribute is `false`.
-
-## Precedence of plugin defaults
-
-Kestra applies non-forced plugin defaults from lowest to highest priority:
-
-1. Global plugin defaults (`kestra.plugins.defaults`)
-2. Namespace-level plugin defaults
-3. Flow-level `pluginDefaults`
-4. Properties defined directly on the task
-
-For forced defaults the direction reverses — the most privileged level wins:
-
-1. Global forced defaults (`kestra.plugins.defaults` with `forced: true`) ← highest priority
-2. Namespace-level forced defaults
-3. Flow-level forced defaults
-
-This means a global forced default cannot be overridden by a namespace-level forced default. Use global forced defaults for platform-wide policies that must apply unconditionally.
-
-## Plugin defaults in a global configuration
-
-Plugin defaults can also be defined globally in your Kestra configuration, applying the same values across all flows. This is useful when you want to apply the same defaults across multiple flows. Let's say that you want to centrally manage the default values for the `io.kestra.plugin.aws` plugin to reuse the same credentials and region across all your flows. You can add the following to your Kestra configuration:
-
-```yaml
-kestra:
- plugins:
- defaults:
- - type: io.kestra.plugin.aws
- values:
- accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
- secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
- region: "us-east-1"
-```
-
-If you want to set defaults only for a specific task, you can do that too:
-
-```yaml
-kestra:
- plugins:
- defaults:
- - type: io.kestra.plugin.aws.s3.Upload
- values:
- accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
- secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
- region: "us-east-1"
-```
-
-### Nested property values
-
-For plugins with nested properties, define the values using the same nested YAML structure you would use in a flow. For example, to set resource limits for the Kubernetes task runner:
-
-```yaml
-kestra:
- plugins:
- defaults:
- - type: io.kestra.plugin.ee.kubernetes.runner.Kubernetes
- forced: true
- values:
- resources:
- limit:
- cpu: "1"
- memory: "128Mi"
-```
-
-This is equivalent to writing the same nested structure directly in a task. The `forced: true` attribute ensures these defaults override any values set at the task level.
-
-## Plugin Defaults Enterprise Edition
-
-:::alert{type="info"}
-In the [Enterprise Edition](../../07.enterprise/index.mdx) or [Kestra Cloud](/cloud), plugin defaults can be configured directly in the UI under the **Plugin Defaults** tab of a Namespace.
-:::
-
-You can create them via form or directly as YAML code for the Namespace:
-
-
-
-Or click on **YAML** and, for example, paste the following:
-
-```yaml
-- type: io.kestra.plugin.aws.s3.Upload
- values:
- accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
- secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
- region: "us-east-1"
-```
-
-### Inherited Plugin Defaults
-
-Plugin Defaults are inherited from the parent Namespace to children Namespaces. In the example above, the image shows the Plugin Default was created in the `kestra.company` Namespace. Navigating to the **Plugin Defaults** tab of a child Namespace, for example `kestra.company.data`, shows the parent Namespace's Plugin Defaults. This avoids having to recreate Plugin Defaults across children Namespaces, but it still allows for the children Namespaces to maintain their own isolated defaults if needed.
-
-
-
-
diff --git a/src/contents/docs/05.workflow-components/09.plugin-defaults/inherited-plugin-defaults.png b/src/contents/docs/05.workflow-components/09.plugin-defaults/inherited-plugin-defaults.png
deleted file mode 100644
index 4752be7ad1e..00000000000
Binary files a/src/contents/docs/05.workflow-components/09.plugin-defaults/inherited-plugin-defaults.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/09.plugin-defaults/plugin-default-creation.png b/src/contents/docs/05.workflow-components/09.plugin-defaults/plugin-default-creation.png
deleted file mode 100644
index f9a2289b3b9..00000000000
Binary files a/src/contents/docs/05.workflow-components/09.plugin-defaults/plugin-default-creation.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/09.plugin-defaults/warning.png b/src/contents/docs/05.workflow-components/09.plugin-defaults/warning.png
deleted file mode 100644
index 0e6fc02c8cc..00000000000
Binary files a/src/contents/docs/05.workflow-components/09.plugin-defaults/warning.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/10.subflows/index.md b/src/contents/docs/05.workflow-components/10.subflows/index.md
index 6f057b9e0da..23fa5925def 100644
--- a/src/contents/docs/05.workflow-components/10.subflows/index.md
+++ b/src/contents/docs/05.workflow-components/10.subflows/index.md
@@ -6,7 +6,7 @@ sidebarTitle: Subflows
icon: /src/contents/docs/icons/flow.svg
---
-Subflows let you build **modular** and **reusable** workflow components.
+Subflows let you build modular and reusable workflow components.
They work like function calls: executing a subflow creates a new flow run from within another flow.
@@ -16,29 +16,23 @@ They work like function calls: executing a subflow creates a new flow run from w
## Why use a subflow?
-Subflows allow you to build modular and reusable components that you can use across multiple flows. For example, you might define a subflow that handles error alerts by posting to Slack and email. By using a Subflow, you can reuse these two tasks together for all flows that you want to send error notifications, instead of having to copy the individual tasks for every flow.
-
-:::alert{type="warning"}
-Recursive flows are not supported. Kestra doesn’t allow a flow to call itself (directly or indirectly). Any cycle **(flowA→flowA)** makes the flow invalid. Recursive execution can create infinite loops and unbounded fan-out.
-
-**Do instead:** Use **[ForEach](/plugins/core/flow/io.kestra.plugin.core.flow.foreach)** and **[branching flowable](../01.tasks/00.flowable-tasks/index.md)** tasks to iterate or split work without creating cycles (e.g., [LoopUntil](/plugins/core/flow/io.kestra.plugin.core.flow.loopuntil)).
-:::
+For example, you might define a subflow that handles error alerts by posting to Slack and email. Any flow that needs error notifications calls this subflow rather than duplicating those tasks.
## How to declare a subflow
-To call a flow from another flow, use the `io.kestra.plugin.core.flow.Subflow` task, and in that task, specify the `flowId` and `namespace` of the subflow that you want to execute. You can also specify custom `inputs`, similar to passing arguments to a function.
+Use the `io.kestra.plugin.core.flow.Subflow` task and specify the `flowId` and `namespace` of the flow to execute. Pass custom `inputs` the same way you would pass arguments to a function.
-The optional properties `wait` and `transmitFailed` control the execution behavior. By default, if `wait` is omitted or set to `false`, the parent flow continues without waiting for the subflow to finish. The `transmitFailed` property determines whether a failure in the subflow execution should cause the parent flow to fail.
+The `wait` and `transmitFailed` properties control execution behavior. If `wait` is `false`, the parent flow continues without waiting for the subflow to finish. `transmitFailed` determines whether a failure in the subflow causes the parent flow to fail.
-:::alert{type="info"}
-A Subflow task acts like a trigger to execute the child flow. While not managed like [Triggers](../07.triggers/index.mdx) in the UI, it is conceptually similar.
+:::alert{type="warning"}
+Recursive flows are not supported. A flow cannot call itself directly or indirectly — any cycle makes the flow invalid and risks infinite loops and unbounded fan-out.
+
+**Do instead:** Use **[Loop](/plugins/core/flow/io.kestra.plugin.core.flow.loop)** and **[branching flowable](../01.tasks/00.flowable-tasks/index.md)** tasks to iterate or split work without creating cycles (e.g., [LoopUntil](/plugins/core/flow/io.kestra.plugin.core.flow.loopuntil)).
:::
## Practical example
-A subflow can encapsulate critical business logic, making it reusable across flows and easier to test in isolation.
-
-Here is a simple example of a subflow:
+A subflow can encapsulate business logic that is reusable across flows and easy to test in isolation:
```yaml
id: critical_service
@@ -60,7 +54,7 @@ outputs:
value: "{{ outputs.return_data.uri }}"
```
-In this example, `return_data` outputs `uri` of the query output. That URI is a reference to the internal storage location of the stored file. This output can be used in the parent flow to perform further processing.
+Here, `return_data` outputs the URI of the stored query result — a reference to its location in Kestra's internal storage. The parent flow can pass this URI to downstream tasks for further processing.
```yaml
id: parent_service
@@ -86,7 +80,7 @@ The `outputs` map task IDs to their results. Here, the parent flow accesses the
## Subflow properties
-Below is a full list of all properties of the `io.kestra.plugin.core.flow.Subflow` task. You don’t need to memorize all properties — the task documentation always lists them.
+`io.kestra.plugin.core.flow.Subflow` task properties:
| Field | Description |
|------------------------|-----------------------------------------------------------------------------|
@@ -95,7 +89,7 @@ Below is a full list of all properties of the `io.kestra.plugin.core.flow.Subflo
| `inheritLabels` | Determines if the subflow inherits labels from the parent (default: false). |
| `inputs` | Inputs passed to the subflow |
| `labels` | Labels assigned to the subflow |
-| `outputs` (deprecated) | Allows passing outputs from the subflow execution to the parent flow. |
+| `outputs` (deprecated) | Passes subflow outputs to the parent flow. Use [flow outputs](../06.outputs/index.md#pass-data-between-flows-using-flow-outputs) instead. |
| `revision` | The subflow revision to execute (defaults to the latest) |
| `scheduleDate` | Schedule subflow execution on a specific date rather than immediately. |
| `transmitFailed` | If true, parent flow fails on subflow failure (requires `wait` to be true). |
@@ -104,15 +98,11 @@ Below is a full list of all properties of the `io.kestra.plugin.core.flow.Subflo
## Passing data between parent and child flows
-Flows can emit outputs that can be accessed by the parent flow. Using the `io.kestra.plugin.core.flow.Subflow` task you can call any flow as a subflow and access its outputs in downstream tasks. For more details and examples, check the [Outputs page](../06.outputs/index.md#pass-data-between-flows-using-flow-outputs).
-
-### Accessing Outputs from a subflow execution
-
-Outputs include the execution ID, extracted outputs, and the final state (if `wait` is true).
+Any flow can emit outputs accessible to the parent flow. For more details and examples, see the [Outputs page](../06.outputs/index.md#pass-data-between-flows-using-flow-outputs).
-Subflows improve maintainability of complex workflows. Use them to build modular, reusable components that can be shared across namespaces, projects, and teams.
+### Accessing outputs from a subflow execution
-Here’s an example of a subflow with explicitly defined outputs.
+Outputs include the execution ID, extracted outputs, and the final state (if `wait` is true). For example:
```yaml
id: flow_outputs
@@ -129,7 +119,7 @@ outputs:
value: "{{ outputs.mytask.value }}"
```
-We can access these outputs from a parent task as seen in the example below:
+Access these outputs in the parent flow:
```yaml
id: parent_flow
@@ -147,11 +137,10 @@ tasks:
message: "{{ outputs.subflow.outputs.final }}"
```
-For more details, see the [sublow outputs documentation](../../11.migration-guide/v0.15.0/subflow-outputs/index.md).
### Passing inputs to a subflow
-You can pass inputs to a Subflow task. The example below passes two inputs to a subflow.
+The example below passes a URL input from the parent flow into the subflow.
Subflow:
```yaml
@@ -200,7 +189,6 @@ tasks:
message: "{{ outputs.subflow.outputs.data }}"
```
-In this example, the parent flow successfully passes an input to the subflow.
#### Nested inputs
diff --git a/src/contents/docs/05.workflow-components/11.errors/index.md b/src/contents/docs/05.workflow-components/11.errors/index.md
index 93b20a18cf1..04b44a37539 100644
--- a/src/contents/docs/05.workflow-components/11.errors/index.md
+++ b/src/contents/docs/05.workflow-components/11.errors/index.md
@@ -14,9 +14,9 @@ Kestra provides multiple ways to handle errors, helping you both identify issues
## `errors` Component
-`errors` is a list of tasks set at the flow level that are executed when an error occurs. You can add multiple tasks, and they are executed sequentially. This is useful for sending alerts when errors occur.
+`errors` is a list of tasks executed at the flow level when an error occurs. Tasks run sequentially.
-The example below sends a flow-level failure alert via Slack using the [SlackIncomingWebhook](/plugins/plugin-slack/slack-notifications/io.kestra.plugin.slack.notifications.slackincomingwebhook) task defined using the `errors` property.
+The following flow fails immediately and sends an alert via Slack:
```yaml
id: errors
@@ -31,7 +31,7 @@ tasks:
errors:
- id: alert_on_failure
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
- url: secret('SLACK_WEBHOOK')
+ url: "{{ secret('SLACK_WEBHOOK') }}"
messageText: "Failure alert for flow {{ flow.namespace }}.{{ flow.id }} with ID {{ execution.id }}"
```
@@ -50,24 +50,10 @@ For post-run actions based on the final execution state, see the [`afterExecutio
| Run different tasks for `SUCCESS`, `FAILED`, or `WARNING` | `afterExecution` |
| Run reports or notifications that depend on the final execution state | `afterExecution` |
-Pros of `errors`:
-
-- Failure-specific by design.
-- Available at the flow level and locally inside flowable tasks.
-- Well suited for remediation, cleanup, or alerts tied to a failure path.
-
-Cons of `errors`:
-
-- It is focused on error paths, not success paths.
-- It is less convenient when you want one block that branches on multiple final states.
-
-Two kinds of error handlers can be defined:
-- **Global**: error handling for the entire flow, defined at the root level
-- **Local**: error handling for a Flowable Task and its children
## Global error handler
-This example shows a global error handler. The first task fails immediately, triggering the handler, which then logs the ID of the failed task using the `tasksWithState()` function.
+The first task fails immediately, triggering the handler, which logs the ID of the failed task using the `tasksWithState()` function:
```yaml
id: errors
@@ -80,15 +66,13 @@ tasks:
errors:
- id: error_handler
type: io.kestra.plugin.core.log.Log
- message: I'm failing task '{{ tasksWithState('failed')[0]['taskId'] }}' # Because tasksWithState() returns an array, the first taskId to fail is retrieved.
+ message: I'm failing task '{{ tasksWithState('FAILED')[0]['taskId'] }}' # tasksWithState() returns an array; [0] gets the first failed task.
level: INFO
```
## Local error handler
-This example demonstrates a local error handler that applies only to the children of `t2`. Errors from other tasks, like `t1`, are not handled here.
-
-This can be useful to restrict error handling for a specific part of the flow and perform specific tasks like resource cleanup.
+A local error handler applies only to the children of the flowable task it is defined on — `t2` in this example. Errors from `t1` are not caught here, making it useful for targeted cleanup within a specific subtree:
```yaml
id: errors
@@ -100,7 +84,7 @@ tasks:
tasks:
- id: t1
type: io.kestra.plugin.core.debug.Return
- format: "{{task.id}} > {{taskrun.startDate}}"
+ format: "{{ task.id }} > {{ taskrun.startDate }}"
- id: t2
type: io.kestra.plugin.core.flow.Sequential
tasks:
@@ -109,22 +93,22 @@ tasks:
errors:
- id: error-t1
type: io.kestra.plugin.core.debug.Return
- format: "Error Trigger ! {{task.id}}"
+ format: "Error in {{ task.id }}"
```
-## `allowFailure` and `allowWarning` Property
+## `allowFailure` and `allowWarning` properties
-When you execute a flow and one of its tasks fails, downstream tasks are not executed. This may not always be desirable, especially for non-critical tasks. You can resolve this by adding the `allowFailure` property to the task, which allows downstream tasks to continue despite an error. In this case, the execution will finish in a `WARNING` state.
+By default, a failed task stops all downstream tasks. Adding `allowFailure: true` lets downstream tasks continue despite the error — the execution finishes in a `WARNING` state.
```yaml
id: allow_failure
namespace: company.team
-description: This flow will allow a failure of a task (imagine a flaky unit test) and will continue processing downstream tasks, but the execution will finish in a `WARNING` state.
+description: Allows a task failure and continues downstream; execution finishes in WARNING state.
tasks:
- id: first
@@ -140,13 +124,13 @@ tasks:
format: "{{ task.id }} > {{ taskrun.startDate }}"
```
-There is also the `allowWarning` property, which works similarly to `allowFailure`, but the execution finishes in a `SUCCESS` state even if warnings occur.
+`allowWarning` works the same way, but the execution finishes in a `SUCCESS` state even if warnings occur:
```yaml
id: allow_warning
namespace: company.team
-description: This flow will allow a warning of a task (imagine a notification task) and will continue processing downstream tasks, with the execution finishing in a `SUCCESS` state even if warnings occurred.
+description: Allows a task warning and continues downstream; execution finishes in SUCCESS state.
tasks:
- id: first
diff --git a/src/contents/docs/05.workflow-components/12.retries/index.md b/src/contents/docs/05.workflow-components/12.retries/index.md
index 5410975959f..a3378a4e45a 100644
--- a/src/contents/docs/05.workflow-components/12.retries/index.md
+++ b/src/contents/docs/05.workflow-components/12.retries/index.md
@@ -6,15 +6,13 @@ sidebarTitle: Retries
icon: /src/contents/docs/icons/flow.svg
---
-Retries handle transient failures in your workflows.
-
-They are defined at the task level and can be configured to retry a task a certain number of times or with a delay between attempts.
+Retries automatically rerun failed tasks. Each retry creates a new task run attempt based on the retry configuration defined in the flow.
-Retries let you automatically rerun failed tasks. Each retry creates a new task run attempt, based on the retry configuration defined in the flow.
+## Task-level retries
### Example
@@ -23,7 +21,7 @@ This task retries up to 5 times with a 15-minute interval between attempts:
```yaml
- id: retry_sample
type: io.kestra.plugin.core.log.Log
- message: my output for task {{task.id}}
+ message: my output for task {{ task.id }}
timeout: PT10M
retry:
type: constant
@@ -44,7 +42,7 @@ tasks:
taskRunner:
type: io.kestra.plugin.core.runner.Process
commands:
- - 'if [ "{{taskrun.attemptsCount}}" -eq 4 ]; then exit 0; else exit 1; fi'
+ - 'if [ "{{ taskrun.attemptsCount }}" -eq 4 ]; then exit 0; else exit 1; fi'
retry:
type: constant
interval: PT0.25S
@@ -55,7 +53,7 @@ tasks:
errors:
- id: never_happen
type: io.kestra.plugin.core.debug.Return
- format: Never happened {{task.id}}
+ format: "Never happened {{ task.id }}"
```
### Timeout vs. Max Retry Duration
@@ -67,14 +65,16 @@ errors:
- Each attempt can last up to 10 minutes.
- The overall retries stop after 30 minutes in total.
-⚠️ Ensure `retry.interval` is smaller than `maxDuration`, or retries may not run.
+:::alert{type="warning"}
+Ensure `retry.interval` is smaller than `maxDuration`, or retries may never run.
+:::
### Retry options
| Name | Type | Description |
|------------------|------------|-------------|
| `type` | string | Retry strategy: `constant`, `exponential`, or `random`. |
-| `maxAttempts` | integer | Number of retry attempts before stopping. |
+| `maxAttempts` | integer | Maximum number of attempts, including the initial run. |
| `maxDuration` | Duration | Maximum total time for the task, across all attempts. |
| `warningOnRetry` | Boolean | Marks execution as `WARNING` if retries occurred (default: false). |
@@ -172,31 +172,16 @@ Flow-level retries also restart Subflows as new executions.
## Retry vs. Restart vs. Replay
-### Automatic vs. manual
-
-- **Retry**: Automatic rerun of failed tasks within the same execution.
-- **Restart**: Manual rerun of failed tasks within the same execution.
-- **Replay**: Manual rerun from any point, creating a new execution.
-
-
-
-### Restart vs. Replay
-
-- **Restart**: Retries only failed tasks in the same execution.
-- **Replay**: Starts a new execution from a chosen task, with a new execution ID. Outputs of previous tasks are reused from cache if needed. Check out the [Replay documentation](../../06.concepts/10.replay/index.md).
-
-
-
-Replays can start from successful or failed tasks but always create a new execution. Restarts keep the same execution ID.
+**Retry** is the only automatic mechanism. **Restart** and **Replay** are both manual, initiated from the UI.
-After a Replay, you can still track which Execution triggered this new run thanks to the `Original Execution` field:
+| Concept | Scope | Trigger | New execution? |
+|---|---|---|---|
+| Retry | Task level | Automatic | No |
+| Restart | Flow level | Manual | No |
+| Replay | Flow or task level | Manual | Yes |
-
+**Restart** reruns only the failed tasks within the same execution, keeping the same execution ID. Use the **Restart** button at the top of the Execution overview page.
-### Summary
+**Replay** starts a new execution from any task — successful or failed — and assigns it a new execution ID. Previous task outputs are reused from cache when available. Trigger a replay from the **Actions** menu at the top of the Execution overview page, or directly from a task node in the **Topology**, **Gantt**, or **Logs** view. See the [Replay documentation](../../06.concepts/10.replay/index.md).
-| Concept | Scope | Trigger | New execution? |
-|---------|------------------------|----------|----------------|
-| Retry | Task level | Automatic| No |
-| Restart | Flow level | Manual | No |
-| Replay | Flow or task level | Manual | Yes |
+After a replay, the new execution's Overview tab shows an **Original Execution** field linking back to the source run.
diff --git a/src/contents/docs/05.workflow-components/12.retries/original_execution.png b/src/contents/docs/05.workflow-components/12.retries/original_execution.png
deleted file mode 100644
index 682f48980ad..00000000000
Binary files a/src/contents/docs/05.workflow-components/12.retries/original_execution.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/12.retries/replay.png b/src/contents/docs/05.workflow-components/12.retries/replay.png
deleted file mode 100644
index 63f6d9e989f..00000000000
Binary files a/src/contents/docs/05.workflow-components/12.retries/replay.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/12.retries/replay_restart.png b/src/contents/docs/05.workflow-components/12.retries/replay_restart.png
deleted file mode 100644
index 8e1b3231d3e..00000000000
Binary files a/src/contents/docs/05.workflow-components/12.retries/replay_restart.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/13.timeout/index.md b/src/contents/docs/05.workflow-components/13.timeout/index.md
index d4edd2dede8..dcd948f22af 100644
--- a/src/contents/docs/05.workflow-components/13.timeout/index.md
+++ b/src/contents/docs/05.workflow-components/13.timeout/index.md
@@ -6,39 +6,32 @@ sidebarTitle: Task Timeout
icon: /src/contents/docs/icons/flow.svg
---
-A timeout defines the maximum duration a [runnable task](../01.tasks/01.runnable-tasks/index.md) is allowed to run.
+A timeout defines the maximum duration a [runnable task](../01.tasks/01.runnable-tasks/index.md) is allowed to run. If a task run exceeds that duration, Kestra stops it and marks it as failed — preventing hangs and bounding costs for long-running cloud jobs like Snowflake queries or AWS Batch tasks.
-## What is a timeout
-
-If a task run exceeds the specified duration, Kestra automatically stops it and marks it as failed. This is useful for tasks that may hang and run indefinitely.
-
-Timeouts are often used as a cost-control mechanism in cloud-based workflows. Imagine a Snowflake query or an AWS Batch job that runs for hours leading to unexpected costs. By setting a timeout, you can ensure that the task run will not exceed a certain duration.
-
## Format
-Similar to [retries](../../05.workflow-components/12.retries/index.md), timeouts use the [ISO 8601 duration](https://en.wikipedia.org/wiki/ISO_8601#Durations) format, but week, month, and year designators are not supported. Below are some examples:
+Timeouts use the [ISO 8601 duration](https://en.wikipedia.org/wiki/ISO_8601#Durations) format. Week, month, and year designators are not supported. See also: [retries](../12.retries/index.md).
-| name | description |
-|----------|--------------------------|
-| PT0.250S | 250 milliseconds delay |
-| PT2S | 2 seconds delay |
-| PT1M | 1 minute delay |
-| PT3.5H | 3 hours and a half delay |
-| P6DT4H | 6 days and 4 hours delay |
+| Value | Duration |
+|----------|-----------------------|
+| PT0.250S | 250 milliseconds |
+| PT2S | 2 seconds |
+| PT1M | 1 minute |
+| PT3.5H | 3 hours, 30 minutes |
+| P6DT4H | 6 days, 4 hours |
## Example
-In this example, the `costly_query` task sleeps for 10 seconds, but the timeout is set to 5 seconds, causing the task to fail.
+The `costly_query` task sleeps for 10 seconds, but the timeout is set to 5 seconds, causing the task to fail.
```yaml
id: timeout
namespace: company.team
-description: This flow will always fail because of a timeout.
tasks:
- id: costly_query
diff --git a/src/contents/docs/05.workflow-components/14.concurrency/concurrency-fail-tab.png b/src/contents/docs/05.workflow-components/14.concurrency/concurrency-fail-tab.png
new file mode 100644
index 00000000000..57935b0d5ba
Binary files /dev/null and b/src/contents/docs/05.workflow-components/14.concurrency/concurrency-fail-tab.png differ
diff --git a/src/contents/docs/05.workflow-components/14.concurrency/concurrency-limits-page.png b/src/contents/docs/05.workflow-components/14.concurrency/concurrency-limits-page.png
new file mode 100644
index 00000000000..855d440af89
Binary files /dev/null and b/src/contents/docs/05.workflow-components/14.concurrency/concurrency-limits-page.png differ
diff --git a/src/contents/docs/05.workflow-components/14.concurrency/concurrency-limits-reset.png b/src/contents/docs/05.workflow-components/14.concurrency/concurrency-limits-reset.png
new file mode 100644
index 00000000000..926005ee307
Binary files /dev/null and b/src/contents/docs/05.workflow-components/14.concurrency/concurrency-limits-reset.png differ
diff --git a/src/contents/docs/05.workflow-components/14.concurrency/concurrency-queue-tab.png b/src/contents/docs/05.workflow-components/14.concurrency/concurrency-queue-tab.png
new file mode 100644
index 00000000000..331e6d50c64
Binary files /dev/null and b/src/contents/docs/05.workflow-components/14.concurrency/concurrency-queue-tab.png differ
diff --git a/src/contents/docs/05.workflow-components/14.concurrency/concurrency.png b/src/contents/docs/05.workflow-components/14.concurrency/concurrency.png
deleted file mode 100644
index 3c832eef7ae..00000000000
Binary files a/src/contents/docs/05.workflow-components/14.concurrency/concurrency.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/14.concurrency/concurrency_fail.png b/src/contents/docs/05.workflow-components/14.concurrency/concurrency_fail.png
deleted file mode 100644
index 8b6272b80be..00000000000
Binary files a/src/contents/docs/05.workflow-components/14.concurrency/concurrency_fail.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/14.concurrency/concurrency_page_1.png b/src/contents/docs/05.workflow-components/14.concurrency/concurrency_page_1.png
deleted file mode 100644
index 124555711d9..00000000000
Binary files a/src/contents/docs/05.workflow-components/14.concurrency/concurrency_page_1.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/14.concurrency/concurrency_page_2.png b/src/contents/docs/05.workflow-components/14.concurrency/concurrency_page_2.png
deleted file mode 100644
index f5f743cbab2..00000000000
Binary files a/src/contents/docs/05.workflow-components/14.concurrency/concurrency_page_2.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/14.concurrency/index.md b/src/contents/docs/05.workflow-components/14.concurrency/index.md
index 5c0456bcdad..ec7b307b822 100644
--- a/src/contents/docs/05.workflow-components/14.concurrency/index.md
+++ b/src/contents/docs/05.workflow-components/14.concurrency/index.md
@@ -7,37 +7,37 @@ icon: /src/contents/docs/icons/flow.svg
version: ">= 0.13.0"
---
-Control how many executions of a flow can run at the same time.
-
-The flow-level `concurrency` property lets you limit how many executions of a flow can run concurrently by setting the `limit` key.
-
-Think of concurrency as a global execution limit for that specific flow. The concurrency limit and behavior is then applied to all executions of that flow, regardless of whether those executions have been started automatically via a trigger, webhook, an API call, or manually created from the UI.
+Concurrency limits control how many executions can run at the same time — at the flow, namespace, or tenant level. When a limit is reached, new executions are queued, cancelled, or failed depending on the configured `behavior`.
Once an execution occupies a concurrency slot, it keeps that slot until it reaches a terminal state. This includes executions in the `PAUSED` state while they wait for manual approval. For example, with `concurrency.limit` set to 1, a flow paused by a `Pause` task blocks all subsequent executions until the paused execution is resumed, killed, or otherwise reaches a terminal state. Resuming the execution continues to use its existing slot, so it cannot exceed the configured limit.
:::alert{type="info"}
-Concurrency limits executions of a flow, not the number of tasks a worker runs. Task processing is still governed by worker thread pools and task runners. Concurrency uses database locks to hold slots, so heavy contention (many executions fighting for the same lock) can increase database load and slow scheduling.
+Concurrency limits executions, not the number of tasks a worker runs. Task processing is governed by worker thread pools and task runners. Concurrency uses database locks to hold slots, so heavy contention (many executions competing for the same lock) can increase database load and slow scheduling.
:::
-Use concurrency when you need to protect downstream systems (rate limits, database load, external APIs) or enforce “only one execution at a time” semantics. Do **not** rely on concurrency to throttle Kestra worker usage; adjust worker threads, task runners, or queue sizing for that.
-
-For example, if you set the concurrency `limit` to 2, only two executions of that flow will be allowed to run at the same time. If you try to trigger a third execution, it will be queued until one of the two running executions is completed.
+## When to use concurrency
-### When to use concurrency
+Use concurrency when you need to:
- Protect a shared target system (databases, SaaS APIs, warehouses) from overload.
- Enforce sequential processing for stateful workloads (one ETL load at a time).
- Keep a small, fixed number of parallel executions within an external rate limit.
-### When **not** to use concurrency
+Do **not** use concurrency to:
+
+- Throttle worker CPU or memory usage — tune worker thread pools or task runners instead.
+- Replace task-level limits — use task runner settings and retry backoff for per-task control.
+- Cap how many executions are **created** over time — use [Quotas](../21.quotas/index.md) (Enterprise Edition) for time-window rate limits.
+
+## Configuring concurrency
-- Throttling worker CPU/RAM usage — tune worker thread pools or task runners instead.
-- Replacing task-level limits — use task runner settings (e.g., container resources) and retry/backoff for per-task control.
-- Broad platform protection — use platform sizing and queue configuration rather than flow-level concurrency locks.
+### Flow level
+
+Set `concurrency.limit` on a flow to cap its parallel executions:
```yaml
id: concurrency_example
@@ -49,38 +49,45 @@ concurrency:
tasks:
- id: wait
type: io.kestra.plugin.scripts.shell.Commands
+ taskRunner:
+ type: io.kestra.plugin.core.runner.Process
commands:
- sleep 10
-
```
-In the UI, the third execution is queued while the first two finish successfully.
+With `limit: 2`, a third execution waits until one of the two running executions completes.
-
+### Namespace level
-## `behavior` property
+Set a concurrency limit on a namespace to cap the total simultaneous executions across all flows within that namespace and its children.
-You can customize the behavior when the concurrency limit is reached by choosing to queue, cancel, or fail the new execution. To do that, set the `behavior` Enum-type property to one of the following values:
+Navigate to **Namespaces**, open the target namespace, click **Edit**, and scroll to the **Concurrency** section. Set the **Limit** and choose a **Behavior** (`Queue`, `Cancel`, or `Fail`).
-- `QUEUE`
-- `CANCEL`
-- `FAIL`
+A namespace concurrency limit applies to every flow whose namespace matches or is a child of the configured namespace. For example, a limit on `company` applies to flows in `company`, `company.team`, and `company.team.project`.
-For example, with `concurrency.limit` set to 2 and `CANCEL` or `FAIL` behavior, the third execution is immediately marked as `CANCELLED` or `FAILED` without running any tasks.
+### Tenant level
-Below is a full flow example that uses the `concurrency` property to limit the number of concurrent executions to 2. The `bash` task sleeps for 10 seconds, so you can trigger multiple executions of that flow and see how the `concurrency` property behaves.
+Set a concurrency limit at the tenant level to cap total simultaneous executions across all namespaces in the tenant.
-:::alert{type="warning"}
-Each execution that waits for a concurrency slot holds a database lock. Large backlogs (many queued executions) can increase lock contention and slow down scheduling. If you expect spikes, combine conservative limits with backoff at the source (e.g., trigger rates) and keep an eye on the Concurrency tab in the UI.
-:::
+In **Instance Owner**, click **Concurrency Limits** in the sidebar, then **Administer** on the target tenant.
+
+## `behavior` property
+
+By default, executions that exceed the limit are queued. Set `behavior` to control what happens instead:
+
+- `QUEUE` — hold the execution until a slot opens (default).
+- `CANCEL` — immediately mark the execution as `CANCELLED`.
+- `FAIL` — immediately mark the execution as `FAILED`.
+
+If you expect execution spikes, combine a conservative limit with backoff at the source (e.g., slower trigger rates) to avoid large queues that increase database lock contention.
```yaml
id: concurrency_limited_flow
namespace: company.team
concurrency:
- behavior: FAIL # QUEUE, CANCEL or FAIL
- limit: 2 # can be any integer >= 1
+ behavior: FAIL # QUEUE, CANCEL, or FAIL
+ limit: 2
tasks:
- id: wait
@@ -91,50 +98,41 @@ tasks:
- sleep 10
```
-As you can see in the UI, the third execution failed as the first two executions were still running.
-
-
-
:::alert{type="warning"}
-When an execution starts from a [Trigger](../07.triggers/index.mdx), the trigger locks until it finishes, preventing multiple executions from that trigger from running concurrently. This means the `behavior` property will not come into effect and instead no new executions will be started.
+When an execution starts from a [Trigger](../07.triggers/index.mdx), the trigger locks until it finishes, preventing multiple executions from that trigger from running concurrently. The `behavior` property does not apply in this case — no new executions start while the trigger is locked.
Read more in the [Locked Triggers](../07.triggers/index.mdx#locked-triggers) section.
:::
-## Tracking concurrency slots from the UI
+## Evaluation order
-The `Concurrency` tab on the `Flow` page lets you track and troubleshoot concurrency issues. It shows a progress bar with the number of active slots compared to the total slots available. Below that progress bar, you can see a table showing executions that occupy slots, including `RUNNING` and `PAUSED` executions, alongside `QUEUED` executions waiting for a slot.
+Kestra evaluates concurrency limits from the most specific scope to the most general:
-
+1. **Flow-level** concurrency is checked first.
+2. **Namespace-level** concurrency is checked next, starting from the flow's own namespace and moving up through each parent namespace to the root.
+3. **Tenant-level** concurrency is checked last.
-To see the concurrency behavior in action, you can configure a flow with a concurrency limit as follows:
+The first limit reached defines the behavior for that execution. When a more specific limit is reached, no slot is consumed at higher levels.
-```yaml
-id: concurrent
-namespace: company.team
+For example, if a flow defines `limit: 5` with `QUEUE` and its parent namespace defines `limit: 10` with `FAIL`, a sixth execution of that flow is queued (flow-level limit is reached first). If the flow has no limit of its own but the parent namespace is at its limit of 10, the execution fails.
-concurrency:
- behavior: QUEUE
- limit: 5
+## Monitoring concurrency
-tasks:
- - id: long_running_task
- type: io.kestra.plugin.scripts.shell.Commands
- commands:
- - sleep 90
- taskRunner:
- type: io.kestra.plugin.core.runner.Process
-```
+### Per-flow Concurrency tab
+
+The **Concurrency** tab on a Flow page shows current slot usage, the configured behavior, and a list of running and queued executions for that flow.
+
+
-Next, trigger multiple Executions of that flow and watch the `Concurrency` tab showing the active slots and queued Executions.
+### Concurrency Limits page
-
+The **Concurrency Limits** page under **Tenant** in the sidebar lists every flow, namespace, and tenant concurrency limit configured, along with its live running count. Use it for a tenant-wide view of concurrency usage across all scopes.
-## Concurrent executions for Triggers
+
-Any [Trigger](../../05.workflow-components/07.triggers/index.mdx) type supports concurrent executions, allowing multiple instances of the same workflow to run simultaneously. This enables more flexible and scalable workflow patterns.
+## Concurrent trigger executions
-For example, consider a workflow that takes 60 seconds to complete but is triggered every second. By default, with `allowConcurrent: false`, only one execution can run at a time. If a trigger fires while a previous execution is still running, the new execution will be skipped:
+Any [Trigger](../07.triggers/index.mdx) type supports concurrent executions through the `allowConcurrent` property. By default, `allowConcurrent: false` — if a trigger fires while a previous execution is still running, the new execution is skipped.
```yaml
id: sleep_concurrent
@@ -151,22 +149,26 @@ triggers:
allowConcurrent: false
```
-In this example, even though the [Schedule trigger](../../05.workflow-components/07.triggers/01.schedule-trigger/index.md) fires every second, only one execution will run at a time. Setting `allowConcurrent: true` would allow multiple executions to run simultaneously.
+Set `allowConcurrent: true` to allow multiple executions to run simultaneously from the same trigger.
-## How to troubleshoot Concurrency issues
-
-Imagine that you encounter a situation where the concurrency limit is reached, and some executions are stuck in the `QUEUED` state. Here are some steps to troubleshoot and resolve the issue.
+## Troubleshooting concurrency issues
### Check the Concurrency tab
-The `Concurrency` tab on the `Flow` UI page described above allows you to see which executions occupy slots, including those in the `RUNNING` and `PAUSED` states, and which are `QUEUED` (i.e., waiting or stuck). This page can help you troubleshoot which Executions are taking concurrency slots and which are waiting to be processed.
+Open the **Concurrency** tab on the Flow page to see which executions are running, queued, or failed. This shows which executions hold slots and which are waiting.
+
+
+
+### Edit the concurrency limit
+
+You can change or remove the `concurrency` property in the flow editor and save. The executor always reads the latest flow revision, so the updated limit takes effect immediately for all in-progress executions.
-In the future, this page will also let you run stuck executions while ignoring concurrency limits.
+### Reset a stuck running counter
-### Edit the Concurrency property
+If executions were deleted while running, their concurrency slots can remain occupied indefinitely. Use the **Concurrency Limits** page to correct this: click the edit icon next to the affected flow and adjust the running counter directly.
-You can edit the `concurrency` property within the flow (or remove that property entirely to get rid of any limits) and `Save` the flow code. The modified concurrency limit and behavior will be immediately taken into account for all Executions in progress because the Executor checks this for the latest flow revision rather than for the revision of the Execution.
+
:::alert{type="warning"}
-Do **not** delete executions, as this makes the issue worse — deleted executions still occupy concurrency slots indefinitely. You can select stuck Executions and hit the `Kill` button to cancel them and free up the concurrency slots, but do not delete them.
+Do **not** delete executions to free stuck slots — deleted executions still hold concurrency slots. Instead, select stuck executions and click **Kill** to cancel them and release their slots. Use the counter reset only when slots remain stuck after killing all relevant executions.
:::
diff --git a/src/contents/docs/05.workflow-components/15.descriptions/description.png b/src/contents/docs/05.workflow-components/15.descriptions/description.png
index 02adce1af0f..45e5e36ee89 100644
Binary files a/src/contents/docs/05.workflow-components/15.descriptions/description.png and b/src/contents/docs/05.workflow-components/15.descriptions/description.png differ
diff --git a/src/contents/docs/05.workflow-components/15.descriptions/index.md b/src/contents/docs/05.workflow-components/15.descriptions/index.md
index 67a1abcb3e0..0d538f2ff11 100644
--- a/src/contents/docs/05.workflow-components/15.descriptions/index.md
+++ b/src/contents/docs/05.workflow-components/15.descriptions/index.md
@@ -6,22 +6,13 @@ sidebarTitle: Descriptions
icon: /src/contents/docs/icons/flow.svg
---
-You can document flows, inputs, outputs, tasks, and triggers by adding a `description` property.
-
-The `description` property is a string field that supports [Markdown](https://en.wikipedia.org/wiki/Markdown) syntax.
+The `description` property accepts [Markdown](https://en.wikipedia.org/wiki/Markdown) and is available on flows, inputs, outputs, tasks, and triggers. Descriptions are rendered in the UI wherever the component appears.
-You can add a `description` property on:
-- Flows
-- Inputs
-- Outputs
-- Tasks
-- Triggers
-
-All Markdown descriptions are rendered directly in the UI. You can get as detailed as you'd like. Take the following example:
+Flow descriptions support full Markdown — headings, lists, bold, code spans, and more:
```yaml
id: data-engineering-pipeline-demo
@@ -65,11 +56,62 @@ description: |
- `INSTALL json; LOAD json;`: Installs and loads the JSON extension for DuckDB to handle JSON files.
- `SELECT brand, round(avg(price), 2) as avg_price FROM read_json_auto('{{ workingDir }}/products.json') GROUP BY brand ORDER BY avg_price DESC;`: Reads the `products.json` file, calculates the average price for each brand, and orders the results by average price in descending order.
- **Fetch Type**: `STORE` ensures that the results of the SQL query are stored as an output file in Kestra's internal storage, making them accessible for further use or inspection.
+
+inputs:
+ - id: columns_to_keep
+ type: ARRAY
+ itemType: STRING
+ defaults:
+ - brand
+ - price
+
+tasks:
+ - id: extract
+ type: io.kestra.plugin.core.http.Download
+ uri: https://dummyjson.com/products
+
+ - id: transform
+ type: io.kestra.plugin.scripts.python.Script
+ taskRunner:
+ type: io.kestra.plugin.scripts.runner.docker.Docker
+ containerImage: python:3.11-alpine
+ inputFiles:
+ data.json: "{{ outputs.extract.uri }}"
+ script: |
+ import json
+
+ columns_to_keep = {{ inputs.columns_to_keep | toJson }}
+
+ with open("data.json") as f:
+ data = json.load(f)
+
+ products = [
+ {col: p[col] for col in columns_to_keep if col in p}
+ for p in data["products"]
+ ]
+
+ with open("products.json", "w") as f:
+ json.dump(products, f)
+ outputFiles:
+ - products.json
+
+ - id: query
+ type: io.kestra.plugin.jdbc.duckdb.Query
+ inputFiles:
+ products.json: "{{ outputs.transform.outputFiles['products.json'] }}"
+ sql: |
+ INSTALL json;
+ LOAD json;
+ SELECT brand, round(avg(price), 2) AS avg_price
+ FROM read_json_auto('products.json')
+ GROUP BY brand
+ ORDER BY avg_price DESC;
+ store: true
```
-
+
-Here is an example flow with descriptions in different components:
+The `description` property works on all supported components in the same flow:
```yaml
id: myflow
diff --git a/src/contents/docs/05.workflow-components/16.disabled/disabled-flow-execute-modal.png b/src/contents/docs/05.workflow-components/16.disabled/disabled-flow-execute-modal.png
new file mode 100644
index 00000000000..76e1e79cf5b
Binary files /dev/null and b/src/contents/docs/05.workflow-components/16.disabled/disabled-flow-execute-modal.png differ
diff --git a/src/contents/docs/05.workflow-components/16.disabled/disabled-subflow-error.png b/src/contents/docs/05.workflow-components/16.disabled/disabled-subflow-error.png
new file mode 100644
index 00000000000..b3400c48cee
Binary files /dev/null and b/src/contents/docs/05.workflow-components/16.disabled/disabled-subflow-error.png differ
diff --git a/src/contents/docs/05.workflow-components/16.disabled/disabled-task-topology.png b/src/contents/docs/05.workflow-components/16.disabled/disabled-task-topology.png
new file mode 100644
index 00000000000..dc3dfc83114
Binary files /dev/null and b/src/contents/docs/05.workflow-components/16.disabled/disabled-task-topology.png differ
diff --git a/src/contents/docs/05.workflow-components/16.disabled/disabled_flag_1.png b/src/contents/docs/05.workflow-components/16.disabled/disabled_flag_1.png
deleted file mode 100644
index 1967e45ba4e..00000000000
Binary files a/src/contents/docs/05.workflow-components/16.disabled/disabled_flag_1.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/16.disabled/disabled_flag_2.png b/src/contents/docs/05.workflow-components/16.disabled/disabled_flag_2.png
deleted file mode 100644
index acf957c064c..00000000000
Binary files a/src/contents/docs/05.workflow-components/16.disabled/disabled_flag_2.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/16.disabled/disabled_flag_3.png b/src/contents/docs/05.workflow-components/16.disabled/disabled_flag_3.png
deleted file mode 100644
index 7805b1db10c..00000000000
Binary files a/src/contents/docs/05.workflow-components/16.disabled/disabled_flag_3.png and /dev/null differ
diff --git a/src/contents/docs/05.workflow-components/16.disabled/index.md b/src/contents/docs/05.workflow-components/16.disabled/index.md
index 9a7e1fa27c2..91185bbb8d7 100644
--- a/src/contents/docs/05.workflow-components/16.disabled/index.md
+++ b/src/contents/docs/05.workflow-components/16.disabled/index.md
@@ -6,20 +6,15 @@ sidebarTitle: Disabled flag
icon: /src/contents/docs/icons/flow.svg
---
-The `disabled` flag is a boolean property that lets you skip a flow, task, or trigger.
-
-This is useful for debugging or testing parts of a flow without removing existing logic. Instead of deleting parts of your YAML, you can add the `disabled` property.
+The `disabled` property is a boolean that skips a flow, task, or trigger without removing it — useful for debugging without deleting YAML.
-
## Disabled flow
-When a flow is disabled, it will not be executed — even if a trigger is set. If you have an active trigger on a disabled flow, it will be ignored. You don’t need to disable the trigger separately – it is ignored automatically. Setting a flow to `disabled` effectively prevents any future executions of the flow until it is re-enabled.
-
-Add the following flow, then attempt to run it and observe the scheduled executions:
+A disabled flow will not execute and its triggers are automatically ignored — you do not need to disable each trigger separately.
```yaml
id: disabled_flow
@@ -37,9 +32,9 @@ triggers:
cron: "*/1 * * * *"
```
-You will see that you cannot run the flow and that the trigger is ignored — no executions are created.
+The Execute dialog warns that the flow is disabled and no executions are created:
-
+
When executing a disabled flow from a subflow:
@@ -53,27 +48,19 @@ tasks:
namespace: company.team
```
-When you execute the parent flow, it immediately fails with the error message: `Cannot execute a flow which is disabled`.
+The parent flow immediately fails with the error: `Cannot execute a flow which is disabled`.
-
-
-Similarly, try running a disabled flow via an API call:
-
-```bash
-curl -X POST http://localhost:8080/api/v1/main/executions/trigger/example/parent_runs_disabled_flow
-```
+
-The API call itself is successful:
+The same error occurs when triggering via API — the execution is created then immediately marked as failed:
```bash
-{"id":"5ScXvrnOkjfKIXqYylRYME","namespace":"example","flowId":"parent_runs_disabled_flow","flowRevision":1,"state":{"current":"CREATED","histories":[{"state":"CREATED","date":"2024-01-19T20:38:48.474047013Z"}],"duration":"PT0.011094958S","startDate":"2024-01-19T20:38:48.474047013Z"},"originalId":"5ScXvrnOkjfKIXqYylRYME"}%
+curl -X POST http://localhost:8080/api/v1/main/executions/trigger/company.team/parent_runs_disabled_flow
```
-That execution is immediately marked as failed with the error message: `Cannot execute a flow which is disabled`.
-
## Disabled trigger
-When using a Schedule trigger, it is often useful to disable it temporarily. For example, you may want to disable a trigger while you are debugging a flow. You can do this by setting the `disabled` flag to `true` on the trigger:
+To disable a trigger without disabling the entire flow, set `disabled: true` on the trigger:
```yaml
id: myflow
@@ -91,26 +78,11 @@ triggers:
disabled: true
```
-You will see that no scheduled executions are created for this flow. Once you are done debugging, you can re-enable the trigger by setting the `disabled` flag to `false` or simply by removing the `disabled` flag:
-
-```yaml
-id: myflow
-namespace: company.team
-
-tasks:
- - id: hello
- type: io.kestra.plugin.core.log.Log
- message: hello from a scheduled flow
-
-triggers:
- - id: daily
- type: io.kestra.plugin.core.trigger.Schedule
- cron: "0 9 * * *"
-```
+No scheduled executions are created while the trigger is disabled. To re-enable it, set `disabled: false` or remove the property entirely.
## Disabled task
-Instead of disabling the entire flow or a trigger, you can also disable a single task. This is useful when you want to temporarily disable a single task without deleting it e.g., when troubleshooting a failure. You can do this by setting the `disabled` flag to `true` on the task:
+You can disable a single task to skip it without deleting it — useful when isolating a failure during debugging:
```yaml
id: myflow
@@ -127,6 +99,6 @@ tasks:
disabled: true
```
-You can see in the UI that disabled tasks are greyed out:
+Disabled tasks appear with a strikethrough on the task name in the Topology view:
-
+
diff --git a/src/contents/docs/05.workflow-components/17.states/index.md b/src/contents/docs/05.workflow-components/17.states/index.md
index efc326f012e..d34073de644 100644
--- a/src/contents/docs/05.workflow-components/17.states/index.md
+++ b/src/contents/docs/05.workflow-components/17.states/index.md
@@ -6,62 +6,55 @@ sidebarTitle: States
icon: /src/contents/docs/icons/flow.svg
---
-States control the status of your workflow execution.
+States represent where an execution or task run is in its lifecycle. Each state determines what Kestra does next — whether to continue, retry, wait for input, or terminate. For a broader overview of executions, see the [Execution documentation](../03.execution/index.md).
-## Overview
-
-An execution is a single run of a flow in a specific state. Each state represents a point in the workflow where Kestra determines what happens next based on the control flow logic defined in the flow.
-
-You can read more about executions in the [workflow components documentation](../03.execution/index.md).
-
## Execution states
-Each Kestra execution can transition through several states during its lifecycle. The following diagram illustrates the possible states an execution can be in:
-
-
-
-Here is a brief description of each state:
-1. **CREATED**: The execution has been created but not yet started. This transient state means the execution is waiting to be processed. It usually transitions quickly to `RUNNING`, `CANCELLED`, or `QUEUED`. If you see executions stuck in this state, it may indicate a problem with the system.
-2. **QUEUED**: The execution is waiting for a free slot to start running. This transient state is only used when the flow has [concurrency](../14.concurrency/index.md) limits, and all available slots are taken.
-3. **RUNNING**: The execution is currently in progress. This transient state continues until all task runs are completed.
-4. **SUCCESS**: The execution has completed successfully. This terminal state indicates that the execution has completed successfully, and all tasks have finished without errors (or were allowed to fail).
-5. **WARNING**: This terminal state is used when the execution has completed successfully, but one or more tasks have emitted warnings.
-6. **FAILED**: This state indicates that one or more tasks have failed and will not be retried. If there is an `errors` branch defined in the flow, the error `tasks` will be executed before permanently ending the execution, e.g., to send an alert about failure. Without additional orchestration, this state is usually considered terminal. However, when the flow has a [flow-level retry policy](../12.retries/index.md#flow-level-retries) set to the `RETRY_FAILED_TASK` behavior, the execution will transition to the `RETRYING` state.
-7. **RETRYING**: This transient state indicates that the execution is currently [retrying](../12.retries/index.md) one or more failed task runs. After all retry attempts are exhausted, the execution will transition to the terminal `SUCCESS`, `WARNING`, or `FAILED` state.
-8. **RETRIED**: This terminal state indicates that the execution has been retried according to the [flow-level retry policy](../12.retries/index.md#flow-level-retries) set to the `CREATE_NEW_EXECUTION` behavior. This means that the original execution (which failed and has been retried) is marked as `RETRIED`, and a new execution is created to run the flow again.
-9. **PAUSED**: This transient state indicates that the execution is awaiting manual approval or has been paused for a fixed duration before continuing the execution. There are no `RESUMING` or `RESUMED` states. A paused execution transitions directly from `PAUSED` to `RUNNING` when resumed.
-10. **RESTARTED**: This transient state is equivalent to the `CREATED` state but for a failed execution that has been restarted e.g., from the UI. These executions transition to `RUNNING` once the restart is processed.
-11. **CANCELLED**: This terminal state indicates that the execution has been automatically cancelled by the system, usually because the `concurrency` limit was reached and the [concurrency](../14.concurrency/index.md) `behavior` was set to `CANCEL`, which cancels all executions that exceed the concurrency limit.
-12. **KILLING**: This transient state indicates that the user has issued a command to kill the execution, e.g., via a task or by clicking on the `Kill` button in the UI. The system is terminating (killing) any task runs still in progress. As soon as all task runs are terminated, the execution will transition to the `KILLED` state.
-13. **KILLED**: This terminal state indicates that the execution has been killed upon request by the user. No more tasks will be able to run, and the execution is considered terminated.
-
-## What is the difference between the `CANCELLED` and `KILLED` states?
-
-1. The `CANCELLED` state is used when the **system** automatically cancels an execution due to the `concurrency` limit being reached.
-2. The `KILLING` state is used when the **user** manually kills an execution and the system is in the process of terminating the task runs associated with the execution.
-3. The `KILLED` state is used when the execution has been killed upon request by the **user**.
-
-## How are task run states different from execution states?
-
-Task run states represent the status of a single task run within an execution.
-
-
-
-Each task run can be in one of the following states:
-1. **CREATED**: The task run has been created but not yet started.
-2. **SUBMITTED**: The task run has been submitted to a Worker but has not started running yet.
-3. **RUNNING**: The task run is currently in progress.
-4. **SUCCESS**: The task run has completed successfully.
-5. **WARNING**: The task run has completed successfully but with warnings.
-6. **FAILED**: The task run has failed.
-7. **RETRYING**: The task run is currently being retried.
-8. **RETRIED**: The task run has been retried.
-9. **RESTARTED**: The task run is currently being restarted.
-10. **KILLING**: The task run is in the process of being killed.
-11. **KILLED**: The task run has been killed upon request by the user.
-
-Note how there is no `QUEUED`, `CANCELLED`, or `PAUSED` states for task runs.
+
+
+| State | Type | Description |
+|-------|------|-------------|
+| `CREATED` | Transient | Created but not yet started. Transitions quickly to `RUNNING`, `QUEUED`, or `CANCELLED`. Executions stuck here may indicate a system issue. |
+| `QUEUED` | Transient | Waiting for a free slot. Only occurs when [concurrency](../14.concurrency/index.md) limits are set and all slots are occupied. |
+| `RUNNING` | Transient | Currently in progress. Continues until all task runs complete. |
+| `PAUSED` | Transient | Awaiting manual approval or a fixed delay before continuing. Transitions directly back to `RUNNING` when resumed — there is no `RESUMING` or `RESUMED` state. |
+| `RESTARTED` | Transient | Equivalent to `CREATED` but for a failed execution that has been manually restarted from the UI. Transitions to `RUNNING` once processed. |
+| `RETRYING` | Transient | One or more failed task runs are being retried under a [flow-level retry policy](../12.retries/index.md#flow-level-retries). Transitions to `SUCCESS`, `WARNING`, or `FAILED` once all attempts are exhausted. |
+| `KILLING` | Transient | The user has issued a kill command. The system is terminating any task runs still in progress. Transitions to `KILLED` once all task runs are terminated. |
+| `SUCCESS` | Terminal | All tasks completed without errors, or any failures were explicitly allowed. |
+| `WARNING` | Terminal | Completed successfully, but one or more tasks emitted warnings. |
+| `FAILED` | Terminal | One or more tasks failed and will not be retried. If an [`errors` handler](../11.errors/index.md) is defined, its tasks run before the execution ends. With a [flow-level retry policy](../12.retries/index.md#flow-level-retries) set to `RETRY_FAILED_TASK`, the execution transitions to `RETRYING` instead. |
+| `RETRIED` | Terminal | The original execution failed and was retried under a flow-level retry policy set to `CREATE_NEW_EXECUTION`. The original execution is marked `RETRIED` and a new execution is created in its place. |
+| `CANCELLED` | Terminal | Automatically cancelled by the system because the [concurrency](../14.concurrency/index.md) limit was reached and `behavior` was set to `CANCEL`. |
+| `KILLED` | Terminal | Killed on request by the user. No further tasks will run. |
+
+## CANCELLED vs. KILLED
+
+Both are terminal states that stop an execution, but they have different causes:
+
+- **`CANCELLED`** — triggered by the **system** when the concurrency limit is reached and `behavior: CANCEL` is configured. No user action required.
+- **`KILLED`** — triggered by the **user** via the **Kill** button in the UI or an API call. The execution first passes through `KILLING` while in-progress task runs are terminated, then settles in `KILLED`.
+
+## Task run states
+
+Task run states represent the status of a single task run within an execution. The lifecycle is similar but not identical — task runs have a `SUBMITTED` state (queued to a Worker) that executions do not, and executions have `QUEUED`, `CANCELLED`, and `PAUSED` states that task runs do not.
+
+
+
+| State | Description |
+|-------|-------------|
+| `CREATED` | Created but not yet started. |
+| `SUBMITTED` | Submitted to a Worker but not yet running. |
+| `RUNNING` | Currently in progress. |
+| `SUCCESS` | Completed successfully. |
+| `WARNING` | Completed with warnings. |
+| `FAILED` | Failed. |
+| `RETRYING` | Being retried. |
+| `RETRIED` | Retried and superseded by a new attempt. |
+| `RESTARTED` | Being restarted. |
+| `KILLING` | Kill in progress. |
+| `KILLED` | Killed on request by the user. |
diff --git a/src/contents/docs/05.workflow-components/18.sla/index.md b/src/contents/docs/05.workflow-components/18.sla/index.md
index 67f6555e4d5..60a46bc8672 100644
--- a/src/contents/docs/05.workflow-components/18.sla/index.md
+++ b/src/contents/docs/05.workflow-components/18.sla/index.md
@@ -7,29 +7,24 @@ icon: /src/contents/docs/icons/flow.svg
version: ">= 0.20.0"
---
-Assert that your workflows meet SLAs.
+SLAs define time or assertion-based checks on flow executions and trigger corrective actions when those checks fail.
-A Service Level Agreement (SLA) is a core property of a flow that defines a `behavior` to trigger if the flow runs too long or fails to meet the defined assertion.
+Use the `sla` property at the root of a flow to declare one or more SLA checks. Each SLA specifies a `type`, the condition to check, and a `behavior` to trigger when the condition is violated.
## SLA types
-Currently, Kestra supports the following SLA types:
-1. **MAX_DURATION** — the maximum allowed execution duration before the SLA is breached
-2. **EXECUTION_ASSERTION** — an assertion defined by a Pebble expression that must be met during the execution. If the assertion doesn't hold true, the SLA is breached.
-
-## How to use SLAs
+Kestra supports two SLA types:
-SLAs are defined using the `sla` property at the root of a flow, and they declare the desired state that must be met during executions of the flow.
+1. **MAX_DURATION** — the maximum allowed execution duration before the SLA is breached
+2. **EXECUTION_ASSERTION** — a Pebble expression that must evaluate to `true` during execution. If it evaluates to `false`, the SLA is breached.
### MAX_DURATION
-If a workflow execution exceeds the expected duration, an SLA can trigger corrective actions, such as cancelling the execution.
-
-The following SLA cancels an execution if it takes more than 8 hours:
+The following SLA cancels an execution if it runs for more than 8 hours:
```yaml
id: sla_example
@@ -60,9 +55,7 @@ tasks:
### EXECUTION_ASSERTION
-An SLA can also be based on an assertion that must hold true during execution. If the assertion fails, the SLA is breached.
-
-The following SLA fails if the output of `mytask` is not equal to `expected output`:
+`EXECUTION_ASSERTION` evaluates a Pebble expression against the execution at runtime. The following SLA fails the execution if `mytask` does not return `expected output`:
```yaml
id: sla_demo
@@ -85,16 +78,17 @@ tasks:
## SLA behavior
-The `behavior` property of an SLA defines the action to take when the SLA is breached. The following behaviors are supported:
+The `behavior` property controls what happens when the SLA is breached:
+
1. **CANCEL** — cancels the execution
2. **FAIL** — fails the execution
-3. **NONE** — logs a message
+3. **NONE** — logs a message without stopping the execution
-In addition, each breached SLA can set labels that can be used to filter executions or trigger follow-up actions.
+Each SLA can also set `labels` on the breached execution, making it easy to filter affected executions or trigger follow-up actions.
## Alerts on SLA breaches
-For example, if you want to receive a Slack alert when an SLA is breached, you can use a Flow trigger to react to cancelled or failed executions labeled with `sla: miss`:
+Use a Flow trigger on the `sla: miss` label to send an alert whenever an SLA is breached:
```yaml
id: sla_miss_alert
@@ -103,20 +97,14 @@ namespace: system
tasks:
- id: send_alert
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
- url: "{{secret('SLACK_WEBHOOK')}}"
- messageText: "SLA breached for flow `{{trigger.namespace}}.{{trigger.flowId}}` with ID `{{trigger.executionId}}`"
+ url: "{{ secret('SLACK_WEBHOOK') }}"
+ messageText: "SLA breached for flow `{{ trigger.namespace }}.{{ trigger.flowId }}` with ID `{{ trigger.executionId }}`"
triggers:
- id: alert_on_failure
type: io.kestra.plugin.core.trigger.Flow
- labels:
- sla: miss
- states:
- - FAILED
- - WARNING
- - CANCELLED
+ dependsOn:
+ - labels:
+ sla: miss
+ states: [FAILED, WARNING, CANCELLED]
```
-
-:::alert{type="info"}
-Best practice: Use labels with SLAs to track SLA breaches across environments, and pair them with alerting or monitoring flows for proactive response.
-:::
diff --git a/src/contents/docs/05.workflow-components/19.finally/index.md b/src/contents/docs/05.workflow-components/19.finally/index.md
index 67cc5193718..893e7e7ffc3 100644
--- a/src/contents/docs/05.workflow-components/19.finally/index.md
+++ b/src/contents/docs/05.workflow-components/19.finally/index.md
@@ -7,42 +7,28 @@ icon: /src/contents/docs/icons/flow.svg
version: "0.21.0"
---
-Define a block of tasks that always run at the end of a flow, regardless of task status.
-
-`finally` tasks are useful for cleanup operations that must run at the end of your flow, whether the execution ends in success or failure.
+The `finally` block defines tasks that always run at the end of a flow — whether the execution succeeds, fails, or is killed — making it the right place for cleanup and resource teardown.
-## `finally` component
-
-`finally` is a block of tasks that execute at the end of your workflow, regardless of the status of prior tasks. This ensures cleanup or teardown steps always occur, no matter how the flow ends.
-
-For example, you might use a `finally` block to turn off a cloud service when the flow finishes, regardless of the outcome.
-
-:::alert{type="info"}
-Note that `finally` tasks run while the execution is still `RUNNING`. If you need to trigger tasks after an execution finishes with a specific status (`SUCCESS` or `FAILED`), use the [`afterExecution` property](../20.afterexecution/index.md).
-:::
-
## `finally` vs `errors`
-`finally` and `errors` can both run near the end of a flow, but they are meant for different jobs.
+`finally` and `errors` both run near the end of a flow, but serve different purposes:
-- Use `finally` for cleanup and teardown that must happen every time.
+- Use `finally` for cleanup and teardown that must happen every time, regardless of outcome.
- Use `errors` for failure-specific handling such as alerts, remediation, or fallback actions.
-Unlike `errors`, `finally` is not tied to a failure path. It runs whether the flow succeeds or fails, and it runs while the execution is still in the `RUNNING` state.
-
-For failure-specific handling, including local handlers inside flowable tasks, see the [`errors` documentation](../11.errors/index.md). For post-run actions based on the final execution state, see the [`afterExecution` documentation](../20.afterexecution/index.md).
+`finally` tasks run while the execution is still `RUNNING`. If you need to react to a specific terminal state (`SUCCESS`, `FAILED`, etc.), use [`afterExecution`](../20.afterexecution/index.md) instead. For failure-specific handling including local handlers inside flowable tasks, see the [`errors` documentation](../11.errors/index.md).
+## Examples
-## `finally` example
+### Cleanup after a failed task
-In the example below, one task is designed to fail, and an `errors` task logs a message to signal the failure. The `finally` task still runs after the other tasks finish. Here it logs another message, but in practice it could be used to shut down resources started for the flow.
+One task is designed to fail and the `errors` block sends an alert. The `finally` task runs regardless — here it logs a message, but in practice it would shut down any resources the flow started:
```yaml
-
id: finally_example
namespace: company.team
@@ -57,22 +43,23 @@ errors:
message: alert on failure
finally:
-- id: cleanup_task
- type: io.kestra.plugin.core.log.Log
- message: cleaning up resources
+ - id: cleanup_task
+ type: io.kestra.plugin.core.log.Log
+ message: cleaning up resources
```
-If you change the example so the first task succeeds, as shown below, the `finally` task still runs in the same way:
+### Cleanup after a successful task
-```yaml
+When the task succeeds, `errors` is skipped but `finally` still runs:
+```yaml
id: finally_example
namespace: company.team
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
- errorMessage: "This flow executes successfully!"
+ message: "This flow executes successfully!"
errors:
- id: send_alert
@@ -80,14 +67,14 @@ errors:
message: alert on failure
finally:
-- id: cleanup_task
- type: io.kestra.plugin.core.log.Log
- message: cleaning up resources
+ - id: cleanup_task
+ type: io.kestra.plugin.core.log.Log
+ message: cleaning up resources
```
-As in the first example, the `finally` task runs at the end even though the `errors` task does not send an alert, ensuring cleanup still happens regardless of status.
+### Managing an external service
-Beyond simple cleanup, `finally` can manage external services. For example, you might spin up Redis, Elasticsearch, or Kafka to run queries or QA checks, and then ensure the service is stopped when the flow ends. The following example shows how to start a Redis Docker container, run some database operations, and then stop the container when the flow finishes.
+Use `finally` to guarantee a service is stopped even if the flow fails mid-run. The following flow starts a Redis container, runs operations against it, and stops the container in the `finally` block:
```yaml
id: dockerRedis
@@ -111,47 +98,43 @@ tasks:
- id: set
type: io.kestra.plugin.redis.string.Set
- url: "redis://:redis@{{vars.host}}:6379/0"
- key: "key_string_{{execution.id}}"
- value: "{{flow.id}}"
+ url: "redis://:redis@{{ vars.host }}:6379/0"
+ key: "key_string_{{ execution.id }}"
+ value: "{{ flow.id }}"
serdeType: STRING
- id: get
type: io.kestra.plugin.redis.string.Get
- url: "redis://:redis@{{vars.host}}:6379/0"
- key: "key_string_{{execution.id}}"
+ url: "redis://:redis@{{ vars.host }}:6379/0"
+ key: "key_string_{{ execution.id }}"
serdeType: STRING
- id: assert
type: io.kestra.plugin.core.execution.Assert
- errorMessage: "Invalid get data {{outputs.get}}"
+ errorMessage: "Invalid get data {{ outputs.get }}"
conditions:
- - "{{outputs.get.data == flow.id}}"
+ - "{{ outputs.get.data == flow.id }}"
- id: delete
type: io.kestra.plugin.redis.string.Delete
- url: "redis://:redis@{{vars.host}}:6379/0"
+ url: "redis://:redis@{{ vars.host }}:6379/0"
keys:
- - "key_string_{{execution.id}}"
+ - "key_string_{{ execution.id }}"
- id: getAfterDelete
type: io.kestra.plugin.redis.string.Get
- url: "redis://:redis@{{vars.host}}:6379/0"
- key: "key_string_{{execution.id}}"
+ url: "redis://:redis@{{ vars.host }}:6379/0"
+ key: "key_string_{{ execution.id }}"
serdeType: STRING
- id: assertAfterDelete
type: io.kestra.plugin.core.execution.Assert
- errorMessage: "Invalid get data {{outputs.getAfterDelete}}"
+ errorMessage: "Invalid get data {{ outputs.getAfterDelete }}"
conditions:
- - "{{(outputs.getAfterDelete contains 'data') == false}}"
+ - "{{ (outputs.getAfterDelete contains 'data') == false }}"
finally:
- id: stop
type: io.kestra.plugin.docker.Stop
- containerId: "{{outputs.start.taskRunner.containerId}}"
+ containerId: "{{ outputs.start.taskRunner.containerId }}"
```
-
-:::alert{type="info"}
-Best practice: Use `finally` for cleanup and resource teardown, not for critical business logic. Business logic dependent on execution outcomes should use `errors` or `afterExecution`.
-:::
diff --git a/src/contents/docs/05.workflow-components/20.afterexecution/after-execution-1.png b/src/contents/docs/05.workflow-components/20.afterexecution/after-execution-1.png
index 13c76f97351..dca2a30e4f8 100644
Binary files a/src/contents/docs/05.workflow-components/20.afterexecution/after-execution-1.png and b/src/contents/docs/05.workflow-components/20.afterexecution/after-execution-1.png differ
diff --git a/src/contents/docs/05.workflow-components/20.afterexecution/index.md b/src/contents/docs/05.workflow-components/20.afterexecution/index.md
index 4ab48c54158..14f0bb5b1ae 100644
--- a/src/contents/docs/05.workflow-components/20.afterexecution/index.md
+++ b/src/contents/docs/05.workflow-components/20.afterexecution/index.md
@@ -7,17 +7,13 @@ icon: /src/contents/docs/icons/flow.svg
version: "0.22.0"
---
-Run tasks after a flow execution completes.
-
-`afterExecution` tasks run once a flow has finished, allowing you to act on the final execution status.
+`afterExecution` tasks run once a flow reaches a terminal state, giving you access to the final execution status for notifications, reporting, or conditional follow-up actions.
-## `afterExecution` property
-
-`afterExecution` is a block of tasks that run after the flow ends. You can use it to run conditional tasks based on the final state, such as **SUCCESS** or **FAILED**. This is especially useful for custom notifications and alerts. For example, you can combine `afterExecution` with the `runIf` property to send different Slack messages depending on the execution state.
+Use `afterExecution` with `runIf` to branch on the final execution state:
```yaml
id: alerts_demo
@@ -29,43 +25,32 @@ tasks:
afterExecution:
- id: onSuccess
- runIf: "{{execution.state == 'SUCCESS'}}"
+ runIf: "{{ execution.state == 'SUCCESS' }}"
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
url: https://hooks.slack.com/services/xxxxx
- messageText: "{{flow.namespace}}.{{flow.id}} finished successfully!"
+ messageText: "{{ flow.namespace }}.{{ flow.id }} finished successfully!"
- id: onFailure
- runIf: "{{execution.state == 'FAILED'}}"
+ runIf: "{{ execution.state == 'FAILED' }}"
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
url: https://hooks.slack.com/services/xxxxx
- messageText: "Oh no, {{flow.namespace}}.{{flow.id}} failed!!!"
+ messageText: "Oh no, {{ flow.namespace }}.{{ flow.id }} failed!!!"
```
## `afterExecution` vs `errors`
-Both constructs are useful for notifications and follow-up actions, but they run at different moments.
-
-- `errors` runs when a task or flow errors and is primarily for failure handling.
-- `afterExecution` runs only after the execution reaches its final state.
-
-For failure-specific handling, including local handlers inside flowable tasks, see the [`errors` documentation](../11.errors/index.md).
-
-Choose `afterExecution` when you need to branch on the final status of the whole execution, for example to send one message for `SUCCESS`, another for `FAILED`, and a third for `WARNING`.
-
-Choose `errors` when you only care about failure handling or when you need local error handling inside a specific flowable task.
-
-Pros of `afterExecution`:
+Both run near the end of a flow, but at different moments and for different purposes:
-- It works naturally with final states such as `SUCCESS`, `FAILED`, and `WARNING`.
-- It keeps all post-run outcome logic in one place.
-- It is well suited for final notifications, reporting, and auditing tasks.
+| | `afterExecution` | `errors` |
+|---|---|---|
+| When it runs | After the execution reaches a terminal state | When a task or flow errors |
+| State visibility | Sees the final execution state (`SUCCESS`, `FAILED`, etc.) | Sees `RUNNING` — the execution hasn't settled yet |
+| Scope | Flow level only | Flow level or local to a flowable task |
-Cons of `afterExecution`:
+Use `afterExecution` when you need to branch on the final status — one message for `SUCCESS`, another for `FAILED`, a third for `WARNING`. Use `errors` when you only need failure handling or local error handling inside a specific flowable task. See the [`errors` documentation](../11.errors/index.md) for details.
-- It cannot be scoped locally to a flowable task the way `errors` can.
-- Errors inside `afterExecution` do not change the final execution state.
-
-Any errors in the `afterExecution` block will not change the state of the flow from `SUCCESS` to `FAILED`, and they will not trigger a flow that relies on `ExecutionStatus = FAILED`. You can force a state change by using a [Sequential flowable task](../01.tasks/00.flowable-tasks/index.md#sequential) with an `errors` block, as in the example below:
+:::alert{type="warning"}
+Errors inside an `afterExecution` block do not change the final execution state. A failing `afterExecution` task will not flip the execution from `SUCCESS` to `FAILED`, and will not trigger flows that listen for `FAILED` executions. To force a state change, use a [Sequential](../01.tasks/00.flowable-tasks/index.md#sequential) task with its own `errors` block:
```yaml
afterExecution:
@@ -83,18 +68,13 @@ afterExecution:
url: https://hooks.slack.com/services/xxxxx
messageText: "Flow {{ flow.namespace }}.{{ flow.id }} with execution ID {{ execution.id }} failed."
```
+:::
## `afterExecution` vs `finally`
-`afterExecution` and `finally` are both end-of-flow constructs, but they serve different purposes.
-
-The `afterExecution` property differs from the `finally` property because:
-1. `finally` runs tasks at the end of the flow while the execution is still in a `RUNNING` state.
-2. `afterExecution` runs tasks after the execution finishes in a terminal state like **SUCCESS** or **FAILED**.
+`finally` runs while the execution is still `RUNNING` — it cannot see the terminal state. `afterExecution` runs after the execution settles, so it sees `SUCCESS`, `FAILED`, or `WARNING`. Use `finally` for cleanup that must always happen; use `afterExecution` when follow-up logic depends on the outcome.
-Use `finally` for cleanup operations that should always run, regardless of the outcome. See the [`finally` documentation](../19.finally/index.md) for examples. When follow-up actions depend on the final state, use `afterExecution` to capture the result.
-
-To demonstrate, take the following flow that uses both `finally` and `afterExecution`:
+The following flow demonstrates the difference:
```yaml
id: state_demo
@@ -119,10 +99,6 @@ afterExecution:
message: Execution {{ execution.state }} # Will show FAILED
```
-After running the example above, the `finally` task appears with a `RUNNING` state while the `afterExecution` task shows `FAILED`.
-
-
+The `finally` task logs `Execution RUNNING` because it runs before the execution reaches its terminal state. The `afterExecution` task logs `Execution FAILED` because it runs after. See the [`finally` documentation](../19.finally/index.md) for more on cleanup patterns.
-:::alert{type="info"}
-Best practice: Use `afterExecution` when you need to act on the final state of an execution. Use `finally` when you need to ensure cleanup happens regardless of state.
-:::
+
diff --git a/src/contents/docs/05.workflow-components/21.quotas/index.md b/src/contents/docs/05.workflow-components/21.quotas/index.md
new file mode 100644
index 00000000000..07ea35a886b
--- /dev/null
+++ b/src/contents/docs/05.workflow-components/21.quotas/index.md
@@ -0,0 +1,128 @@
+---
+title: "Flow Quotas in Kestra: Rate-Limit Execution Creation"
+h1: Rate-Limit Execution Creation with Quotas
+description: Use flow-level quotas in Kestra Enterprise to cap how many executions a flow can create within a time period. Set CANCEL or FAIL behavior when the limit is exceeded.
+sidebarTitle: Quotas
+icon: /src/contents/docs/icons/flow.svg
+editions: ["EE", "Cloud"]
+---
+
+Cap the number of executions created within a time window, at the flow, namespace, or tenant level. This is different from [`concurrency`](../14.concurrency/index.md), which limits how many executions **run simultaneously**.
+
+Each quota specifies a `duration` (the time window and the unique identifier for that quota entry), a `limit` (the maximum executions allowed in that window), and a `behavior` (what happens when the limit is exceeded).
+
+:::alert{type="warning"}
+Quotas are an Enterprise Edition and Cloud feature. Adding a `quotas` block to a flow on an Open Source installation causes a validation error at save time.
+:::
+
+## Quota properties
+
+Each entry in the `quotas` list has three required properties:
+
+| Property | Type | Required | Description |
+| --- | --- | --- | --- |
+| `duration` | string | Yes | ISO 8601 duration defining the time window, e.g. `PT1H` (one hour) or `P1D` (one day). Minimum: `PT1M`. Maximum: `P1D`; durations above one day are not supported. Acts as the unique identifier; each duration value must appear at most once in the list. |
+| `limit` | integer | Yes | Maximum number of executions allowed within the window. Must be `>= 1`. |
+| `behavior` | enum | Yes | Action taken when the limit is reached. One of `CANCEL` or `FAIL`. |
+
+:::alert{type="info"}
+Windows are **fixed and UTC-aligned**, not rolling. `PT1H` covers the current UTC clock-hour (e.g. 14:00–15:00 UTC), not the preceding 60 minutes. The maximum duration is `P1D` (one day); durations above one day are not supported. Prefer durations that divide evenly into 24 hours (`PT1M`, `PT5M`, `PT15M`, `PT30M`, `PT1H`, `PT2H`, `PT3H`, `PT4H`, `PT6H`, `PT8H`, `PT12H`, `PT24H`). Durations that don't divide evenly (e.g. `PT7H`) produce a shorter final window each day.
+:::
+
+## Behavior options
+
+- **`CANCEL`** — the execution is immediately marked as `CANCELLED` before any tasks run.
+- **`FAIL`** — the execution is immediately marked as `FAILED` before any tasks run.
+
+:::alert{type="info"}
+`QUEUE` behavior is not supported for quotas. To hold executions until capacity is available rather than dropping them, use [`concurrency`](../14.concurrency/index.md) with `behavior: QUEUE`.
+:::
+
+## Quota levels
+
+### Flow level
+
+Define quotas directly on a flow using the `quotas` property in the flow YAML. These quotas apply only to that specific flow.
+
+```yaml
+id: my_flow
+namespace: company.team
+
+quotas:
+ - behavior: CANCEL
+ limit: 10
+ duration: PT1H
+```
+
+### Namespace level
+
+Define quotas on a namespace to apply limits to all flows within that namespace. Navigate to **Namespaces**, open the target namespace, click **Edit**, and scroll to the **Quotas** section.
+
+Namespace quotas apply to every flow whose namespace matches or is a child of the configured namespace.
+
+### Tenant level
+
+Define quotas on a tenant to apply limits across all flows in the entire tenant. In **Instance Owner**, click **Quota Limits** in the sidebar, then **Administer** on the target tenant.
+
+## Evaluation order
+
+When an execution is triggered, quotas are evaluated from the most specific level to the most general:
+
+1. **Flow-level** quotas are checked first.
+2. **Namespace-level** quotas are checked next, from the most general ancestor namespace down to the flow's own namespace.
+3. **Tenant-level** quotas are checked last.
+
+If a quota at a more specific level is exceeded, the execution is cancelled or failed immediately and no slot is consumed at higher levels. This prevents a flow-level quota breach from also counting against shared namespace or tenant limits.
+
+## Flow-level examples
+
+The flow below defines two quotas: a short-window limit of 10 executions per hour and a longer-window limit of 100 executions per day. Executions that exceed either limit are cancelled immediately.
+
+```yaml
+id: quotas_flow
+namespace: company.team
+
+quotas:
+ - behavior: CANCEL
+ limit: 10
+ duration: PT1H
+
+ - behavior: CANCEL
+ limit: 100
+ duration: P1D
+
+tasks:
+ - id: hello
+ type: io.kestra.plugin.core.log.Log
+ message: Hello World!
+```
+
+You can mix behaviors across quotas in the same flow, for example cancel executions that exceed the hourly limit but fail those that exceed the daily limit:
+
+```yaml
+quotas:
+ - behavior: CANCEL
+ limit: 10
+ duration: PT1H
+
+ - behavior: FAIL
+ limit: 100
+ duration: P1D
+```
+
+Each `duration` value must be unique within the `quotas` list. Defining two quotas with the same `duration` string causes a validation error at save time.
+
+## Monitoring quota state
+
+The **Quota Limits** page (left sidebar, under **Tenant**) lists active quota counters across the tenant. Each row shows the namespace, flow ID, quota duration, the start of the current window, and the current execution count. The **Actions** column contains a reset button to manually clear a quota counter before the window expires.
+
+For namespace-level quotas, the flow column shows ``; for tenant-level quotas, both columns show ``.
+
+Rows for expired windows are automatically hidden. Use the refresh button (top right of the page) to reload the current state. Columns are sortable by namespace and flow ID.
+
+## When to use quotas
+
+- **Flow level** — cap how often a specific flow can be triggered by external events or webhooks to prevent runaway execution chains, or enforce a cost policy on flows that call expensive external APIs.
+- **Namespace level** — apply a shared execution budget across all flows in a team or environment namespace, without configuring each flow individually.
+- **Tenant level** — enforce an organization-wide ceiling on execution creation, for example to stay within an infrastructure or cost constraint that applies across all namespaces.
+- **Complement concurrency** — quotas cap the creation rate; concurrency caps simultaneous parallelism.
diff --git a/src/contents/docs/05.workflow-components/22.reusable-inputs/index.md b/src/contents/docs/05.workflow-components/22.reusable-inputs/index.md
new file mode 100644
index 00000000000..06a098b425e
--- /dev/null
+++ b/src/contents/docs/05.workflow-components/22.reusable-inputs/index.md
@@ -0,0 +1,128 @@
+---
+title: "Reusable Inputs in Kestra Enterprise"
+h1: Reusable Inputs
+sidebarTitle: Reusable Inputs
+description: Define a named input group once at the namespace level and reference it across multiple flows to eliminate duplicated input declarations.
+icon: /src/contents/docs/icons/flow.svg
+editions: ["EE", "Cloud"]
+---
+
+Reusable inputs let you define a named group of typed inputs once at the namespace level and reference them in any flow.
+
+## Define a reusable input set
+
+Open the namespace where you want the reusable input set to live, select the **Reusable Inputs** tab, and click **Create**. Give the set an `id`, an optional description, and declare the inputs the same way you would in a flow.
+
+```yaml
+id: ad_credentials
+namespace: company.team
+description: Active Directory credentials required by provisioning flows.
+inputs:
+ - id: username
+ type: STRING
+ required: true
+ displayName: AD Username
+
+ - id: domain
+ type: SELECT
+ required: true
+ values:
+ - corp.example.com
+ - eu.example.com
+
+ - id: password
+ type: SECRET
+ required: true
+ displayName: AD Password
+```
+
+The set's `id` must start with a letter or digit and may contain letters, digits, dots, hyphens, and underscores. All flow input types are valid inside a set, including `FORM` groups.
+
+Each time you save, Kestra stores a new revision.
+
+## Reference a set in a flow
+
+Add an input of `type: REUSABLE_INPUTS` to any flow and set `ref` to the set's `id`. Kestra inlines each child input before the execution starts, prefixing its id with the reference id (`.`).
+
+```yaml
+id: provision_ad_account
+namespace: company.team
+
+inputs:
+ - id: environment
+ type: SELECT
+ values: [staging, production]
+
+ - id: ad
+ type: REUSABLE_INPUTS
+ ref: ad_credentials
+
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: |
+ User: {{ inputs.ad.username }}
+ Domain: {{ inputs.ad.domain }}
+```
+
+The set expands into three inputs — `ad.username`, `ad.domain`, and `ad.password` — available alongside `environment` in the execution form.
+
+### Access input values
+
+Reference each child input as `{{ inputs.. }}`, where `refId` is the `id` of the `REUSABLE_INPUTS` input in the flow and `childId` is the `id` from the set definition.
+
+```
+{{ inputs.ad.username }}
+{{ inputs.ad.domain }}
+{{ inputs.ad.password }}
+```
+
+## Set properties
+
+| Property | Required | Description |
+| --- | --- | --- |
+| `ref` | Yes | The `id` of the reusable input set to include. |
+| `namespace` | No | Namespace where the set is defined. Defaults to the flow's own namespace. Resolution walks the hierarchy — a set in a parent namespace is available to child namespaces. |
+| `revision` | No | Integer revision to pin. Omit to use the latest revision at execution time. |
+
+### Pin a specific revision
+
+To lock a flow to a particular version of a set — for example, to avoid pulling in a breaking change before your team has reviewed it — set `revision`:
+
+```yaml
+inputs:
+ - id: ad
+ type: REUSABLE_INPUTS
+ ref: ad_credentials
+ revision: 3
+```
+
+
+### Use a set from a parent namespace
+
+A set defined in `company` is available to any flow in `company`, `company.team`, `company.team.infra`, and any other descendant namespace. If a child namespace defines a set with the same `id`, the child's definition takes precedence over the parent's. To use a set from a different branch of the hierarchy, set `namespace` explicitly:
+
+```yaml
+inputs:
+ - id: ad
+ type: REUSABLE_INPUTS
+ ref: ad_credentials
+ namespace: company
+```
+
+## RBAC
+
+The `REUSABLE_INPUTS` resource controls who can manage reusable input sets. Flows that reference a set require only normal flow-execution permissions — the expansion happens server-side before the execution form is rendered.
+
+| Role | Permissions |
+| --- | --- |
+| Viewer | View, list |
+| Launcher | View, list |
+| Editor | View, list, create, update, delete |
+| Admin | All actions |
+
+## Validation
+
+Kestra does not check whether a referenced set exists at flow save time — the existence check runs at execution creation. If the set is missing, execution creation fails with an error naming the missing set and the namespace hierarchy that was searched.
+
+A reusable input set's inputs are expanded in a single pass. If a set's inputs list contains a `REUSABLE_INPUTS` entry, that nested reference is not recursively expanded and the execution form will fail to render it correctly. Keep set definitions flat.
diff --git a/src/contents/docs/05.workflow-components/task-cache/index.md b/src/contents/docs/05.workflow-components/task-cache/index.md
index ea1878bd931..96c4e5d9fa8 100644
--- a/src/contents/docs/05.workflow-components/task-cache/index.md
+++ b/src/contents/docs/05.workflow-components/task-cache/index.md
@@ -9,9 +9,7 @@ version: "0.24.0"
Cache the status and outputs of computationally expensive operations.
-The `taskCache` property stores a task’s status and outputs in Kestra’s database. When the same execution runs again with identical inputs, Kestra skips the task and reuses the cached outputs. You can enable caching on any task, but it is most effective for heavy operations such as large data extractions or long-running scripts.
-
-Using task caching can significantly speed up workflows and reduce resource consumption.
+The `taskCache` property stores a task’s status and outputs in Kestra’s database. When the same execution runs again with identical inputs, Kestra skips the task and reuses the cached outputs. It is most effective for heavy operations such as large data extractions or long-running scripts.
:::alert{type="info"}
Task caching is only supported for [Runnable Tasks](../01.tasks/01.runnable-tasks/index.md).
@@ -19,8 +17,6 @@ Task caching is only supported for [Runnable Tasks](../01.tasks/01.runnable-task
## `taskCache` syntax
-The syntax of the `taskCache` property is as follows:
-
```yaml
taskCache:
enabled: true
@@ -31,7 +27,7 @@ The `ttl` (time-to-live) property defines how long cached outputs are kept befor
## `taskCache` example
-In the example below, the flow caches the outputs of a computationally expensive task, extracting a large dataset from a production database. This flow downloads product data once per day, caches it for 24 hours, and reuses it in joins with frequently updated transaction data.
+This flow downloads product data once per day, caches it for 24 hours, and reuses it in joins with frequently updated transaction data.
```yaml
id: caching
diff --git a/src/contents/docs/06.concepts/02.namespace-files/index.md b/src/contents/docs/06.concepts/02.namespace-files/index.md
index d559bb14326..ecbffcb2dda 100644
--- a/src/contents/docs/06.concepts/02.namespace-files/index.md
+++ b/src/contents/docs/06.concepts/02.namespace-files/index.md
@@ -6,7 +6,7 @@ sidebarTitle: Namespace Files
icon: /src/contents/docs/icons/concepts.svg
---
-Manage Namespace Files and how to use them in your flows.
+Namespace Files are files tied to a namespace — scripts, queries, configs, and other assets you can reference in any flow within that namespace.
@@ -30,12 +30,12 @@ namespace: company.team
tasks:
- id: for_each_row
- type: io.kestra.plugin.core.flow.ForEach
+ type: io.kestra.plugin.core.flow.Loop
values: "{{ trigger.rows }}"
tasks:
- id: return
type: io.kestra.plugin.core.debug.Return
- format: "{{ json(taskrun.value) }}"
+ format: "{{ fromJson(item.value) }}"
triggers:
- id: query_trigger
@@ -44,7 +44,7 @@ triggers:
url: jdbc:clickhouse://127.0.0.1:56982/
username: "{{ secret('CLICKHOUSE_USERNAME') }}"
password: "{{ secret('CLICKHOUSE_PASSWORD') }}"
- sql: "{{ read('queries/my_query.sql') }}" # 🚀 The read() function reads the content of the file as a string!
+ sql: "{{ read('queries/my_query.sql') }}"
fetchType: FETCH
```
@@ -54,21 +54,20 @@ The `namespaceFiles.enabled: true` property is not required here — it is only
## Why use Namespace Files
-Namespace Files offer a simple way to organize your code and configuration files. Before Namespace Files, you had to store your code and configuration files in a Git repository and then clone that repository at runtime using the `git.Clone` task. With Namespace Files, you can store your code and configuration files directly in the Kestra's internal storage backend. That storage backend can be your local directory or an S3 bucket to ensure maximum security and privacy.
+Namespace Files let you store scripts, queries, and configs directly in Kestra rather than cloning a Git repository at runtime. Files live in Kestra's internal storage backend and are shared across all flows in the namespace, so you maintain one copy instead of duplicating code between flows.
-Namespace Files make it easy to:
-- orchestrate Python, R, Node.js, SQL, and more without having to worry about code dependencies, packaging, and deployments — simply add your code in the embedded Code Editor or sync your Git repository with a given namespace
-- manage your code for a given project or team in one place, even if those files are stored in different Git repositories or even different Git providers
-- share your code and configuration files between workflows and team members in your organization
-- orchestrate complex projects that require the code to be separated into multiple scripts, queries, or modules.
+Common use cases:
+- Centralize SQL queries, Python scripts, or config files used by multiple flows
+- Sync a full Git project (dbt, Terraform, Ansible) to a namespace and orchestrate it without per-flow cloning
+- Share code across teams whose files live in different repositories or Git providers
## How to add Namespace Files
### Embedded code editor
-While creating or editing a Flow, you can access Namespace Files from the **Namespace Files** tab. You can easily write, import, or paste custom scripts, queries, and configuration files.
+Access Namespace Files from the **Files** tab while creating or editing a flow. From there you can write, import, or paste scripts, queries, and configuration files directly.
-To start, add a new file (e.g., a Python script). Add a folder named `scripts` and a file called `hello.py` with the following content:
+To try it, create a folder named `scripts` and a file called `hello.py`:
```python
print("Hello from the Editor!")
@@ -89,7 +88,7 @@ tasks:
- python scripts/hello.py
```
-The **Execute** button allows you to run your flow directly from the Code Editor. Click on the **Execute** button to run your flow. You then see the Execution running, and once you navigate to the **Logs** tab, you should see a friendly message ``Hello from the Editor!`` in the logs.
+The **Execute** button allows you to run your flow directly from the Code Editor. Click **Execute** to run the flow. The **Logs** tab shows a friendly message ``Hello from the Editor!``.
### Namespace Files Revision History
@@ -101,11 +100,11 @@ Namespace Files include revision history just like flows, so you can inspect or
To access a file's revision history, right-click on the file.
-
+
From the history, view, compare, and restore prior versions.
-
+
From the **Revisions** list, you can delete a given revision or all revisions older than the selected one. You will be prompted to confirm this choice, as there is no possible way to restore a revision once is has been deleted.
@@ -239,11 +238,9 @@ You can also use the `io.kestra.plugin.core.flow.WorkingDirectory` task to read
### The `read()` function
-The script in the first section used the `read()` function to read the content of the `scripts/hello.py` file as a string using the expression `"{{ read('scripts/hello.py') }}"`. It's important to remember that this function reads **the content of the file as a string**. Therefore, you should use that function only in tasks that expect a string as an input like `io.kestra.plugin.scripts.python.Script` or `io.kestra.plugin.scripts.node.Script`, rather than `io.kestra.plugin.scripts.python.Commands` or `io.kestra.plugin.scripts.node.Commands`.
+`read()` returns the **contents** of a namespace file as a string. Use it in tasks that accept string input — `io.kestra.plugin.scripts.python.Script`, `io.kestra.plugin.scripts.node.Script`, SQL query properties, and similar — not in `Commands` tasks that expect a file path on disk. The path must point to a file in the same namespace as the flow.
-The `read()` function allows you to read the content of a Namespace File stored in the Kestra's internal storage backend. The `read()` function takes a single argument, which is the absolute path to the file you want to read. The path must point to a file stored in the **same namespace** as the flow you are executing.
-
-In this example, we have a namespace file called `example.txt` that contains the text `Hello, World!`. We can print the content to the logs by using `{{ read('example.txt') }}`:
+This example logs the contents of `example.txt`:
```yaml
id: files
@@ -295,12 +292,12 @@ The example above uses the `include` field to only allow the `scripts/weather.py
We can control what namespace files are available to our flow with the `namespaceFiles` property.
`namespaceFiles` has several configurable attributes:
-- `enabled`: when set to true enables all files in that namespace to be visible to the task
-- `include`: specifies files you want to be accessible by the task
-- `exclude`: specifies files you don't want to be accessible by the task
-- `namespaces`: specifies a list of namespaces to search for files.
-- `ifExists`: specifies what to do in the instance a Namespace file already exists in the working directory
-- `folderPerNamespace`: a boolean property that mounts namespace files in separate directories (set to `false` by default) rather than all files to the root of the working directory
+- `enabled`: when set to `true`, makes all files in the namespace visible to the task.
+- `include`: restricts which files are accessible — only the listed paths are mounted.
+- `exclude`: mounts all namespace files except those listed.
+- `namespaces`: a list of additional namespaces to load files from.
+- `ifExists`: controls what happens when a namespace file conflicts with an existing file in the working directory.
+- `folderPerNamespace`: when `true`, mounts each namespace's files into a separate subdirectory instead of the working directory root.
The `namespaces` attribute can be used like in the following example:
@@ -328,7 +325,7 @@ tasks:
script: "{{ read('test.py') }}"
```
-The files are loaded in the namespace order, and only the latest version of a file is kept. Meaning if a file is present in the first and second namespace, only the file present on the second namespace will be loaded. In the first task, the `test.py` file from the `company` namespace will be used because priority is given from top to bottom of the list of namespaces. In the case of multiple files of the same name, the last listed namespace holds priority.
+Namespaces are loaded in list order, but when the same file exists in multiple namespaces, the last listed namespace wins. In the first task, `dev.test` is listed first and `company` is listed second, so `company`'s `test.py` takes precedence.
For the second task, the `test.py` file in the `dev.test` namespace will be used because no namespace has been defined in the `read()` function. If you want to fetch the `test.py` script from a different namespace, you need to explicitly define it as follows: `"{{ read('test.py', namespace='company.team') }}"`.
@@ -373,7 +370,7 @@ tasks:
### Namespace tasks
-You can use the Namespace Tasks to upload, download, and delete tasks in Kestra.
+Use the Namespace Tasks to upload, download, and delete files in Kestra.
In the example below, we have a namespace file called `example.ion` that we want to convert to a `.csv` file. We can use the `DownloadFiles` task to generate an output that contains the file so we can easily pass it dynamically to the `IonToCsv` task.
@@ -473,5 +470,5 @@ namespaceFiles:
```
:::alert{type="warning"}
-Patterns without a leading `/` are automatically prefixed with `**/`. Use `/…` or explicit `glob:`/`regex:` patterns if you want root-only matching. Patterns that already contain `**` (for example `sg_base_etl/**`) may be unintentionally transformed; use `/sg_base_etl/**` or `glob:/sg_base_etl/**` as a workaround.
+Patterns without a leading `/` are automatically prefixed with `**/`, which makes them recursive. Use a leading `/` or explicit `glob:`/`regex:` to restrict matching to the namespace root. Patterns that already contain `**` (e.g. `my_dir/**`) are still prefixed, producing `**/my_dir/**`; use `/my_dir/**` or `glob:/my_dir/**` to avoid the double prefix.
:::
diff --git a/src/contents/docs/06.concepts/02.namespace-files/namespace-file-restore-2-0.png b/src/contents/docs/06.concepts/02.namespace-files/namespace-file-restore-2-0.png
new file mode 100644
index 00000000000..8055b716dee
Binary files /dev/null and b/src/contents/docs/06.concepts/02.namespace-files/namespace-file-restore-2-0.png differ
diff --git a/src/contents/docs/06.concepts/02.namespace-files/namespace-file-restore.png b/src/contents/docs/06.concepts/02.namespace-files/namespace-file-restore.png
deleted file mode 100644
index 864ea03906c..00000000000
Binary files a/src/contents/docs/06.concepts/02.namespace-files/namespace-file-restore.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/02.namespace-files/namespace-file-revision-history-2-0.png b/src/contents/docs/06.concepts/02.namespace-files/namespace-file-revision-history-2-0.png
new file mode 100644
index 00000000000..30005163bf6
Binary files /dev/null and b/src/contents/docs/06.concepts/02.namespace-files/namespace-file-revision-history-2-0.png differ
diff --git a/src/contents/docs/06.concepts/02.namespace-files/namespace-file-revision-history.png b/src/contents/docs/06.concepts/02.namespace-files/namespace-file-revision-history.png
deleted file mode 100644
index c488998ab28..00000000000
Binary files a/src/contents/docs/06.concepts/02.namespace-files/namespace-file-revision-history.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/03.revision/index.md b/src/contents/docs/06.concepts/03.revision/index.md
index 60b6b166c7a..d110c4cc9d7 100644
--- a/src/contents/docs/06.concepts/03.revision/index.md
+++ b/src/contents/docs/06.concepts/03.revision/index.md
@@ -6,14 +6,26 @@ sidebarTitle: Revision
icon: /src/contents/docs/icons/concepts.svg
---
-Manage versions of flows.
+Every time you save a flow, Kestra creates a new revision. The **Revisions** tab lists all revisions for a flow — compare any two side-by-side or line-by-line, and roll back to a previous version at any time.
-Flows are versioned by default. Whenever you make any changes to your flows, a new revision is created. This allows you to rollback to a previous version of your flow if needed.
+
-If you navigate to a specific flow and go to the **Revisions** tab, you will see a list of all revisions of that flow. You can then compare the differences between two revisions side-by-side or line-by-line and rollback to a previous revision if needed.
+## Draft revisions
-
+When you click **Save as draft** in the flow editor, Kestra saves your changes as a draft revision rather than publishing them immediately.
+
+:::alert{type="warning"}
+Executions do not run against a draft revision. If the latest revision of a flow is a draft, any execution — whether triggered manually, by a schedule, or by an event — will run against the last published revision instead. A warning banner in the run panel makes this explicit.
+:::
+
+To make your latest changes active, open the run panel — it displays a draft warning banner with a **Publish** button that promotes the draft to a published revision.
+
+Use **Save as draft** when you want to stage changes without affecting running executions — for example, while iterating on a flow that is already in production.
+
+:::alert{type="info"}
+On Kestra Enterprise, you can promote a published revision to another environment (staging, production) directly from the flow's **Deploy** tab, without copy-pasting YAML or setting up a CI/CD pipeline. See [Promote flows across environments](../../version-control-cicd/06.promote/index.md).
+:::
diff --git a/src/contents/docs/06.concepts/03.revision/revisions.png b/src/contents/docs/06.concepts/03.revision/revisions.png
index f5a033fcbec..4a8244de171 100644
Binary files a/src/contents/docs/06.concepts/03.revision/revisions.png and b/src/contents/docs/06.concepts/03.revision/revisions.png differ
diff --git a/src/contents/docs/06.concepts/04.secret/index.md b/src/contents/docs/06.concepts/04.secret/index.md
index 377dd8f9242..8f5f92712a4 100644
--- a/src/contents/docs/06.concepts/04.secret/index.md
+++ b/src/contents/docs/06.concepts/04.secret/index.md
@@ -12,32 +12,19 @@ Secrets let you store sensitive values (API keys, passwords, certificates) outsi
-How secrets are stored and managed depends on your edition. In **Enterprise Edition**, Kestra connects to a dedicated Secrets Manager (namespace-scoped, backed by AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, or Kestra's own store). In **Open-Source**, there is no secret store — `secret()` reads a base64-encoded environment variable instead.
+How secrets are stored depends on your edition. Enterprise Edition connects to a dedicated Secrets Manager backed by AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, or Kestra's own store. Open-Source has no secret store — `secret()` reads a base64-encoded environment variable instead.
## Enterprise Edition
-Use secrets for static sensitive values such as API keys, passwords, webhook URLs, certificates, and long-lived tokens. Use [Credentials](../../07.enterprise/03.auth/credentials/index.md) when Kestra needs to manage reusable server-to-server authentication for supported integrations — for example, minting or refreshing short-lived access tokens at runtime. Credentials can reference secrets for sensitive inputs such as client secrets and private keys.
+Secrets are the right choice for static sensitive values: API keys, passwords, webhook URLs, certificates, and long-lived tokens. For reusable server-to-server authentication — where Kestra needs to mint or refresh short-lived tokens at runtime — use [Credentials](../../07.enterprise/03.auth/credentials/index.md) instead. Credentials can reference secrets for sensitive inputs such as client secrets and private keys.
-For available backends (AWS Secrets Manager, Azure Key Vault, HashiCorp Vault, and Kestra's built-in store), see the [Secrets Manager](../../07.enterprise/02.governance/secrets-manager/index.md) page. For best practices, see [Secrets management](../../14.best-practices/9.secrets-management/index.md) and [Choosing where to store sensitive and shared values](../../14.best-practices/10.credentials-vs-secrets-vs-kv-store/index.md).
+Secrets are available under **Namespaces → [namespace] → Secrets** or under **Tenant → Secrets** in the sidebar. Click **New secret**, set a key name such as `MY_SECRET`, and optionally add a description and tags. From the same tab you can edit, delete, or copy a secret as a Pebble expression — for example, `"{{ secret('API_TOKEN') }}"` — ready to paste into a flow.
-From the **Secrets** tab, you can edit, delete, and copy your secret to your clipboard as a Pebble expression for use in a flow, such as `"{{ secret('API_TOKEN') }}"`.
+For available backends, see the [Secrets Manager](../../07.enterprise/02.governance/secrets-manager/index.md) page. For best practices, see [Secrets management](../../14.best-practices/9.secrets-management/index.md) and [Choosing where to store sensitive and shared values](../../14.best-practices/10.credentials-vs-secrets-vs-kv-store/index.md).
-
+### Reading secrets from another namespace
-### Adding a new secret from the UI
-
-Go to **Namespaces** in the left navigation menu and select the namespace where you want to add a secret. Open the **Secrets** tab and add a new secret.
-
-
-
-Set a key name such as `MY_SECRET`. You can also include a short description and tags.
-
-
-
-
-### Reading secrets from another namespace (EE)
-
-By default, `secret()` reads from the flow's own namespace. In Enterprise Edition, you can pass a `namespace` argument to read a secret stored in a different namespace:
+By default, `secret()` reads from the flow's own namespace. You can pass a `namespace` argument to read a secret stored in a different namespace:
```yaml
tasks:
@@ -46,11 +33,11 @@ tasks:
message: "{{ secret('SHARED_TOKEN', namespace='shared.secrets') }}"
```
-The secret is resolved using the target namespace's own secret backend, so a flow can read a value from a namespace backed by a different secrets manager. Cross-namespace reads stay within the same tenant. Access to another namespace's secrets is allowed by default; restrict it by configuring `allowedNamespaces` on the target namespace.
+The secret resolves using the target namespace's own backend, so a flow can read a value from a namespace backed by a different secrets manager. Cross-namespace reads stay within the same tenant. Access is allowed by default; restrict it by configuring `allowedNamespaces` on the target namespace.
-## Environment variables as secrets (OSS)
+## Secrets in Open-Source
-The Open-Source Edition has no dedicated secret store. As a workaround, Kestra reads base64-encoded environment variables prefixed with `SECRET_` and exposes them via the `secret()` function. This keeps sensitive values out of flow YAML, but it is not a secrets manager — there is no encryption at rest, no audit trail, and no access control beyond what your host environment provides.
+Open-Source has no dedicated secret store. Kestra reads base64-encoded environment variables prefixed with `SECRET_` and exposes them via the `secret()` function. This keeps sensitive values out of flow YAML, but offers no encryption at rest, no audit trail, and no access control beyond what your host environment provides.
See [Configure secrets in Kestra (OSS)](../../15.how-to-guides/secrets/index.md) for step-by-step instructions on encoding values and wiring them into your Docker Compose file.
diff --git a/src/contents/docs/06.concepts/04.secret/secrets-ee-0.png b/src/contents/docs/06.concepts/04.secret/secrets-ee-0.png
deleted file mode 100644
index 549fd47c65a..00000000000
Binary files a/src/contents/docs/06.concepts/04.secret/secrets-ee-0.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/04.secret/secrets-ee-1.png b/src/contents/docs/06.concepts/04.secret/secrets-ee-1.png
deleted file mode 100644
index 22a768942c5..00000000000
Binary files a/src/contents/docs/06.concepts/04.secret/secrets-ee-1.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/04.secret/secrets-ee-2.png b/src/contents/docs/06.concepts/04.secret/secrets-ee-2.png
deleted file mode 100644
index 3b01f69eafc..00000000000
Binary files a/src/contents/docs/06.concepts/04.secret/secrets-ee-2.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/05.kv-store/create_kv_pair.png b/src/contents/docs/06.concepts/05.kv-store/create_kv_pair.png
deleted file mode 100644
index bdf777da48d..00000000000
Binary files a/src/contents/docs/06.concepts/05.kv-store/create_kv_pair.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/05.kv-store/edit_delete_kv_pair.png b/src/contents/docs/06.concepts/05.kv-store/edit_delete_kv_pair.png
deleted file mode 100644
index d1465c0acf5..00000000000
Binary files a/src/contents/docs/06.concepts/05.kv-store/edit_delete_kv_pair.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/05.kv-store/index.md b/src/contents/docs/06.concepts/05.kv-store/index.md
index f6abb8e085b..853c3ffe23b 100644
--- a/src/contents/docs/06.concepts/05.kv-store/index.md
+++ b/src/contents/docs/06.concepts/05.kv-store/index.md
@@ -8,36 +8,19 @@ version: ">= 0.18.0"
docId: kv
---
-Build stateful workflows with the KV Store.
+The KV Store lets you persist and share key-value data across executions and flows — beyond what task outputs alone can carry.
-Kestra's workflows are stateless by design. All workflow executions and task runs are isolated from each other by default to avoid any unintended side effects. When you pass data between tasks, you do so explicitly by passing outputs from one task to another, and that data is stored transparently in Kestra's internal storage. This stateless execution model ensures that workflows are idempotent and can be executed anywhere in parallel at scale.
+Kestra's execution model is stateless by design — task runs are isolated and data moves between tasks via explicit outputs. The KV Store extends this with a namespace-scoped key-value layer for cases where you need to persist state across executions or share values between flows. Values are stored in Kestra's internal storage (your private cloud bucket); the database holds only metadata such as the key, TTL, and timestamps.
-However, in certain scenarios, your workflow might need to share data beyond passing outputs from one task to another. For example, you might want to persist data across executions or even across different workflows. This is where the Key Value (KV) store comes into play.
+## Keys and values
-KV Store allows you to store any data in a convenient key-value format. You can create them directly from the UI, via dedicated tasks, Terraform, or through the API.
+Keys are arbitrary strings containing uppercase or lowercase letters and standard ASCII characters.
-The KV store is a powerful tool that allows you to build stateful workflows and share data across executions and workflows.
-
-## How KV store fits into Kestra's architecture
-
-Kestra's architecture has been designed to offer a transparent separation between the orchestration and data processing capabilities. Kestra's [Executor](../../08.architecture/02.server-components/index.md#executor) is responsible for executing tasks and workflows without directly interacting with the user's infrastructure. The Executor relies on [Workers](../../08.architecture/02.server-components/index.md#worker), which are stateless processes that carry out the computation of runnable tasks and polling triggers. For privacy reasons, workers are the only components that interact with the user's infrastructure, including the internal storage and external services.
-
-Given that data persisted in the KV Store might contain sensitive information, the **KV Store has been built on top of Kestra's internal storage**. This ensures that all values are stored in your private cloud storage bucket, and Kestra's database only contains metadata about the object, such as the key, file URI, any attached metadata about the object like TTL, creation date, last updated timestamp, etc.
-
-In short, the KV Store gives you full control and privacy over your data, and Kestra only stores metadata about the KV pairs.
-
-## Keys and Values
-
-`Keys` are arbitrary strings. Keys can contain:
-
-- characters in uppercase and or lowercase
-- standard ASCII characters
-
-`Values` are stored as ION files in Kestra's internal storage. Values are strongly typed and can be of one of the following types:
+Values are stored as ION files in Kestra's internal storage and are strongly typed. Supported types:
- string
- number
@@ -45,56 +28,41 @@ In short, the KV Store gives you full control and privacy over your data, and Ke
- datetime
- date
- duration
-- JSON.
+- JSON
-For each KV pair, you can set a `Time to Live` (TTL) to avoid cluttering your storage with data that may only be relevant for a limited time.
+Each KV pair can have a Time to Live (TTL) to automatically expire data that is only relevant for a limited period.
## Namespace binding
-Key value pairs are defined at a namespace level, and you can access them from the namespace page in the UI in the KV Store tab.
+KV pairs are scoped to a namespace. Access them under **Namespaces → [namespace] → KV Store** or under **Tenant → KV Store** in the sidebar for a tenant-wide view.
You can create and read KV pairs across namespaces as long as those namespaces are [allowed](../../07.enterprise/02.governance/07.namespace-management/index.md#allowed-namespaces).
-## UI: How to Create, Read, Update and Delete KV pairs from the UI
-
-Kestra follows a philosophy of Everything as Code and also from the UI. Therefore, you can create, read, update, and delete KV pairs both from the UI and Code.
-
-Here is a list of the different ways to manage KV pairs:
-1. **Kestra UI**: select a Namespace and go to the KV Store tab — from here, you can create, edit, and delete KV pairs.
-2. **Task in a flow**: use the `io.kestra.plugin.core.kv.Set`, `io.kestra.plugin.core.kv.Get`, and `io.kestra.plugin.core.kv.Delete` tasks to create, read, and delete KV pairs in a flow.
-3. **Kestra's API**: use our HTTP REST API to create, read, and delete KV pairs.
-4. **Kestra's Terraform provider**: use the `kestra_kv` resource to create, read, and delete KV pairs.
-5. **Pebble function**: use the `kv()` function to retrieve a value by key in a flow.
-6. **GitHub Actions**: create, read, and delete KV pairs in your CI/CD pipeline.
-7. **kestractl**: use `kestractl kv` to list, set, update, get, and delete KV pairs from the command line. See the [kestractl docs](../../kestra-cli/kestractl/index.md) for setup.
-
-The sections below provide detailed instructions on how to create and manage KV pairs using each of these methods.
+## Managing KV pairs
-### Create new KV pairs from the UI
+KV pairs can be managed from the UI, in flows via tasks, through the REST API, via Terraform, or from the command line:
-You can create, read, update, and delete KV pairs from the UI in the following way:
+1. **Kestra UI**: create, edit, and delete KV pairs directly (see [Namespace binding](#namespace-binding) for navigation paths).
+2. **Tasks in a flow**: use `io.kestra.plugin.core.kv.Set`, `Get`, and `Delete` tasks.
+3. **REST API**: create, read, and delete KV pairs via the HTTP API.
+4. **Terraform**: use the `kestra_kv` resource.
+5. **Pebble function**: use `kv()` to retrieve a value inline in a flow expression.
+6. **GitHub Actions**: manage KV pairs in your CI/CD pipeline.
+7. **kestractl**: use `kestractl kv` to list, set, get, and delete from the command line. See the [kestractl docs](../../kestra-cli/kestractl/index.md) for setup.
-1. Navigate to the `Namespaces` page from the left navigation menu and select the namespace where you want to create the KV pair.
- 
-2. Go to the `KV Store` tab. This is where you can see all the KV pairs associated with this namespace.
- 
-3. Click on `New Key-Value` button in the top right corner to create a new KV pair. Enter a name for the `Key` and assign a suitable `Type` for the value — it can be a string, number, boolean, datetime, date, duration, or JSON.
- 
-4. Enter the value in the `Value` field.
-5. Optionally, you can configure a Time to Live (TTL) for the KV pair. The dropdown contains some standard durations. You can also select `Custom duration` to enter a custom duration as a string in ISO 8601 duration format.
-6. Finally, `Save` the changes. Your new KV pair should now be displayed in the list of KV pairs for that namespace.
+### Create a KV pair from the UI
-### Update, Delete, and Copy KV pairs from the UI
+Open the **KV Store** tab and click **New Key-Value**. Enter a key name, select a type (string, number, boolean, datetime, date, duration, or JSON), and enter the value. Optionally set a TTL — choose a standard duration from the dropdown or select **Custom duration** to enter an ISO 8601 duration string. Click **Save**.
-You can edit, delete, or copy any KV pair by clicking on the associated button on the right side of each KV pair. The copy option copies the [Pebble expression for the KV pair](#read-kv-pairs-with-pebble) (i.e., `{{ kv('YOUR_KEY') }}`) so you can use it directly in your flow.
+### Update, delete, and copy KV pairs
-
+Edit, delete, or copy any KV pair using the action buttons on the right. The copy option copies the [Pebble expression for the KV pair](#read-kv-pairs-with-pebble) (`{{ kv('YOUR_KEY') }}`) ready to paste into a flow.
-## CODE: How to Create, Read, Update and Delete KV pairs in your flow code
+## KV tasks in flows
-### Create a new KV pair with the `Set` task in a flow
+### Create a KV pair with the `Set` task
-To create a KV pair from a flow, you can use the `io.kestra.plugin.core.kv.Set` task. Below is an example of how to create a KV pair in a flow:
+Use `io.kestra.plugin.core.kv.Set` to create or update a KV pair from a flow:
```yaml
id: add_kv_pair
@@ -132,22 +100,20 @@ tasks:
values:
my_key: "{{ kv('my_key') }}"
simple_string: "{{ kv('simple_string') }}"
- favorite_song: "{{ json(kv('json_kv')).song }}"
+ favorite_song: "{{ fromJson(kv('json_kv')).song }}"
```
-You can use the `io.kestra.plugin.core.kv.Set` task to create or modify any KV pair. When modifying existing values, you can leverage the `overwrite` boolean parameter to control whether to overwrite the existing value or fail if a value for that key already exists. By default, the `overwrite` parameter is set to `true` so that the existing value is always updated.
+Set `overwrite: false` to fail instead of silently replacing an existing value. The default is `true`.
### Read KV pairs with Pebble
-The easiest way to retrieve a value by key is to use the `{{ kv('YOUR_KEY'') }}` Pebble function.
+Use `{{ kv('YOUR_KEY') }}` to retrieve a value inline. The full signature is:
-Below is the full syntax of that function:
-
-```twig
+```
{{ kv(key='your_key_name', namespace='your_namespace_name', errorOnMissing=false) }}
```
-Assuming that you retrieve the key in a flow in the same namespace as the one for which the key was created, you can simply use `"{{ kv('my_key') }}"` to retrieve the value:
+When reading from the same namespace as the flow, the namespace argument is optional:
```yaml
id: read_kv_pair
@@ -158,7 +124,7 @@ tasks:
message: "{{ kv('my_key') }}"
```
-When retrieving the key from another namespace, you can use the following syntax:
+To read from another namespace, pass it as the second argument:
```yaml
id: read_kv_pair_from_another_namespace
@@ -169,7 +135,7 @@ tasks:
message: "{{ kv('my_key', 'kestra.engineering.myproject') }}"
```
-By default, when you try to retrieve a key that doesn't exist, the task using the `"{{ kv('non_existing_key') }}"` expression will run with an error. If you prefer to instead run without error when the key doesn't exist, you can set the `errorOnMissing` parameter to `false` (that expression will simply return `null`):
+By default, referencing a missing key causes the task to fail. Set `errorOnMissing=false` to return `null` instead:
```yaml
id: read_non_existing_kv_pair
@@ -180,22 +146,9 @@ tasks:
format: "{{ kv('non_existing_key', errorOnMissing=false) }}"
```
-The function arguments such as the `errorOnMissing` keyword can be skipped for brevity as long as you fill in all positional arguments i.e., `{{ kv(key='your_key_name', namespace='your_namespace_name', errorOnMissing=false) }}` — the version below has the same effect:
-`{{ kv(key='my_key', namespace='company.team') }}`
-```yaml
-id: read_non_existing_kv_pair
-namespace: company.team
-tasks:
- - id: log_key_from_another_namespace
- type: io.kestra.plugin.core.debug.Return
- format: "{{ kv('my_key', 'kestra.engineering.myproject', false) }}"
-```
-
### Read KV pairs with the `Get` task
-You can also retrieve the value of any KV pair using the `Get` task. The `Get` task produces the `value` output, which you can use in subsequent tasks. This option is a little more verbose, but it has two benefits:
-1. More declarative syntax
-2. Useful when you need to pass the current state of that value to multiple downstream tasks
+The `Get` task produces a `value` output you can reference in downstream tasks — useful when you need to pass the same KV value to multiple steps:
```yaml
id: get_kv_pair
@@ -215,9 +168,9 @@ tasks:
### Read and parse JSON-type values from KV pairs
-To parse JSON values in Kestra's templated expressions, make sure to wrap the `kv()` call in the `json()` function like the following: `"{{ json(kv('your_json_key')).json_property }}"`.
+To parse JSON values in Kestra's templated expressions, wrap the `kv()` call in the `fromJson()` function: `"{{ fromJson(kv('your_json_key')).json_property }}"`.
-The following example demonstrates how to parse values from JSON-type KV pairs in a flow:
+This example sets a JSON KV pair and reads individual fields using `fromJson()`:
```yaml
id: kv_json_flow
namespace: company.team
@@ -239,10 +192,10 @@ tasks:
- id: parse_json_kv
type: io.kestra.plugin.core.log.Log
message:
- - "Author: {{ json(kv('favorite_song')).author }}"
- - "Song: {{ json(kv('favorite_song')).song }}"
- - "Album name: {{ json(kv('favorite_song')).album.name }}"
- - "Album release date: {{ json(kv('favorite_song')).album.release_date }}"
+ - "Author: {{ fromJson(kv('favorite_song')).author }}"
+ - "Song: {{ fromJson(kv('favorite_song')).song }}"
+ - "Album name: {{ fromJson(kv('favorite_song')).album.name }}"
+ - "Album release date: {{ fromJson(kv('favorite_song')).album.release_date }}"
- id: get
type: io.kestra.plugin.core.kv.Get
@@ -250,13 +203,13 @@ tasks:
- id: parse_json_from_kv
type: io.kestra.plugin.core.log.Log
- message: "Country: {{ json(outputs.get.value).album.name }}"
+ message: "Album name: {{ fromJson(outputs.get.value).album.name }}"
```
### Read keys by prefix with the `GetKeys` task
-If you want to check if some values already exist for a given key, you can search keys by prefix:
+Search for keys matching a prefix with `GetKeys`:
```yaml
id: get_keys_by_prefix
@@ -273,11 +226,11 @@ tasks:
message: "{{ outputs.get.keys }}"
```
-The output is a list of keys - if no keys were found, an empty list will be returned.
+The output is a list of matching keys, or an empty list if none are found.
### Delete a KV pair with the `Delete` task
-The `io.kestra.plugin.core.kv.Delete` task produces the boolean output `deleted` to confirm whether a given KV pair was deleted or not.
+`Delete` produces a boolean `deleted` output confirming whether the key existed and was removed:
```yaml
id: delete_kv_pair
@@ -297,53 +250,28 @@ tasks:
---
-## API: How to Create, Read, Update and Delete KV pairs via REST API
-
-Let's look at how you can interact with the KV Store via the REST API.
+## REST API
### Create a KV pair
-The API call to set the KV pair follows the structure:
-
-```bash
-curl -X PUT -H "Content-Type: application/json" http://localhost:8080/api/v1/main/namespaces/{namespace}/kv/{key} -d ''
-```
-
-For example:
+Use a `PUT` request to set a KV pair:
```bash
curl -X PUT -H "Content-Type: application/json" http://localhost:8080/api/v1/main/namespaces/company.team/kv/my_key -d '"Hello World"'
```
-The above `curl` command creates the KV pair with key `my_key` and the `Hello World` string value in the `company.team` namespace. The API does not return any response.
-
-### Read all keys in the namespace
-
-You can get all KV pairs using:
+The API returns no response body on success.
-```bash
-curl -X GET -H "Content-Type: application/json" http://localhost:8080/api/v1/main/kv/
-```
-
-You can also use the `filters` to get all KV pairs from a specific Namespace (replace `namespace-name`):
+### Read all keys in a namespace
```bash
curl -G "http://localhost:8080/api/v1/main/kv" \
- --data-urlencode "filters[namespace][EQUALS]= namespace-name" \
+ --data-urlencode "filters[namespace][EQUALS]=company.team" \
-H "Authorization: Bearer "
```
-Older versions of Kestra may use the path to specify a Namespace:
-
-```bash
-curl -X GET -H "Content-Type: application/json" http://localhost:8080/api/v1/main/namespaces/{namespace}/kv/{key}
-```
-
-:::alert{type="info"}
-As a general tip, your Kestra instance exposes an interactive API reference at https:///api which lists all available endpoints for your installed version.
-:::
+The response is a JSON array of keys with their creation and update timestamps:
-The output is returned as a JSON array of all keys in the namespace:
```json
[
{"key":"my_key","creationDate":"2024-07-27T06:10:33.422Z","updateDate":"2024-07-27T06:11:08.911Z"},
@@ -351,37 +279,23 @@ The output is returned as a JSON array of all keys in the namespace:
]
```
-### Delete a KV pair
-
-You can delete any KV pair using the following API call:
-
-```bash
-curl -X DELETE -H "Content-Type: application/json" http://localhost:8080/api/v1/main/namespaces/{namespace}/kv/{key}
-```
-
-This call returns a boolean indicating whether the key was deleted.
-
-For example, the following `curl` command returns `false` because the key `non_existing_key` does not exist:
-
-```bash
-curl -X DELETE -H "Content-Type: application/json" http://localhost:8080/api/v1/main/namespaces/company.team/kv/non_existing_key
-```
+:::alert{type="info"}
+Your Kestra instance exposes an interactive API reference at `https:///api` listing all available endpoints.
+:::
-However, when we try to delete a key `my_key` which exists in the `company.team` namespace, the same API call returns `true`:
+### Delete a KV pair
```bash
curl -X DELETE -H "Content-Type: application/json" http://localhost:8080/api/v1/main/namespaces/company.team/kv/my_key
```
----
-
-## TERRAFORM: How to Create, Read, Update and Delete KV pairs via Terraform
+Returns `true` if the key existed and was deleted, `false` if it did not exist.
-### Create a KV pair
+---
-You can create a KV pair via Terraform by using the `kestra_kv` resource.
+## Terraform
-Below is an example of how to create a KV pair:
+Use the `kestra_kv` resource to create or update a KV pair:
```hcl
resource "kestra_kv" "my_key" {
@@ -392,11 +306,7 @@ resource "kestra_kv" "my_key" {
}
```
-### Read a KV pair
-
-You can read a KV pair via Terraform by using the `kestra_kv` data source.
-
-Below is an example of how to read a KV pair:
+Use the `kestra_kv` data source to read a KV pair:
```hcl
data "kestra_kv" "new" {
@@ -405,4 +315,4 @@ data "kestra_kv" "new" {
}
```
-As with anything in Terraform, you can manage the state of your KV resources by adjusting the Terraform code and running the `terraform apply` command to create, update, or delete your KV pairs.
+Run `terraform apply` to create, update, or delete KV pairs from your Terraform state.
diff --git a/src/contents/docs/06.concepts/05.kv-store/navigate_to_keystore.png b/src/contents/docs/06.concepts/05.kv-store/navigate_to_keystore.png
deleted file mode 100644
index 8ff33d6e2eb..00000000000
Binary files a/src/contents/docs/06.concepts/05.kv-store/navigate_to_keystore.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/05.kv-store/navigate_to_namespace.png b/src/contents/docs/06.concepts/05.kv-store/navigate_to_namespace.png
deleted file mode 100644
index 97370540203..00000000000
Binary files a/src/contents/docs/06.concepts/05.kv-store/navigate_to_namespace.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/06.pebble/index.md b/src/contents/docs/06.concepts/06.pebble/index.md
index ca083d11485..f24960f1e6d 100644
--- a/src/contents/docs/06.concepts/06.pebble/index.md
+++ b/src/contents/docs/06.concepts/06.pebble/index.md
@@ -6,9 +6,7 @@ sidebarTitle: Pebble Templating Engine
icon: /src/contents/docs/icons/concepts.svg
---
-Dynamically render variables, inputs and outputs.
-
-Pebble is a Java templating engine inspired by [Twig](https://twig.symfony.com/) and similar to the [Python Jinja Template Engine](https://palletsprojects.com/p/jinja/) syntax. Kestra uses it to dynamically render variables, inputs, and outputs within the execution context.
+Pebble is a Java templating engine inspired by [Twig](https://twig.symfony.com/) and similar to [Jinja](https://palletsprojects.com/p/jinja/). Kestra uses it to dynamically render variables, inputs, and outputs within the execution context. For the full list of available variables, filters, and functions, see the [Expressions](../../expressions/index.md) reference.
@@ -16,7 +14,7 @@ Pebble is a Java templating engine inspired by [Twig](https://twig.symfony.com/)
## Reading inputs
-When using `inputs` property in a Flow, you can access the corresponding values by using `inputs` variable in your tasks.
+Access input values in tasks using the `inputs` variable:
```yaml
id: input_string
@@ -34,9 +32,9 @@ tasks:
## Reading task outputs
-Most of Kestra's tasks expose output values. You can access those outputs in other tasks by using `outputs..`. Every task output can be found in the corresponding task documentation.
+Most tasks expose output values accessible as `outputs..`. The available outputs for each task are listed in its plugin documentation.
-In the example below, we use the `value` outputs of the `io.kestra.plugin.core.debug.Return` task in the downstream task.
+This example reads the `value` output of the `Return` task in a downstream `Log` task:
```yaml
id: input_string
@@ -51,7 +49,6 @@ tasks:
type: io.kestra.plugin.core.debug.Return
format: "Hello 👋, my name is {{ inputs.name }}"
-
- id: can_you_repeat
type: io.kestra.plugin.core.log.Log
message: '{{ outputs.say_hello.value }}'
@@ -59,9 +56,7 @@ tasks:
## Dynamically render a task with `TemplatedTask`
-Since Kestra 0.16.0, you can use the `TemplatedTask` task to fully template all task properties using Pebble. This way, all task properties and their values can be dynamically rendered based on your custom inputs, variables, and outputs from other tasks.
-
-Below is an example of how to use the [TemplatedTask](/plugins/core/templating/io.kestra.plugin.core.templating.templatedtask) to create a Databricks job using dynamic properties:
+`TemplatedTask` lets you fully template all task properties using Pebble — including properties that are not natively dynamic. This example uses [TemplatedTask](/plugins/core/templating/io.kestra.plugin.core.templating.templatedtask) to create a Databricks job with inputs controlling the cluster, task key, and wait time:
```yaml
id: templated_databricks_job
@@ -103,24 +98,17 @@ tasks:
waitForCompletion: "{{ inputs.maxWaitTime }}"
```
-Note how in this example, the `waitForCompletion` property is templated using Pebble even though that property is not dynamic. The same is true for the `sparkPythonTaskSource` property. Without the `TemplatedTask` task, you would not be able to pass those values from inputs.
+`waitForCompletion` and `sparkPythonTaskSource` are not natively dynamic properties — `TemplatedTask` makes it possible to drive them from inputs.
---
## Date formatting
-Pebble can be very useful for making small transformations on the fly without the need to use Python or another dedicated programming language.
-
-For instance, we can use the `date` filter to format date values: `'{{ inputs.my_date | date("yyyyMMdd") }}'`
+Use the `date` filter to format date values inline: `'{{ inputs.my_date | date("yyyyMMdd") }}'`
## Coalesce operator to conditionally use trigger or execution date
-Most of the time, a flow will be triggered automatically. Either on schedule or based on external events. It’s common to use the date of the execution to process the corresponding data and make the flow dependent on time.
-
-With Pebble, you can use the `trigger.date` to get the date of the executed trigger.
-Still, sometimes you may want to manually execute a flow. In this case, the `trigger.date` variable won’t be suitable. In this scenario, you can use the `execution.startDate` variable that returns the execution start date.
-
-To support both use cases, use the coalesce operator `??`. The example below shows how to apply it in a flow.
+Scheduled flows can use `trigger.date` to get the trigger’s date, but that variable is not set on manual executions. Use the coalesce operator `??` to fall back to `execution.startDate` when the trigger date is unavailable:
```yaml
id: pebble_date_trigger
@@ -129,7 +117,7 @@ namespace: company.team
tasks:
- id: return_date
type: io.kestra.plugin.core.debug.Return
- format: '{{ trigger.date ?? execution.startDate | date("yyyy-MM-dd")}}'
+ format: '{{ trigger.date ?? execution.startDate | date("yyyy-MM-dd") }}'
triggers:
- id: schedule
@@ -137,11 +125,9 @@ triggers:
cron: "* * * * *"
```
-## Parsing objects & lists using jq
-
-Sometimes, outputs return nested objects or lists. To parse those elements, you may leverage `jq`. You can use jQuery to slice, filter, map, and transform structured data with the same ease that `sed`, `awk`, `grep`, and similar Linux commands let you manipulate strings.
+## Parsing objects and lists using jq
-Consider the following flow:
+Use the `jq` filter to slice, filter, and transform nested objects or lists returned by task outputs — similar to how `sed`, `awk`, and `grep` work on strings.
```yaml
id: object_example
@@ -166,12 +152,12 @@ The expression `{{ inputs.data.value | jq(".[1]") | first }}` returns `2`.
`{{ inputs | jq(".data.value[1]") | first }}` also works — jq can parse any object in the Kestra context.
-You can troubleshoot complex Pebble expressions using the **Debug Expression** button in the outputs tab of a Flow execution page in the UI. It's helpful to validate how complex objects will be parsed.
+Use the **Debug Expression** button in the **Outputs** tab of an execution to troubleshoot complex expressions and validate how objects will be parsed.
## Using conditions in Pebble
-In some tasks, such as the `If` or `Switch` tasks, you need to provide some conditions. You can use the Pebble syntax to use previous task outputs within those conditions:
+Tasks like `If` and `Switch` accept Pebble expressions as conditions, letting you branch on inputs or previous task outputs:
```yaml
id: test-object
@@ -186,7 +172,7 @@ tasks:
- id: if
type: io.kestra.plugin.core.flow.If
- condition: '{{ inputs.data.value | jq(".[2]") | first == 3}}'
+ condition: '{{ inputs.data.value | jq(".[2]") | first == 3 }}'
then:
- id: when_true
type: io.kestra.plugin.core.log.Log
diff --git a/src/contents/docs/06.concepts/07.blueprints/blueprints.png b/src/contents/docs/06.concepts/07.blueprints/blueprints.png
index ec915efd4f2..adaacbd86d6 100644
Binary files a/src/contents/docs/06.concepts/07.blueprints/blueprints.png and b/src/contents/docs/06.concepts/07.blueprints/blueprints.png differ
diff --git a/src/contents/docs/06.concepts/07.blueprints/blueprints2.png b/src/contents/docs/06.concepts/07.blueprints/blueprints2.png
deleted file mode 100644
index 37b80d36263..00000000000
Binary files a/src/contents/docs/06.concepts/07.blueprints/blueprints2.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/07.blueprints/blueprints3.png b/src/contents/docs/06.concepts/07.blueprints/blueprints3.png
index d5add0cdb36..39257bd7460 100644
Binary files a/src/contents/docs/06.concepts/07.blueprints/blueprints3.png and b/src/contents/docs/06.concepts/07.blueprints/blueprints3.png differ
diff --git a/src/contents/docs/06.concepts/07.blueprints/index.md b/src/contents/docs/06.concepts/07.blueprints/index.md
index fb90cbfe679..8b8ddb6d6b6 100644
--- a/src/contents/docs/06.concepts/07.blueprints/index.md
+++ b/src/contents/docs/06.concepts/07.blueprints/index.md
@@ -8,50 +8,33 @@ version: ">= 0.10.0"
docId: blueprints
---
-Ready-to-use examples designed to kickstart your workflow.
+Blueprints are a searchable catalog of validated, documented flow examples. Each blueprint combines code and documentation with tags for discoverability. Click **Use** to copy any blueprint into your editor and customize it from there.
-Blueprints are a curated, organized, and searchable catalog of ready-to-use examples designed to help you kickstart your workflow.
-
-Each Blueprint combines code and documentation and can be assigned several tags for organization and discoverability.
-
-All Blueprints are validated and documented. You can easily customize and integrate them into your new or existing flows with a single click on the "Use" button.
-
-To see more, check out the [Blueprints library](/blueprints).
+Browse the full catalog at the [Blueprints library](/blueprints).
:::alert{type="info"}
-The [Kestra MCP server](../../ai-tools/kestra-mcp-resources/index.md) exposes the blueprints library directly to AI coding agents like Claude Code and Cursor. Ask your agent to find a blueprint by use case and it will retrieve the full flow YAML for you.
+The [Kestra MCP server](../../ai-tools/02.kestra-mcp-resources/index.md) exposes the blueprints library directly to AI coding agents like Claude Code and Cursor. Ask your agent to find a blueprint by use case and it will retrieve the full flow YAML for you.
:::
-
+
## Community blueprints
-We refer to all Blueprints available in the open-source product as Community Blueprints, as they are guided by the community feedback and represent common usage patterns we see among open-source users and contributors.
-
-Community Blueprints are particularly helpful when you're getting started with a new use case, integration, or with Kestra in general because they reflect fairly standardized workflow patterns. All Blueprints are verified by the Kestra team, but everyone is welcome to contribute new Blueprints or suggest improvements to the existing ones using [the following GitHub issue template](https://github.com/kestra-io/kestra/issues/new?assignees=&labels=blueprint&projects=&template=blueprint.yml).
-
-### Where to find Blueprints
-
-Blueprints are accessible from two places in the UI:
-
-1. The left navigation sidebar
-
-
+Community blueprints are available in the open-source product and reflect common workflow patterns across the Kestra user base. All blueprints are verified by the Kestra team. To contribute a new blueprint or suggest improvements, use the [GitHub issue template](https://github.com/kestra-io/kestra/issues/new?assignees=&labels=blueprint&projects=&template=blueprint.yml).
-2. A dedicated tab in the flow code editor named **Blueprints**, showing your source code and Blueprints side by side.
+### Where to find blueprints
-
+Blueprints are available from the **Blueprints** item in the left sidebar. Each blueprint shows its full YAML and topology before you commit — click **Open in Edit Flow** to load it directly into the editor.
-### How to find the right Blueprint
+
-Once you are on the Blueprints page, you can:
+### How to find the right blueprint
-- **Search** Blueprints for a specific use case or integration, e.g., Snowflake, BigQuery, DuckDB, Slack, ETL, ELT, Pandas, GPU, Git, Python, Docker, Redis, MongoDB, dbt, Airbyte, Fivetran, etc.
-- **Filter** by one or multiple tags, e.g., filter for Docker to see various ways to run containers in your flow, or filter for Notifications to see several options for configuring alerts on success or failure.
+From the Blueprints page, **search** by use case or integration (Snowflake, DuckDB, Slack, dbt, Docker, etc.) or **filter** by tag to narrow results.
## Custom blueprints
@@ -59,4 +42,4 @@ Once you are on the Blueprints page, you can:
This feature requires the [Enterprise Edition](../../07.enterprise/index.mdx).
:::
-Apart from Community Blueprints, you can create custom Blueprints available only to your organization. You can use them to share, centralize, and document commonly used workflows in your team. Read more in the [Custom Blueprints](../../07.enterprise/02.governance/custom-blueprints/index.md) documentation.
+Custom Blueprints are private blueprints available only to your organization. Beyond static templates, you can create **Templated Blueprints** — form-driven blueprints that generate complete flows from user inputs using Pebble-style templating, without requiring users to edit YAML directly. Custom Blueprints can also be version-controlled with Git using the `PushBlueprints` and `SyncBlueprints` tasks. See the [Custom Blueprints](../../07.enterprise/02.governance/custom-blueprints/index.md) documentation for details.
diff --git a/src/contents/docs/06.concepts/08.backfill/backfill1.png b/src/contents/docs/06.concepts/08.backfill/backfill1.png
index bb458ed2f2e..49fc2129efa 100644
Binary files a/src/contents/docs/06.concepts/08.backfill/backfill1.png and b/src/contents/docs/06.concepts/08.backfill/backfill1.png differ
diff --git a/src/contents/docs/06.concepts/08.backfill/backfill2.png b/src/contents/docs/06.concepts/08.backfill/backfill2.png
index ccedb4e0861..fd789143e1c 100644
Binary files a/src/contents/docs/06.concepts/08.backfill/backfill2.png and b/src/contents/docs/06.concepts/08.backfill/backfill2.png differ
diff --git a/src/contents/docs/06.concepts/08.backfill/backfill3.png b/src/contents/docs/06.concepts/08.backfill/backfill3.png
deleted file mode 100644
index 6e871c6ddca..00000000000
Binary files a/src/contents/docs/06.concepts/08.backfill/backfill3.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/08.backfill/delete-backfills.png b/src/contents/docs/06.concepts/08.backfill/delete-backfills.png
index c63e0e664c3..f2f58671818 100644
Binary files a/src/contents/docs/06.concepts/08.backfill/delete-backfills.png and b/src/contents/docs/06.concepts/08.backfill/delete-backfills.png differ
diff --git a/src/contents/docs/06.concepts/08.backfill/index.md b/src/contents/docs/06.concepts/08.backfill/index.md
index 69607862260..a7f9d380b91 100644
--- a/src/contents/docs/06.concepts/08.backfill/index.md
+++ b/src/contents/docs/06.concepts/08.backfill/index.md
@@ -8,7 +8,7 @@ icon: /src/contents/docs/icons/concepts.svg
Backfills are replays of missed schedule intervals between a defined start and end date.
-Let's take the following flow as an example:
+Consider a flow that runs every 30 minutes:
```yaml
id: scheduled_flow
@@ -18,13 +18,13 @@ tasks:
- id: label
type: io.kestra.plugin.core.execution.Labels
labels: # label to track scheduled date
- scheduledDate: "{{trigger.date ?? execution.startDate}}"
+ scheduledDate: "{{ trigger.date ?? execution.startDate }}"
- id: external_system_export
type: io.kestra.plugin.scripts.shell.Commands
taskRunner:
type: io.kestra.plugin.core.runner.Process
commands:
- - echo "processing data for {{trigger.date ?? execution.startDate}}"
+ - echo "processing data for {{ trigger.date ?? execution.startDate }}"
- sleep $((RANDOM % 5 + 1))
triggers:
@@ -33,31 +33,25 @@ triggers:
cron: "*/30 * * * *"
```
-This flow runs every 30 minutes. However, imagine that your source system had an outage for 5 hours. The flow will miss 10 executions. To replay these missed executions, you can use the backfill feature.
-
-Ensure the backfill’s start and end dates encompass every missed schedule, so the trigger can replay each execution. Note that Backfill does not only replay missed executions in the time window. If there are successful executions, then these are also replayed. To target specific executions, rather than a time window, to avoid duplication use [Replay](../10.replay/index.md).
+If the source system had a 5-hour outage, this flow would miss 10 executions. A backfill replays all schedule intervals in the specified time window — including any that succeeded — so set the start and end dates precisely. To replay specific executions rather than a full time window, use [Replay](../10.replay/index.md) instead.
:::alert{type="info"}
**All missed schedules are automatically recovered by default** if the Kestra server is down. The missed schedules will be executed as soon as Kestra is back up because of the `recoverMissedSchedules: ALL` property default. If you have configured this differently in your global Kestra configuration or specifically on a trigger, a Backfill achieves the same behavior. Read more about `recoverMissedSchedules` in the [dedicated documentation](../../05.workflow-components/07.triggers/01.schedule-trigger/index.md#recover-missed-schedules).
:::
-To backfill the missed executions, go to the **Triggers** tab on the Flow's detail page and click on the **Backfill executions** button.
+To backfill the missed executions, use **Backfill executions** on the **Triggers** tab of the flow's detail page.
-
+
-You can then select the start and end date for the backfill. Additionally, you can set custom labels for the backfill executions to help you identify them in the future.
+Select the start and end date for the backfill and optionally add custom labels to the executions for tracking.
-You can pause and resume the backfill process at any time:
-
-
-
-And by clicking on the **Details** button, you can see more details about that backfill process:
+You can pause and resume the backfill at any time. Click **Details** to see progress and execution status:
-
+
:::alert{type="info"}
Backfill executions will not be processed if the associated trigger is disabled.
@@ -65,9 +59,9 @@ Backfill executions will not be processed if the associated trigger is disabled.
## Delete a backfill
-You can delete a Backfill from the **Administrations - Triggers** view. Select the triggers you'd like to delete Backfills for if you do not want to replay any missed executions.
+Delete a backfill from **Tenant → Triggers**. Select the trigger and remove the backfill to stop pending replays.
-
+
Deleting a backfill only cancels the scheduled catch-up executions. For example, if you defined a `* * * * *` schedule and backfilled the last five minutes, removing that backfill prevents those five replayed runs from being emitted. This is different from **Delete trigger**, which clears the trigger state itself — effectively recreating the trigger so it starts evaluating from the current time. Use **Delete backfill** to stop pending replays, and **Delete trigger** when you need to reset a stuck trigger or start it fresh.
@@ -75,8 +69,6 @@ Deleting a backfill only cancels the scheduled catch-up executions. For example,
### Using cURL
-You can invoke the backfill executions using the `cURL` call as follows:
-
```sh
curl -X PUT http://localhost:8080/api/v1/main/triggers \
-H "Authorization: Bearer $KESTRA_API_TOKEN" \
@@ -98,18 +90,14 @@ curl -X PUT http://localhost:8080/api/v1/main/triggers \
}'
```
-In the `backfill` attribute, you need to provide the start time for the backfill; the end time can be optionally provided. You can provide inputs to the flow with `inputs`, as well as assign labels to the backfill executions by providing key-value pairs in the `labels` section. In the example `reason:outage` is labelled to make it clear what caused the need to backfill.
-
-Other attributes in this PUT call are `flowId`, `namespace`, and `triggerId`, corresponding to the flow to backfill.
-
-Check out the [API Reference](../../api-reference/02.open-source/index.mdx) for further backfill operations via the API.
+`start` is required; `end` defaults to the current time if omitted. Use `inputs` to pass flow inputs and `labels` to tag the resulting executions for tracking. See the [API Reference](../../api-reference/02.open-source/index.mdx) for all available backfill operations.
### Using a service account
:::badge{version=">=0.15" editions="EE,Cloud"}
:::
-For Enterprise and Cloud users, the same process as above can be done with [Service Accounts](../../07.enterprise/03.auth/service-accounts/index.md), so no human user needs to be involved. In this case, you must specify the Tenant to use in the request header and definition: `X-KESTRA-TENANT` and `tenantId`. In the example, we use a Tenant named `production`.
+Use a [Service Account](../../07.enterprise/03.auth/service-accounts/index.md) token instead of a user token, and include the tenant in the request header and body:
```sh
curl -X PUT http://localhost:8080/api/v1/main/triggers \
@@ -134,16 +122,8 @@ curl -X PUT http://localhost:8080/api/v1/main/triggers \
}'
```
-To use a Service Account, go to **Administration -> IAM -> Service Accounts**. From the Service Accounts tab, create a Service Account, generate an API Token, copy the token, and give the Service Account the appropriate access to backfill a flow. Use this API token in your `cURL` instead of a user's token.
-
-The interactive demo below walks through the steps one-by-one.
-
-
-
### Using Python requests
-You can invoke the backfill executions using Python `requests` as follows:
-
```python
import requests
import json
@@ -177,4 +157,4 @@ print(response.status_code)
print(response.text)
```
-With this code, you will be invoking the backfill for `scheduled_flow` flow under `company.team` namespace based on `schedule` trigger ID within the flow. The number of backfills that will be executed will depend on the schedule present in the `schedule` trigger and the `start` and `end` times mentioned in the backfill. When the `end` time is null, as in this case, the `end` time would be considered as the present time.
+When `end` is `None`, the backfill runs up to the current time.
diff --git a/src/contents/docs/06.concepts/10.replay/ai-suggestion.png b/src/contents/docs/06.concepts/10.replay/ai-suggestion.png
deleted file mode 100644
index acf640237bb..00000000000
Binary files a/src/contents/docs/06.concepts/10.replay/ai-suggestion.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/10.replay/index.md b/src/contents/docs/06.concepts/10.replay/index.md
index a5d45a215cc..16ae2164f12 100644
--- a/src/contents/docs/06.concepts/10.replay/index.md
+++ b/src/contents/docs/06.concepts/10.replay/index.md
@@ -6,158 +6,52 @@ sidebarTitle: Replay
icon: /src/contents/docs/icons/dev.svg
---
-Replay allows you to re-run a workflow execution from any chosen task run.
+Replay re-runs a workflow execution from any chosen task — skipping tasks that already completed successfully. Use it to recover from failures without reprocessing upstream work, or to iterate on a specific task without re-running the full flow.
-By using Replay, you can re-run a workflow execution from any selected task run. To do that, simply go to the Gantt view of the chosen workflow execution (it doesn't need to be a Failed execution, it can be an execution in any state) and click on the task run you want to re-run. Additionally, you can re-run an execution or bulk executions from the **Executions** tab with the option to use the latest revision.
+To replay from a specific task, open the **Gantt** or **Logs** tab of any execution and use the three-dot menu on the task run. You can also replay a single execution or bulk-replay from the **Executions** page, with the option to use the latest flow revision.
-
+## Example: fixing a failed task and replaying
-Replays are extremely useful for iterative development and reprocessing data.
-
-Imagine the following scenario: you have a workflow that extracts a large compressed CSV dataset and you want to transform it into a Parquet file with a specific schema.
+Consider a flow that downloads a file and then validates its schema:
```yaml
-id: divvy_tripdata
+id: replay_demo
namespace: company.team
-variables:
- file_id: "{{ execution.startDate | dateAdd(-3, 'MONTHS') | date('yyyyMM') }}"
-
tasks:
- - id: get_zipfile
+ - id: download
type: io.kestra.plugin.core.http.Download
- uri: "https://divvy-tripdata.s3.amazonaws.com/{{ render(vars.file_id) }}-divvy-tripdata.zip"
-
- - id: unzip
- type: io.kestra.plugin.compress.ArchiveDecompress
- algorithm: ZIP
- from: "{{ outputs.get_zipfile.uri }}"
-
- - id: convert
- type: io.kestra.plugin.serdes.csv.CsvToIon
- from: "{{outputs.unzip.files[render(vars.file_id) ~ '-divvy-tripdata.csv']}}"
-
- - id: to_parquet
- type: io.kestra.plugin.serdes.avro.IonToAvro # render(vars.file_id)
- from: "{{ outputs.convert.uri }}"
- datetimeFormat: "yy-MM-dd' 'HH:mm:ss"
- schema: |
- {
- "type": "record",
- "name": "Ride",
- "namespace": "com.example.bikeshare",
- "fields": [
- {"name": "ride_id", "type": "string"},
- {"name": "rideable_type", "type": "string"},
- {"name": "started_at", "type": {"type": "long", "logicalType": "timestamp-millis"}},
- {"name": "ended_at", "type": {"type": "long", "logicalType": "timestamp-millis"}},
- {"name": "start_station_name", "type": "string"},
- {"name": "start_station_id", "type": "string"},
- {"name": "end_station_name", "type": "string"},
- {"name": "end_station_id", "type": "string"},
- {"name": "start_lat", "type": "double"},
- {"name": "start_lng", "type": "double"},
- {
- "name": "end_lat",
- "type": ["null", "double"],
- "default": null
- },
- {
- "name": "end_lng",
- "type": ["null", "double"],
- "default": null
- },
- {"name": "member_casual", "type": "string"}
- ]
- }
-```
-
-When you run the above workflow, you should see an error in the `to_parquet` task.
-
-From the logs, you are able to see that the error is due to a misconfigured date format in the `datetimeFormat` field — in fact, the date format should have a full year, not just a two-digit year: `"yyyy-MM-dd' 'HH:mm:ss"`.
-
-You ask [AI](../../ai-tools/ai-copilot/index.md) to fix the flow for you, or you correct the error yourself in the workflow code and save it.
-
-
+ uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
-
+ - id: validate
+ type: io.kestra.plugin.core.execution.Assert
+ errorMessage: "Schema validation failed — unexpected number of columns"
+ conditions:
+ - "{{ 8 >= 5 }}"
-:::collapse{title="Full corrected flow code"}
-```yaml
-id: divvy_tripdata
-namespace: company.team
-
-variables:
- file_id: "{{ execution.startDate | dateAdd(-3, 'MONTHS') | date('yyyyMM') }}"
-
-tasks:
- - id: get_zipfile
- type: io.kestra.plugin.core.http.Download
- uri: "https://divvy-tripdata.s3.amazonaws.com/{{ render(vars.file_id) }}-divvy-tripdata.zip"
-
- - id: unzip
- type: io.kestra.plugin.compress.ArchiveDecompress
- algorithm: ZIP
- from: "{{ outputs.get_zipfile.uri }}"
-
- - id: convert
- type: io.kestra.plugin.serdes.csv.CsvToIon
- from: "{{outputs.unzip.files[render(vars.file_id) ~ '-divvy-tripdata.csv']}}"
-
- - id: to_parquet
- type: io.kestra.plugin.serdes.parquet.IonToParquet
- from: "{{ outputs.convert.uri }}"
- datetimeFormat: "yyyy-MM-dd HH:mm:ss.SSS"
- schema: |
- {
- "type": "record",
- "name": "Ride",
- "namespace": "com.example.bikeshare",
- "fields": [
- {"name": "ride_id", "type": "string"},
- {"name": "rideable_type", "type": "string"},
- {"name": "started_at", "type": {"type": "long", "logicalType": "timestamp-millis"}},
- {"name": "ended_at", "type": {"type": "long", "logicalType": "timestamp-millis"}},
- {"name": "start_station_name", "type": "string"},
- {"name": "start_station_id", "type": "string"},
- {"name": "end_station_name", "type": "string"},
- {"name": "end_station_id", "type": "string"},
- {"name": "start_lat", "type": "double"},
- {"name": "start_lng", "type": "double"},
- {
- "name": "end_lat",
- "type": ["null", "double"],
- "default": null
- },
- {
- "name": "end_lng",
- "type": ["null", "double"],
- "default": null
- },
- {"name": "member_casual", "type": "string"}
- ]
- }
+ - id: notify
+ type: io.kestra.plugin.core.log.Log
+ message: Validation passed, data is ready for processing.
```
-:::
-Now, you can go to the previously failed Execution and click on the `to_parquet` task run to re-run it (either from the Gantt or from the Logs view).
+Run the flow once with `8 == 5` as the condition — `download` succeeds, then `validate` fails because the assertion is always false.
-
+Open the failed execution and go to the **Gantt** tab. Use the three-dot menu on `validate` to select **Fix with AI** or correct the condition yourself — change `8 == 5` to `8 >= 5` and save as a new revision. Then select **Replay** from the same menu.
-Now select the latest revision of the flow code that contains the fix.
+
-
+In the confirmation dialog, select **Latest flow revision** to use the revision containing your fix.
-This re-runs the task with the new (corrected!) revision of the flow code. You can inspect the logs and verify that the task now completes successfully. The attempt number increments to show that this is a new run of the task.
+
-
+The `download` task is skipped — Kestra reuses its output from the original execution. Only `validate` and `notify` run again. The **Attempt 2/2** label on `validate` confirms this is the replayed run.
-The **Overview** tab will additionally show the new attempt number and the new revision of the flow code that was used during Replay.
+
-
+The **Overview** tab shows the attempt number, the revision used, a `system.replay: true` label marking this as a replay, and a `system.correlationId` label linking back to the original execution.
-Replay lets you re-run a failed task with the corrected flow code without rerunning tasks that already completed successfully.
+
diff --git a/src/contents/docs/06.concepts/10.replay/latest-revision.png b/src/contents/docs/06.concepts/10.replay/latest-revision.png
index 54ef30f421d..14ae92e5cee 100644
Binary files a/src/contents/docs/06.concepts/10.replay/latest-revision.png and b/src/contents/docs/06.concepts/10.replay/latest-revision.png differ
diff --git a/src/contents/docs/06.concepts/10.replay/replay-ai-fix.png b/src/contents/docs/06.concepts/10.replay/replay-ai-fix.png
deleted file mode 100644
index bcdf8ba174c..00000000000
Binary files a/src/contents/docs/06.concepts/10.replay/replay-ai-fix.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/10.replay/replay-execution-overview.png b/src/contents/docs/06.concepts/10.replay/replay-execution-overview.png
index 25f422ab989..4cdca404c3b 100644
Binary files a/src/contents/docs/06.concepts/10.replay/replay-execution-overview.png and b/src/contents/docs/06.concepts/10.replay/replay-execution-overview.png differ
diff --git a/src/contents/docs/06.concepts/10.replay/replay-task.png b/src/contents/docs/06.concepts/10.replay/replay-task.png
index 530110127b9..b5059530fa4 100644
Binary files a/src/contents/docs/06.concepts/10.replay/replay-task.png and b/src/contents/docs/06.concepts/10.replay/replay-task.png differ
diff --git a/src/contents/docs/06.concepts/10.replay/replay6.png b/src/contents/docs/06.concepts/10.replay/replay6.png
deleted file mode 100644
index 61c9f0d9628..00000000000
Binary files a/src/contents/docs/06.concepts/10.replay/replay6.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/10.replay/task-count.png b/src/contents/docs/06.concepts/10.replay/task-count.png
index b873ec258da..8b4951e4dc8 100644
Binary files a/src/contents/docs/06.concepts/10.replay/task-count.png and b/src/contents/docs/06.concepts/10.replay/task-count.png differ
diff --git a/src/contents/docs/06.concepts/11.storage/index.md b/src/contents/docs/06.concepts/11.storage/index.md
index 46cbb4771f8..2e67ea31a25 100644
--- a/src/contents/docs/06.concepts/11.storage/index.md
+++ b/src/contents/docs/06.concepts/11.storage/index.md
@@ -6,11 +6,9 @@ sidebarTitle: Data storage and processing
icon: /src/contents/docs/icons/concepts.svg
---
-Manage data processed by tasks.
+Kestra stores task data in the execution context, in internal storage, or in the KV store — each suited to different data sizes and lifetimes.
-Kestra's primary purpose is to orchestrate data processing via tasks, so data is central to each flow's execution.
-
-Depending on the task, data can be stored inside the execution context or inside Kestra's internal storage. You can also manually store data inside Kestra's KV store by using [dedicated tasks](/plugins/core/kv/io.kestra.plugin.core.kv.set).
+Depending on the task, data can be stored inside the execution context or inside Kestra's internal storage. You can also store data in the KV store using [dedicated tasks](/plugins/core/kv/io.kestra.plugin.core.kv.set).
Some tasks give you the choice of where you want to store the data, usually using a `fetchType` property or the three `fetch`/`fetchOne`/`store` properties.
@@ -40,9 +38,9 @@ The three `fetch`/`fetchOne`/`store` properties do the same but using three diff
Data can be stored as variables inside the flow execution context. This can be convenient for sharing data between tasks.
-To do so, tasks store data as [output attributes](../../05.workflow-components/06.outputs/index.md) that are then available inside the flow via Pebble expressions like `{{outputs.taskName.attributeName}}`.
+To do so, tasks store data as [output attributes](../../05.workflow-components/06.outputs/index.md) that are then available inside the flow via Pebble expressions like `{{ outputs.taskName.attributeName }}`.
-Be careful, the size of the data is significant, this increases the size of the flow execution context, which can lead to slow execution and increase the size of the execution storage inside Kestra's repository.
+Large values stored in the execution context increase its size, which can slow execution and grow Kestra's repository storage.
:::alert{type="warning"}
Depending on the Kestra internal queue and repository implementation, there can be a hard limit on the size of the flow execution context as it is stored as a single row/message. Usually, this limit is around 1MB, so this is important to avoid storing large amounts of data inside the flow execution context.
@@ -85,8 +83,7 @@ Dedicated tasks allow managing the files stored inside the internal storage:
- [Split](/plugins/core/storage/io.kestra.plugin.core.storage.split): split a file into multiple files depending on the size of the file or the number of rows.
:::alert{type="warning"}
-This should be the main method for storing and carrying large data from task to task.
-As an example, if you know that a [HTTP Request](/plugins/core/http/io.kestra.plugin.core.http.request) returns a heavy payload, you should consider using [HTTP Download](/plugins/core/http/io.kestra.plugin.core.http.download) along with a [Serdes](/plugins/plugin-serdes) instead of carrying raw data in [Flow Execution Context](#storing-data-inside-the-flow-execution-context)
+Use internal storage for large data. If an [HTTP Request](/plugins/core/http/io.kestra.plugin.core.http.request) returns a heavy payload, use [HTTP Download](/plugins/core/http/io.kestra.plugin.core.http.download) with a [Serdes](/plugins/plugin-serdes) task instead of carrying the raw data in the [execution context](#storing-data-inside-the-flow-execution-context).
:::
### Storing data inside the KV store
@@ -190,7 +187,7 @@ tasks:
- id: convertToCsv
type: io.kestra.plugin.serdes.csv.IonToCsv
- from: "{{outputs.query.uri}}"
+ from: "{{ outputs.query.uri }}"
- id: convertBackToIon
type: io.kestra.plugin.serdes.csv.CsvToIon
@@ -199,9 +196,9 @@ tasks:
### Processing data using scripts
-Kestra can launch Python, R, Node.js, Shell, Powershell, and Go scripts. Depending on the `runner`, they can run directly in a local process on the host or inside Docker containers.
+Kestra can run Python, R, Node.js, Shell, PowerShell, and Go scripts in a local process or inside a Docker container, depending on the configured `runner`.
-Those script tasks are available in the [Scripts Plugin](https://github.com/kestra-io/plugin-scripts). Below is documentation for each of them:
+These script tasks are available in the [Scripts Plugin](https://github.com/kestra-io/plugin-scripts):
- The [Python](/plugins/plugin-script-python/io.kestra.plugin.scripts.python.script) task runs a Python script in a Docker container or in a local process.
- The [Node](/plugins/plugin-script-node/io.kestra.plugin.scripts.node.script) task runs a Node.js script in a Docker container or in a local process.
- The [R](/plugins/plugin-script-r/io.kestra.plugin.scripts.r.script) task runs an R script in a Docker container or in a local process.
@@ -226,16 +223,16 @@ tasks:
LIMIT 10
store: true
projectId: geller
- serviceAccount: "{{envs.gcp_creds}}"
+ serviceAccount: "{{ envs.gcp_creds }}"
- id: write-csv
type: io.kestra.plugin.serdes.csv.IonToCsv
- from: "{{outputs.query.uri}}"
+ from: "{{ outputs.query.uri }}"
- id: wdir
type: io.kestra.plugin.core.flow.WorkingDirectory
inputFiles:
- data.csv: "{{outputs['write-csv'].uri}}"
+ data.csv: "{{ outputs['write-csv'].uri }}"
tasks:
- id: pandas
type: io.kestra.plugin.scripts.python.Script
@@ -264,7 +261,7 @@ Kestra can process data **row by row** using file transform tasks. The transform
- The [GraalVM JavaScript FileTransform](/plugins/plugin-graalvm/js-graalvm/io.kestra.plugin.graalvm.js.filetransform) task allows transforming rows with JavaScript.
- The [Groovy Script](/plugins/plugin-script-groovy/io.kestra.plugin.scripts.groovy.script) task allows running scripts with Groovy.
-The following example queries the BigQuery public dataset for Wikipedia pages, convert it row by row with the Nashorn FileTransform, and write it in a CSV file.
+The following example queries the BigQuery public dataset for Wikipedia pages, transforms each row with the GraalVM Python FileTransform, and writes the result to a CSV file.
```yaml
id: wikipedia-top-ten-file-transform
@@ -282,7 +279,7 @@ tasks:
- id: file-transform
type: io.kestra.plugin.graalvm.python.FileTransform
- from: "{{outputs['query-top-ten'].uri}}"
+ from: "{{ outputs['query-top-ten'].uri }}"
script: |
logger.info('row: {}', row)
@@ -298,7 +295,7 @@ tasks:
- id: write-csv
type: io.kestra.plugin.serdes.csv.IonToCsv
- from: "{{outputs['file-transform'].uri}}"
+ from: "{{ outputs['file-transform'].uri }}"
```
:::alert{type="info"}
@@ -374,28 +371,27 @@ tasks:
```
:::
-:::collapse{title="Example with the ForEachItem task reading file's content as a string"}
-When using the `ForEachItem` task, you can use the `read()` function to read the content of a file as a string. This is especially useful when you want to pass the content of a file as a raw string as an input to a subflow.
-
+:::collapse{title="Example with the Loop task passing row data to a subflow"}
+When using the `Loop` task with a storage URI, each iteration receives one row from the file as `item.value`. You can pass that value directly as an input to a subflow.
Below is a simple subflow example that uses a string input:
```yaml
-id: subflow_raw_string_input
+id: subflow_row_input
namespace: company.team
inputs:
- - id: string_input
+ - id: row
type: STRING
- defaults: hey there
+ defaults: "{}"
tasks:
- - id: for_each_item
+ - id: log
type: io.kestra.plugin.core.debug.Return
- format: "{{ inputs.string_input }}"
+ format: "{{ inputs.row }}"
```
-Because the `ForEachItem` task splits the `items` file into batches of smaller files (one file per row by default), you can use the `read()` function to read the content of that file for a given batch as a string value and pass it as an input to that subflow shown above.
+The parent flow queries a dataset, stores the result as a URI, then loops over each row and passes it to the subflow:
```yaml
id: parent_flow
@@ -411,13 +407,17 @@ tasks:
FROM read_csv_auto('https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv', header=True);
store: true
- - id: each_raw
- type: io.kestra.plugin.core.flow.ForEachItem
- items: "{{ outputs.extract.outputs[0].uri }}"
- namespace: company.team
- flowId: subflow_raw_string_input
- inputs:
- string_input: "{{ read(taskrun.items) }}"
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: "{{ outputs.extract.outputs[0].uri }}"
+ tasks:
+ - id: process
+ type: io.kestra.plugin.core.flow.Subflow
+ namespace: company.team
+ flowId: subflow_row_input
+ wait: true
+ inputs:
+ row: "{{ item.value | toJson }}"
```
:::
@@ -447,13 +447,13 @@ You can use the Pebble function `{{ fromJson(myvar) }}` and a `{{ myvar | toJson
:::collapse{title="The fromJson() function"}
-The function is used to convert a string to a JSON object. For example, the following Pebble expression converts the string `{"foo": [666, 1, 2]}` to a JSON object and then returns the first value of the `foo` key, which is `42`:
+`fromJson()` converts a JSON string to an object. For example, this expression returns `42`:
```yaml
-{{ json('{"foo": [42, 43, 44]}').foo[0] }}
+{{ fromJson('{"foo": [42, 43, 44]}').foo[0] }}
```
-You can use the `read()` function to read the content of a file as a string and then apply the `json()` function to convert it to a JSON object. Afterwards, you can read the value of a specific key in that JSON object. For example, the following Pebble expression reads the content of a file named `my.json` and then returns the value of the `foo` key, which is `42`:
+Combine `read()` with `fromJson()` to load a file from internal storage and parse its contents:
```yaml
id: extract_json
@@ -469,26 +469,22 @@ tasks:
- id: read_as_json
type: io.kestra.plugin.core.log.Log
- message: "{{ json(read(outputs.extract.uri)) }}"
+ message: "{{ fromJson(read(outputs.extract.uri)) }}"
- id: parse_json_elements
type: io.kestra.plugin.core.log.Log
- message: "{{ json(read(outputs.extract.uri)) | jq('map(.detail | fromjson | .message)') | first }}"
+ message: "{{ fromJson(read(outputs.extract.uri)) | jq('map(.detail | fromjson | .message)') | first }}"
```
-
-The above flow downloads a JSON file via an HTTP Request, reads its content as a string, converts it to a JSON object, and then in another task, it parses the JSON object and returns the value of a nested key.
:::
-:::collapse{title="The json filter"}
-
-You can use the `json` filter to convert any variable to a JSON string. You can think of it as a reverse process to what the `json()` function does.
+:::collapse{title="The toJson filter"}
-The example below shows how you can convert a list of numbers to a JSON string `'[1, 2, 3]'` using the `| json` filter:
+The `| toJson` filter converts any variable to a JSON string — the reverse of `fromJson()`.
```yaml
-{{ [1, 2, 3] | json }}
+{{ [1, 2, 3] | toJson }}
```
:::alert{type="info"}
-You typically would never used the `| json` filter in combination with the `read()` function. Anytime you need to read a file's content and then convert it to a JSON object, use a combination of the `read()` function and the `json()` function instead.
+To read a file's content as a JSON object, use `read()` combined with `fromJson()`, not `| toJson`.
:::
diff --git a/src/contents/docs/06.concepts/12.caching/index.md b/src/contents/docs/06.concepts/12.caching/index.md
index 7f962e1f7b5..9e39a643ef0 100644
--- a/src/contents/docs/06.concepts/12.caching/index.md
+++ b/src/contents/docs/06.concepts/12.caching/index.md
@@ -6,23 +6,14 @@ sidebarTitle: Caching
icon: /src/contents/docs/icons/concepts.svg
---
-Manage file caching inside Kestra.
+Kestra supports two complementary caching strategies: file caching via `WorkingDirectory` and output caching via `taskCache`.
-Kestra provides file caching, which is especially useful when you work with sizable package dependencies that don't change often.
+- **File caching** stores files (dependencies, build artifacts) in internal storage and restores them at the start of the next run.
+- **Output caching** stores a task's status and outputs in the database and skips re-execution entirely when inputs have not changed. See [Task Cache](../../05.workflow-components/task-cache/index.md).
## Cache files in a `WorkingDirectory` task
-The file caching functionality on the `WorkingDirectory` task allows you to cache a subset of files to speed up your workflow execution. This is especially useful when you work with sizable package dependencies that don't change often.
-
-:::alert{type="info"}
-Kestra can only cache files installed or created as part of the script tasks if the script uses a `PROCESS` runner. If the script uses a `DOCKER` runner, the files will not be cached and the `WorkingDirectory` task will [throw an error](https://github.com/kestra-io/kestra/issues/2233): `Unable to execute WorkingDirectory post actions`.
-:::
-
-### Use cases for file caching
-
-The file caching is useful if you want to install some `pip` or `npm` packages before running your script. You can cache the `node_modules` or Python `venv` folder to avoid re-installing the dependencies on each run.
-
-To do that, add a `cache` to your `WorkingDirectory` task. The `cache` property accepts a list of glob `patterns` to match files to cache. The cache will be automatically invalidated after a specified time-to-live using the `ttl` property accepting a duration.
+Add a `cache` block to a `WorkingDirectory` task to persist files across executions.
```yaml
id: caching_files
@@ -37,13 +28,46 @@ tasks:
ttl: PT1H
```
-### How does it work under the hood
+The `cache` property accepts a list of glob `patterns` and a `ttl` duration after which the cached files are invalidated.
+
+### How caching works
+
+Kestra packages the matched files and stores them in internal storage at the end of each run. On the next run, those files are restored before any task executes.
+
+### Runner compatibility
+
+File caching works when the script runs in a `PROCESS` runner. If you use a `DOCKER` runner, Kestra cannot cache files that exist only inside the container — attempting to do so [throws an error](https://github.com/kestra-io/kestra/issues/2233): `Unable to execute WorkingDirectory post actions`.
+
+To cache pip packages with a Docker runner, install into a subdirectory of the working directory using `--cache-dir` and cache that directory instead:
+
+```yaml
+id: python_cached_pip
+namespace: company.team
+
+tasks:
+ - id: working_dir
+ type: io.kestra.plugin.core.flow.WorkingDirectory
+ cache:
+ patterns:
+ - cache/pip/**
+ ttl: PT24H
+ tasks:
+ - id: python_script
+ type: io.kestra.plugin.scripts.python.Script
+ taskRunner:
+ type: io.kestra.plugin.scripts.runner.docker.Docker
+ beforeCommands:
+ - pip install --cache-dir cache/pip pandas
+ script: |
+ import pandas as pd
+ print(pd.__version__)
+```
-Kestra packages the files that need to be cached and stores them in the internal storage. When the task is executed again, the cached files are retrieved, initializing the working directory with their contents.
+Because `cache/pip` lives inside the working directory (not inside the container), Kestra can read and restore it between runs.
### Node.js example
-Below is an example of a flow that installs the `colors` package before running a Node.js script. The `node_modules` folder is cached for one hour.
+This flow installs the `colors` package and caches `node_modules` for one hour. Use a `PROCESS` runner when caching `node_modules` directly.
```yaml
id: node_cached_dependencies
@@ -66,9 +90,9 @@ tasks:
console.log(colors.red("Hello"));
```
-### Python example
+### Python example (Process runner)
-Below is an example of a flow that installs the `pandas` package before running a Python script. The `deps` folder is cached for one day.
+This flow installs `pandas` into a `deps` folder and caches it for one day.
```yaml
id: python_cached_dependencies
@@ -97,13 +121,8 @@ tasks:
### How to invalidate the cache
-Below are the details how to invalidate the cache:
-- After the first run, the files are cached
-- The next time the task is executed:
- - If the `ttl` didn't pass, then the files are retrieved from cache.
- - If the `ttl` passed, then the cache is invalidated and no files will be retrieved from cache; because cache is no longer present, the `npm install` command from the `beforeCommands` property will take a bit longer to execute.
-- If you edit the task and change the `ttl` to:
- - a longer duration e.g., `PT5H` — the files will be cached for five hours using the new `ttl` duration
- - a shorter duration e.g., `PT5M` — the cache will be invalidated after five minutes using the new `ttl` duration.
+- After the first run, files are cached.
+- On subsequent runs, if the `ttl` has not elapsed, the cached files are restored. If it has elapsed, the cache is cleared and `beforeCommands` (e.g. `npm install`) runs in full.
+- Changing the `ttl` takes effect on the next run.
-The `ttl` is evaluated at runtime. If the most recently set `ttl` duration has passed as compared to the last task run execution date, the cache is invalidated and the files are no longer retrieved from cache.
+The `ttl` is evaluated at runtime against the last task execution date.
diff --git a/src/contents/docs/06.concepts/file-access/index.md b/src/contents/docs/06.concepts/file-access/index.md
index 2af4f759177..29f12cbfce7 100644
--- a/src/contents/docs/06.concepts/file-access/index.md
+++ b/src/contents/docs/06.concepts/file-access/index.md
@@ -7,11 +7,9 @@ icon: /src/contents/docs/icons/concepts.svg
version: ">= 0.24.0"
---
-Access local and namespace files in Kestra with universal file protocol.
+Kestra supports a universal file protocol for referencing local and [namespace files](../02.namespace-files/index.md) in your flows using consistent URI schemes.
-Kestra supports a universal file protocol that simplifies how to reference files in your flows. This protocol provides more consistent and flexible handling of local and [namespace files](../02.namespace-files/index.md) in your flows.
-
-You can still reference files inline by defining the file name and its content directly in YAML, but you can now also use `nsfile:///` and `file:///` URIs to reference files stored as namespace files or on the host machine. The example flow below shows a task demonstrating the various file access methods:
+You can reference files inline in YAML, or use `nsfile:///` and `file:///` URIs to point to namespace files or files on the host machine. The flow below demonstrates all three approaches:
```yaml
id: protocol
@@ -33,20 +31,17 @@ tasks:
hello.py: nsfile:///scripts/hello.py
- id: namespace_file_from_other_namespace
type: io.kestra.plugin.scripts.python.Commands
+ taskRunner:
+ type: io.kestra.plugin.core.runner.Process
+ commands:
+ - python hello.py
inputFiles:
hello.py: nsfile://company/scripts/hello.py
-pluginDefaults:
- - type: io.kestra.plugin.scripts.python.Commands
- values:
- taskRunner:
- type: io.kestra.plugin.core.runner.Process
- commands:
- - python hello.py
```
-### Allowed paths
+## Allowed paths
-Note that to use the `file:///` scheme, you will need to bind-mount the host directory containing the files into the Docker container running Kestra, as well as set the `kestra.local-files.allowed-paths` configuration property to allow access to that directory. For example, if you want to read files from the `scripts` folder on your host machine, you can add the following to your `kestra.yml` configuration:
+To use the `file:///` scheme, bind-mount the host directory into the Kestra container and set the `kestra.local-files.allowed-paths` configuration property. For example, to allow access to a `scripts` folder:
```yaml
kestra:
@@ -68,19 +63,22 @@ If you see the following error:
java.lang.SecurityException: The path /scripts/hello.py is not authorized. Only files inside the working directory are allowed by default, other paths must be allowed either globally inside the Kestra configuration using the `kestra.local-files.allowed-paths` property, or by plugin using the `allowed-paths` plugin configuration.`.
```
-It means that you have not configured the allowed paths correctly. Ensure that the host directory is bind-mounted into the container and that the `kestra.local-files.allowed-paths` configuration property includes the path to that directory.
+This means the allowed paths are not configured correctly. Confirm that the host directory is bind-mounted into the container and that `kestra.local-files.allowed-paths` includes that path.
+
+## Protocol reference
-### Protocol reference
+| Scheme | Purpose |
+|--------|---------|
+| `file:///path/to/file.txt` | Local file on the host machine from an explicitly allowed path |
+| `nsfile:///path/to/file.txt` | File in the current namespace (three slashes; no namespace inheritance) |
+| `nsfile://other.namespace/path/to/file.txt` | File in another namespace (two slashes + namespace name); requires that namespace to be allowed |
+| `kestra:///path/to/file.txt` | File in Kestra's internal storage |
-Here is a reference of the new file protocol:
-1. Use `file:///path/to/file.txt` to reference local files on the host machine from explicitly allowed paths.
-2. Use `nsfile:///path/to/file.txt` to reference files stored in the current namespace. Note that this protocol uses three slashes after `nsfile://` to indicate that you are referencing a file in the current namespace. The namespace inheritance doesn't apply here, i.e., if you specify `nsfile:///path/to/file.txt` in a flow from `company.team` namespace and Kestra can't find it there, Kestra won't look for that file in the parent namespace, i.e., the `company` namespace, unless you explicitly specify the parent namespace in the path, e.g., `nsfile://company/path/to/file.txt`.
-3. Use `nsfile://your.infinitely.nested.namespace/path/to/file.txt` to reference files stored in another namespace, provided that the current namespace has permission to access it. Note how this protocol uses two slashes after `nsfile://`, followed by the namespace name, to indicate that you are referencing a file in a different namespace. Under the hood, Kestra EE uses the Allowed Namespaces concept to check permissions to read that file.
-4. Kestra also uses the `kestra:///` scheme for internal storage files. If you need to reference files stored in the internal storage, you can use the `kestra:///path/to/file.txt` protocol.
+`nsfile:///` does not inherit from parent namespaces. If `nsfile:///scripts/hello.py` is not found in `company.team`, Kestra will not fall back to `company`. To reference a parent namespace explicitly, use `nsfile://company/scripts/hello.py`.
-### Usage with `read()` function
+## Usage with `read()`
-You can also use the `read()` function to read namespace files or local files in tasks that expect content rather than a path to a script or SQL query. For example, if you want to read a SQL query from a namespace file, you can use the `read()` function as follows:
+Use `read()` in tasks that expect file content rather than a path — for example, to load a SQL query from a namespace file:
```yaml
id: query
@@ -91,7 +89,7 @@ tasks:
type: io.kestra.plugin.jdbc.duckdb.Query
sql: "{{ read('nsfile:///query.sql') }}"
```
-For local files on the host, you can use the `file:///` scheme:
+For local files on the host:
```yaml
id: query
@@ -102,9 +100,9 @@ tasks:
sql: "{{ read('file:///query.sql') }}"
```
-### Namespace files as default FILE-type inputs
+## Namespace files as default FILE-type inputs
-One of the benefits of this protocol is that you can reference Namespace Files as default FILE-type inputs in your flows. See the example below, which reads a local file, `hello.txt`, from the `demo` namespace and logs its content.
+You can reference a namespace file as the default value for a `FILE`-type input. This flow reads `hello.txt` from the `demo` namespace and logs its content:
```yaml
id: file_input
diff --git a/src/contents/docs/06.concepts/system-flows/index.md b/src/contents/docs/06.concepts/system-flows/index.md
index 98894230bf2..79b2ff327e2 100644
--- a/src/contents/docs/06.concepts/system-flows/index.md
+++ b/src/contents/docs/06.concepts/system-flows/index.md
@@ -8,13 +8,13 @@ editions: ["OSS", "EE"]
version: ">= 0.19.0"
---
-Automate maintenance workflows with System Flows.
+System Flows are flows in the `system` namespace that automate recurring platform maintenance — running on a fixed cadence and kept separate from your user-facing executions.
-System Flows periodically execute background operations that keep your platform running but which you would generally prefer to keep out of sight. These flows automate maintenance workflows, such as:
+Common uses include:
1. Sending [alert notifications](/blueprints/failure-alert-slack)
2. Creating automated support tickets when critical workflows fail
@@ -22,7 +22,7 @@ System Flows periodically execute background operations that keep your platform
4. Syncing code from Git or pushing code to Git
5. Automatically [releasing flows](/blueprints/copy-flows-to-new-tenant) from development to QA and staging environments
-We refer to these as **System Flows** because by default they are only visible within the `system` namespace and to users with appropriate access. If you prefer, you can use a different namespace name instead of `system` by overwriting the following [Plugins and Execution configuration](../../configuration/04.plugins-and-execution/index.md):
+By default, System Flows are only visible within the `system` namespace and to users with appropriate access. To use a different namespace name, set the following in your [Plugins and Execution configuration](../../configuration/04.plugins-and-execution/index.md):
```yaml
kestra:
@@ -30,20 +30,20 @@ kestra:
namespace: system
```
-To access System Flows, navigate to the **Namespaces** section in the UI. The `system` namespace is pinned at the top for quick access.
+System Flows live in the `system` namespace, pinned at the top of **Namespaces**.
-
+
-From this section, you’ll find the **System Blueprints** tab, which provides fully customizable templates that you can modify to suit your organization’s needs.
+The **Blueprints** tab inside the system namespace provides ready-to-use templates you can customize for your organization.
-
+
:::alert{type="info"}
-Keep in mind that System Flows are not restricted to System Blueprints — any valid Kestra flow can become a System Flow if it's added to the `system` namespace.
+System Flows are not restricted to System Blueprints — any valid Kestra flow added to the `system` namespace becomes a System Flow.
:::
-System flow executions appear across the Dashboard, Flows, and Executions pages, each with a multi-select **Scope** filter (`User`, `System`) so you can view user-facing and system executions separately or together.
+System flow executions appear across the Dashboard, Flows, and Executions pages. Use the **Scope** filter to view user and system executions separately or together.
-
+
-In terms of permissions, `system` namespace is open by default, but using the namespace-level RBAC functionality in the Enterprise Edition, you can restrict access to the `system` namespace only to Admins, while assigning `company.*` namespaces to your general user base.
+The `system` namespace is open by default. Using namespace-level RBAC in the Enterprise Edition, you can restrict it to admins while assigning `company.*` namespaces to your general user base.
diff --git a/src/contents/docs/06.concepts/system-flows/system-blueprints.png b/src/contents/docs/06.concepts/system-flows/system-blueprints.png
index 4458f88c07e..2171d915e02 100644
Binary files a/src/contents/docs/06.concepts/system-flows/system-blueprints.png and b/src/contents/docs/06.concepts/system-flows/system-blueprints.png differ
diff --git a/src/contents/docs/06.concepts/system-flows/system-filters.png b/src/contents/docs/06.concepts/system-flows/system-filters.png
index d13e053af07..52eaa061471 100644
Binary files a/src/contents/docs/06.concepts/system-flows/system-filters.png and b/src/contents/docs/06.concepts/system-flows/system-filters.png differ
diff --git a/src/contents/docs/06.concepts/system-flows/system-namespace.png b/src/contents/docs/06.concepts/system-flows/system-namespace.png
index 9d993f45e5a..f85a4884912 100644
Binary files a/src/contents/docs/06.concepts/system-flows/system-namespace.png and b/src/contents/docs/06.concepts/system-flows/system-namespace.png differ
diff --git a/src/contents/docs/06.concepts/system-labels/correlationId-filter-result.png b/src/contents/docs/06.concepts/system-labels/correlationId-filter-result.png
deleted file mode 100644
index fe19a22148c..00000000000
Binary files a/src/contents/docs/06.concepts/system-labels/correlationId-filter-result.png and /dev/null differ
diff --git a/src/contents/docs/06.concepts/system-labels/correlationId.png b/src/contents/docs/06.concepts/system-labels/correlationId.png
index 4d8cceb16b2..066ce1df388 100644
Binary files a/src/contents/docs/06.concepts/system-labels/correlationId.png and b/src/contents/docs/06.concepts/system-labels/correlationId.png differ
diff --git a/src/contents/docs/06.concepts/system-labels/index.md b/src/contents/docs/06.concepts/system-labels/index.md
index ad533e022cb..3a5e2695070 100644
--- a/src/contents/docs/06.concepts/system-labels/index.md
+++ b/src/contents/docs/06.concepts/system-labels/index.md
@@ -8,15 +8,11 @@ editions: ["OSS", "EE"]
version: ">= 0.20.0"
---
-Special labels for system use only.
+System Labels and Hidden Labels are reserved metadata labels used to manage and monitor Kestra. They are hidden in the UI by default. To find executions tagged with a specific hidden label, use the **Labels** filter with the key and value explicitly — for example, `system.correlationId: o8014I6GBOxqcQg8guKRP`.
-System Labels and Hidden Labels are reserved for storing metadata used by administrators to manage and monitor Kestra. These labels are hidden in the UI by default. To view executions with a specific Hidden Label, you must explicitly filter for it using the `Labels` filter, such as `system.correlationId: 6WuLA1vh9lpFsGyrkuVRYb`.
+
-
-
-The table will then show the execution connected to that ID.
-
-
+After applying the filter, the executions table shows only the matching execution.
## Hidden labels
@@ -31,15 +27,15 @@ kestra:
- admin.
```
-By default, System Labels (prefixed with `system.`) are hidden. To display them, simply remove the `system.` prefix from the list of hidden prefixes.
+By default, System Labels (prefixed with `system.`) are hidden. To display them, remove `system.` from the list of hidden prefixes.
## System labels
-System Labels are labels prefixed with `system.` that serve specific purposes. Below are the available System Labels. For a step-by-step guide on using `system.correlationId` specifically as an idempotency key, see [Idempotency with correlation IDs](../../15.how-to-guides/idempotency/index.md).
+System Labels are labels prefixed with `system.` that serve specific purposes. The labels below are set automatically by Kestra. For a step-by-step guide on using `system.correlationId` as an idempotency key, see [Idempotency with correlation IDs](../../15.how-to-guides/idempotency/index.md).
### `system.correlationId`
-- Automatically set for every execution and propagated to downstream executions created by `Subflow` or `ForEachItem` tasks
+- Automatically set for every execution and propagated to downstream executions created by `Subflow` or `Loop` tasks
- Represents the ID of the first execution in a chain of executions, enabling tracking of execution lineage
- Can also be set to a stable business key and used as an idempotency key for flows that must not process the same event twice
- Use this label to filter all executions originating from a specific parent execution or business event.
@@ -50,8 +46,6 @@ For example, if a parent flow triggers multiple subflows, filtering by the paren
The Execution API supports setting this label at execution creation but not modification.
:::
----
-
### `system.username`
- Automatically set for every execution and contains the username of the user who triggered the execution
@@ -77,10 +71,26 @@ tasks:
message: Hello from a read-only flow!
```
-Once this label is set, the editor for this flow will be disabled in the UI.
-
-
+
:::alert{type="info"}
In the Enterprise Edition, updating a read-only flow server-side is restricted to service accounts or API keys.
:::
+
+### `system.from`
+
+- Automatically set on every execution to indicate how it was triggered
+- Common values include `ui` (triggered from the Kestra UI) and `mcp` (triggered by a Kestra MCP server)
+- Use this label to filter executions by their trigger source
+
+### `system.mcpServerId`
+
+- Automatically set on every execution created by a Kestra MCP server
+- Value is the `id` of the MCP server that invoked the tool
+- Use this label together with `system.from: mcp` to identify which server triggered a specific execution
+
+### `system.mcpSessionId`
+
+- Automatically set on every execution created by a Kestra MCP server
+- Value is the session ID of the MCP client connection that triggered the execution
+- Use this label to correlate multiple executions that originated from the same agent session
diff --git a/src/contents/docs/06.concepts/system-labels/readOnly.png b/src/contents/docs/06.concepts/system-labels/readOnly.png
index 8e5047946ab..737bcf6d4d7 100644
Binary files a/src/contents/docs/06.concepts/system-labels/readOnly.png and b/src/contents/docs/06.concepts/system-labels/readOnly.png differ
diff --git a/src/contents/docs/07.enterprise/01.overview/01.enterprise-edition/index.md b/src/contents/docs/07.enterprise/01.overview/01.enterprise-edition/index.md
index 4edcf88bf09..ce570d943d4 100644
--- a/src/contents/docs/07.enterprise/01.overview/01.enterprise-edition/index.md
+++ b/src/contents/docs/07.enterprise/01.overview/01.enterprise-edition/index.md
@@ -7,31 +7,25 @@ icon: /src/contents/docs/icons/kestra.svg
editions: ["EE", "Cloud"]
---
-Learn about the Enterprise Edition and how it can help you run Kestra securely and reliably at scale.
+Kestra Enterprise Edition is built for production workloads with high security and compliance requirements, deployable wherever you need.
-## Kestra Enterprise features – high-availability platform
+## Key features
-Designed for production workloads with high security and compliance requirements, deployed wherever you need.
+Kestra Enterprise is an additive overlay on the [Open Source Edition](https://github.com/kestra-io/kestra) — it shares one codebase and preserves all open-source behavior while adding enterprise capabilities on top.
-## Key Features
+**High availability**: Designed to be highly available and fault-tolerant. Supports **Kafka**, **Redis**, **AMQP**, or **GCP Pub/Sub** as the queue backend, and **Elasticsearch** for the search and read model — eliminating single points of failure and enabling horizontal scaling for large workloads.
-Kestra Enterprise is built on top of the [Open Source Edition](https://github.com/kestra-io/kestra) but features a different architecture. Below are the key differences between the two.
+**Multi-tenancy**: Separate environments for different teams or projects. Each tenant is fully isolated, with its own access control policies, and can optionally run with Worker Isolation and dedicated worker groups to prevent cross-tenant contention.
-⚡️**High Availability**: Kestra Enterprise is designed to be highly available and fault-tolerant. It uses a **Kafka** cluster as a backend for event-driven orchestration and **Elasticsearch** for storing logs and metrics. This not only improves performance but also eliminates single points of failure and enables the system to scale for large workloads.
+**Security and access control**: SSO and RBAC let you integrate with your existing identity provider and manage user access to workflows and resources. Enforce plugin allow-lists, apply read-only secrets for least privilege, and use audit logs for full traceability.
-⚡️**Multi-Tenancy**: The Enterprise Edition supports multi-tenancy, enabling separate environments for different teams or projects. Each tenant is fully isolated, can have its own access control policies, and can optionally run with Worker Isolation and dedicated worker groups to prevent cross-tenant contention.
+**Enterprise features**: Audit Logs, Custom Blueprints, namespace-level secrets, variables and plugin defaults, Assets packaging, declarative Unit Tests for flows, Versioned Plugins for safe upgrades, operational safeguards like the Kill Switch and in-product Announcements, and [Promote](../../../version-control-cicd/06.promote/index.md) for moving flows across environments from the UI with a review gate.
-⚡️**Security and Access Control**: Kestra Enterprise supports Single Sign-On (SSO) and Role-Based Access Control (RBAC), enabling you to integrate with your existing identity provider and manage user access to workflows and resources. Enforce plugin allow-lists, apply read-only secrets for least privilege, and lean on audit logs for full traceability.
+**Secrets management**: Securely stores and manages secrets. Supports read-only secrets for sensitive values and integrates with AWS Secrets Manager, Azure Key Vault, Elasticsearch, Google Secret Manager, HashiCorp Vault, Doppler, 1Password, and more.
-⚡️**Enterprise Features**: Audit Logs, Custom Blueprints, Namespace-level secrets/variables and plugin defaults, Assets packaging, declarative Unit Tests for flows, Versioned Plugins for safe upgrades, and operational safeguards like the Kill Switch and in-product Announcements.
+**Support**: Guaranteed SLAs and priority support, with onboarding and training to ensure a fast and confident start.
-⚡️**Secrets Management**: Kestra Enterprise securely stores and manages secrets. It supports read-only secrets for sensitive values and integrates with existing secret managers such as AWS Secrets Manager, Azure Key Vault, Elasticsearch, Google Secret Manager, HashiCorp Vault, Doppler, 1Password, and more to come.
-
-⚡️**Support**: The Enterprise Edition comes with guaranteed SLAs and priority support.
-
-⚡️**Onboarding**: We provide onboarding and training for your team to ensure a fast and confident start.
-
-If you're interested to learn more, [get in touch!](/demo)
+[Get in touch](/demo) to learn more.
:::alert{type="info"}
**Kestra Cloud:** If you’re unable to host Kestra Enterprise yourself, you can try Kestra Cloud — a fully managed SaaS solution hosted by the Kestra team. Kestra Cloud is currently in early access. If you are interested in trying it out, [sign up here](/cloud).
diff --git a/src/contents/docs/07.enterprise/01.overview/02.setup/index.md b/src/contents/docs/07.enterprise/01.overview/02.setup/index.md
index 0c24270744f..d6cfa88594b 100644
--- a/src/contents/docs/07.enterprise/01.overview/02.setup/index.md
+++ b/src/contents/docs/07.enterprise/01.overview/02.setup/index.md
@@ -7,17 +7,11 @@ icon: /src/contents/docs/icons/admin.svg
editions: ["EE", "Cloud"]
---
-How to set up Kestra Enterprise Edition.
-
-## Set up Kestra Enterprise – license and first tenant
-
-These setup instructions guide you through the initial configuration of your instance.
-
-When you launch Kestra Enterprise Edition for the first time, Kestra will prompt you to configure your instance. This includes setting up your first tenant, creating your first user, and starting the Kestra UI.
+When you launch Kestra Enterprise Edition for the first time, Kestra walks you through configuring your instance: setting up your first tenant, creating your first user, and starting the Kestra UI.
## Prerequisites
-To use Kestra Enterprise Edition, you will need a valid license configured under the `kestra.ee.license` configuration. The license is unique to your organization. If you need a license, please reach out to our Sales team at [sales@kestra.io](mailto:sales@kestra.io).
+Kestra Enterprise Edition requires a valid license under `kestra.ee.license`. Contact [sales@kestra.io](mailto:sales@kestra.io) to obtain one.
The license is set up using three configuration properties: `id`, `fingerprint`, and `key`.
@@ -53,9 +47,7 @@ This step asks you to confirm whether your configuration is valid. If not, you c
## Step 2: Create your first tenant
-If `multitenancy` is enabled, Kestra, will prompt you to create your first tenant.
-
-If you choose to create a tenant, you will be asked to input the Tenant ID and Tenant Name, for example:
+If `multitenancy` is enabled, Kestra will prompt you to create your first tenant. Enter a Tenant ID and Tenant Name, for example:
- tenant id: `stage`
- tenant name: `Staging Environment`
@@ -66,9 +58,9 @@ If you enabled a default tenant, you can skip this step.
## Step 3: Create your first user
-Now that you have your instance configured, you will create your first user. This user will have a [Superadmin](../../03.auth/rbac/index.md#super-admin) role for the instance and will be able to manage tenants, users, and roles.
+Create the first user. This user receives the [Instance Owner](../../03.auth/rbac/index.md#instance-owner) privilege and can manage tenants, users, and roles across the instance.
-
+
## Step 4: Start Kestra UI
diff --git a/src/contents/docs/07.enterprise/01.overview/03.migrate-from-oss/index.md b/src/contents/docs/07.enterprise/01.overview/03.migrate-from-oss/index.md
index 9bddce879cf..ad39af6a014 100644
--- a/src/contents/docs/07.enterprise/01.overview/03.migrate-from-oss/index.md
+++ b/src/contents/docs/07.enterprise/01.overview/03.migrate-from-oss/index.md
@@ -7,11 +7,7 @@ icon: /src/contents/docs/icons/admin.svg
editions: ["EE"]
---
-How to migrate your flows and data from Kestra Open Source to Enterprise Edition.
-
-## Migrate from Open Source to Enterprise Edition
-
-When you start **Kestra Enterprise Edition**, you can bring your existing flows from the open-source version. This guide covers how to export and import flows, and what to keep in mind for other resources.
+When you move to Kestra Enterprise Edition, you can bring your existing flows from the open-source version. This guide covers how to export and import flows, and what to migrate separately.
## Export and import flows
diff --git a/src/contents/docs/07.enterprise/01.overview/index.mdx b/src/contents/docs/07.enterprise/01.overview/index.mdx
index a57feffe8bd..466109bbca9 100644
--- a/src/contents/docs/07.enterprise/01.overview/index.mdx
+++ b/src/contents/docs/07.enterprise/01.overview/index.mdx
@@ -9,12 +9,8 @@ editions: ["EE", "Cloud"]
import ChildCard from "~/components/docs/ChildCard.astro"
-Overview of the Enterprise Edition with an introduction to our enterprise-level features and initial setup guide.
+Kestra Enterprise Edition builds on the open-source version with granular access control, data isolation, high-availability architecture, and enterprise support.
-## Kestra Enterprise overview – architecture and setup
-
-Kestra Enterprise Edition builds on the open-source version by offering more granular access control, enhanced data isolation, improved performance and high-availability architecture, and enterprise-level support from our team.
-
-To learn more, explore the sections below or follow the setup guide to get started.
+Explore the sections below or follow the setup guide to get started.
\ No newline at end of file
diff --git a/src/contents/docs/07.enterprise/01.overview/standalone-server-installation/index.md b/src/contents/docs/07.enterprise/01.overview/standalone-server-installation/index.md
index 885da646e73..2dd644b212b 100644
--- a/src/contents/docs/07.enterprise/01.overview/standalone-server-installation/index.md
+++ b/src/contents/docs/07.enterprise/01.overview/standalone-server-installation/index.md
@@ -7,15 +7,7 @@ icon: /src/contents/docs/icons/installation.svg
editions: ["EE"]
---
-Install Kestra on a standalone server with a simple executable file.
-
-## Run Kestra Enterprise from a standalone JAR
-
-To deploy Kestra without Docker, there's a standalone JAR available that allows deployment in any environment that has JVM version 21+.
-
-## Instructions
-
-The following is a quick start guide to get your Kestra Enterprise Edition up and running in standalone mode.
+To deploy Kestra without Docker, use the standalone JAR — a single executable that runs in any environment with JVM 21+.
## Standalone JAR
@@ -34,7 +26,7 @@ Make sure to store your credentials in an `application.yaml` file.
This provides a single JAR file that can be used to start Kestra. Store the file in your execution environment as `kestra` (make it executable).
-To make the file executable, Linux or MacOS users use the following with filename:
+Make the file executable (Linux or macOS):
```bash
chmod +x kestra-ee-VERSION # Replace VERSION with your version
@@ -47,7 +39,7 @@ mv kestra-ee-VERSION /usr/local/bin/kestra # Replace with your version and execu
chmod +x /usr/local/bin/kestra
```
-The file is then executable with:
+Then run:
```bash
./kestra-ee-VERSION server standalone # Replace VERSION with your version
@@ -82,7 +74,7 @@ kestra plugins install --all
This installs task plugins in the `plugins` directory. To install them elsewhere, specify a path with the `-p` argument.
-Additional Enterprise Edition plugins that are not task related may also be required -- such as secrets or storage plugins.
+Additional Enterprise Edition plugins that are not task-related may also be required — such as secrets or storage plugins.
## Secret plugins
@@ -115,7 +107,7 @@ To enable Kestra Enterprise features, configure the following parameters:
| Multi-tenancy | Yes | [Enterprise and Advanced Features](../../../configuration/06.enterprise-and-advanced/index.md) | Enables/disables multi-tenancy (required for SCIM) |
| Secret Manager | Yes | [Security and Secrets](../../../configuration/05.security-and-secrets/index.md) | Configure a secret manager in RW or RO mode |
| Encryption Key | Yes | [Security and Secrets](../../../configuration/05.security-and-secrets/index.md) | Key to encrypt inputs/outputs in flows |
-| Security | No | [Security and Secrets](../../../configuration/05.security-and-secrets/index.md) | Configure Super Admin (also settable in UI on startup) |
+| Security | No | [Security and Secrets](../../../configuration/05.security-and-secrets/index.md) | Configure Instance Owner (also settable in UI on startup) |
| User Invitations | No | [Runtime and Storage](../../../configuration/02.runtime-and-storage/index.md), [Observability and Networking](../../../configuration/03.observability-and-networking/index.md) | Required for email invitations (not needed with LDAP/SCIM) |
| SSO | No | [SSO](../../03.auth/sso/index.md) | Configure OIDC provider |
| LDAP | No | [LDAP](../../03.auth/sso/ldap/index.md) | Connect to an existing LDAP provider |
@@ -125,7 +117,7 @@ To enable Kestra Enterprise features, configure the following parameters:
Kestra can be started in **standalone mode** or in a **distributed setup** for production.
-Make sure to have a database configured and your Enterprise credentials stored in the `application.yaml` file.
+Ensure a database is configured and your Enterprise credentials are stored in `application.yaml`.
## Standalone server
@@ -137,7 +129,7 @@ This starts Kestra as a standalone service on port `8080`.
## Distributed mode
-For production usage, Kestra should run in distributed mode for scalability and high availability.
+For production, run Kestra in distributed mode for scalability and high availability.
Each component can run independently across servers, with shared access to the same database (no TCP communication is required between components).
diff --git a/src/contents/docs/07.enterprise/02.governance/01.assets/index.md b/src/contents/docs/07.enterprise/02.governance/01.assets/index.md
index 199a70259b0..6946f4d7829 100644
--- a/src/contents/docs/07.enterprise/02.governance/01.assets/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/01.assets/index.md
@@ -24,7 +24,7 @@ For an end-to-end architecture walkthrough with diagrams, see [Assets for infras
Assets are captured automatically when tasks declare `assets.inputs` or `assets.outputs`; you can also add them manually from the **Assets** tab. Once created, you can view asset details, check which workflow runs created or modified them, and see how assets connect to each other across your workflows.
-Assets enables:
+Assets enable:
- Shipping metadata to lineage providers (e.g., OpenLineage).
- Populating dropdowns or Pebble inputs with live assets (e.g., available VMs).
@@ -57,7 +57,7 @@ Asset types fall into two categories:
- **Kestra-defined asset types**: These predefined types use the `io.kestra.core.models.assets` model and provide structured metadata fields specific to each asset type. Plugins that support auto-generation populate these fields automatically during task execution — for example, a JDBC plugin creates a `Table` asset with `system`, `database`, and `schema` filled in from the connection details.
-The current Kestra-defined asset types are the following:
+Kestra provides these built-in asset types:
- `io.kestra.plugin.ee.assets.Dataset`
- Represents a dataset asset managed by Kestra.
@@ -373,6 +373,67 @@ tasks:
:::
+## Locking assets
+
+A lock prevents concurrent writes to a shared asset while a flow operates on it. Locks are TTL-bounded: they expire automatically when their duration elapses and can also be released explicitly. Reads are always open — only writes (edit, delete) are blocked while a lock is held.
+
+Two owner types exist:
+
+| Owner type | Acquired by | Behavior while held |
+|---|---|---|
+| `EXECUTION` | `Acquire` task | Blocks other executions' writes. The lock-holding execution can still write to the asset, and each write extends the lease. |
+| `USER` | UI or REST API | Blocks all execution writes. Use for manual maintenance windows. |
+
+### Locking from a flow
+
+The `Acquire` and `Release` tasks wrap the work that needs exclusive write access. Both require the `LOCK` permission on the `ASSET` resource (`UNLOCK` for `Release`).
+
+If another execution already holds the lock when `Acquire` runs, the task fails with a 423 error. Add a `Retry` to the `Acquire` task to wait for the lock to become available.
+
+```yaml
+id: update_customer_asset
+namespace: company.team
+
+tasks:
+ - id: acquire
+ type: io.kestra.plugin.kestra.ee.locks.Acquire
+ assetId: customers_by_country
+ ttl: PT1H
+
+ - id: write
+ type: io.kestra.plugin.core.log.Log
+ message: Writing to the locked asset
+
+ - id: release
+ type: io.kestra.plugin.kestra.ee.locks.Release
+ assetId: customers_by_country
+```
+
+**`Acquire`** properties:
+
+| Property | Required | Description |
+|---|---|---|
+| `assetId` | Yes | ID of the asset to lock. |
+| `ttl` | No | How long to hold the lock before it expires automatically. ISO-8601 duration (e.g. `PT1H`). Defaults to 5 minutes when unset. |
+
+**`Acquire`** outputs:
+
+| Output | Description |
+|---|---|
+| `lockedUntil` | When the lock expires. |
+| `ownerType` | Always `EXECUTION` for a task-acquired lock. |
+| `executionId` | ID of the execution holding the lock. |
+
+`Release` is owner-checked: it removes the lock only if the current execution holds it. If the lock has already expired or belongs to a different owner, `Release` is a no-op — safe to call unconditionally.
+
+### Locking from the UI
+
+From any asset's detail page, users with the `LOCK` permission can lock the asset manually. Choose from preset durations (5 minutes to 24 hours) or enter a custom ISO-8601 duration. The page shows who holds the lock and when it expires. Users with the `UNLOCK` permission can release any lock regardless of owner.
+
+When an asset is locked, the detail page shows a banner: *You might be seeing outdated metadata as this asset is currently locked for writing.*
+
+The asset list supports filtering by lock status.
+
## Data pipeline use cases
:::collapse{title="Advanced: data pipeline examples"}
@@ -430,6 +491,8 @@ namespace: kestra.company.data
tasks:
- id: create_staging_layer_asset
type: io.kestra.plugin.jdbc.duckdb.Query
+ url: "jdbc:duckdb:md:my_db?motherduck_token={{ secret('MOTHERDUCK_TOKEN') }}"
+ fetchType: STORE
sql: |
CREATE TABLE IF NOT EXISTS trips AS
select VendorID, passenger_count, trip_distance from sample_data.nyc.taxi limit 10;
@@ -444,28 +507,25 @@ tasks:
model_layer: staging
- id: for_each
- type: io.kestra.plugin.core.flow.ForEach
+ type: io.kestra.plugin.core.flow.Loop
values:
- passenger_count
- trip_distance
tasks:
- id: create_mart_layer_asset
type: io.kestra.plugin.jdbc.duckdb.Query
- sql: SELECT AVG({{taskrun.value}}) AS avg_{{taskrun.value}} FROM trips;
+ url: "jdbc:duckdb:md:my_db?motherduck_token={{ secret('MOTHERDUCK_TOKEN') }}"
+ fetchType: STORE
+ sql: SELECT AVG({{item.value}}) AS avg_{{item.value}} FROM trips;
assets:
inputs:
- id: trips
outputs:
- - id: avg_{{taskrun.value}}
+ - id: avg_{{item.value}}
type: io.kestra.plugin.ee.assets.Table
namespace: "{{flow.namespace}}"
metadata:
model_layer: mart
-pluginDefaults:
- - type: io.kestra.plugin.jdbc.duckdb
- values:
- url: "jdbc:duckdb:md:my_db?motherduck_token={{ secret('MOTHERDUCK_TOKEN') }}"
- fetchType: STORE
```
**What's happening in this pipeline**:
@@ -474,7 +534,7 @@ pluginDefaults:
2. **Staging Layer**: The `trips` table is created and registered with `model_layer: staging` metadata. This becomes an intermediate asset that mart layers will consume.
-3. **Dynamic Mart Creation**: The `ForEach` task generates two mart tables:
+3. **Dynamic Mart Creation**: The `Loop` task generates two mart tables:
- `avg_passenger_count`
- `avg_trip_distance`
@@ -488,9 +548,6 @@ pluginDefaults:
- **Dependency Tracking**: Know exactly which tables depend on others before making schema changes
- **Audit Trail**: Track which workflows created each table and when
-See the flow in action in this interactive demo:
-
-
:::
@@ -520,28 +577,24 @@ inputs:
tasks:
- id: for_each
- type: io.kestra.plugin.core.flow.ForEach
+ type: io.kestra.plugin.core.flow.Loop
values: "{{ inputs.teams }}"
tasks:
- id: create_bucket
type: io.kestra.plugin.aws.cli.AwsCLI
+ accessKeyId: "{{ secret('AWS_ACCESS_KEY') }}"
+ secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
+ region: "{{ secret('AWS_REGION') }}"
+ allowFailure: true
commands:
- - aws s3 mb s3://kestra-{{ taskrun.value | slugify }}-bucket
+ - aws s3 mb s3://kestra-{{ item.value | slugify }}-bucket
assets:
outputs:
- - id: kestra-{{ taskrun.value | slugify }}-bucket
+ - id: kestra-{{ item.value | slugify }}-bucket
type: AWS_BUCKET
metadata:
provider: s3
- address: s3://kestra-{{ taskrun.value | slugify }}-bucket
-
-pluginDefaults:
- - type: io.kestra.plugin.aws
- values:
- accessKeyId: "{{ secret('AWS_ACCESS_KEY') }}"
- secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
- region: "{{ secret('AWS_REGION') }}"
- allowFailure: true
+ address: s3://kestra-{{ item.value | slugify }}-bucket
```
This flow dynamically creates buckets (e.g., `kestra-data-bucket`, `kestra-finance-bucket`) and registers each as an `AWS_BUCKET` asset with relevant metadata.
@@ -559,6 +612,9 @@ tasks:
- id: aws_upload
type: io.kestra.plugin.aws.s3.Upload
+ accessKeyId: "{{ secret('AWS_ACCESS_KEY') }}"
+ secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
+ region: "{{ secret('AWS_REGION') }}"
bucket: kestra-data-bucket
from: '{{ outputs.download.uri }}'
key: raw_customer.csv
@@ -570,13 +626,6 @@ tasks:
type: io.kestra.plugin.ee.assets.File
metadata:
owner: data
-
-pluginDefaults:
- - type: io.kestra.plugin.aws
- values:
- accessKeyId: "{{ secret('AWS_ACCESS_KEY') }}"
- secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
- region: "{{ secret('AWS_REGION') }}"
```
In this workflow:
@@ -656,12 +705,12 @@ inputs:
tasks:
- id: for_each
- type: io.kestra.plugin.core.flow.ForEach
+ type: io.kestra.plugin.core.flow.Loop
values: "{{inputs.assets}}"
tasks:
- id: log
type: io.kestra.plugin.core.log.Log
- message: "{{taskrun.value}}"
+ message: "{{item.value}}"
```
**Filter assets by namespace:**
@@ -780,3 +829,9 @@ tasks:
- io.kestra.plugin.ee.assets.VM
endDate: "{{ now() | dateAdd(-180, 'DAYS') }}"
```
+
+## Visualizing assets in dashboards
+
+Use the `io.kestra.plugin.ee.dashboard.data.Assets` data source to build charts over your asset inventory directly in a custom dashboard. Asset charts are not filtered by the dashboard time range — they always reflect the current state of your inventory.
+
+See [Assets (EE and Cloud only)](../../../09.ui/00.dashboard/index.md#assets-ee-and-cloud-only) in the Dashboards documentation for available fields, chart type compatibility, and configuration examples.
diff --git a/src/contents/docs/07.enterprise/02.governance/06.audit-logs/audit-logs-filter.png b/src/contents/docs/07.enterprise/02.governance/06.audit-logs/audit-logs-filter.png
index ed90c084d04..8d8577ff1ae 100644
Binary files a/src/contents/docs/07.enterprise/02.governance/06.audit-logs/audit-logs-filter.png and b/src/contents/docs/07.enterprise/02.governance/06.audit-logs/audit-logs-filter.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/06.audit-logs/audit_logs.png b/src/contents/docs/07.enterprise/02.governance/06.audit-logs/audit_logs.png
index 8eadd083fae..27bdf4d7c27 100644
Binary files a/src/contents/docs/07.enterprise/02.governance/06.audit-logs/audit_logs.png and b/src/contents/docs/07.enterprise/02.governance/06.audit-logs/audit_logs.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/06.audit-logs/changes_diff.png b/src/contents/docs/07.enterprise/02.governance/06.audit-logs/changes_diff.png
index a179df4fde5..b3c3729a59b 100644
Binary files a/src/contents/docs/07.enterprise/02.governance/06.audit-logs/changes_diff.png and b/src/contents/docs/07.enterprise/02.governance/06.audit-logs/changes_diff.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/06.audit-logs/index.md b/src/contents/docs/07.enterprise/02.governance/06.audit-logs/index.md
index afb5642efae..af2e781af11 100644
--- a/src/contents/docs/07.enterprise/02.governance/06.audit-logs/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/06.audit-logs/index.md
@@ -8,25 +8,25 @@ editions: ["EE", "Cloud"]
docId: auditlogs
---
-How to use Audit Logs to govern activities in your Kestra instance.
+Audit Logs record every action taken in your Kestra instance by users and service accounts.
-## Audit logs – governance and compliance guide
+## Audit logs
-Audit Logs record all activities performed in your Kestra instance by users and service accounts. By reviewing Audit Logs, system administrators can track user activity, and security teams can investigate incidents and ensure compliance with regulatory requirements.
+By reviewing Audit Logs, system administrators can track user activity, and security teams can investigate incidents and ensure compliance with regulatory requirements.
-## Why are Audit Logs important
+## Why audit logs matter
-The audit log table in Kestra serves as a historical record that developers and system administrators can use to track changes, monitor system usage, and verify system activity. It's a transparency tool that tracks the sequence of activities, ensuring accountability for actions taken and providing data for troubleshooting and analysis. Given that Audit Logs are immutable, they can also be used to detect and investigate security incidents. If you leverage Kestra edition with Elasticsearch backend, you can also use Kibana to search and visualize your logs.
+Audit logs are a historical record that developers and system administrators can use to track changes, monitor system usage, and verify system activity. They track the sequence of activities, ensuring accountability and providing data for troubleshooting and analysis. Because audit logs are immutable, they can also be used to detect and investigate security incidents. If you use the Elasticsearch backend, you can use Kibana to search and visualize your logs.
## How to access Audit Logs
You can access Audit Logs from the **Tenant** section in the UI. That UI page provides a detailed table of recorded events, capturing the actions taken within the system:
-
+
Each row in the table represents a distinct event with several columns providing specific details:
@@ -39,21 +39,19 @@ Each row in the table represents a distinct event with several columns providing
## How to see a full diff of a specific event
-To see a full diff of a specific event, click on the icon in the **Changes** column. The expanded view shows the full diff of the event side-by-side, including the `before` and `after` states of a given resource:
+To see a full diff of a specific event, click the icon in the **Changes** column. The expanded view shows the full diff of the event side-by-side, including the `before` and `after` states of a given resource:
-## How to use the Details filter to search for specific Audit Log events
+## How to filter audit logs
-The `Details` filter allows you to flexibly search for any Audit Log event using the `key:value` format. It's a tag-based system which works the same way as [Execution Labels](../../../05.workflow-components/08.labels/index.md).
+Click **Add filters** to open the Advanced filter dialog. You can combine multiple conditions — for example, filter by Interval (Last 7 days) and Resource type (NAMESPACE) to narrow the table:
-For example, you can filter for all events related to a specific namespace by typing `namespace:your_namespace`:
+
-
+To filter for a specific event, click any tag in the **Details** column to add it as a filter condition.
-To further filter for a specific event, you can click on the relevant tag in the `Details` column, and it automatically adds the filter to the view.
-
-## How to Purge Audit Logs
+## How to purge audit logs
The Enterprise Edition of Kestra generates an audit log for _every action_ taken on the platform. While these logs are essential for tracking changes and ensuring compliance, they can accumulate over time and take up a significant amount of space in the database.
diff --git a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/add_secret.png b/src/contents/docs/07.enterprise/02.governance/07.namespace-management/add_secret.png
deleted file mode 100644
index 4c98aa434bb..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/add_secret.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/allowed-namespaces-2.png b/src/contents/docs/07.enterprise/02.governance/07.namespace-management/allowed-namespaces-2.png
deleted file mode 100644
index f6a72c81c6b..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/allowed-namespaces-2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/allowed-namespaces.png b/src/contents/docs/07.enterprise/02.governance/07.namespace-management/allowed-namespaces.png
deleted file mode 100644
index 9779a449399..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/allowed-namespaces.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/create_new_secret.png b/src/contents/docs/07.enterprise/02.governance/07.namespace-management/create_new_secret.png
deleted file mode 100644
index 0d4eb52a54b..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/create_new_secret.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/define_variables.png b/src/contents/docs/07.enterprise/02.governance/07.namespace-management/define_variables.png
deleted file mode 100644
index 63fc8fe8553..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/define_variables.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/index.md b/src/contents/docs/07.enterprise/02.governance/07.namespace-management/index.md
index 04fea8ccdf8..ef1a0d18861 100644
--- a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/07.namespace-management/index.md
@@ -8,13 +8,13 @@ editions: ["EE", "Cloud"]
docId: namespace.management
---
-How to manage secrets, variables, and plugin defaults at the Namespace level.
+Namespaces provide an additional layer of isolation for secrets, variables, and plugin defaults within a tenant.
-## Namespace management – secure configuration
+## Namespace management — secure configuration
Kestra is a [multi-tenant](../../02.governance/tenants/index.md) platform. Each tenant can have multiple Namespaces, and each Namespace provides additional isolation and security.
@@ -34,13 +34,9 @@ Since Kestra supports [everything as code and from the UI](https://youtu.be/dU3p
### Secrets
-On the Namespaces page, select the Namespace where you want to define the secrets and go to the **Secrets** tab. Here, you will see all existing secrets associated with this Namespace. Click on **Add a secret** button on the top right corner of the page.
+On the namespace page, go to the **Secrets** tab, click **Add a secret**, enter a key and value, and save.
-
-
-Define the secret by entering its key and value. Save the secret by clicking on the **Save** button at the bottom.
-
-The secret key should now start appearing on the **Secrets** tab. You can edit the secret's value or delete the secret by clicking on the appropriate button towards the right of the secret row. You can reference the secret in the flow by using the key, for example, `"{{ secret('MYSQL_PASSWORD') }}"`.
+The secret key now appears on the **Secrets** tab. Edit or delete it using the action buttons on the right. Reference the secret in flows using its key, for example, `"{{ secret('MYSQL_PASSWORD') }}"`.
For APIs that issue short-lived access tokens (e.g., OAuth2), create a [Credential](../../03.auth/credentials/index.md) that relies on these secrets and fetch the token in flows with `{{ credential('your_credential_key') }}`.
@@ -66,17 +62,32 @@ Make sure to only use the secret in flows defined in the same Namespace (or chil
When building new flows in a Namespace, Namespace secrets are accessible from the **Secrets** tab. Open the tab to view all available Namespace secret key names.
-### Plugin defaults
+### Policies
-Plugin Defaults can also be defined at the Namespace level. These plugin defaults are then applied for all tasks of the corresponding type defined in the flows under the same Namespace.
+[Policies](../policies/index.md) can be defined at the Namespace level to inject, restrict, or validate configuration for all flows in the Namespace. On the namespace page, open the **Policies** tab to create and manage Policies.
-On the Namespaces page, select the Namespace where you want to define the plugin defaults and navigate to the **Plugin defaults** tab. You can add the plugin defaults here and save the changes by clicking on the **Save** button at the bottom of the page.
+Policies can reference secrets and variables defined in the same Namespace.
-
+For example, a namespace-scoped Policy can inject database credentials into every MySQL task so flows don't need to declare them individually:
-You can reference secrets and variables defined with the same Namespace in the plugin defaults.
+```yaml
+id: mysql-credentials
+description: "Inject MySQL credentials for all MySQL tasks in this namespace."
+enforcement: ACTIVE
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.jdbc.mysql
+ values:
+ url: jdbc:mysql://localhost:3306/test
+ username: root
+ password: "{{ secret('MYSQL_PASSWORD') }}"
+```
-In the example below, you no longer need to add the `password` property for the MySQL query task as it's defined in your Namespace-level `pluginDefaults`:
+With this Policy applied, flows in the Namespace need no credentials on the task:
```yaml
id: query-mysql
@@ -85,12 +96,12 @@ namespace: company.team
tasks:
- id: query
type: io.kestra.plugin.jdbc.mysql.Query
- url: jdbc:mysql://localhost:3306/test
- username: root
sql: select * from employees
fetchOne: true
```
+Namespace-level Policies are inherited by child Namespaces. A Policy created in a parent Namespace applies to all flows in the parent and every child Namespace under it. See [Policies](../policies/index.md) for the full DSL reference, enforcement modes, and inheritance behavior.
+
### Default service account for SDK plugins
Namespaces can now provide **default authentication credentials** that [SDK-based plugins](/plugins/plugin-kestra) use to run tasks such as [List all Namespaces](/plugins/plugin-kestra/kestra-namespaces/io.kestra.plugin.kestra.namespaces.list). This allows tasks relying on the [Kestra SDK](../../../api-reference/kestra-sdk/index.mdx) to call the API without hard-coding credentials inside the flow.
@@ -103,9 +114,7 @@ On the Namespace **Edit** page, open the **Default authentication** section and
Variables defined at the Namespace level can be used in any flow defined under the same Namespace using the syntax: `{{ namespace.variable_name }}`.
-On the Namespaces page, select the Namespace where you want to define the variables. Go to the **Variables** tab. You can now define the variables on this page. Save the changes by clicking the **Save** button at the bottom of the page.
-
-
+On the namespace page, go to the **Variables** tab, define the variables, and save.
Here is an example flow where the Namespace variable is used:
@@ -122,15 +131,13 @@ tasks:
fetchOne: true
```
-When building new flows in a Namespace, Namespace variables are accessible from the **Variables** tab. Open the tab to view all available Namespace variables and their associated values.
-
-
+When building new flows in a Namespace, Namespace variables are accessible from the **Variables** tab.
## Creating Namespaces
### From the UI
-The video below shows how you can create a Namespace from the Kestra UI. After creating a Namespace, we're adding:
+The video below shows how to create a namespace and add:
- several new secrets
- a nested Namespace variable that references one of these secrets
- a list of plugin defaults helping to use those pre-configured secrets and variables in all the tasks from the AWS and Git plugins.
@@ -141,7 +148,7 @@ The video below shows how you can create a Namespace from the Kestra UI. After c
### From Terraform
-The following example reproduces the UI steps using Terraform, so that you know how to perform the same steps both from the UI and programmatically.
+The following example reproduces those steps in Terraform.
To create a Namespace from Terraform, use the [kestra_namespace](https://registry.terraform.io/providers/kestra-io/kestra/latest/docs) resource.
@@ -192,7 +199,7 @@ github:
token: "{{ secret('GITHUB_TOKEN') }}"
```
-Then, create another file for `task_defaults_marketing.yml`:
+Then, create another file for `plugin_defaults_marketing.yml`:
```yaml
- type: io.kestra.plugin.aws
@@ -213,7 +220,7 @@ resource "kestra_namespace" "marketing" {
namespace_id = "marketing"
description = "Namespace for the marketing team"
variables = file("variables_marketing.yml")
- task_defaults = file("task_defaults_marketing.yml")
+ plugin_defaults = file("plugin_defaults_marketing.yml")
}
```
@@ -293,10 +300,7 @@ kestra_password = "your-kestra-password"
```
## Allowed Namespaces
-When you navigate to any Namespace and go to the Edit tab, you can explicitly configure which Namespaces are allowed to access flows and other resources related to that Namespace. By default, all Namespaces are allowed:
-
-
-However, you can restrict that access if you want only specific Namespaces (or no Namespace at all) to trigger its corresponding resources.
+On the **Edit** tab of any namespace, configure which namespaces are allowed to access its flows and resources.
-
+By default, **all Namespaces** are allowed. To restrict access, **select specific Namespaces** — access automatically extends to each selected namespace's children.
diff --git a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/namespace-variable-tab.png b/src/contents/docs/07.enterprise/02.governance/07.namespace-management/namespace-variable-tab.png
deleted file mode 100644
index a6516b5cb36..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/namespace-variable-tab.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/plugindefaults-namespaces.png b/src/contents/docs/07.enterprise/02.governance/07.namespace-management/plugindefaults-namespaces.png
deleted file mode 100644
index f9a2289b3b9..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/07.namespace-management/plugindefaults-namespaces.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/allowed-plugins/index.md b/src/contents/docs/07.enterprise/02.governance/allowed-plugins/index.md
index 1a85202c581..09564b76990 100644
--- a/src/contents/docs/07.enterprise/02.governance/allowed-plugins/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/allowed-plugins/index.md
@@ -8,9 +8,7 @@ editions: ["EE", "Cloud"]
version: ">= 0.19.0"
---
-How to configure Kestra to allow or restrict specific plugins.
-
-## Allowed & Restricted Plugins
+Configure Kestra to allow or restrict specific plugins.
Kestra comes with the full library of official plugins by default. However, in some cases you may want to restrict which plugins are available to specific teams or users. For example, you might allow a team to use only BigQuery tasks while blocking script execution. Kestra enables this by letting you define allowlists (`includes`) and blocklists (`excludes`) in your [Plugins and Execution configuration](../../../configuration/04.plugins-and-execution/index.md).
diff --git a/src/contents/docs/07.enterprise/02.governance/cases/cases-board-view.png b/src/contents/docs/07.enterprise/02.governance/cases/cases-board-view.png
new file mode 100644
index 00000000000..8facd7bad11
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/cases/cases-board-view.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/cases/cases-detail.png b/src/contents/docs/07.enterprise/02.governance/cases/cases-detail.png
new file mode 100644
index 00000000000..12cc88ad0a3
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/cases/cases-detail.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/cases/cases-templates.png b/src/contents/docs/07.enterprise/02.governance/cases/cases-templates.png
new file mode 100644
index 00000000000..3c98d14a634
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/cases/cases-templates.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/cases/index.md b/src/contents/docs/07.enterprise/02.governance/cases/index.md
new file mode 100644
index 00000000000..5c102cf24e6
--- /dev/null
+++ b/src/contents/docs/07.enterprise/02.governance/cases/index.md
@@ -0,0 +1,295 @@
+---
+title: "Cases in Kestra: Incident Management for Executions"
+h1: Track and Resolve Incidents with Cases
+description: Use Cases in Kestra Enterprise to track and resolve incidents directly in Kestra. Create cases automatically from failed executions, deduplicate alerts, and manage SLAs.
+sidebarTitle: Cases
+icon: /src/contents/docs/icons/admin.svg
+editions: ["EE", "Cloud"]
+version: ">= 2.0.0"
+---
+
+Track and resolve incidents directly in Kestra, next to the executions that caused them.
+
+When an execution fails, it is often an incident. Cases lets you track and resolve those incidents directly in Kestra, without a separate tool.
+
+Cases are not limited to failures; you can open one for executions that need approvals, or for successful executions that produced unexpected outputs.
+
+The `CreateCase` task lets you open cases automatically from your flow YAML:
+
+- in the `errors`, `finally`, or `afterExecution` blocks, to auto-create custom cases when the execution moves to a terminal state like Failed, Killed, Warning, or Success
+- anywhere in the regular `tasks` block, to create cases based on custom conditions within the flow, for example if a given output is bad (error status code on some API request).
+
+Not every execution needs to be treated as an independent case, though. Often a single production issue (e.g. an external API that is temporarily unreachable) can easily create 10 to 100 failed executions within an hour. Sending Slack alerts for each failed execution can quickly get so noisy that users mute the channel, making the alerting useless. Cases solve this with **deduplication**: with `linkMatchingExecutions: true`, new failing executions are attached to the already-open case for the same flow and task instead of opening a new one.
+
+:::alert{type="warning"}
+Cases require the [Kestra plugin](https://kestra.io/plugins/plugin-kestra) to be installed on your Kestra EE instance, because the `CreateCase` task used under the hood by Cases comes from this plugin. The default Docker image with plugins included already contains it. If you manage plugins yourself, install the Kestra plugin through [Versioned Plugins](../../05.instance/versioned-plugins/index.md) to use Cases reliably.
+:::
+
+## What a case contains
+
+- Title and description: the description supports Markdown
+- Severity: `Critical`, `High`, `Medium`, or `Low`; defaults to `Medium`
+- Status: `Open`, `Acknowledged`, `Investigating`, `Resolved`, or `Cancelled`; `Resolved` and `Cancelled` are terminal
+- Namespace: every case belongs to a namespace, and all permission checks are scoped to it
+- Assignees and watchers: both accept users and groups; assignees are responsible for the case, watchers only receive notifications. The same user or group cannot be both at once — assigning them as an assignee removes them from the watchers
+- Custom fields: typed fields (Text, Select, Multi-select) defined per case or inherited from a template; cases use custom fields instead of labels
+- SLAs: optional acknowledgement and resolution targets
+- Linked executions and assets: the executions and [assets](../01.assets/index.md) affected by the incident
+- Case actions: flows attached to the case as one-click remediation buttons
+- Origin: the flow, task, and execution that created the case (set only when created by the `CreateCase` task); this is the deduplication key
+- Template reference: the case template the case was created from, if any
+
+## Case lifecycle
+
+There is no restrictive state machine: any status can be changed to any other, including reopening a resolved or cancelled case. A few rules apply:
+
+1. Resolving requires a **resolution reason**; the note is optional. When the case was created from a template, the template can restrict the allowed reasons and make the note mandatory. Without a template, the built-in reasons are offered: Fixed, Workaround applied, Configuration change, Duplicate, No action needed, Won't fix.
+2. Cancelling accepts an optional reason and note.
+3. The first time the case moves from `Open` to any other status, the acknowledgement time is recorded (this satisfies the acknowledgement SLA and is kept even if the case is later reopened).
+4. Reopening a terminal case clears the resolution, and the case's auto-attach flow (if any) is recreated.
+
+## SLAs
+
+Each case can have two optional SLA targets, set directly or inherited from a template:
+
+1. Acknowledgement: met the first time the case moves from `Open` to any other status
+2. Resolution: met when the case is resolved
+
+Both clocks start at case creation. Each of the two SLAs can be in one of six states: `Not started`, `Running`, `Overdue`, `Met`, `Missed`, or `Voided` (cancelling a case voids its pending SLAs). States are computed at read time and never persisted. The UI shows a live countdown ("Due in 2h", "Due 30m ago") on the case detail page, board cards, and the list's Resolution SLA column.
+
+When a target is missed, a background check records an `SLA_ACKNOWLEDGEMENT_BREACHED` or `SLA_RESOLUTION_BREACHED` event in the case's timeline and sends an in-app notification to the case's assignees and watchers. The check runs every 5 minutes by default and can be tuned or disabled through the `kestra.cases.sla-breach-check` configuration; each of the two SLAs is notified at most once per case.
+
+## The CreateCase task
+
+The task type is `io.kestra.plugin.kestra.ee.cases.CreateCase`. It is an SDK-based task that calls the Kestra API, so it can be placed in any task block: `tasks`, `errors`, `finally`, or `afterExecution`.
+
+```yaml
+id: orders_sync
+namespace: company.team
+
+tasks:
+ - id: sync
+ type: io.kestra.plugin.core.http.Request
+ uri: https://api.example.com/orders
+
+errors:
+ - id: open_case
+ type: io.kestra.plugin.kestra.ee.cases.CreateCase
+ kestraUrl: http://localhost:8080
+ auth:
+ apiToken: "{{ secret('KESTRA_API_TOKEN') }}"
+ title: "Orders sync failed: {{ flow.id }}"
+ severity: HIGH
+ linkMatchingExecutions: true
+ sla:
+ acknowledgement: PT1H
+ resolution: PT8H
+```
+
+The task calls the Kestra API, so it needs an endpoint and credentials. `kestraUrl` defaults to the URL of the current instance, and `auth` accepts either an `apiToken` or a `username` and `password` pair. Both can also be omitted: an administrator can set **Default authentication credentials** when editing a namespace or tenant, and those credentials are then used automatically (the instance configuration `kestra.tasks.sdk.authentication` works as a global fallback). Without `auth` on the task and without such defaults, the task fails at runtime.
+
+For condition-based cases inside the regular `tasks` block, combine it with `runIf`:
+
+```yaml
+tasks:
+ - id: check
+ type: io.kestra.plugin.core.http.Request
+ uri: https://api.example.com/health
+ allowFailed: true
+
+ - id: open_case
+ type: io.kestra.plugin.kestra.ee.cases.CreateCase
+ kestraUrl: http://localhost:8080
+ auth:
+ apiToken: "{{ secret('KESTRA_API_TOKEN') }}"
+ runIf: "{{ outputs.check.code != 200 }}"
+ title: "Health check returned {{ outputs.check.code }}"
+ severity: CRITICAL
+```
+
+### Properties
+
+| Property | Description |
+|---|---|
+| `title` | Case title; required unless `caseId` is set. Supports expressions. |
+| `caseDescription` | Markdown description (named this way because `description` is reserved for the task itself). |
+| `severity` | `CRITICAL`, `HIGH`, `MEDIUM`, or `LOW`; defaults to `MEDIUM`. |
+| `status` | Initial status; defaults to `OPEN`. |
+| `namespace` | Namespace the case belongs to; defaults to the flow's namespace. |
+| `sla` | `acknowledgement` and `resolution` durations (ISO-8601, e.g. `PT1H`). |
+| `linkMatchingExecutions` | When `true`, the execution is attached to an already-open case with the same origin instead of creating a new one; defaults to `false`. |
+| `caseId` | Attach the execution to this exact case instead of creating one (used by generated auto-attach flows). |
+| `executionId` | Execution to link; defaults to the current execution. |
+| `assignees` / `watchers` | User emails and group names, resolved to stable IDs on the server. |
+| `labels` | Key/value pairs, stored as Text custom fields on the case. |
+| `assetIds` | Assets to link to the case. |
+| `actions` | Case actions to attach, each with `label`, `namespace`, and `flowId`. |
+
+Connection properties (`kestraUrl`, `auth`, `tenantId`) are shared by all tasks from the Kestra plugin; `tenantId` defaults to the execution's tenant. Authentication is resolved as described above: `auth` set on the task takes precedence, then the namespace's default authentication credentials, then the tenant's, then the instance configuration.
+
+### Outputs
+
+The task returns `caseId` and `created`. `created: false` means the execution was attached to an existing case (via deduplication or `caseId`) rather than opening a new one.
+
+Note that the task cannot reference a case template. Case templates are mainly helpers to make it easier to configure cases from the UI; when creating cases from code, all properties such as severity or SLAs are declared explicitly on the task, following Kestra's declarative nature.
+
+## Deduplication
+
+With `linkMatchingExecutions: true`, the server first looks for an active case with the same **origin**: the combination of flow namespace, flow ID, and task ID (within the tenant). The title is not part of the key. A case counts as active when its status is `Open`, `Acknowledged`, or `Investigating`.
+
+- On a match: the triggering execution is appended to the existing case's linked executions (visible in the timeline as an `EXECUTION_LINKED` event), no new case is created, and the task returns `created: false`. Fields such as title or severity are not merged, and no new "created" notification is sent.
+- On no match: a new case is created with the origin recorded, and the execution is linked to it.
+
+The same behavior can also be enabled from the UI on an existing case with [auto-attach](#auto-attach).
+
+One caveat: the deduplication check is not atomic. The server first checks for an existing case and then creates one, so two executions of the same task failing at exactly the same moment can each create their own case.
+
+## Auto-attach
+
+Auto-attach allows you to avoid getting a new case and a new notification for every single failed execution. A case is created for the first failure, and all following executions matching the same flow and state are automatically attached to that same case until the case is resolved or cancelled.
+
+Deduplication with `linkMatchingExecutions` achieves the same from the flow YAML; auto-attach is its UI counterpart, configured on an existing case. It can be enabled with the "Auto-link matching executions" toggle when attaching executions to a case, from the case detail page or in the Create Case modal. A banner then lists exactly what will be matched, for example "New executions matching company.team/orders_sync (FAILED) will automatically attach to this case." Several executions can be selected at once, even across different flows: the selection is grouped into one rule per flow, and the states of that flow's selected executions are combined.
+
+Under the hood, enabling it generates a flow named `attach_executions_` in the system namespace, with one Flow trigger per (namespace, flow, states) rule and a single `CreateCase` task that passes the `caseId`. Multiple rules accumulate on the same generated flow. The flow is deleted when the case reaches a terminal status or is deleted, and recreated on reopen. These generated flows are visible on the [System Flows](../../../06.concepts/system-flows/index.md) page.
+
+Enabling auto-attach is gated by RBAC: it requires the `UPDATE` action on the `CASE` permission in the case's namespace, plus the `EXECUTE` action on the `FLOW` permission in the namespace of the flow whose executions will be attached.
+
+## Creating cases from the UI
+
+Besides the task, cases can be created in four places:
+
+1. Cases page: the Create button opens a modal with template selection, title, severity, initial status, namespace, assignees and watchers, both SLA targets (presets from 1 hour to 72 hours, or a custom duration), description, custom fields, and executions to link.
+2. Executions page (bulk): select executions and use "Create case from selection" or "Add to existing case". Both also work with "select all matching filter", so a case can be created from everything matching the current query.
+3. Execution's Overview tab: a "Linked cases" panel lists the cases the execution belongs to and offers a Create case button pre-linked to that execution.
+4. Asset page: the same panel exists on the asset overview; a case created there is linked to the asset automatically.
+
+## Cases in the UI
+
+Cases sit in the left menu between Executions and System Flows.
+
+- Board view (default): a kanban board grouped by status, severity, or assignee. Dragging a card between columns updates the case; dropping onto Resolved opens the resolve modal, because a reason is mandatory. Cards show the case ID, severity, title, assignees, and a live SLA countdown.
+- List view: columns for Case, Title, Severity, Status, Resolution SLA, and Assignee, with an inline next-step button per row (Acknowledge, Investigating, Resolve) and bulk Acknowledge/Delete.
+- Toolbar: full-text search; filters on namespace, status, severity, assignee, and time range; an "Assigned to me" toggle; and one chip per status with live counts.
+
+
+
+The case detail page shows the editable header (title, severity, status, quick transitions, Resolve/Reopen), the Markdown description, custom fields, a resolution card once resolved, SLA countdowns, assignees and watchers (with a Watch/Unwatch toggle and "Assign with note"), case actions, linked executions, linked assets, and the activity timeline.
+
+
+
+## Comments and activity timeline
+
+Every change to a case is recorded as a timeline event: creation, field updates, status and severity changes, assignments (with optional note), execution and asset links, action runs, auto-attach changes, and SLA breaches. Comments support **Markdown** and up to 5 file attachments of 10 MB each (drag-and-drop and clipboard paste work).
+
+## Case actions
+
+A case action is a flow attached to the case as a one-click button, intended for remediation or diagnostics (e.g. "Restart service"). Clicking an action opens the Execute flow modal so you can review and fill the flow's inputs, and optionally set labels, a schedule date, or breakpoints before running. The resulting execution is automatically linked back to the case and labeled `system.caseId: ` and `system.from: case`, and the run is recorded in the timeline. Attaching or running an action requires the `EXECUTE` action on the `FLOW` permission in the target flow's namespace.
+
+## Linked executions and assets
+
+- Executions can be linked by the task, by auto-attach, manually from the case detail page, or in bulk by ID or by filter query. The linked-executions card shows each execution's live state and keeps a row (marked as no longer existing) even if the execution was purged.
+- Assets come in two kinds: explicitly attached ones, and derived ones (assets used by the case's linked executions, shown with an "Auto-detected" tag). Derived assets cannot be unlinked; they disappear when the executions that reference them are unlinked.
+
+## Case templates
+
+Templates standardize how cases are created. A template can define a default severity, default assignees and watchers, a title pattern (supporting the same expressions as the task title), a description, both SLA targets, custom field definitions, default case actions, allowed resolution reasons, and whether a resolution note is required. One template per tenant can be marked as the default; it is preselected in the Create Case modal.
+
+Templates are managed in **Cases → Settings** and require the `TEMPLATE` action on the `CASE` permission.
+
+ A template can be scoped to a namespace or left tenant-wide (tenant-wide templates and the default flag require a global grant).
+
+Every tenant is seeded with a built-in "Execution failure incident" template: severity High, title pattern `{{ execution.id }} failed for {{ flow.id }}`, 1 hour acknowledgement and 8 hour resolution SLA, and the built-in resolution reasons.
+
+Template defaults apply when creating cases via the UI or the API; they cannot be used by the `CreateCase` task, where all properties are declared explicitly in the flow YAML.
+
+## Notifications
+
+Case events are wired into the in-app notification system (the bell icon). Seven events send notifications: case created, assigned, status changed, severity changed, commented, acknowledgement SLA breached, and resolution SLA breached. Recipients are the case's assignees and watchers, with groups expanded to their members; assignment notifications go only to the users whose assignment actually changed, with the assignment note included. Each notification contains a link to the case.
+
+## Permissions and audit
+
+- RBAC: a new `CASE` permission resource with actions `VIEW`, `LIST`, `CREATE`, `UPDATE`, `DELETE`, `FOLLOW`, and `TEMPLATE`, optionally scoped to a namespace (_creating a case requires `CREATE` on its namespace, transitions and linking require `UPDATE`, commenting only requires `VIEW`_).
+- Audit: create, update, and delete of cases and case templates are recorded in the [audit log](../06.audit-logs/index.md) (resource types `CASE` and `CASE_TEMPLATE`). Timeline events and comments are not separately audited.
+- Backup: cases, templates, timeline events, and execution links are included in backup and restore.
+
+## API
+
+All case operations are available via REST under `/api/v1/{tenant}/cases`. The calls below cover the most common integration patterns. For the full endpoint list, request shapes, and filter DSL, see the [Enterprise API reference](../../../api-reference/01.enterprise/index.mdx).
+
+**Search cases**
+
+```bash
+curl -X GET "https://{host}/api/v1/{tenant}/cases/search?page=1&size=25" \
+ -H "Authorization: Bearer {token}"
+```
+
+Accepts `filters` (a list of structured query filter objects), `sort`, and `dateFilter` as query parameters. Use `GET /cases/counts` to get a `{status: count}` breakdown without pagination.
+
+**Create a case**
+
+```bash
+curl -X POST "https://{host}/api/v1/{tenant}/cases" \
+ -H "Authorization: Bearer {token}" \
+ -H "Content-Type: application/json" \
+ -d '{
+ "namespace": "company.team",
+ "title": "Orders sync failed",
+ "severity": "HIGH",
+ "sla": { "acknowledgement": "PT1H", "resolution": "PT8H" }
+ }'
+```
+
+Returns the full case object including the generated `id`.
+
+**Acknowledge and resolve**
+
+```bash
+# Acknowledge
+curl -X POST "https://{host}/api/v1/{tenant}/cases/{id}/acknowledge" \
+ -H "Authorization: Bearer {token}"
+
+# Resolve
+curl -X POST "https://{host}/api/v1/{tenant}/cases/{id}/resolve" \
+ -H "Authorization: Bearer {token}" \
+ -H "Content-Type: application/json" \
+ -d '{ "reason": "Fixed", "note": "Deployed hotfix v1.2.3" }'
+```
+
+A `reason` is required to resolve. The reason is a free-form string; case templates can configure a `resolutionReasons` list to restrict which values are accepted.
+
+**Link executions**
+
+```bash
+curl -X POST "https://{host}/api/v1/{tenant}/cases/{id}/executions" \
+ -H "Authorization: Bearer {token}" \
+ -H "Content-Type: application/json" \
+ -d '["execution-id-1", "execution-id-2"]'
+```
+
+Use `POST /cases/{id}/executions/by-query` to link everything matching a filter query (capped at 1,000).
+
+**Look up cases by execution**
+
+```bash
+curl -X POST "https://{host}/api/v1/{tenant}/cases/by-executions" \
+ -H "Authorization: Bearer {token}" \
+ -H "Content-Type: application/json" \
+ -d '["execution-id-1", "execution-id-2"]'
+```
+
+Returns `{ "executionId": [caseSummary, ...] }`, useful for annotating an execution list with its linked cases.
+
+**Add a comment**
+
+```bash
+curl -X POST "https://{host}/api/v1/{tenant}/cases/{id}/comments" \
+ -H "Authorization: Bearer {token}" \
+ -F 'comment={"message":"Restarted the service, monitoring now."};type=application/json'
+```
+
+Supports up to 5 file attachments (10 MB each) as additional `-F file=@path` parts.
+
+## Current limitations
+
+- Notifications are in-app only: case events appear in the notification bell, but no Slack message or email is sent by Cases. External alerting can be added with standard notification tasks alongside `CreateCase`, or as a case action.
+- Deduplication is not atomic: concurrent failures of the same task can occasionally create duplicate cases.
diff --git a/src/contents/docs/07.enterprise/02.governance/custom-blueprints/blueprint-org-2.png b/src/contents/docs/07.enterprise/02.governance/custom-blueprints/blueprint-org-2.png
deleted file mode 100644
index bb7351f82bd..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/custom-blueprints/blueprint-org-2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/custom-blueprints/index.md b/src/contents/docs/07.enterprise/02.governance/custom-blueprints/index.md
index 833e09c56a9..b6be5d2aac8 100644
--- a/src/contents/docs/07.enterprise/02.governance/custom-blueprints/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/custom-blueprints/index.md
@@ -8,25 +8,17 @@ editions: ["EE", "Cloud"]
docId: custom
---
-How to create and manage Custom Blueprints.
-
-# Custom Blueprints in Kestra Enterprise – Private Templates
+Custom Blueprints are private, reusable workflow templates that extend the publicly available [Community Blueprints](../../../06.concepts/07.blueprints/index.md).
-In addition to the publicly available [Community Blueprints](../../../06.concepts/07.blueprints/index.md), Kestra allows you to create **Custom Blueprints**—private, reusable workflow templates tailored to your team. These blueprints help centralize orchestration patterns, document best practices, and streamline collaboration across your organization.
-
-You can think of Custom Blueprints as your team's internal App Store, offering a wide range of integrations and validated workflow patterns tailored to your needs.
+Custom Blueprints help centralize orchestration patterns, document best practices, and streamline collaboration across your organization.
### How to create a new custom blueprint
-From the left navigation menu, go to **Blueprints**. Then, select the **Custom Blueprints** tab. Click on **Create**.
-
-Add a title, description, and the contents of the flow. You can add as many tags as you want. Then click on the **Create** button.
-
-
+From the left navigation menu, go to **Blueprints**, select the **Custom Blueprints** tab, and click **Create**. Add a title, description, and the flow YAML, then click **Create**.
You can edit Blueprints at any time, for example, to add new tasks or expand the documentation.
@@ -36,7 +28,7 @@ Templated Blueprints allow you to create reusable, configurable workflows that u
Platform teams build templates once; business users instantiate them by filling in a form rather than editing YAML.
-**How It Works:** Templated Blueprints use [Pebble templating](../../../06.concepts/06.pebble/index.md), with custom delimiters to avoid conflicts with Kestra expressions.
+**How it works:** Templated Blueprints use [Pebble templating](../../../06.concepts/06.pebble/index.md), with custom delimiters to avoid conflicts with Kestra expressions.
### Define Template Arguments
@@ -98,7 +90,6 @@ tasks:
This allows you to dynamically generate tasks or include them conditionally.
-Solutions such as templatized Terraform configurations or using the Python SDK to make DAG factories are still valid ways to address similar templating needs. Templated Custom Blueprints offer a more direct, simpler and integrated approach within the Kestra platform.
### Example: Data Ingestion Template
@@ -144,33 +135,29 @@ tasks:
tasks:
- id: << domain | slugify >>-download
type: io.kestra.plugin.jdbc.postgresql.CopyOut
+ url: jdbc:postgresql://sample_postgres:5432/<>
+ username: '{{ secret("POSTGRES_USERNAME") }}'
+ password: '{{ secret("POSTGRES_PASSWORD") }}'
+ format: CSV
sql: SELECT * FROM public.<< domain | slugify >>
- id: << domain | slugify >>-ingest
<% if arg.target == 'Oracle' %>
type: io.kestra.plugin.jdbc.oracle.Batch
+ url: jdbc:oracle:thin:@<< arg.env >>:49161:XE
+ username: '{{ secret("ORACLE_USERNAME") }}'
+ password: '{{ secret("ORACLE_USERNAME") }}'
from: "{{ << domain | slugify >>-download.uri }}"
table: public.< domain | slugify >>
<% elseif arg.target == 'Postgres' %>
type: io.kestra.plugin.jdbc.postgresql.CopyIn
- from: "{{ outputs.<< domain | slugify >>-download.uri }}"
url: jdbc:postgres://sample_<< arg.target | lower>>:5432/<>
+ username: '{{ secret("POSTGRES_USERNAME") }}'
+ password: '{{ secret("POSTGRES_PASSWORD") }}'
+ format: CSV
+ from: "{{ outputs.<< domain | slugify >>-download.uri }}"
table: public.< domain | slugify >>
<% endif %>
<% endfor %>
-
-pluginDefaults:
- - type: io.kestra.plugin.jdbc.postgresql
- values:
- url: jdbc:postgresql://sample_postgres:5432/<>
- username: '{{ secret("POSTGRES_USERNAME") }}'
- password: '{{ secret("POSTGRES_PASSWORD") }}'
- format: CSV
-
- - type: io.kestra.plugin.jdbc.oracle.Batch
- values:
- url: jdbc:oracle:thin:@<< arg.env >>:49161:XE
- username: '{{ secret("ORACLE_USERNAME") }}'
- password: '{{ secret("ORACLE_USERNAME") }}'
```
:::
@@ -193,9 +180,16 @@ tasks:
tasks:
- id: hr-download
type: io.kestra.plugin.jdbc.postgresql.CopyOut
+ url: jdbc:postgresql://sample_postgres:5432/dev
+ username: '{{ secret("POSTGRES_USERNAME") }}'
+ password: '{{ secret("POSTGRES_PASSWORD") }}'
+ format: CSV
sql: SELECT * FROM public.hr
- id: hr-ingest
type: io.kestra.plugin.jdbc.oracle.Batch
+ url: jdbc:oracle:thin:@dev:49161:XE
+ username: '{{ secret("ORACLE_USERNAME") }}'
+ password: '{{ secret("ORACLE_USERNAME") }}'
from: "{{ hr-download.uri }}"
table: public.< domain | slugify >>
@@ -204,24 +198,105 @@ tasks:
tasks:
- id: manufacture-download
type: io.kestra.plugin.jdbc.postgresql.CopyOut
+ url: jdbc:postgresql://sample_postgres:5432/dev
+ username: '{{ secret("POSTGRES_USERNAME") }}'
+ password: '{{ secret("POSTGRES_PASSWORD") }}'
+ format: CSV
sql: SELECT * FROM public.manufacture
- id: manufacture-ingest
type: io.kestra.plugin.jdbc.oracle.Batch
+ url: jdbc:oracle:thin:@dev:49161:XE
+ username: '{{ secret("ORACLE_USERNAME") }}'
+ password: '{{ secret("ORACLE_USERNAME") }}'
from: "{{ manufacture-download.uri }}"
table: public.< domain | slugify >>
-
-pluginDefaults:
- - type: io.kestra.plugin.jdbc.postgresql
- values:
- url: jdbc:postgresql://sample_postgres:5432/dev
- username: '{{ secret("POSTGRES_USERNAME") }}'
- password: '{{ secret("POSTGRES_PASSWORD") }}'
- format: CSV
-
- - type: io.kestra.plugin.jdbc.oracle.Batch
- values:
- url: jdbc:oracle:thin:@dev:49161:XE
- username: '{{ secret("ORACLE_USERNAME") }}'
- password: '{{ secret("ORACLE_USERNAME") }}'
```
:::
+
+## Version control for Custom Blueprints
+
+Custom Blueprints can be version-controlled with Git using two dedicated tasks from the `plugin-ee-git` plugin:
+
+- [PushBlueprints](/plugins/plugin-ee-git/io.kestra.plugin.ee.git.PushBlueprints) commits and pushes blueprints from Kestra to a Git repository.
+- [SyncBlueprints](/plugins/plugin-ee-git/io.kestra.plugin.ee.git.SyncBlueprints) syncs blueprints from a Git repository into Kestra, treating Git as the single source of truth.
+
+These tasks mirror the [PushFlows and SyncFlows patterns](../../../version-control-cicd/04.git/index.md) used for flows, applied to Custom Blueprints.
+
+### Push blueprints to Git
+
+Use `PushBlueprints` to export your blueprints from Kestra into a Git repository. This is useful for creating backups, reviewing changes via pull requests, or promoting blueprints across environments.
+
+Each blueprint is written as a YAML file to the target `gitDirectory` (default: `_blueprints`). Use the `blueprints` property with glob patterns to push only a subset of blueprints.
+
+```yaml
+id: push_blueprints
+namespace: system
+
+tasks:
+ - id: commit_and_push
+ type: io.kestra.plugin.ee.git.PushBlueprints
+ url: https://github.com/your-org/blueprints-repo
+ username: git_username
+ password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
+ branch: main
+ commitMessage: "push blueprints from {{ flow.namespace ~ '.' ~ flow.id }}"
+
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 * * * *"
+```
+
+The task outputs a `commitId`, a `commitURL`, and a `blueprints` URI pointing to a diff report that lists the number of lines added, deleted, and changed per file.
+
+### Sync blueprints from Git
+
+Use `SyncBlueprints` to pull blueprints from Git into Kestra. This is the recommended pattern when Git is your single source of truth, for example when platform teams manage approved blueprint libraries centrally and deploy them across multiple Kestra instances.
+
+By default, `SyncBlueprints` only adds and updates blueprints. Set `delete: true` to also remove any blueprints present in Kestra but absent in Git.
+
+```yaml
+id: sync_blueprints_from_git
+namespace: system
+
+tasks:
+ - id: git
+ type: io.kestra.plugin.ee.git.SyncBlueprints
+ url: https://github.com/your-org/blueprints-repo
+ branch: main
+ username: git_username
+ password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
+ delete: true
+ dryRun: true
+
+triggers:
+ - id: every_full_hour
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 * * * *"
+```
+
+Set `dryRun: true` to preview what would change without applying it. The `blueprints` output URI contains a row-per-blueprint report showing each blueprint's `syncState`: `ADDED`, `UPDATED`, `UNCHANGED`, or `DELETED`.
+
+Use caution with `delete: true`: it removes all blueprints not present in Git, not just those that differ.
+
+### Blueprint YAML file format
+
+Both tasks read and write blueprints as YAML files. Each file represents one blueprint:
+
+```yaml
+id: my-blueprint-id
+title: My Blueprint Title
+description: Optional description of what this blueprint does
+tags:
+ - tag1
+ - tag2
+flow: |
+ id: my-flow
+ namespace: company.team
+ tasks:
+ - id: hello
+ type: io.kestra.plugin.core.log.Log
+ message: Hello World
+```
+
+The `id` field controls how blueprints are matched on sync: if a blueprint with that ID already exists in Kestra, it is updated; if not, it is created with that ID. If `id` is omitted, a new blueprint is created with an auto-generated ID.
diff --git a/src/contents/docs/07.enterprise/02.governance/index.mdx b/src/contents/docs/07.enterprise/02.governance/index.mdx
index 73730ead372..51239853d8f 100644
--- a/src/contents/docs/07.enterprise/02.governance/index.mdx
+++ b/src/contents/docs/07.enterprise/02.governance/index.mdx
@@ -10,10 +10,6 @@ docId: governance
import ChildCard from "~/components/docs/ChildCard.astro"
-Give your team secured, isolated environments and control over workflows with tenants, audit logs, secrets and more.
-
-## Governance – security and control
-
-With tailored automation and precise access management, you can ensure compliance and efficiency at scale.
+Governance features let you control what runs, who can see it, and where sensitive data lives — across tenants, namespaces, and external systems.
\ No newline at end of file
diff --git a/src/contents/docs/07.enterprise/02.governance/logshipper/index.md b/src/contents/docs/07.enterprise/02.governance/logshipper/index.md
index 1ff24ba6f7a..5bf76c0fe61 100644
--- a/src/contents/docs/07.enterprise/02.governance/logshipper/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/logshipper/index.md
@@ -10,15 +10,19 @@ version: "0.21.0"
Log Shipper exports Kestra execution and audit logs to external observability platforms and SIEMs.
+:::alert{type="info"}
+**Log Shipper vs External Log Data Store** — these solve different problems. Log Shipper copies logs to third-party observability platforms (Datadog, Splunk, CloudWatch, etc.) for alerting and search. The [External Log Data Store](../../../10.administrator-guide/log-data-store/index.md) routes Kestra's primary log storage to a separate database or Elasticsearch, reducing main database size without any flow or sidecar. Both can be active at the same time.
+:::
+
-## Log shipper – centralize logs
+## Log shipper — centralize logs
Log Shipper can distribute Kestra logs from across your instance to an external logging platform. Log synchronization fetches logs and batches them into optimized chunks automatically. The batch process is done intelligently through defined synchronization points. Once batched, the Log Shipper delivers consistent and reliable data to your monitoring platform.
-Log Shipper is built on top of [Kestra plugins](/plugins), ensuring it can integrate with popular logging platforms and expand as more plugins are developed. Supported observability platforms include ElasticSearch, Datadog, New Relic, Azure Monitor, Google Operational Suite, AWS Cloudwatch, Splunk, OpenSearch, Huawei Cloud LTS, OpenTelemetry, Graylog, and Syslog (CEF).
+Log Shipper is built on top of [Kestra plugins](/plugins), ensuring it can integrate with popular logging platforms and expand as more plugins are developed. Supported observability platforms include ElasticSearch, Datadog, New Relic, Azure Monitor, Google Operational Suite, AWS Cloudwatch, Splunk, OpenSearch, Huawei Cloud LTS, OpenTelemetry, Graylog, Dash0, and Syslog (CEF).
## Log shipper properties
@@ -80,8 +84,6 @@ The logs are then distributed to the exporters in chunks of 5000, 2000, and 1000
## Log shipper examples
-The Log Shipper integrates with many popular observability platforms. Below are a couple of example flows using a Kestra core plugin as well as external platform plugins.
-
### Kestra `FileLogExporter`
The following example uses Kestra's core `FileLogExporter` plugin to synchronize the logs of the `company.team` namespace. The `synchronize_logs` task outputs a file, and the log file `uri` is passed as an expression in the `upload` task to then upload the logs to an S3 bucket.
@@ -116,7 +118,7 @@ tasks:
### Datadog
-The below example demonstrates an execution that runs a daily log synchronization and distribution of logs with [Datadog](https://www.datadoghq.com/) using the default property settings.
+This example exports logs to [Datadog](https://www.datadoghq.com/) using the default property settings:
```yaml
id: log_shipper
@@ -144,9 +146,9 @@ The batched logs directly populate your Datadog instance like in the following s
-### AWS Cloudwatch
+### AWS CloudWatch
-This example exports logs to [AWS Cloudwatch](https://aws.amazon.com/cloudwatch/). The following example flow triggers a daily batch and exports to AWS's service [Amazon CloudWatch](https://docs.aws.amazon.com/cloudwatch/):
+This example exports logs to [AWS CloudWatch](https://aws.amazon.com/cloudwatch/):
```yaml
id: log_shipper
@@ -180,7 +182,7 @@ The logs are viewable in the interface of the specified Log Group and can be exa
### AWS S3
-This example exports logs to [AWS S3](https://aws.amazon.com/s3/). The following example flow triggers a daily batch and exports to AWS's S3 object storage:
+This example exports logs to [AWS S3](https://aws.amazon.com/s3/):
```yaml
id: log_shipper
@@ -209,7 +211,7 @@ tasks:
```
### Google Operational Suite
-This example exports logs to [Google Cloud Observability](https://cloud.google.com/products/observability). The following example flow triggers a daily batch and exports to Google Cloud Platform's observability monitor:
+This example exports logs to [Google Cloud Observability](https://cloud.google.com/products/observability):
```yaml
id: log_shipper
@@ -233,7 +235,7 @@ tasks:
projectId: my-gcp-project
```
-This example exports logs to [Google Cloud Storage](https://cloud.google.com/storage?hl=en). The following example flow triggers a daily batch and exports to Google Cloud Storage:
+This example exports logs to [Google Cloud Storage](https://cloud.google.com/storage?hl=en):
```yaml
id: log_shipper
@@ -261,7 +263,7 @@ tasks:
### Azure Monitor
-This example exports logs to [Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/overview). The following example flow triggers a daily batch and export to Azure Monitor:
+This example exports logs to [Azure Monitor](https://learn.microsoft.com/en-us/azure/azure-monitor/overview):
```yaml
id: log_shipper
@@ -292,7 +294,7 @@ tasks:
### Azure Blob Storage
-This example exports logs to [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs/). The following example flow triggers a daily batch and export to Azure Blob Storage:
+This example exports logs to [Azure Blob Storage](https://azure.microsoft.com/en-us/products/storage/blobs/):
```yaml
id: log_shipper
@@ -323,7 +325,7 @@ tasks:
### Elasticsearch
-This example exports logs to [Elasticsearch](https://www.elastic.co). The following example flow triggers a daily batch and export to [Elasticsearch Observability platform](https://www.elastic.co/observability).
+This example exports logs to [Elasticsearch](https://www.elastic.co/observability):
```yaml
id: logShipper
@@ -355,7 +357,7 @@ tasks:
### New Relic
-This example exports logs to [New Relic](https://newrelic.com/). The following example flow triggers a daily batch and export to the [New Relic Observability Platform](https://newrelic.com/platform).
+This example exports logs to [New Relic](https://newrelic.com/platform):
```yaml
id: logShipper
@@ -382,7 +384,7 @@ tasks:
### Splunk
-This example exports logs to [Splunk](https://www.splunk.com/). The following example flow triggers a daily batch and export to [Splunk Observability Cloud](https://www.splunk.com/en_us/products/observability-cloud.html).
+This example exports logs to [Splunk Observability Cloud](https://www.splunk.com/en_us/products/observability-cloud.html):
```yaml
id: log_shipper
@@ -393,23 +395,23 @@ triggers:
type: io.kestra.plugin.core.trigger.Schedule
cron: "@daily"
- tasks:
- - id: log_export
- type: io.kestra.plugin.ee.core.log.LogShipper
- logLevelFilter: INFO
- lookbackPeriod: P1D
- offsetKey: logShipperOffset
- delete: false
- logExporters:
- - id: SplunkLogExporter
- type: io.kestra.plugin.ee.splunk.LogExporter
- host: https://example.splunkcloud.com:8088
- token: "{{ secret('SPLUNK_API_KEY') }}"
+tasks:
+ - id: log_export
+ type: io.kestra.plugin.ee.core.log.LogShipper
+ logLevelFilter: INFO
+ lookbackPeriod: P1D
+ offsetKey: logShipperOffset
+ delete: false
+ logExporters:
+ - id: SplunkLogExporter
+ type: io.kestra.plugin.ee.splunk.LogExporter
+ host: https://example.splunkcloud.com:8088
+ token: "{{ secret('SPLUNK_API_KEY') }}"
```
### OpenSearch
-This example exports logs to [OpenSearch](https://opensearch.org/) database. The following example flow triggers a daily batch and export to [OpenSearch Observability platform](https://opensearch.org/platform/observability/index.html).
+This example exports logs to the [OpenSearch observability platform](https://opensearch.org/platform/observability/index.html):
```yaml
id: log_shipper
@@ -438,7 +440,7 @@ tasks:
### OpenTelemetry
-This example exports logs to [OpenTelemetry](https://opentelemetry.io/). The following example flow triggers a daily batch and export to an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/).
+This example exports logs to an [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/):
```yaml
id: logShipper
@@ -466,10 +468,10 @@ tasks:
### Graylog
-This example exports logs to [Graylog](https://graylog.org/). The following example flow triggers a daily batch and sends logs to Graylog using a GELF HTTP input. Refer to the [Graylog Plugin Documentation](/plugins/plugin-ee-graylog) for more property details.
+This example exports logs to [Graylog](https://graylog.org/) using a GELF HTTP input. See the [Graylog Plugin Documentation](/plugins/plugin-ee-graylog) for the full property reference.
```yaml
- id: log_shipper
+id: log_shipper
namespace: system
triggers:
@@ -492,7 +494,7 @@ tasks:
### Huawei Cloud LTS
-This example exports logs to [Huawei Cloud Log Tank Service (LTS)](https://www.huaweicloud.com/intl/en-us/product/lts.html). The following example flow triggers a daily batch and ships logs to an LTS log stream. Refer to the [Huawei EE Plugin Documentation](/plugins/plugin-ee-huawei) for more property details.
+This example exports logs to [Huawei Cloud Log Tank Service (LTS)](https://www.huaweicloud.com/intl/en-us/product/lts.html). See the [Huawei EE Plugin Documentation](/plugins/plugin-ee-huawei) for the full property reference.
```yaml
id: log_shipper
@@ -583,6 +585,32 @@ tasks:
To ship [Audit Logs](../06.audit-logs/index.md) to your SIEM, use the same exporter with `AuditLogShipper` — replace `logLevelFilter` with `resources`.
+### Dash0
+
+This example exports logs to [Dash0](https://www.dash0.com/) via OTLP/HTTP. Set `endpoint` to the ingestion URL for your Dash0 region. Set `dataset` to route logs to a named dataset, or omit it to use the Dash0 `default` dataset.
+
+```yaml
+id: log_shipper
+namespace: company.team
+
+triggers:
+ - id: daily
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "@daily"
+
+tasks:
+ - id: log_export
+ type: io.kestra.plugin.ee.core.log.LogShipper
+ logLevelFilter: INFO
+ lookbackPeriod: P1D
+ logExporters:
+ - id: dash0LogExporter
+ type: io.kestra.plugin.ee.dash0.LogExporter
+ endpoint: https://ingress.eu-west-1.aws.dash0.com/v1/logs
+ authToken: "{{ secret('DASH0_AUTH_TOKEN') }}"
+ dataset: my-dataset
+```
+
## Audit log shipper
Use [Audit Log Shipper](../06.audit-logs/index.md) to ship audit logs from the Kestra backend to external destinations including Datadog, Elasticsearch, New Relic, OpenTelemetry, AWS CloudWatch, Google Operational Suite, Azure Monitor, and Syslog (CEF).
diff --git a/src/contents/docs/07.enterprise/02.governance/policies/index.md b/src/contents/docs/07.enterprise/02.governance/policies/index.md
new file mode 100644
index 00000000000..1453c89784e
--- /dev/null
+++ b/src/contents/docs/07.enterprise/02.governance/policies/index.md
@@ -0,0 +1,562 @@
+---
+title: Policies
+h1: Enforce Governance Rules with Policies
+description: Use Policies to inject, validate, and enforce plugin and flow configuration across namespaces and tenants. Policies give platform administrators centralized, observable control over how flows run.
+sidebarTitle: Policies
+icon: /src/contents/docs/icons/admin.svg
+editions: ["EE", "Cloud"]
+---
+
+Policies let platform administrators inject, validate, and enforce configuration rules across namespaces and tenants.
+
+A Policy bundles rules that inject property values, strip properties, restrict allowed values, or block non-compliant flows from saving or running.
+
+## Why Policies
+
+Without Policies, ensuring that flows across many namespaces use approved configurations requires manual coordination; authors must set values correctly on every task, and administrators have no way to enforce or verify compliance.
+
+Policies address this with a model that supports:
+
+- **Mutate rules** — inject or remove property values before execution, on flows and plugins alike
+- **Validate rules** — block or warn when flows violate configuration requirements
+- **Enforcement modes** — control whether a Policy actively blocks, only reports, or is opt-in
+- **Inheritance** — rules from parent namespaces apply to children, and children can only tighten them
+
+## Policy structure
+
+A Policy has an `id`, an optional `description`, an optional `displayName` shown in the Governance UI, an `enforcement` mode, an optional `target` that narrows which tenants or namespaces the policy applies to within its scope, and a list of `rules`.
+
+```yaml
+id: prod-cost-controls
+displayName: "Prod Cost Controls"
+description: "Cost guardrails for prod flows."
+enforcement: ACTIVE
+
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: FLOW
+ values:
+ concurrency:
+ limit: 5
+ - type: io.kestra.plugin.ee.rules.Restrict
+ on: FLOW
+ property: concurrency.limit
+ max: 10
+ errorMessage: "concurrency.limit cannot exceed 10 in prod."
+```
+
+A Policy has no `type` field of its own — each rule has a `type` that selects one of the five rule types. A single policy can mix mutate and validate rules, which lets a coherent intent (such as "all prod flows: inject a default concurrency, then block values above 10") live in one place.
+
+## Enforcement modes
+
+| Mode | Behavior |
+|------|----------|
+| `ACTIVE` | Rules are enforced on save and before execution. A `block` violation on a caller-initiated route (API, UI, webhook) returns `403` with the policy reason and creates no execution. System-initiated routes (schedule, flow trigger) skip the execution silently and log the block. Default. |
+| `EVALUATE` | Validate rule violations are reported in the Governance UI without blocking. Mutate rules (`Add`, `Delete`) are suppressed — no values are injected or removed, and the Governance UI shows no violations for them. Use this mode to audit validate rule compliance before enabling enforcement; it is not useful for auditing mutate rule coverage. |
+| `DISABLED` | The Policy is inactive. Rules are not checked. |
+| `REFERENCE` | The Policy is opt-in. It only applies to flows or tasks that explicitly list it via `policyRefs:`. Use reference policies for opt-in configuration injection (`Add` rules); validate rules are not enforced. |
+
+Start new validate policies in `EVALUATE` mode to see which flows already violate the rule before blocking saves. Once violation counts are acceptable, switch to `ACTIVE`.
+
+```yaml
+id: label-audit
+enforcement: EVALUATE
+rules:
+ - type: io.kestra.plugin.ee.rules.Require
+ on: FLOW
+ properties:
+ - labels.team
+ errorMessage: "Every flow must declare labels.team."
+```
+
+## Rule targeting
+
+Every rule has two targeting fields: `on` selects whether the rule applies to the flow itself or to plugin instances within it, and `where` filters which plugin instances match.
+
+### `on` field
+
+Each rule declares `on: FLOW` to target the flow's own properties, or `on: PLUGIN` to target every plugin instance in the flow — tasks including nested ones, triggers, and task runners.
+
+There is no flow-level `where:` filter. To target flows, use scope placement: place the Policy at the namespace that owns the subtree, and inheritance carries it down. A tenant-scoped policy applies to every flow in the tenant.
+
+### `where` clause
+
+The `where` clause narrows which plugin instances a rule applies to. It is only valid on `on: PLUGIN` rules. Each condition specifies a `field`, an `operator`, and a `value`. Multiple conditions combine with AND.
+
+| Operator | Behavior |
+|----------|----------|
+| `EQUAL_TO` | Exact match |
+| `NOT_EQUAL_TO` | Not an exact match |
+| `STARTS_WITH` | Prefix match |
+| `ENDS_WITH` | Suffix match |
+| `CONTAINS` | Substring match |
+| `IN` | Value is in a list |
+| `NOT_IN` | Value is not in a list |
+| `REGEX` | Full regular expression match |
+| `IS_NULL` | Property is null or absent |
+| `IS_NOT_NULL` | Property is set and non-null |
+
+```yaml
+# Prefix match — all Python script plugins
+where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.scripts.python
+
+# Match a specific set of plugin types
+where:
+ - field: type
+ operator: IN
+ value:
+ - io.kestra.plugin.gcp.bigquery.Query
+ - io.kestra.plugin.gcp.bigquery.Load
+ - io.kestra.plugin.gcp.bigquery.ExtractToGcs
+```
+
+`field` is a property path on the matched target — `type` is the most common, matching on the plugin's class name.
+
+## Rule types
+
+Rules come in two families: mutate rules change the resolved configuration before execution, and validate rules check it at save time and before execution.
+
+### Mutate rules
+
+Mutate rules change the resolved configuration before execution. They do not alter the stored flow YAML — the injection or deletion happens at resolution time. Every mutation is visible to the author in the flow editor's merged preview.
+
+**`io.kestra.plugin.ee.rules.Add`**: inject values into flows or plugin instances. With `override: false` (the default), the author's explicit value wins and the policy injects only when the property is absent. With `override: true`, the policy value always wins.
+
+Add rules can target flow-level properties (like `retry`, `concurrency`, or `labels`) with `on: FLOW`, or individual plugin instances with `on: PLUGIN`.
+
+```yaml
+# Inject flow-level defaults — retry, concurrency, and a team label
+- type: io.kestra.plugin.ee.rules.Add
+ on: FLOW
+ values:
+ retry:
+ type: constant
+ maxAttempts: 3
+ interval: PT1M
+ concurrency:
+ limit: 5
+ labels:
+ team: data-platform
+
+# Inject credentials into every AWS plugin instance
+- type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.aws
+ values:
+ accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
+ secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
+
+# Force a region value — override any author-supplied value
+- type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ override: true
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.aws
+ values:
+ region: eu-west-1
+```
+
+Injected `values` may contain Pebble expressions such as `{{ secret('…') }}`. These are evaluated at runtime as plugin properties.
+
+:::alert{type="info"}
+When injecting a nested object, keys from the flow and the policy are merged. If both supply a list at the same path, the policy's list replaces the flow's list entirely — lists are not combined.
+:::
+
+**`io.kestra.plugin.ee.rules.Delete`**: remove specific properties from matching flows or plugin instances at resolution time. Delete always wins over an author-supplied value, and is always visible in the merged preview.
+
+```yaml
+# Strip hardcoded credentials — a central injection policy supplies them
+- type: io.kestra.plugin.ee.rules.Delete
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.aws
+ properties:
+ - accessKeyId
+ - secretKeyId
+```
+
+:::alert{type="warning"}
+If an `Add` and a `Delete` rule both target the same property in the same scope chain, Kestra raises a conflict error at save time. Express delete-then-re-inject as a single `Add` with `override: true` instead.
+:::
+
+### Validate rules
+
+Validate rules check flow or plugin properties at save time and before execution. Each validate rule has an `action` of `block` (default) or `warn`.
+
+:::alert{type="info"}
+Validation runs after all `Add` and `Delete` rules have been applied. An `Add` rule can satisfy a `Require` rule, and a `Restrict` rule catches a bad value regardless of whether the author or a Policy supplied it.
+:::
+
+**`io.kestra.plugin.ee.rules.Deny`**: reject a plugin type wholesale. Requires `on: PLUGIN`. Matches every instance of the plugin wherever it appears — tasks, error handlers, triggers, and task runners.
+
+```yaml
+- type: io.kestra.plugin.ee.rules.Deny
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.gcp.bigquery
+ errorMessage: "BigQuery plugins are disabled — contact the data platform team."
+```
+
+**`io.kestra.plugin.ee.rules.Restrict`**: constrain a property to an allowed range or set of values. Requires at least one of: `min`, `max`, `enum`, `regex`, `equals`. If the property is absent, no violation is raised — pair with a `Require` or `Add` rule for guaranteed presence.
+
+```yaml
+# Cap concurrency on flow-level properties
+- type: io.kestra.plugin.ee.rules.Restrict
+ on: FLOW
+ property: concurrency.limit
+ max: 10
+ errorMessage: "concurrency.limit cannot exceed 10 in prod."
+
+# Allow only approved task runners
+- type: io.kestra.plugin.ee.rules.Restrict
+ on: PLUGIN
+ property: taskRunner.type
+ enum:
+ - io.kestra.plugin.scripts.runner.docker.Docker
+ - io.kestra.plugin.scripts.runner.kubernetes.Kubernetes
+ errorMessage: "Only Docker and Kubernetes task runners are allowed in prod."
+
+# Enforce a naming convention — warn only
+- type: io.kestra.plugin.ee.rules.Restrict
+ on: FLOW
+ property: id
+ regex: "^[a-z][a-z0-9]*(-[a-z0-9]+)*$"
+ action: warn
+ errorMessage: "Flow ids should be kebab-case (e.g. daily-orders-sync)."
+```
+
+**`io.kestra.plugin.ee.rules.Require`**: mandate that one or more properties are set and non-empty. Empty string, whitespace-only, `[]`, and `{}` do not satisfy the rule; `0` and `false` do.
+
+```yaml
+# Every flow must carry a team label
+- type: io.kestra.plugin.ee.rules.Require
+ on: FLOW
+ properties:
+ - labels.team
+ errorMessage: "Every flow must declare labels.team."
+
+# Every script task must declare an explicit task runner
+- type: io.kestra.plugin.ee.rules.Require
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.scripts
+ properties:
+ - taskRunner
+ errorMessage: "Every script task must declare an explicit taskRunner."
+```
+
+#### Combining Add and Require
+
+Use `Add` with `Require` when you want tasks that don't declare a property to receive a sensible default, while still blocking tasks that explicitly omit it. Validation runs after all mutate rules, so an `Add` rule can satisfy a `Require` rule.
+
+```yaml
+id: log-level-default
+enforcement: ACTIVE
+rules:
+ # Inject a default log level when the author hasn't set one
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: EQUAL_TO
+ value: io.kestra.plugin.core.log.Log
+ values:
+ level: WARN
+ # Require level to be set — satisfied by the Add rule above for tasks that omit it
+ - type: io.kestra.plugin.ee.rules.Require
+ on: PLUGIN
+ where:
+ - field: type
+ operator: EQUAL_TO
+ value: io.kestra.plugin.core.log.Log
+ properties:
+ - level
+ errorMessage: "Log tasks must declare an explicit level."
+```
+
+Log tasks without a `level` receive `WARN` from the `Add` rule, satisfying the `Require`. Tasks that explicitly set `level` keep their value.
+
+## Override behavior
+
+For `Add` rules:
+
+- `override: false` (default) — if the author sets a value, it is used. The policy injects only when the property is absent.
+- `override: true` — the policy value always wins. The replacement is annotated in the merged preview so the author can see it.
+
+When multiple policies with `override: true` apply to the same property, the outermost scope wins: a tenant-level policy with `override: true` cannot be overridden by a namespace-level policy.
+
+### Choosing between `override: true` and `Restrict(equals:)`
+
+Both approaches pin a property to a specific value. The difference is posture:
+
+| | `Add(override: true)` | `Add` + `Restrict(equals:)` |
+|-|----------------------|----------------------------|
+| Author sets a different value | Silently corrected at runtime; visible in merged preview | Rejected at save; author is told to fix it |
+| Requires | One rule | Two rules in the same policy |
+| Personas | Platform teams — "just make it right" | Compliance teams — "make the author fix it" |
+
+Both can live in the same policy.
+
+## Reference policies
+
+A Policy with `enforcement: REFERENCE` is not applied automatically. It applies only to flows or tasks that explicitly list it using `policyRefs:`. This lets teams share opt-in configuration bundles — named runner profiles, database connections, or compliance defaults that flows can adopt voluntarily.
+
+```yaml
+# Policy: db-analytics
+id: db-analytics
+description: "Analytics warehouse connection."
+enforcement: REFERENCE
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.jdbc.postgresql
+ values:
+ url: jdbc:postgresql://analytics:5432/dw
+ username: "{{ secret('DW_USER') }}"
+ password: "{{ secret('DW_PASSWORD') }}"
+```
+
+```yaml
+# Policy: db-orders
+id: db-orders
+description: "Orders OLTP connection."
+enforcement: REFERENCE
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.jdbc.postgresql
+ values:
+ url: jdbc:postgresql://orders:5432/oltp
+ username: "{{ secret('ORDERS_USER') }}"
+ password: "{{ secret('ORDERS_PASSWORD') }}"
+```
+
+```yaml
+# Attach the bundle per task — each task opts into the policy it needs
+id: orders-to-warehouse
+namespace: acme.data
+tasks:
+ - id: extract
+ type: io.kestra.plugin.jdbc.postgresql.Query
+ policyRefs:
+ - db-orders
+ - id: load
+ type: io.kestra.plugin.jdbc.postgresql.Query
+ policyRefs:
+ - db-analytics
+```
+
+`policyRefs` is valid at the flow level, on individual tasks, and on triggers. The attachment level determines which rules in the bundle activate:
+
+- **Flow-level `policyRefs`** — activates all rules in the bundle: both `on: FLOW` and `on: PLUGIN` rules apply across the entire flow.
+- **Task- or trigger-level `policyRefs`** — activates only `on: PLUGIN` rules, scoped to that plugin and its nested configuration. `on: FLOW` rules in the same bundle are filtered out at rule-scoping time and are never enforced.
+
+Flow-level `policyRefs` applies the bundle to all tasks in the flow:
+
+```yaml
+# Flow-level policyRefs — applies the bundle to all tasks
+id: daily-report
+namespace: company.team
+policyRefs:
+ - pydata-defaults
+tasks:
+ - id: transform
+ type: io.kestra.plugin.scripts.python.Script
+ script: |
+ print("done")
+```
+
+Triggers also support `policyRefs`:
+
+```yaml
+id: event-pipeline
+namespace: acme.data
+tasks:
+ - id: process
+ type: io.kestra.plugin.jdbc.postgresql.Query
+ sql: SELECT 1
+triggers:
+ - id: on-file
+ type: io.kestra.plugin.aws.s3.Trigger
+ policyRefs:
+ - aws-s3-credentials
+ bucket: my-bucket
+ prefix: "data/"
+```
+
+:::alert{type="warning"}
+If a referenced bundle contains `on: FLOW` rules and `policyRefs` is on a task or trigger, those rules are silently dropped. Place `policyRefs` at the flow level to activate them.
+:::
+
+Reference policies do not propagate through namespace inheritance — flows and tasks must opt in explicitly using `policyRefs:`. Reference policies are designed for opt-in configuration injection: use `Add` rules to supply credentials, defaults, or runner configuration that teams can adopt voluntarily. Validate rules (`Deny`, `Restrict`, `Require`) are not enforced in `REFERENCE` mode.
+
+## Visibility
+
+The Policy overview page (accessible from the tenant **Policies** menu or **Namespaces → [namespace] → Policies**) lists every flow violating the policy's validate rules, with a finding count per flow and a **Fix** button.
+
+The Fix dialog shows:
+- The policy and flow names
+- The rule action (`block` or `warn`) and your `errorMessage`
+- The rule type and the property path that triggered the violation
+- A diff preview of the suggested change
+
+`block` and `warn` findings require manual corrections — use **Open flow** to go directly to the flow editor. The **Apply** button applies auto-fixable suggestions where available.
+
+When saving a flow violates an `ACTIVE` Policy, Kestra rejects the save and shows the policy name, which rule was violated, and the admin-authored `errorMessage`. The flow is never auto-disabled; it must be corrected before it can be saved. Warnings from `action: warn` rules are surfaced separately without blocking the save.
+
+When a flow is open in the editor, the Policies panel shows a **Mutation preview** toggle that surfaces injected values from `Add` rules, `override: true` replacements, and `Delete` removals, each annotated with the source policy that supplied them.
+
+## Policy scope and inheritance
+
+Policies apply along a scope chain, from outermost to innermost:
+
+| Scope | Where defined | Who can edit |
+|---|---|---|
+| `STATIC` | `kestra.policies` in server configuration | Infrastructure / platform team |
+| `INSTANCE` | Tenant root (no specific namespace) | Instance Owner |
+| `TENANT` | Tenant-level via API or UI | Tenant admin |
+| `NAMESPACE` | Namespace-level via API or UI | Namespace admin with `POLICY` permission |
+
+An optional `target` field narrows a policy's reach within its scope. Which sub-field is valid depends on the scope:
+
+| Scope | Valid `target` sub-field | Effect |
+|---|---|---|
+| `STATIC`, `INSTANCE` | `tenants` | Apply only to the listed tenants; absent = every tenant |
+| `TENANT` | `namespaces` | Apply only to the listed namespace subtrees; absent = whole tenant |
+| `NAMESPACE` | — | `target` is not valid and will be rejected |
+
+**`target.namespaces` uses ancestor-chain matching.** Listing `analytics` covers `analytics` itself and every descendant (`analytics.finance`, `analytics.finance.reports`, and so on). Listing `analytics.finance` covers only that subtree. You do not need to list child namespaces explicitly.
+
+```yaml
+# TENANT policy scoped to the `analytics` subtree and the `ml` namespace
+id: data-team-policy
+enforcement: ACTIVE
+target:
+ namespaces:
+ - analytics
+ - ml
+rules:
+ - type: io.kestra.plugin.ee.rules.Require
+ on: FLOW
+ properties:
+ - labels.team
+ errorMessage: "Every flow must declare labels.team."
+```
+
+```yaml
+# INSTANCE policy scoped to specific tenants
+id: prod-only-controls
+enforcement: ACTIVE
+target:
+ tenants:
+ - prod
+ - staging
+rules:
+ - type: io.kestra.plugin.ee.rules.Restrict
+ on: FLOW
+ property: concurrency.limit
+ max: 20
+ errorMessage: "concurrency.limit cannot exceed 20."
+```
+
+A `target` must carry at least one entry — an empty `tenants: []` or `namespaces: []` is invalid.
+
+Policies from parent namespaces automatically apply to all child namespaces. Children can add stricter validate rules but cannot relax rules inherited from a parent.
+
+For `override: false` `Add` rules, the innermost scope wins — a namespace policy fills a property that a tenant policy left unset. For `override: true` `Add` rules, the outermost scope wins — a static or tenant policy with `override: true` cannot be overridden by a namespace policy.
+
+## Static policies
+
+Static policies are declared in server configuration under `kestra.policies`. They form the outermost scope, are cross-tenant, and are read-only through the API. Use them for installation-wide governance that no namespace or tenant can override.
+
+```yaml
+kestra:
+ policies:
+ instance-defaults:
+ description: "Global task runner and cost controls."
+ rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ override: true
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.aws
+ values:
+ region: eu-west-1
+ - type: io.kestra.plugin.ee.rules.Restrict
+ on: FLOW
+ property: concurrency.limit
+ max: 20
+ errorMessage: "concurrency.limit cannot exceed 20."
+```
+
+- `kestra.policies` is a map. The key (`instance-defaults` above) is the policy identity — do not add an `id` field inside the body.
+- A malformed static policy prevents server startup (fail-closed). Validate in a staging environment before deploying.
+
+Static policies are the replacement for the removed `kestra.plugins.defaults` server configuration key. See the [pluginDefaults Removed migration guide](../../../11.migration-guide/v2.0.0/plugin-defaults-removed/index.md) for the full conversion.
+
+## Creating and managing policies
+
+Create and manage Policies from the Kestra UI in two places:
+
+- **Tenant-level**: navigate to the tenant administration menu and open **Policies** to manage tenant-scoped policies.
+- **Namespace-level**: open **Namespaces → [your namespace] → Policies** to manage policies scoped to that namespace.
+
+You can also create and manage Policies via the API:
+
+```
+POST /api/v1/{tenant}/policies # tenant-scoped
+POST /api/v1/{tenant}/namespaces/{namespace}/policies # namespace-scoped
+```
+
+Policies can also be exported as YAML and imported into other namespaces or environments.
+
+## Verifying policy behavior
+
+### Preview the effective policy chain
+
+Before saving a flow, preview what policies will inject and which rules will flag violations:
+
+```
+POST /api/v1/{tenant}/flows/policies/preview
+```
+
+Send the flow source YAML in the request body. The response returns the mutated source with per-property attribution — each injected value is annotated with the policy ID and scope that supplied it.
+
+### Dry-run a policy against saved flows
+
+Evaluate a policy against flows already in a namespace without enforcing it:
+
+```
+GET /api/v1/{tenant}/policies/{id}/evaluate
+GET /api/v1/{tenant}/namespaces/{namespace}/policies/{id}/evaluate
+```
+
+The response lists every flow that would violate the policy's validate rules, grouped by rule. Use this to audit your compliance baseline before switching a policy from `EVALUATE` to `ACTIVE`.
+
+## RBAC
+
+Managing Policies requires the `POLICY` permission on the target resource. This permission is separate from namespace edit rights — a user who can edit flows in a namespace does not automatically have permission to manage its Policies.
+
+Permissions: `VIEW`, `CREATE`, `UPDATE`, `DELETE`.
diff --git a/src/contents/docs/07.enterprise/02.governance/promote/index.md b/src/contents/docs/07.enterprise/02.governance/promote/index.md
new file mode 100644
index 00000000000..4af7503a669
--- /dev/null
+++ b/src/contents/docs/07.enterprise/02.governance/promote/index.md
@@ -0,0 +1,146 @@
+---
+title: "Promote Flows Across Environments in Kestra"
+h1: Promote Flows Between Environments from the UI
+sidebarTitle: Promote
+icon: /src/contents/docs/icons/admin.svg
+editions: ["EE", "Cloud"]
+description: Move flows from dev to staging to production directly from the Kestra UI, with a diff review, confirmation gate, and full promotion history.
+---
+
+Promote copies a flow from one Kestra instance to another, with a source-to-target diff review and an optional confirmation gate before anything lands in production.
+
+## When to use Promote
+
+Use Promote when your team authors flows in the Kestra UI, runs separate instances per environment (dev, staging, production), and does not want to build or maintain a Git pipeline to move flows between them.
+
+If you already treat Git as the source of truth and deploy flows on merge via CI/CD, continue with that path. Promote and [Git-based deployment](../../../version-control-cicd/04.git/index.md) solve the same problem with different tradeoffs: Promote is UI-first and requires no pipeline; Git-based deployment is automated and auditable at the repository level. See [Version Control & CI/CD](../../../version-control-cicd/index.mdx) for a comparison of all deployment paths.
+
+---
+
+## Promotion targets
+
+A promotion target is a remote Kestra instance that flows are promoted into. Each target has a name, a base URL, an optional target tenant (for multi-tenant instances), and a connection mode. Targets are managed at the tenant level and are available to all users with the appropriate permissions.
+
+
+
+### Connection modes
+
+Each target uses one of two connection modes that control how the promote action reaches the remote instance.
+
+**SERVER mode**: the Kestra backend holds an encrypted API token for the remote instance. When a user promotes a flow, the source Kestra server makes the API call to the target on their behalf. Users never see or handle the token.
+
+Use SERVER mode when you want to centralize credential management and prevent users from needing direct access to the target instance.
+
+**CLIENT mode**: no token is stored on the source instance. When a user promotes a flow, they supply their own API token for the target at promote time. The browser calls the target instance directly, then reports the result back to the source for history and audit purposes. Tokens are stored in the user's browser for convenience and are never sent to the source server.
+
+Use CLIENT mode when users already have personal API tokens on the target, or when you want each promotion to be attributable to the individual user's identity on the target.
+
+
+
+### Confirmation gate
+
+Each target can optionally require explicit confirmation before any promotion runs. When **Require a confirmation gate before promoting** is enabled on a target, the UI presents the diff and requires the user to acknowledge before the flow is copied to the target. If a promote request is submitted to a gated target without confirmation, no promotion is attempted and nothing is recorded. The user must confirm and resubmit.
+
+Use gated targets for production environments where you want a deliberate review step before deployment.
+
+
+
+### Disabling a target
+
+Targets can be disabled without being deleted. A disabled target is hidden from the promote UI (users cannot select it when promoting flows) but its configuration is preserved.
+
+---
+
+## Promote a flow
+
+### From the Deploy tab
+
+Each flow has a **Deploy** tab alongside the editor. From the Deploy tab, select a target, review the diff between the local revision and what is currently running on the target, and confirm. If the target is gated, an explicit confirmation is required before the promotion proceeds.
+
+
+
+Only the flow's latest published revision can be promoted. Draft revisions are not eligible.
+
+### From the flow editor
+
+You can also promote a flow by clicking the **Promote** tab directly from the flow editor, without navigating away. The same diff review and gate confirmation apply.
+
+### Bulk promote
+
+From the flows list, select up to 100 flows and choose **Promote** from the action bar. The bulk promote dialog shows a drift summary per target (how many are out of sync, how many have never been promoted) and requires gate confirmation if any selected target has a gate enabled. Results are reported per flow; a failure on one flow does not block the others.
+
+
+
+
+
+---
+
+## Drift detection
+
+The flows list includes a **Deploy** column that shows the sync state of each flow relative to a selected target. The active target for drift comparison is selected in the flows list and persisted per user session.
+
+| State | Meaning |
+|---|---|
+| `IN_SYNC` | The target is running the same revision as the local instance |
+| `OUT_OF_SYNC` | The target has the flow but is running a different revision |
+| `NOT_PROMOTED` | The flow has never been promoted to this target |
+| `UNREACHABLE` | The target could not be reached to compare hashes |
+| `NEEDS_AUTH` | The target is CLIENT mode and no token has been provided for it |
+
+Drift is computed by comparing each flow's source YAML between the local instance and the target. Only the latest published revision is included.
+
+
+
+---
+
+## Promotion history
+
+Every promotion (including who initiated it, which revision was promoted, which target it went to, and whether a gate was confirmed) is recorded as an auditable event. The history for a specific flow is visible on its Deploy tab. For tenant-wide audit purposes, promotions are also available in [Audit Logs](../06.audit-logs/index.md), filtered by resource type `FLOW` and action `PROMOTE`.
+
+From the history, you can recompute the diff of any past promotion to see exactly what changed at the time it was deployed.
+
+
+
+---
+
+## What is and is not promoted
+
+Promote copies the flow's YAML definition from the selected revision to the target instance. It does not copy namespace-level resources. If the flow depends on KV pairs, secrets, variables, or namespace files that differ between environments, those must be managed separately on the target. See [Namespace Management](../07.namespace-management/index.md) and [Secrets Manager](../secrets-manager/index.md) for managing these resources per environment.
+
+---
+
+## Access control
+
+Promote uses two distinct RBAC resources.
+
+**Promoting flows** requires the `FLOW: PROMOTE` permission on the source flow's namespace. This permission is namespace-scoped: a user with `FLOW: PROMOTE` on `company.dev` can promote flows in that namespace, but not flows in other namespaces unless they also have the permission there.
+
+**Managing promotion targets** (creating, editing, deleting, viewing targets) requires `PROMOTION_TARGET` permissions. Unlike flow permissions, `PROMOTION_TARGET` is not namespace-scoped; it applies at the tenant level.
+
+| Action | Required permission |
+|---|---|
+| Promote a flow | `FLOW: PROMOTE` on the flow's namespace |
+| View available targets when promoting | `PROMOTION_TARGET: VIEW` |
+| Create or edit targets | `PROMOTION_TARGET: CREATE` / `UPDATE` |
+| Delete targets | `PROMOTION_TARGET: DELETE` |
+| List all targets | `PROMOTION_TARGET: LIST` |
+
+See [RBAC](../../03.auth/rbac/index.md) for how to assign permissions to roles and groups.
+
+---
+
+## Configuration
+
+### Allow insecure target URLs
+
+By default, promotion target URLs must use HTTPS. To allow HTTP URLs, for example when targets are on a private network without TLS, set the following in your Kestra configuration:
+
+```yaml
+kestra:
+ ee:
+ promote:
+ allow-insecure-url: true
+```
+
+This setting applies instance-wide. Target URL validation is enforced at create and update time.
+
diff --git a/src/contents/docs/07.enterprise/02.governance/promote/promote-bulk-dialog.png b/src/contents/docs/07.enterprise/02.governance/promote/promote-bulk-dialog.png
new file mode 100644
index 00000000000..0b53f55f562
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/promote/promote-bulk-dialog.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/promote/promote-deploy-tab-diff.png b/src/contents/docs/07.enterprise/02.governance/promote/promote-deploy-tab-diff.png
new file mode 100644
index 00000000000..7d74e782a96
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/promote/promote-deploy-tab-diff.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/promote/promote-flows-list.png b/src/contents/docs/07.enterprise/02.governance/promote/promote-flows-list.png
new file mode 100644
index 00000000000..e304a4fef3c
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/promote/promote-flows-list.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/promote/promote-gate-confirmation.png b/src/contents/docs/07.enterprise/02.governance/promote/promote-gate-confirmation.png
new file mode 100644
index 00000000000..a542602268d
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/promote/promote-gate-confirmation.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/promote/promote-history.png b/src/contents/docs/07.enterprise/02.governance/promote/promote-history.png
new file mode 100644
index 00000000000..eb08fe327dc
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/promote/promote-history.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/promote/promote-target-form.png b/src/contents/docs/07.enterprise/02.governance/promote/promote-target-form.png
new file mode 100644
index 00000000000..b3d79575367
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/promote/promote-target-form.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/promote/promote-targets-list.png b/src/contents/docs/07.enterprise/02.governance/promote/promote-targets-list.png
new file mode 100644
index 00000000000..34537b9b7a0
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/promote/promote-targets-list.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-1.png b/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-1.png
deleted file mode 100644
index a865b6f5a04..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-1.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-2.png b/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-2.png
deleted file mode 100644
index c666693c804..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-3.png b/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-3.png
deleted file mode 100644
index 2877b63bbed..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-3.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-4.png b/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-4.png
deleted file mode 100644
index 58673847c40..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-4.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-5.png b/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-5.png
deleted file mode 100644
index 86a2ebab1cc..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-5.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-6.png b/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-6.png
deleted file mode 100644
index fe9f673eb76..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-6.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-7.png b/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-7.png
deleted file mode 100644
index 60611b7eee8..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-7.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-8.png b/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-8.png
deleted file mode 100644
index a14ce2b8099..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/read-only-secrets/read-only-secrets-8.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/secrets-manager/gcp-secret-configuration.png b/src/contents/docs/07.enterprise/02.governance/secrets-manager/gcp-secret-configuration.png
deleted file mode 100644
index fc66afc78cd..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/secrets-manager/gcp-secret-configuration.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/secrets-manager/index.md b/src/contents/docs/07.enterprise/02.governance/secrets-manager/index.md
index df3e8cdb364..bb4be70fe88 100644
--- a/src/contents/docs/07.enterprise/02.governance/secrets-manager/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/secrets-manager/index.md
@@ -58,17 +58,11 @@ kestra:
region: us-east-1
```
-**Enable per tenant** — toggle the setting in the **Dedicated secrets manager** configuration under the tenant settings:
+**Enable per tenant** — toggle the setting in the **Dedicated secrets manager** configuration under the tenant settings.
-
+**Enable per namespace** — toggle the setting in the **Dedicated secrets manager** section of the namespace **Edit** tab.
-**Enable per namespace** — toggle the setting in the **Dedicated secrets manager** section of the namespace **Edit** tab:
-
-
-
-Once enabled, secrets display a lock icon and the **Create New Secret** button is hidden:
-
-
+Once enabled, secrets display a lock icon and the **Create New Secret** button is hidden.
Required tags that must be set externally in read-only mode:
- `namespace` — the namespace the secret belongs to.
@@ -420,9 +414,7 @@ kestra:
2. `GOOGLE_APPLICATION_CREDENTIALS` environment variable pointing to a key file.
3. Google default application credentials (Workload Identity, GCE metadata server, gcloud CLI).
-When configuring via the Kestra UI, only `project` and `service-account` fields appear:
-
-
+When configuring via the Kestra UI, only `project` and `service-account` fields are shown.
### Read-only mode
@@ -560,13 +552,7 @@ kestra:
| `filter-on-tags` | map | No | — | Read-only mode: filter visible secrets by matching tags. |
| `excluded-tags` | map | No | — | Secrets with these tags are excluded from Kestra's view. |
-In Vault, secrets created through Kestra are stored under `TENANT_ID/NAMESPACE_PARENT/.../NAMESPACE_CHILD/SECRET_NAME`. For a tenant `internal` and namespace `company.team`:
-
-
-
-When using `root-engine: dev`, the engine path appears in Vault as:
-
-
+In Vault, secrets created through Kestra are stored under `TENANT_ID/NAMESPACE_PARENT/.../NAMESPACE_CHILD/SECRET_NAME`. For a tenant `internal` and namespace `company.team`, the path would be `internal/company/team/SECRET_NAME` under the configured `root-engine`.
### Read-only mode
@@ -1006,6 +992,224 @@ kestra:
environment: prod
```
+### Filter secrets by tags (read-only mode)
+
+When integrating an external secrets manager in read-only mode, you can filter which secrets are visible in Kestra by matching tags. Set `read-only: true` and configure `filter-on-tags` with the key/value pairs to match.
+
+:::alert{type="info"}
+AWS Secrets Manager, Azure Key Vault, and Google Secret Manager use a nested `tags` sub-key under `filter-on-tags`. All other providers accept `filter-on-tags` as a flat map of key/value pairs.
+:::
+
+```yaml
+kestra:
+ secret:
+ type: aws-secret-manager
+ read-only: true
+ aws-secret-manager:
+ filter-on-tags:
+ tags:
+ application: kestra-production
+```
+
+```yaml
+kestra:
+ secret:
+ type: azure-key-vault
+ read-only: true
+ azure-key-vault:
+ filter-on-tags:
+ tags:
+ application: kestra-production
+```
+
+```yaml
+kestra:
+ secret:
+ type: google-secret-manager
+ read-only: true
+ google-secret-manager:
+ filter-on-tags:
+ tags:
+ application: kestra-production
+```
+
+```yaml
+kestra:
+ secret:
+ type: vault
+ read-only: true
+ vault:
+ filter-on-tags:
+ application: kestra-production
+```
+
+```yaml
+kestra:
+ secret:
+ type: cyberark
+ read-only: true
+ cyberark:
+ filter-on-tags:
+ application: kestra-production
+```
+
+```yaml
+kestra:
+ secret:
+ type: doppler
+ read-only: true
+ doppler:
+ filter-on-tags:
+ application: kestra-production
+```
+
+```yaml
+kestra:
+ secret:
+ type: 1password
+ read-only: true
+ 1password:
+ filter-on-tags:
+ application: kestra-production
+```
+
+```yaml
+kestra:
+ secret:
+ type: beyondtrust
+ read-only: true
+ beyondtrust:
+ filter-on-tags:
+ application: kestra-production
+```
+
+```yaml
+kestra:
+ secret:
+ type: delinea
+ read-only: true
+ delinea:
+ filter-on-tags:
+ application: kestra-production
+```
+
+### Exclude secrets by tags (read-only mode)
+
+Use `excluded-tags` to hide secrets from Kestra based on their tags. Any secret whose tags match at least one key-value pair in `excluded-tags` is excluded from Kestra's view, even if it would otherwise be included by `filter-on-tags`. This filter applies only when `read-only: true` is set.
+
+When both `filter-on-tags` and `excluded-tags` are configured, a secret must match all entries in `filter-on-tags` and must not match any entry in `excluded-tags`.
+
+The following examples exclude secrets tagged `hidden: "true"` for each supported provider:
+
+```yaml
+kestra:
+ secret:
+ type: aws-secret-manager
+ read-only: true
+ aws-secret-manager:
+ excluded-tags:
+ hidden: "true"
+```
+
+```yaml
+kestra:
+ secret:
+ type: azure-key-vault
+ read-only: true
+ azure-key-vault:
+ excluded-tags:
+ hidden: "true"
+```
+
+```yaml
+kestra:
+ secret:
+ type: google-secret-manager
+ read-only: true
+ google-secret-manager:
+ excluded-tags:
+ hidden: "true"
+```
+
+```yaml
+kestra:
+ secret:
+ type: vault
+ read-only: true
+ vault:
+ excluded-tags:
+ hidden: "true"
+```
+
+```yaml
+kestra:
+ secret:
+ type: cyberark
+ read-only: true
+ cyberark:
+ excluded-tags:
+ hidden: "true"
+```
+
+```yaml
+kestra:
+ secret:
+ type: doppler
+ read-only: true
+ doppler:
+ excluded-tags:
+ hidden: "true"
+```
+
+```yaml
+kestra:
+ secret:
+ type: 1password
+ read-only: true
+ 1password:
+ excluded-tags:
+ hidden: "true"
+```
+
+```yaml
+kestra:
+ secret:
+ type: beyondtrust
+ read-only: true
+ beyondtrust:
+ excluded-tags:
+ hidden: "true"
+```
+
+```yaml
+kestra:
+ secret:
+ type: delinea
+ read-only: true
+ delinea:
+ excluded-tags:
+ hidden: "true"
+```
+
+:::alert{type="info"}
+AWS Secrets Manager does not support negative tag filtering in its `ListSecrets` API. Kestra evaluates `excluded-tags` client-side after fetching the secret list from AWS. For CyberArk, Doppler, 1Password, Vault, BeyondTrust, and Delinea, both `filter-on-tags` and `excluded-tags` are also evaluated client-side.
+:::
+
+### Filter secrets by prefix (AWS, read-only mode)
+
+For AWS Secrets Manager, you can filter secrets by a name prefix when using read-only mode. Use `filter-on-prefix.prefix` to select secrets whose names start with the given prefix and `filter-on-prefix.keep-prefix` to control whether the prefix is kept in the Kestra secret key.
+
+```yaml
+kestra:
+ secret:
+ type: aws-secret-manager
+ read-only: true
+ aws-secret-manager:
+ filter-on-prefix:
+ prefix: prod_
+ keep-prefix: true
+```
+
### Secret caching
For cloud-backed secrets managers, enable caching to reduce the number of API calls per execution.
diff --git a/src/contents/docs/07.enterprise/02.governance/secrets-manager/kv-secret-engine.png b/src/contents/docs/07.enterprise/02.governance/secrets-manager/kv-secret-engine.png
deleted file mode 100644
index 3ecef50faf1..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/secrets-manager/kv-secret-engine.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/secrets-manager/read-only-secrets-1.png b/src/contents/docs/07.enterprise/02.governance/secrets-manager/read-only-secrets-1.png
deleted file mode 100644
index a865b6f5a04..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/secrets-manager/read-only-secrets-1.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/secrets-manager/read-only-secrets-4.png b/src/contents/docs/07.enterprise/02.governance/secrets-manager/read-only-secrets-4.png
deleted file mode 100644
index 58673847c40..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/secrets-manager/read-only-secrets-4.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/secrets-manager/read-only-secrets-8.png b/src/contents/docs/07.enterprise/02.governance/secrets-manager/read-only-secrets-8.png
deleted file mode 100644
index a14ce2b8099..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/secrets-manager/read-only-secrets-8.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/secrets-manager/secret-structure.png b/src/contents/docs/07.enterprise/02.governance/secrets-manager/secret-structure.png
deleted file mode 100644
index 2c256d4aedf..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/secrets-manager/secret-structure.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/tenants/index.md b/src/contents/docs/07.enterprise/02.governance/tenants/index.md
index c0e81702e8e..229e7831f12 100644
--- a/src/contents/docs/07.enterprise/02.governance/tenants/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/tenants/index.md
@@ -9,13 +9,13 @@ version: ">= 0.13.0"
docId: tenants
---
-How to enable multi-tenancy in your Kestra instance.
+Multi-tenancy lets you run isolated environments for different teams, projects, or customers within a single Kestra instance.
-## Multi-tenancy – configure and manage tenants
+## Multi-tenancy — configure and manage tenants
A tenant represents an **isolated environment within a single Kestra instance**.
@@ -29,11 +29,11 @@ All resources (such as [flows](../../../05.workflow-components/01.flow/index.md)
Data stored inside the internal storage is also separated by tenants.
-End-users can use the tenant selection dropdown menu from the [UI](../../../09.ui/index.mdx) to see tenants they have access to. It allows users to switch between tenants easily. Each UI page includes the tenant ID in the URL (e.g., `https://demo.kestra.io/ui/yourTenantId/executions/namespace/flow/executionId`.)
+Users switch between tenants using the tenant dropdown in the bottom-left corner of the UI. The dropdown shows every tenant the user has access to; the active tenant is indicated with a checkmark. Each UI page includes the tenant ID in the URL (e.g., `https://demo.kestra.io/ui/yourTenantId/executions/namespace/flow/executionId`).
-
+
-Most [API](../../../api-reference/index.mdx) endpoints also include the tenant identifier. The exception to that is instance-level endpoints such as `/configs`, `/license-info` or `/banners` that require Superadmin access.
+Most [API](../../../api-reference/index.mdx) endpoints also include the tenant identifier. The exception to that is instance-level endpoints such as `/configs`, `/license-info` or `/banners` that require Instance Owner access.
For example, the URL of the API operation to list flows of the `products` namespace is `/api/v1/{your_tenant_id}/flows/products`. You can check the [Enterprise Edition API Guide](../../../api-reference/01.enterprise/index.mdx) for more information.
@@ -47,21 +47,17 @@ Tenants must be created upfront, and a user needs to be granted access to use a
4. **Intuitive UI Navigation**: the UI provides a dropdown as well as tenant identifiers included in the URL to make switching between tenants seamless.
-## Creating and Managing Tenants
+## Creating and managing tenants
-Tenants in Kestra can be managed in various ways: from the UI, CLI, API, or Terraform.
+Tenants are created and managed through the **Instance Owner console** — only users with the Instance Owner privilege can create, edit, or delete tenants. The console is accessible from **Instance Owner → Tenants** in the UI. Tenants can also be managed via the CLI, API, or Terraform.
-### Creating a Tenant from the UI
+### Creating a tenant from the UI
-Tenants can be created and managed directly through Kestra's user interface. Go to **Instance -> Tenants**. Then, click on the **Create** button:
-
-
-Fill in the form and click **Save**:
-
+Go to **Instance Owner → Tenants**, click **Create**, fill in the form, and click **Save**.
The user who creates a tenant is automatically granted the Admin Role for that tenant. You may need to refresh the UI to see updated Roles.
-### Creating a Tenant from the CLI
+### Creating a tenant from the CLI
Kestra provides CLI commands for tenant creation. The following command creates a tenant with the identifier `stage` and the name `Staging`:
@@ -99,7 +95,7 @@ create a tenant and assign admin roles to an existing admin user
-V, --version Print version information and exit.
```
-### Creating a Tenant from the API
+### Creating a tenant from the API
Tenants can be managed programmatically via Kestra's [API](../../../api-reference/01.enterprise/index.mdx#post-/api/v1/tenants). Here is an example of an API call for creating a tenant:
@@ -110,7 +106,7 @@ curl -X POST "https://demo.kestra.io/api/v1/tenants" \
-d "{ \"id\": \"stage\", \"name\": \"staging\", \"deleted\": false}"
```
-### Creating a Tenant from Terraform
+### Creating a tenant from Terraform
Tenants can be managed via Infrastructure as Code using [Kestra's Terraform provider](../../../13.terraform/resources/tenant/index.md).
@@ -141,21 +137,15 @@ Key-value pairs and namespace files will not be deleted as they are persisted in
Regardless of which of the above methods you use to create a tenant, the User who creates the tenant automatically gets the Admin Role assigned. That role grants admin rights to that user on that tenant.
-Note that there is an exception to this rule if a tenant is created by a Superadmin. In that case, the Superadmin has to explicitly assign the Admin Role for that tenant to themselves or any other User, Service Account, or Group.
+Note that there is an exception to this rule if a tenant is created by an Instance Owner. In that case, the Instance Owner has to explicitly assign the Admin Role for that tenant to themselves or any other User, Service Account, or Group.
### Dedicated storage and secrets backend per tenant
By default, each tenant uses the same [runtime and storage configuration](../../../configuration/02.runtime-and-storage/index.md) and [secrets backend](../secrets-manager/index.md) configured for your Kestra instance. If you need more isolation, you can configure a dedicated storage and secrets backend per tenant. This can be useful if each of your tenants serves different customers and you need to ensure complete data isolation between them.
-To configure a dedicated storage and secrets backend per tenant, navigate to the **Instance - Tenants** in the UI and click on the **Details** button of the tenant you'd like to configure. Then, select the storage and secrets backend you want to use for that tenant:
-
-
-
-For storage configuration examples, refer to [Runtime and Storage](../../../configuration/02.runtime-and-storage/index.md) in the configuration guide.
-
-
+To configure a dedicated storage and secrets backend per tenant, open the tenant's **Settings** page and scroll to the **Dedicated internal storage** and **Dedicated secrets manager** sections. Each section has a storage type selector and a YAML configuration editor.
-For the different secret managers' configurations, refer to the [Secret Managers documentation](../secrets-manager/index.md).
+For storage configuration examples, refer to [Runtime and Storage](../../../configuration/02.runtime-and-storage/index.md). For secrets backend options, refer to the [Secret Managers documentation](../secrets-manager/index.md).
:::alert{type="warning"}
Make sure to use `camelCase` notation. For example, if you want to use the `GCS` storage backend, you should use `projectId` as the value rather than `project-id`.
diff --git a/src/contents/docs/07.enterprise/02.governance/tenants/tenant-create-2.png b/src/contents/docs/07.enterprise/02.governance/tenants/tenant-create-2.png
deleted file mode 100644
index 3c9fbffcc88..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/tenants/tenant-create-2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/tenants/tenant-create.png b/src/contents/docs/07.enterprise/02.governance/tenants/tenant-create.png
deleted file mode 100644
index 49d071e5604..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/tenants/tenant-create.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/tenants/tenant-switcher.png b/src/contents/docs/07.enterprise/02.governance/tenants/tenant-switcher.png
new file mode 100644
index 00000000000..1fab9f9f8f3
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/tenants/tenant-switcher.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/tenants/tenants-dedicated-internal-storage.png b/src/contents/docs/07.enterprise/02.governance/tenants/tenants-dedicated-internal-storage.png
deleted file mode 100644
index d5e77afa300..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/tenants/tenants-dedicated-internal-storage.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/tenants/tenants-dedicated-secrets-manager.png b/src/contents/docs/07.enterprise/02.governance/tenants/tenants-dedicated-secrets-manager.png
deleted file mode 100644
index a461470b78a..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/tenants/tenants-dedicated-secrets-manager.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/tenants/tenants.png b/src/contents/docs/07.enterprise/02.governance/tenants/tenants.png
deleted file mode 100644
index 2c474441a15..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/tenants/tenants.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/unit-tests/index.md b/src/contents/docs/07.enterprise/02.governance/unit-tests/index.md
index f1d9af0ee7c..05532783764 100644
--- a/src/contents/docs/07.enterprise/02.governance/unit-tests/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/unit-tests/index.md
@@ -8,9 +8,7 @@ editions: ["EE", "Cloud"]
version: ">= 0.23.0"
---
-Build tests to ensure proper flow behavior.
-
-Tests let you verify that your flow behaves as expected, without cluttering your instance with test executions that run every task. For example, a unit test designed to mock the notification task of a flow ensures the configuration is correct without spamming dummy notifications to the recipient. They also let you isolate testing to specific changes to a task, rather than executing the entire flow.
+Unit tests verify that flows behave as expected without running every task or producing real side effects — for example, mocking a notification task to confirm the configuration is correct without sending dummy alerts, or isolating a single changed task without re-executing the whole flow.
@@ -26,18 +24,17 @@ Unit tests are configured for and connected to their respective flows. To create
Once tests are created, they can all be viewed from the **Tests** tab with their respective Id, Namespace, Tested Flow, and current State listed. Additionally, tests can be run from this view with expandable results.
-
-
The following diagram illustrates the structure of flows and unit tests together in Kestra:
## Configuration
-Unit tests are written in YAML like flows. A test is made up of `testCases`, and each test case is made up of `fixtures` and `assertions`. Fixtures can target **files**, **inputs**, **tasks**, or **triggers** depending on what you need to mock or override. Like flows, you can write unit tests as code, in No Code, or with the [AI Copilot](../../../ai-tools/ai-copilot/index.md).
+Unit tests are written in YAML like flows. A test is made up of `testCases`, and each test case is made up of `fixtures`, `assertions`, and an optional `expectedState`. Fixtures can target **files**, **inputs**, **tasks**, or **triggers** depending on what you need to mock or override. Like flows, you can write unit tests as code, in No Code, or with the [AI Copilot](../../../ai-tools/01.ai-copilot/index.md).
- A **fixture** refers to the setup required before a test runs, such as initializing objects or configuring environments, to ensure the test has a consistent starting state.
- An **assertion** is a statement that checks if a specific condition is true during the test. If the condition is false, the test fails, indicating an issue with the code being tested, while true indicates the expectation is met.
+- **expectedState** sets the terminal state the flow must reach for the test to pass. It defaults to `SUCCESS`; set it to `FAILED`, `WARNING`, `KILLED`, or any other valid state to test intentional failure paths. Note that `assertions` is required on every test case — a case with only `expectedState` and no `assertions` is rejected.
Common fixture types:
- **files**: provide inline files or namespace file URIs the flow can read.
@@ -131,21 +128,15 @@ The `id` is unique to the test suite, and the `namespace` and `flowId` must matc
In the first test case, `extract_should_return_data`, the `fixtures` include tasks to replace the Slack alert and BigQuery data load so as to not clutter a Slack channel with test alert messages or a BigQuery table with test data but still test the overall design of the flow.
-The `assertions` property defines the conditions for success or failure. In the example, the test aims to ensure that the outputs from the `transform_to_uppercase` task are not null. After running the test, we can see the results for the `extract_should_return_data` test by expanding the results.
-
-
-
-The assertion passed as the `extract` task downloading data from the API returned product names and was not null. Additionally, since we did not include a fixture for the `transform_to_uppercase` task, we can see that the returned product names were also transformed successfully to uppercase in the assertion's actual result.
+The `assertions` property defines the conditions for success or failure. In the example, the test aims to ensure that the outputs from the `transform_to_uppercase` task are not null.
-Because we wrote the test suite with two test cases, both executed during the run. For more isolation, you could separate test cases into multiple tests of the flow as needed. While we know from the previous test that the uppercase transformation was successful, you may not want to extract actual data during testing, as it could add load to an external service or send unnecessary alerts. To mitigate this and solely test the transformation, we added the `extract` and `transform_to_products_name` fixtures in the second test case, `extract_should_transform_product_names_to_uppercase_mocked`. The `extract` fixture prevents the API call, and the `transform_to_products_name` fixture simulates the return of the flow task with a mock output, `my-product-1`, all in lowercase.
+Because we wrote the test suite with two test cases, both execute during the run. For more isolation, you could separate test cases into multiple tests of the flow as needed. While we know from the first test that the uppercase transformation was successful, you may not want to extract actual data during testing, as it could add load to an external service or send unnecessary alerts. To mitigate this, the second test case `extract_should_transform_product_names_to_uppercase_mocked` mocks `extract` to prevent the API call and mocks `transform_to_products_name` to return `my-product-1` in lowercase — then asserts that `transform_to_uppercase` produced `MY-PRODUCT-1`.
-After running, we can see that the assertion was successful and the actual result `MY-PRODUCT-1` was successfully transformed and matches the expected result defined in the `assertions` property of the test.
+After running, both assertions pass with their actual and expected values visible in the expanded results:
-
+
-Execution details are not stored in the Executions page like normally run flows to avoid cluttering that space with unnecessary execution details. To view an execution made from a test, you can open the test case and click on the link for the ExecutionId.
-
-
+Execution details are not stored in the Executions page like normally run flows to avoid cluttering that space with unnecessary execution details. To view an execution made from a test, open the test case and click the ExecutionId link.
## Unit test with a namespace file
@@ -242,7 +233,7 @@ testCases:
description: "don't send end output"
assertions:
- value: "{{outputs.transform_to_uppercase.value}}"
- equalsTo: "[BOWLER HAT, TRILBY HAT]"
+ equalTo: "[BOWLER HAT, TRILBY HAT]"
```
With a combination of namespace files and tests, you can target specific components of your flow for correct functionality without using up any external resources or unnecessarily communicating with external hosts for scripts or files.
@@ -419,6 +410,47 @@ In this example:
This approach allows you to test the complete flow logic while avoiding the overhead and complexity of executing actual scripts during testing.
+## Assert expected failure state
+
+Some flows are designed to fail when conditions are not met — for example, a validation guard that uses `io.kestra.plugin.core.execution.Fail` to reject invalid inputs. The `expectedState` property on a test case lets you assert that a flow ends in a specific terminal state. It defaults to `SUCCESS`; set it to `FAILED`, `WARNING`, `KILLED`, or any other valid state.
+
+The following flow fails when the supplied quantity is not positive:
+
+```yaml
+id: order_validation
+namespace: company.team
+
+inputs:
+ - id: quantity
+ type: INT
+
+tasks:
+ - id: validate_quantity
+ type: io.kestra.plugin.core.execution.Fail
+ condition: "{{ inputs.quantity <= 0 }}"
+ errorMessage: "Order quantity must be greater than zero"
+```
+
+The test asserts that passing a negative value causes the expected failure:
+
+```yaml
+id: order_validation_tests
+namespace: company.team
+flowId: order_validation
+testCases:
+ - id: invalid_quantity_should_fail
+ type: io.kestra.core.tests.flow.UnitTest
+ expectedState: FAILED
+ fixtures:
+ inputs:
+ quantity: -1
+ assertions:
+ - value: "{{ inputs.quantity }}"
+ lessThan: 1
+```
+
+When `expectedState` is set, the test passes only if the execution ends in exactly that state. If it ends in a different state, the test fails and reports both the expected and actual states.
+
## Available assertion operators
While the above example uses `isNotNull` and `contains` as assertion operators, there are many more that can be used when designing unit tests for your flows. The complete list is as follows:
diff --git a/src/contents/docs/07.enterprise/02.governance/unit-tests/test-case-1.png b/src/contents/docs/07.enterprise/02.governance/unit-tests/test-case-1.png
deleted file mode 100644
index 3caee1fadf5..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/unit-tests/test-case-1.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/unit-tests/test-case-2.png b/src/contents/docs/07.enterprise/02.governance/unit-tests/test-case-2.png
deleted file mode 100644
index e1c6e4853e6..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/unit-tests/test-case-2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/unit-tests/test-cases.png b/src/contents/docs/07.enterprise/02.governance/unit-tests/test-cases.png
new file mode 100644
index 00000000000..a9e36da22d8
Binary files /dev/null and b/src/contents/docs/07.enterprise/02.governance/unit-tests/test-cases.png differ
diff --git a/src/contents/docs/07.enterprise/02.governance/unit-tests/test-execution.png b/src/contents/docs/07.enterprise/02.governance/unit-tests/test-execution.png
deleted file mode 100644
index 5b75fd3509f..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/unit-tests/test-execution.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/unit-tests/unit-test-interface.png b/src/contents/docs/07.enterprise/02.governance/unit-tests/unit-test-interface.png
deleted file mode 100644
index 688525ba2fc..00000000000
Binary files a/src/contents/docs/07.enterprise/02.governance/unit-tests/unit-test-interface.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/02.governance/worker-isolation/index.md b/src/contents/docs/07.enterprise/02.governance/worker-isolation/index.md
index f2a9362bed6..0c30f9b1714 100644
--- a/src/contents/docs/07.enterprise/02.governance/worker-isolation/index.md
+++ b/src/contents/docs/07.enterprise/02.governance/worker-isolation/index.md
@@ -7,9 +7,9 @@ icon: /src/contents/docs/icons/admin.svg
editions: ["EE"]
---
-How to configure worker isolation in Kestra.
+Configure worker isolation to separate file systems, restrict thread creation, and enforce script task isolation in multi-tenant Kestra instances.
-## Worker isolation – enforce separation
+## Worker isolation — enforce separation
When dealing with multiple teams, you can add extra security measures to your Kestra instance to isolate access so that there is no shared file system, only certain plugins can create worker threads, and script tasks are isolated.
@@ -61,35 +61,29 @@ Currently, all the official Kestra plugins are safe to be authorized **except**
## Scripting isolation
-You can provide global plugin defaults using the `kestra.plugins.defaults` configuration. Those will be applied to each task on your cluster **if a property is not defined** on flows or tasks. Plugin defaults ensure a property is defined at a default value for these tasks.
+Use a [Policy](../policies/index.md) to enforce Docker isolation for script tasks. For installation-wide enforcement, declare a static policy in server configuration (Enterprise Edition):
```yaml
kestra:
- plugins:
- defaults:
- - type: io.kestra.plugin.core.log.Log
- values:
- level: ERROR
+ policies:
+ - id: enforce-docker-isolation
+ description: "Force Docker isolation for all shell script tasks."
+ rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ override: true
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.scripts.shell
+ values:
+ containerImage: ubuntu:latest
+ taskRunner:
+ type: io.kestra.plugin.scripts.runner.docker.Docker
```
-For [Bash tasks](/plugins/plugin-script-shell/io.kestra.plugin.scripts.shell.script) and other script tasks in the core, we advise you to force `io.kestra.plugin.scripts.runner.docker.Docker` isolation and to configure global cluster `pluginDefaults`:
-
-```yaml
-kestra:
- tasks:
- defaults:
- - type: io.kestra.plugin.scripts.shell.Commands
- forced: true
- values:
- containerImage: ubuntu:latest
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
-```
-
-Forced plugin defaults:
-- Ensure a property is set globally for a task, and no task can override it.
-- Are critical for security and governance — for example, to enforce Shell tasks to run as Docker containers.
+Static policies apply across all tenants and cannot be overridden by namespace-level policies, making them suitable for cluster-wide security requirements.
:::alert{type="warning"}
-You will need to add all script plugins tasks (like Python and Node) to be sure that no tasks can bypass the docker isolation.
+Add rules for all script plugin types (Python, Node, and others) to ensure no tasks can bypass Docker isolation.
:::
diff --git a/src/contents/docs/07.enterprise/03.auth/04.authentication/index.md b/src/contents/docs/07.enterprise/03.auth/04.authentication/index.md
index 0ae71cec14c..65d0777827a 100644
--- a/src/contents/docs/07.enterprise/03.auth/04.authentication/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/04.authentication/index.md
@@ -7,20 +7,13 @@ icon: /src/contents/docs/icons/admin.svg
editions: ["EE", "Cloud"]
---
-How to configure authentication for your Kestra instance.
-
-## Authentication – configure login and OIDC
+Kestra supports two authentication methods: Basic Auth (enabled by default) and OpenID Connect (OIDC).
-Kestra provides two authentication methods:
-
-- Basic Auth – enabled by default
-- OpenID Connect (OIDC)
-
-By default, JWT token security is configured to use the default Kestra encryption key. If you haven't already configured it, generate a secret that is at least 256 bits and add it to your [Kestra Security and Secrets configuration](../../../configuration/05.security-and-secrets/index.md) as follows:
+Kestra uses the default encryption key for JWT sessions. Generate a secret of at least 256 bits and add it to your [Kestra Security and Secrets configuration](../../../configuration/05.security-and-secrets/index.md):
```yaml
kestra:
@@ -30,7 +23,7 @@ kestra:
This secret must be the same across all your webserver instances and will be used to sign the JWT cookie and encode the refresh token.
-If you want to use different keys, you can configure the key using the following configuration:
+To use separate keys for the signature and refresh token:
```yaml
micronaut:
@@ -48,9 +41,7 @@ micronaut:
:::alert{type="info"}
-**JWT configuration**
-
-It is possible to change the JWT cookie behavior using [Micronaut Cookie Token Reader](https://micronaut-projects.github.io/micronaut-security/latest/guide/#cookieToken) configuration. For example, you can define the cookie's maximum lifetime using `micronaut.security.token.cookie.cookie-max-age: P2D`.
+You can change the JWT cookie behavior using [Micronaut Cookie Token Reader](https://micronaut-projects.github.io/micronaut-security/latest/guide/#cookieToken) configuration. For example, define the cookie's maximum lifetime with `micronaut.security.token.cookie.cookie-max-age: P2D`.
:::
## Basic authentication
@@ -61,17 +52,17 @@ The default installation comes with no users defined. To create an administrator
./kestra auths users create --admin --username= --password= --tenant=
```
-If you do not have multi-tenancy enabled, you can omit the `--tenant` parameter.
+Without multi-tenancy, omit the `--tenant` parameter.
:::alert{type="info"}
-Multi-tenancy is enabled by default, so make sure to include the `--tenant` parameter.
+Multi-tenancy is enabled by default. Include the `--tenant` parameter.
:::
## Single sign-on (SSO)
Single Sign-On (SSO) is an authentication process that allows users to access multiple applications with one set of login credentials (e.g., Sign in with Google). Kestra supports SSO using the OpenID Connect (OIDC) protocol, which is a simple identity layer built on top of the OAuth 2.0 protocol.
-To enable OIDC in the application, make sure to enable OIDC in Micronaut:
+Enable OIDC in your Micronaut configuration:
```yaml
micronaut:
@@ -86,6 +77,4 @@ micronaut:
issuer: "{{ issuerUrl }}"
```
-More information can be found in the [Micronaut OIDC configuration](https://micronaut-projects.github.io/micronaut-security/latest/guide/#openid-configuration).
-
-Check the [Single Sign-On documentation](../sso/index.md) for more details on how to configure SSO with Google, Microsoft, and other providers.
+See the [Micronaut OIDC configuration guide](https://micronaut-projects.github.io/micronaut-security/latest/guide/#openid-configuration) for full details. See [Single Sign-On](../sso/index.md) to configure SSO with Google, Microsoft, and other providers.
diff --git a/src/contents/docs/07.enterprise/03.auth/api-tokens/api-token.png b/src/contents/docs/07.enterprise/03.auth/api-tokens/api-token.png
deleted file mode 100644
index 46b337b75ad..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/api-tokens/api-token.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/api-tokens/api-token2.png b/src/contents/docs/07.enterprise/03.auth/api-tokens/api-token2.png
deleted file mode 100644
index 0d281d41c46..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/api-tokens/api-token2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/api-tokens/api-token3.png b/src/contents/docs/07.enterprise/03.auth/api-tokens/api-token3.png
deleted file mode 100644
index 8d3a99280c4..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/api-tokens/api-token3.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/api-tokens/copy-and-save.png b/src/contents/docs/07.enterprise/03.auth/api-tokens/copy-and-save.png
deleted file mode 100644
index 56000f5ca1b..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/api-tokens/copy-and-save.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/api-tokens/create-api-token.png b/src/contents/docs/07.enterprise/03.auth/api-tokens/create-api-token.png
deleted file mode 100644
index 96c7ef4f06c..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/api-tokens/create-api-token.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/api-tokens/index.md b/src/contents/docs/07.enterprise/03.auth/api-tokens/index.md
index 38b884bb17b..67f06481dd5 100644
--- a/src/contents/docs/07.enterprise/03.auth/api-tokens/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/api-tokens/index.md
@@ -8,19 +8,15 @@ editions: ["EE", "Cloud"]
version: ">= 0.15.0"
---
-How to manage API tokens in Kestra.
+API tokens grant programmatic access to the Kestra API for users and [service accounts](../service-accounts/index.md).
-## API tokens – manage programmatic access
-
-API tokens authenticate requests to the Kestra API. You can create an API token for a user or a [service account](../service-accounts/index.md).
-
## Where you can use API tokens
-API tokens are used anytime you want to grant programmatic access to the Kestra API. To authenticate your custom API calls, you can pass a bearer token to the request header. For example, you can use API tokens to authenticate with the Kestra API from a CI/CD pipeline or from a custom application.
+Pass a token as a `Bearer` header to authenticate any Kestra API call — from a CI/CD pipeline, a custom application, or any of the following:
- [GitHub Actions](https://github.com/kestra-io/github-actions)
- [Terraform Provider](https://registry.terraform.io/providers/kestra-io/kestra/latest/docs)
@@ -28,52 +24,22 @@ API tokens are used anytime you want to grant programmatic access to the Kestra
- [kestractl](../../../kestra-cli/kestractl/index.md)
- [Kestra API](../api/index.md)
-## How to create a User API token
-
-To create an API token, navigate to your profile in the bottom left corner of the Kestra UI and click on **+ Create API Token**.
-
-
-
-Once in your profile, click **+ Create API Token** in the **Manage your API Tokens** section.
-
-
-
-Fill in the form with the required information, including the `Name`, `Description`, and `Max age`. Once satisfied, click `Generate`:
-
-
-
-:::alert{type="info"}
-**Note:** you can configure the token to expire after a certain period of time or to never expire. Also, there is a toggle called `Extended` that automatically prolongs the token's expiration date by the specified number of days (`Max Age`) if the token is actively used. This toggle is disabled by default.
-:::
-
-Once you confirm the API token creation, the token will be generated and displayed in the UI. Make sure to copy the token and store it in a secure location, as it will not be displayed again.
-
-
-
-## How to create a Service Account API token
-
-To create an API token for a Service Account, navigate to the `Administration` section and click the `Service Accounts` page.
-
-Then, go to the `API Tokens` tab and click the `Create` button:
-
-
-
-Fill in the form with the required information including the `Name`, `Description`, and `Max age`. Once satisfied, click `Generate`:
+## How to create a user API token
-
+Two ways to reach the token creation form:
-:::alert{type="info"}
-**Note:** same as for a user token, you can configure the token to expire after a certain period of time or to never expire. Also, there is a toggle called `Extended` that will automatically prolong the token's expiration date by the specified number of days (`Max Age`) if the token is actively used. That toggle is disabled by default.
-:::
+- Click your user avatar at the bottom-left and select **Create API Token**.
+- Go to **Settings → API Tokens** and click **+ Create API Token**.
-Once you confirm the API token creation via the **Generate** button, the token will be generated and displayed in the UI. Make sure to copy the token and store it in a secure location as it will not be displayed again.
+Fill in a **Name**, optional **Description**, and **Max age** (leave blank for a non-expiring token). Enable **Extended** to automatically reset the expiry each time the token is used. Click **Generate**, then copy the token immediately — it is shown only once.
-
+## How to create a service account API token
+Go to **IAM → Service Accounts**, open the service account, click the **API Tokens** tab, and click **Create**. The form fields and expiry options are the same as for user tokens.
## How to use an API token in an API request
-To authenticate your custom API calls, pass a `Bearer` token to the request's `Authorization` header. Here is an example that will trigger a flow execution using the Kestra API:
+To authenticate your custom API calls, pass a `Bearer` token to the request's `Authorization` header. The following example triggers a flow execution via the Kestra API:
```bash
curl -X POST http://localhost:8080/api/v1/executions/dev/hello-world \
diff --git a/src/contents/docs/07.enterprise/03.auth/api-tokens/new-token-details.png b/src/contents/docs/07.enterprise/03.auth/api-tokens/new-token-details.png
deleted file mode 100644
index 7081a9250bf..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/api-tokens/new-token-details.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/api-tokens/user-api-token.png b/src/contents/docs/07.enterprise/03.auth/api-tokens/user-api-token.png
deleted file mode 100644
index e36c3abd11d..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/api-tokens/user-api-token.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/api/index.md b/src/contents/docs/07.enterprise/03.auth/api/index.md
index 7c8d89b9ceb..4f5cd9cf2a1 100644
--- a/src/contents/docs/07.enterprise/03.auth/api/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/api/index.md
@@ -7,9 +7,7 @@ icon: /src/contents/docs/icons/admin.svg
editions: ["EE", "Cloud"]
---
-How to interact with the Kestra Enterprise Edition using the API.
-
-## Kestra Enterprise API – endpoints and authentication
+The Kestra Enterprise API exposes endpoints for managing executions, flows, tenants, and more — all authenticated with API tokens.
@@ -17,15 +15,13 @@ How to interact with the Kestra Enterprise Edition using the API.
## Authentication
-To authenticate with the Kestra API, you need to create an [API token](../api-tokens/index.md). You can create it directly from the Kestra UI.
-
-Once you have your API token, use it to authenticate with the API by passing it in the `Authorization` header as a `Bearer` token.
+Authenticate with an [API token](../api-tokens/index.md). Pass the token in the `Authorization` header:
```bash
curl -X POST http://localhost:8080/api/v1/executions/company.team/hello_world \
-H "Authorization: Bearer YOUR_API_TOKEN"
```
-## Browse the API Reference
+## Browse the API reference
-For a full list of available API endpoints, refer to the [Enterprise Edition API Reference](../../../api-reference/01.enterprise/index.mdx).
+For a full list of available endpoints, see the [Enterprise Edition API Reference](../../../api-reference/01.enterprise/index.mdx).
diff --git a/src/contents/docs/07.enterprise/03.auth/credentials/index.md b/src/contents/docs/07.enterprise/03.auth/credentials/index.md
index 18f63c3f1e2..9826fcdc648 100644
--- a/src/contents/docs/07.enterprise/03.auth/credentials/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/credentials/index.md
@@ -4,24 +4,16 @@ h1: Securely Connect Flows to External Systems with Credentials
description: Authenticate to external systems securely with Kestra Credentials. Store and manage server-to-server auth tokens for use across flows and namespaces.
sidebarTitle: Credentials
icon: /src/contents/docs/icons/concepts.svg
-editions: ["EE"]
+editions: ["EE", "Cloud"]
---
-Authenticate to external systems securely.
-
-## Credentials – Server to Server authentication for Flows
-
-Credentials are a reusable way to configure server-to-server authentication credentials once and use it across tasks.
-
-Instead of embedding token minting/refresh logic in each plugin, Kestra can mint and refresh access tokens at runtime and you reference them in your workflow with a simple expression.
+Credentials let you configure server-to-server authentication once and reuse it across tasks. Instead of embedding token minting and refresh logic in each plugin, Kestra handles this at runtime and you reference the current token with a simple expression.
Many APIs are moving away from long-lived static API keys toward **short-lived tokens** (e.g. OAuth 2.0), which improves security and simplifies rotation and revocation.
For simple static values (API keys, usernames/passwords), use [Secrets](../../../06.concepts/04.secret/index.md) directly.
-Sensitive material used by a credential (client secrets, private keys, certificates) is referenced via [Secrets](../../../06.concepts/04.secret/index.md) so it can be stored in external or read-only secret managers (e.g., [Secrets Manager](../../02.governance/secrets-manager/index.md) or [Read-only Secrets](../../02.governance/secrets-manager/index.md)) and never appears in plain text in the credential config.
-
----
+Sensitive material used by a credential (client secrets, private keys, certificates) is referenced via [Secrets](../../../06.concepts/04.secret/index.md) so it can be stored in an external secret manager and never appears in plain text in the credential config.
## Availability and scope
@@ -32,8 +24,6 @@ Credentials can be accessed and created at:
During setup, Kestra lets you **test token retrieval** from the UI to ensure your configuration is correct.
----
-
## Use a credential in a flow
Use the `credential()` Pebble function to retrieve the **current access token** for a credential key.
@@ -56,8 +46,6 @@ tasks:
For non-sensitive configuration (e.g., hostnames, table names, feature flags), prefer [Variables](../../../05.workflow-components/04.variables/index.md).
----
-
## Credential types
Credentials cover common server-to-server authentication patterns, including:
@@ -69,8 +57,6 @@ Credentials cover common server-to-server authentication patterns, including:
Credentials can reference sensitive inputs via existing [Secrets](../../../06.concepts/04.secret/index.md) (e.g., client secrets, private keys, certificates), including secrets stored in an external or [read-only secrets manager](../../02.governance/secrets-manager/index.md).
----
-
## Example: Google service account with JWT Bearer
The following example shows how to use a Google Cloud service account with an OAuth2 JWT Bearer credential in Kestra.
@@ -79,7 +65,7 @@ The following example shows how to use a Google Cloud service account with an OA
In Google Cloud:
-1. Go to **IAM & Admin** -> **Service Accounts**.
+1. Go to **IAM & Admin → Service Accounts**.
2. Create a new service account and grant it only the roles required for your use case.
3. Open the service account, go to **Keys**, and create a new **JSON** key.
4. Download the JSON key file.
@@ -158,8 +144,6 @@ tasks:
If the service account has the required permissions on the target project, the request should return `200` and the project metadata in the response body.
----
-
## Token lifecycle and caching
- Tokens are **not persisted**.
@@ -171,10 +155,8 @@ If the service account has the required permissions on the target project, the r
Avoid storing long-lived secrets directly in flow YAML. Prefer credentials + secrets so Kestra can handle token minting/refresh and reduce exposure risk.
:::
----
-
## Credential hygiene
-- **Least privilege:** scope credentials to the smallest set of permissions required.
-- **Rotate regularly:** prefer short-lived tokens where possible; rotate long-lived keys.
-- **Avoid leaking values:** don’t print tokens or derived values (e.g., substrings) to logs; see [Best Practices for Secrets](../../../14.best-practices/9.secrets-management/index.md).
+- Scope credentials to the smallest set of permissions required.
+- Prefer short-lived tokens where possible; rotate long-lived keys.
+- Don’t print tokens or derived values to logs — see [Best Practices for Secrets](../../../14.best-practices/9.secrets-management/index.md).
diff --git a/src/contents/docs/07.enterprise/03.auth/invitations/index.md b/src/contents/docs/07.enterprise/03.auth/invitations/index.md
index a6c1cdc7b1f..67dab1be09c 100644
--- a/src/contents/docs/07.enterprise/03.auth/invitations/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/invitations/index.md
@@ -9,43 +9,36 @@ version: ">= 0.20.0"
docId: users
---
-Add new users to your Tenant or Instance by using the invitation process.
+Add users to a tenant or instance by invitation.
-## Invitations – onboard users
+Administrators can invite users with pre-configured RBAC permissions. If an [email server is configured](../../../configuration/03.observability-and-networking/index.md), Kestra sends the invitation link by email. Otherwise, you can copy and share the link manually.
-Administrators can invite users with pre-configured RBAC permissions. Invitations can be emailed directly, and users can set up their accounts upon acceptance.
+## How to invite users
-By default, if the [email server is configured in Kestra EE](../../../configuration/03.observability-and-networking/index.md), an email with an invitation link is sent. If the email server is not configured, you can manually share the link with invited users.
-
-## How to Invite Users
-
-1. Navigate to the **IAM** page in the **Tenant** section
-2. Click on the **Users** tab
-3. Click on the **+ Add** button
-4. Fill in the user's email address, and select the desired group or attach the role directly — optionally restricting the permission to one or more namespaces
-5. Click the **Add** button — this will send an email to the user with an invitation link, or display the link you can share with the user manually.
-
-
+1. Go to **IAM** in the sidebar and open the **Users** tab.
+2. Click **+ Add**.
+3. Enter the user's email address and select a group (via the **Groups** tab) or assign a role directly (via the **Roles** tab).
+4. Click **+ Add** to send the invitation email or display the shareable link.
:::alert{type="info"}
-You can check the box to **Create user directly (skip invitation)** if one is not required. This action is recommended only with third-party authentication such as SSO or LDAP.
+Check **Create user directly (skip invitation)** to bypass the email flow. Use this only with third-party authentication such as SSO or LDAP.
:::
-
-
## Accepting invitations
-When a user receives an invitation, they can click on the link in the email to accept it. The user will be redirected to the Kestra login page, where they set up their account (i.e., create a password), or log in using SSO if it's enabled.
+When a user receives an invitation, they can click the link in the email to accept it. The user is redirected to the Kestra login page, where they create a password or log in with SSO if enabled.
+
+If password-based login is enabled, the password they choose must satisfy the instance password policy configured under `kestra.security.basic-auth`. See [Security and Secrets configuration](../../../configuration/05.security-and-secrets/index.md) for the available password policy settings.
## Invite expiration time
-Users have 7 days to accept the invitation. After this period, the invitation will expire and must be reissued.
+Users have 7 days to accept the invitation. After this period, the invitation expires and must be reissued.
-If you want to change the default expiration time, you can do so by setting the `expireAfter` property in the `kestra.security.invitation` section of your `application.yaml` file. For example, to set the expiration time to 30 days, add the following configuration:
+To change the default expiration, set `expireAfter` in `kestra.security.invitations`. For example, to set 30 days:
```yaml
kestra:
diff --git a/src/contents/docs/07.enterprise/03.auth/invitations/invite1.png b/src/contents/docs/07.enterprise/03.auth/invitations/invite1.png
deleted file mode 100644
index 5cd3c237603..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/invitations/invite1.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/invitations/invite2.png b/src/contents/docs/07.enterprise/03.auth/invitations/invite2.png
deleted file mode 100644
index 879b2c9259f..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/invitations/invite2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/rbac/index.md b/src/contents/docs/07.enterprise/03.auth/rbac/index.md
index c7edd563b4a..7ec5b049db4 100644
--- a/src/contents/docs/07.enterprise/03.auth/rbac/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/rbac/index.md
@@ -8,28 +8,20 @@ editions: ["EE", "Cloud"]
docId: iam
---
-How to manage access and permissions to your instance.
+Kestra Enterprise supports Role-Based Access Control (RBAC) to manage access to tenants, namespaces, flows, and resources.
-
-
-
+Kestra has three types of entities:
-## RBAC – manage roles and permissions
-
-Kestra Enterprise supports Role-Based Access Control (RBAC), allowing you to manage access to Tenants, Namespaces, Flows and resources.
-
-In Kestra you will find three types of entities:
-
-* Users: Represents a **person**. To add users to your Kestra instance, you can do one of the following:
+- Users: Represent a **person**. To add users to your Kestra instance, you can do one of the following:
- [Invite users](../invitations/index.md) to your instance or tenant from the UI
- Sync users from an external identity provider using [SCIM](../scim/index.mdx)
- Create users directly using [Terraform](../../../13.terraform/index.mdx)
- Automate user onboarding, offboarding, and group management from inside a flow using [IAM tasks](../../../15.how-to-guides/iam-automation/index.md)
-* Groups: Represent a collection of **Users** and **Service Accounts**. Groups are a useful mechanism for providing the same roles to multiple Users or Service Accounts at once by binding a role to a Group.
-* Service Accounts: Represents an **application**. They are considered Users when binding Role assignments.
+- Groups: Represent a collection of **Users** and **Service Accounts**. Groups are a useful mechanism for providing the same roles to multiple users or service accounts at once by binding a role to a group.
+- Service Accounts: Represent an **application**. They are treated as users when binding role assignments.
-All theses entities can be assigned to a Role, which define what resources the User, Group, or Service Account can access. Note that these entities don’t belong to Namespaces, but their permissions can be limited to specific namespaces via Bindings (**IAM** page).
+All these entities can be assigned to a role, which defines what resources the user, group, or service account can access. These entities do not belong to namespaces, but their permissions can be limited to specific namespaces via bindings on the **IAM** page.
The image below shows the relationship between Users, Groups, Service Accounts, Roles, and Bindings:
@@ -37,197 +29,178 @@ The image below shows the relationship between Users, Groups, Service Accounts,
## Roles and Bindings
-A Role is a collection of permissions that can be assigned to Users, Service Accounts, or Groups. These permissions are defined by a combination of a **Permission** (e.g., `FLOWS`, `NAMESPACE`, `SECRET`, etc.) and an **Action** (
-e.g., `CREATE`). The **Role** itself does not grant any permissions. Through the **IAM** page, you are able to assign a Role to a User, Service Account, or Group, which creates a **Binding**.
-
-This Binding grants the permissions defined by that Role to the User, Service Account, or Group. Select any IAM entity (User, Group, etc.), and assign the desired Role. There is no limit to the number of Roles that can be bound to an entity. They can have zero, one, or more Roles attached, giving specific permissions, optionally tied to one or more namespaces; make sure to test their access with the [Impersonate](../rbac/index.md#impersonate) feature.
+A role is a collection of permissions that can be assigned to users, service accounts, or groups. Each permission is a combination of a **resource** (e.g., `FLOW`, `EXECUTION`, `SECRET`) and one or more **actions** (e.g., `EXECUTE`, `VIEW`, `DELETE`). A role alone grants nothing — it must be attached to a user, service account, or group via a **binding** on the **IAM** page.
-Once a Role has been created, you can assign that Role to Users and Groups. Optionally, when you assign the Role to an entity (User, Group, or Service Account), you can specify the Binding to a specific Namespace(s). A Binding can be optionally limited to specific namespaces. When a Binding is tied to a namespace, it automatically grants permissions to all child namespaces. For example, a User assigned to a Role specifying the `prod` namespace automatically grants access to the `prod.engineering` namespace as well. Note that you can [configure a default role](../../../configuration/05.security-and-secrets/index.md) so that all new Users are automatically assigned that Role. This is especially useful to grant a default set of permissions to all new Users who join your Kestra instance via [SSO](../sso/index.md).
+Users, service accounts, and groups can hold any number of roles simultaneously. Bindings can be scoped to one or more namespaces — scoped access automatically extends to all child namespaces (for example, binding to `prod` also grants access to `prod.engineering`). You can [configure a default role](../../../configuration/05.security-and-secrets/index.md) to assign it automatically to new users joining via [SSO](../sso/index.md). Use [Impersonate](#impersonate) to verify a user's effective permissions after assigning roles.
## Impersonate
-After assigning permissions to a User, Superadmins can impersonate Users to ensure their access is as intended. Impersonation switches your view immediately to that User's perspective and can be easily closed back to Superadmin view – a seamless way to test RBAC in one context.
+After assigning permissions to a user, Instance Owners can impersonate users to verify their access is correct. Impersonation switches your view to that user's perspective and can be closed back to the Instance Owner view at any time.
-### Permissions
-
-A Permission is a resource that can be accessed by a User or Group. Open the following to view all supported permissions:
-
-:::collapse{title="Permissions"}
-- `FLOW`
-- `EXECUTION`
-- `TEMPLATE`
-- `NAMESPACE`
-- `KVSTORE`
-- `DASHBOARD`
-- `SECRET`
-- `CREDENTIAL`
-- `GROUP`
-- `ROLE`
-- `BINDING`
-- `AUDITLOG`
-- `BLUEPRINT`
-- `IMPERSONATE`
-- `SETTING`
-- `APP`
-- `AI_COPILOT`
-- `APPEXECUTION`
-- `TEST`
-- `ASSET`
-- `USER`
-- `SERVICE_ACCOUNT`
-- `TENANT_ACCESS`
-- `INVITATION`
-- `GROUP_MEMBERSHIP`
-- `CREDENTIALS`
-- `AI_COPILOT`
-
-:::alert{type="warning"}
-The `ME` and `APITOKEN` are removed in [Kestra 0.24](../../../11.migration-guide/v0.24.0/endpoint-changes/index.md#rbac-updates)
-:::
+### Resources
+
+A resource is a category of product entity or capability that can be controlled through RBAC. Each resource has its own set of allowed actions.
+
+**Core resources** (namespace-scoped — bindings can restrict access to specific namespaces):
+
+| Resource | Description |
+|---|---|
+| `FLOW` | Flows, their revisions, graphs, and dependencies |
+| `EXECUTION` | Executions, their state, logs, outputs, and files |
+| `TRIGGER` | Triggers attached to flows |
+| `NAMESPACE` | Namespaces and their files |
+| `KVSTORE` | Key-value store entries |
+| `SECRET` | Secrets stored in the namespace |
+| `CREDENTIAL` | Credentials for external integrations (namespace-level and tenant-level) |
+
+**Apps and features** (tenant-scoped):
+
+| Resource | Description |
+|---|---|
+| `DASHBOARD` | Custom dashboards |
+| `BLUEPRINT` | Custom blueprints |
+| `APP` | Apps and their executions |
+| `TESTSUITE` | Unit tests |
+| `ASSET` | Data assets and lineage |
+| `MCP_SERVER` | MCP servers exposing flows as AI tools |
+| `COPILOT` | AI Copilot flow generation |
+
+**Administration** (tenant-scoped):
+
+| Resource | Description |
+|---|---|
+| `USER` | Users in the tenant |
+| `GROUP` | Groups and their members |
+| `ROLE` | RBAC roles |
+| `BINDING` | Role-to-entity bindings |
+| `SERVICE_ACCOUNT` | Service accounts |
+| `INVITATION` | User invitations |
+| `AUDITLOG` | Audit log entries |
+| `POLICY` | Governance policies controlling flow and task behavior (namespace-scope and tenant-scope) |
+| `SYSTEM_SETTINGS` | Instance-level settings |
+| `TENANT_SETTINGS` | Tenant-level settings |
### Actions
-An Action is the CRUD verb allowed on a given resource (Flow, Execution, Secret, KV, Namespace, etc.). Supported Actions map directly to HTTP operations:
-
-- `CREATE` → typically `POST` the resource (e.g., create a flow, secret, KV entry).
-- `READ` → `GET` to list or view the resource; no writes.
-- `UPDATE` → `PUT`/`PATCH` to modify an existing resource; cannot create new ones.
-- `DELETE` → `DELETE` to remove the resource.
-
-Example (Flows):
-- `CREATE` lets you `POST /api/v1/{tenant}/flows`
-- `READ` lets you `GET /api/v1/{tenant}/flows/*`
-- `UPDATE` lets you `PUT /api/v1/{tenant}/flows/{flowId}`
-- `DELETE` lets you `DELETE /api/v1/{tenant}/flows/delete/by-ids`
+Each resource defines its own set of allowed actions. Not every action applies to every resource.
+
+**Common actions** (available on most resources):
+
+| Action | Meaning |
+|---|---|
+| `VIEW` | Read a single item's details |
+| `LIST` | Search or browse items |
+| `CREATE` | Create a new item |
+| `UPDATE` | Modify an existing item |
+| `DELETE` | Remove an item |
+
+**Resource-specific actions:**
+
+| Resource | Additional actions |
+|---|---|
+| `FLOW` | `EXECUTE` (trigger an execution), `DISABLE`, `ENABLE`, `VALIDATE`, `EXPORT`, `IMPORT` |
+| `EXECUTION` | `RESTART`, `KILL`, `REPLAY`, `PAUSE`, `RESUME`, `CHANGE_LABELS`, `ACCESS_LOGS`, `ACCESS_OUTPUTS`, `ACCESS_FILES`, `FOLLOW` (live SSE stream), `EXPORT`, `UNQUEUE`, `FORCE_RUN` |
+| `TRIGGER` | `UNLOCK`, `RESTART`, `DISABLE`, `ENABLE`, `EXPORT`, `BACKFILL` |
+| `NAMESPACE` | `MANAGE_FILES` (all namespace file operations) |
+| `POLICY` | `EXECUTE` (dry-run evaluate a policy against its scope), `EXPORT`, `IMPORT` |
+| `APP` | `EXECUTE`, `ACCESS_FILES`, `ACCESS_LOGS` |
+| `TESTSUITE` | `EXECUTE` |
+| `AUDITLOG` | `EXPORT` |
+| `USER` | `MANAGE_GROUP_MEMBERSHIP`, `IMPERSONATE` |
+| `GROUP` | `MANAGE_MEMBERS` |
+| `COPILOT` | `USE` (only action) |
+| `SYSTEM_SETTINGS` | — (`VIEW` and `UPDATE` only; no `CREATE`, `DELETE`, or `LIST`) |
+| `TENANT_SETTINGS` | — (`VIEW` and `UPDATE` only; no `CREATE`, `DELETE`, or `LIST`) |
:::alert{type="info"}
-For a complete CRUD-to-endpoint mapping for every permission, see the [Permissions Reference](./permissions-reference/index.md).
-:::
-
-### Currently supported roles
-
-Currently, Kestra only creates an **Admin** role by default. That role grants full access to **all resources**.
-
-Apart from **Admin**, Kestra has the managed Roles: Developer, Editor, Launcher, and Viewer. Each Role's permissions can be viewed from **IAM - Roles**. Superadmins can create additional Roles with custom permission combinations in addition to Kestra-managed roles. Users can be assigned multiple Roles.
-
-## Superadmin and Admin
-
-Kestra provides two roles for managing your instance: Superadmin and Admin.
+For a complete resource-to-endpoint mapping, see the [Permissions reference](./permissions-reference/index.md).
-- Superadmin is a user type with elevated privileges for global control
-- Admin is a customizable role that grants full access to all resources (scoped to a tenant if multi-tenancy is enabled).
-
-:::collapse{title="Summary"}
-Here's a table summarizing the key differences between an Admin and a Super Admin:
-
-| Feature | Admin (scoped to a tenant if enabled) | Super Admin |
-|-------------------------------------|----------------------------------------------------|------------------------------------------------------|
-| Access Level | By default as all permissions, depends on the Role | Manages tenants and IAM across all tenants |
-| Tenant Management | No | Create/Update/Read/Delete tenants across all tenants |
-| User/Role/Group/Bindings Management | Has the permission by default | Create/Update/Read/Delete across all tenants |
-| Flow/Execution Management | Has the permission by default | No |
-| Set Super Admin privilege | No | Yes |
+If you are upgrading from Kestra 1.x, see the [RBAC action model migration guide](../../../11.migration-guide/v2.0.0/rbac-action-model/index.md) for how old CRUD permissions map to the new actions and what was dropped.
:::
-## Super Admin
-
-Super Admin is a powerful type of user. Use the role sparingly and only for use cases that require it, such as creating a new tenant, troubleshooting tenant issues, or helping a user with a problem.
-
-Without any Role or Binding, Super Admin has access to manage tenants, users, roles, and groups within a Kestra Enterprise instance. There are multiple methods to create a Superadmin user.
-
-### Through the UI
-
-When you launch Kestra for the first time, if no prior action has been made through the CLI, you will be invited to setup Kestra through the [Setup Page](../../01.overview/02.setup/index.md).
+### MCP server permissions
-This interface invites you to create your first User which will be automatically assigned the `Superadmin` privilege.
+`MCP_SERVER` is a first-class RBAC resource that controls access to [Kestra MCP servers](../../../ai-tools/03.mcp-server/index.md). Supported actions are `VIEW`, `LIST`, `CREATE`, `UPDATE`, and `DELETE`.
-### Through the CLI
+Default role assignments:
-To create a User with a Superadmin privilege from the [CLI](../../../kestra-cli/kestra-server/index.md), use the `--superadmin` option:
+| Role | Actions granted |
+|---|---|
+| Admin | All (`VIEW`, `LIST`, `CREATE`, `UPDATE`, `DELETE`) |
+| Developer / Editor | All (`VIEW`, `LIST`, `CREATE`, `UPDATE`, `DELETE`) |
+| Launcher | — (not included) |
+| Viewer | `VIEW`, `LIST` |
-```bash
-kestra auths users create admin@kestra.io TopSecret42 --superadmin
+In addition to these permissions, access to a **private** MCP server is also flow-scoped: a user can connect to a private server only if they have `FLOW: EXECUTE` on at least one namespace that contains a flow with an `McpToolTrigger` pointing at that server.
-## schema:
-kestra auths users create \
---tenant= --superadmin
-```
+### Managed roles
-To set or revoke Superadmin privileges, use the following in the CLI:
+Kestra ships five managed roles. Each role's full permission set is visible under **IAM → Roles**. Instance Owners can create additional custom roles on top of these. Users can hold multiple roles.
-```bash
-kestra auths users set-superadmin user@email.com true # (use false to revoke)
-```
+| Role | Description |
+|---|---|
+| **Admin** | All actions on all resources. |
+| **Developer** | Everything Editor has, plus: full namespace management (including file management and plugin default import), secrets, credentials, and full blueprint CRUD. For engineers who also need platform-level access. |
+| **Editor** | Full flow and execution management (create, update, delete, execute, restart, kill, etc.), triggers, KV, dashboards, apps, test suites, assets, MCP servers, settings, and Copilot. No namespace file management, no secrets or credentials, blueprint read-only. No IAM resources. |
+| **Launcher** | Execute flows and monitor executions (`EXECUTE`, `REPLAY`, `RESTART`, `CHANGE_LABELS`, `ACCESS_LOGS`, `ACCESS_OUTPUTS`, `ACCESS_FILES`, `FOLLOW`, `EXPORT`). Read-only on triggers, KV, dashboards, and assets. No flow write access, no namespace management. |
+| **Viewer** | `VIEW`, `LIST`, and `EXPORT` on flows, executions, triggers, and namespaces. Can access execution logs, outputs, files, and live-follow executions. No execution state changes (no restart, kill, replay, etc.). No write access anywhere. |
-### Configuration
+## Instance Owner and Admin
-A Super Admin can also be created from the configuration file using the configuration below:
+Kestra provides two roles for managing your instance: Instance Owner and Admin.
-```yaml
-kestra:
- security:
- superAdmin:
- username:
- password:
- tenantAdminAccess:
- -
-```
-
-For more details, check the [Security and Secrets configuration](../../../configuration/05.security-and-secrets/index.md) page.
+- Instance Owner is a user type with elevated privileges for global control.
+- Admin is a customizable role that grants full access to all resources (scoped to a tenant if multi-tenancy is enabled).
-## Grant/Revoke Super Admin permissions
+:::collapse{title="Summary"}
+Key differences between Admin and Instance Owner:
-:::alert{type="info"}
-Note that you need to be a Superadmin yourself.
+| Feature | Admin (scoped to a tenant if enabled) | Instance Owner |
+|-------------------------------------|----------------------------------------------------|------------------------------------------------------|
+| Access Level | By default as all permissions, depends on the Role | Manages tenants and IAM across all tenants |
+| Tenant Management | No | View, create, update, delete tenants across all tenants |
+| User/Role/Group/Bindings Management | Has the permission by default | View, create, update, delete across all tenants |
+| Flow/Execution Management | Has the permission by default | No |
+| Set Instance Owner privilege | No | Yes |
:::
-### Through the UI
-
-You can grant or revoke the Superadmin privilege using the switch in the User Edit page.
+## Instance Owner
-
+Instance Owner is a powerful user type with instance-wide privileges. Use it sparingly — only for tasks that require it, such as creating tenants, troubleshooting, or helping a user.
-### Through the CLI
-
-To set an existing User with a Superadmin privilege from the [CLI](../../../kestra-cli/kestra-server/index.md), use the dedicated command:
-
-```bash
-## Set a user as Super Admin
-kestra auths users set-superadmin admin@kestra.io true
+Unlike tenant-scoped roles, Instance Owner operates across all tenants and does not require any Role or Binding. Instance Owners access instance-wide controls through the [Instance Owner console](../../05.instance/00.instance-owner/index.md), which covers tenant management, instance IAM, infrastructure, and governance.
-## Revoke Super Admin privilege
-kestra auths users set-superadmin admin@kestra.io false
-```
+For how to create Instance Owner users and manage the privilege, see [Instance Owner](../../05.instance/00.instance-owner/index.md).
## Admin
-In Kestra, the notion of Admin user does not exist; instead we create an **Admin** Role with all permissions.
+Kestra has no Admin user type; Admin is a role with full permissions.
-This role can be assigned to any User, Service Account, or Group. This allows you to have different types of admins, to grant admin permissions to a whole group, and to revoke those admin permissions at any time without having to delete any group or user.
+This role can be assigned to any user, service account, or group. This allows you to have different types of admins, grant admin permissions to a whole group, and revoke those permissions at any time without deleting any group or user.
-When using multi-tenancy, Kestra assigns the Admin Role to the user who created the tenant by default.
+When using multi-tenancy, Kestra assigns the Admin role to the user who created the tenant by default.
:::alert{type="info"}
If you see an error when creating a new User or Service Account, it might be caused by a limit on your license. In that case, [reach out to us](/contact-us) to validate and optionally upgrade your license.
:::
-## Creating a User with an Admin Role
+## Creating a user with an Admin role
### Through the UI
-When launching Kestra for the first time, if no prior action has been made through the CLI, you will be invited to setup Kestra through the [Setup Page](../../01.overview/02.setup/index.md).
+When launching Kestra for the first time with no prior CLI setup, you are prompted to set up Kestra through the [Setup Page](../../01.overview/02.setup/index.md).
-This interface invites you to create the first User which will automatically create the role Admin and bind the User to the role.
+This creates the first user, automatically assigns the Admin role, and binds it.
-Later, you can create a new User or pick an existing User and assign the Admin role to it from the Access page.
+Later, create a new user or select an existing user and assign the Admin role from the **Access** tab in IAM.
### Through the CLI
-To create a User with an Admin Role from the CLI, use the `--admin` option:
+To create a user with the Admin role from the CLI, use the `--admin` option:
```bash
kestra auths users create prod.admin@kestra.io TopSecret42 --admin
@@ -235,9 +208,10 @@ kestra auths users create prod.admin@kestra.io TopSecret42 --admin
## schema:
kestra auths users create --admin
```
+
## User lockout
-Use the following configuration to change the lockout behavior after too many failed login attempts. By default, Kestra >= 0.22 will lock the user for the `lock-duration` period after a `threshold` number of failed attempts performed within the `monitoring-window` duration. The snippet below lists the default values for those properties — you can adjust them based on your preferences:
+Use the following configuration to change the lockout behavior after too many failed login attempts. By default, Kestra locks the user for the `lock-duration` period after a `threshold` number of failed attempts within the `monitoring-window` duration. The snippet below lists the default values — adjust them based on your preferences:
```yaml
kestra:
@@ -252,14 +226,14 @@ kestra:
The key attributes are:
- `threshold`: Sets the number of allowed failed attempts before a user is locked out.
-- `monitoring-window`: Defines the period during which failed login attempts are counted before triggering a lock. Super Admin can unlock the user manually by resetting their password from the user's detail page.
+- `monitoring-window`: Defines the period during which failed login attempts are counted before triggering a lock.
- `lock-duration`: Defines how long the account remains locked.
-In the above configuration, a user is allotted 10 failed login attempts in a 5-minute window before they are locked out. They must wait 30 minutes to try again, be unlocked by an Admin, or reset their password by clicking on the "Forgot password" link and following the instructions in the email.
+With the configuration above, a user gets 10 failed login attempts in a 5-minute window before lockout. They must wait 30 minutes, be unlocked by an Admin, or reset their password using the Forgot Password link. An Instance Owner can also unlock a user manually from the user's detail page.
## Change password
-If a user wants to change their password, they can do it on their profile. This page can be accessed through the profile in the bottom left corner of the UI. "Forgot Password" settings can be configured in your Kestra configuration under `basic-auth.password-reset`. Settings to consider are the cooldown time between reset requests and how many requests can be made in a given time window.
+Users can change their password from their profile, accessible via the user avatar in the bottom-left corner of the UI. "Forgot Password" settings can be configured in your Kestra configuration under `basic-auth.password-reset`. Settings to consider are the cooldown time between reset requests and how many requests can be made in a given time window.
```yaml
kestra:
@@ -272,39 +246,32 @@ kestra:
window: PT1H # Time window during which password reset requests are counted for rate limiting
```
-### Reset password (by a Super Admin)
+### Reset a password as an Instance Owner
-Kestra provides a "forgot password" functionality that your users can leverage to reset their password. This functionality is available on the login page, where users can click on the "Forgot password?" link. On top of that, a Super Admin can reset a user's password from the User Edit page by going to **Instance** - **IAM - Users**.
+Users can reset their password via the Forgot Password link on the login page. An Instance Owner can also reset a user's password from the User Edit page at **Instance Owner → Instance IAM → Users**.
-
+
## RBAC FAQ
:::collapse{title="Why is Admin a Role rather than User type?"}
-The Admin role is a collection of permissions that can be assigned to Users, Service Accounts, or Groups. This allows you to
-grant multiple users with admin permissions if needed, and you can revoke only specific admin permissions at any time
-without having to delete the user.
+The Admin role is a collection of permissions that can be assigned to users, service accounts, or groups. This lets you grant admin permissions to multiple users or groups, and revoke them at any time without deleting anything.
-Admin roles can be assumed by multiple users or groups, and some user may later be granted a lower or a higher
-permission boundary. In the same way, some users may initially be Admins but then their permission may be revoked. The
-Admin role enables all these patterns in a flexible way.
+A user can start as Admin and later have that permission narrowed or removed. Multiple users can share the Admin role, or the same user can hold multiple roles with different scopes.
-You can think of Users as **authentication** mechanism (who you are), and Roles as **authorization** mechanism (what you
-are allowed to do). Decoupling authentication from authorization allows you to grant permissions to multiple users or
-groups at once by attaching a single Role to a Group.
+Users represent authentication (who you are); roles represent authorization (what you can do). Decoupling the two lets you grant permissions to multiple users or groups at once by attaching a single role to a group.
:::
:::collapse{title="Why can't I edit an existing Binding?"}
-A Binding is an immutable object. If a Binding no longer reflects the desired permissions, you can delete the existing
-Binding and create a new one for the same User, Service Account, or Group but with different Roles and/or namespaces.
-This is a safety feature to prevent accidental changes to existing permissions.
+A binding is immutable. If a binding no longer reflects the desired permissions, delete it and create a new one for the same user, service account, or group with different roles and/or namespaces.
+This prevents accidental changes to existing permissions.
:::
:::collapse{title="What happens if you delete a Group?"}
-All Users and Service Accounts assigned to that Group will lose permissions that were binds to the groups. However, Users and Services Accounts will still exist.
+All users and service accounts in that group lose the permissions granted by bindings attached to it. The users and service accounts themselves still exist.
:::
diff --git a/src/contents/docs/07.enterprise/03.auth/rbac/permissions-reference/index.md b/src/contents/docs/07.enterprise/03.auth/rbac/permissions-reference/index.md
index 3474e72fa5a..9ebf6ccf2ec 100644
--- a/src/contents/docs/07.enterprise/03.auth/rbac/permissions-reference/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/rbac/permissions-reference/index.md
@@ -1,789 +1,914 @@
---
title: RBAC Permissions Reference for Kestra Enterprise
-h1: RBAC Permissions Mapped to API Endpoints
-description: Reference for Kestra RBAC permissions and CRUD actions mapped to API endpoints. Use this to configure precise access controls for users and service accounts.
+h1: RBAC Resources and Actions Mapped to API Endpoints
+description: Reference for Kestra RBAC resources and actions mapped to API endpoints. Use this to design least-privilege roles and troubleshoot authorization errors.
sidebarTitle: Permissions Reference
icon: /src/contents/docs/icons/admin.svg
editions: ["EE", "Cloud"]
docId: iam
---
-This reference maps each RBAC Permission and Action to the Enterprise API endpoints that enforce it. Use it to design least-privilege roles and troubleshoot authorization errors.
+This reference maps each RBAC resource and action to the API endpoints that enforce it. Use it to design least-privilege roles and troubleshoot authorization errors.
## How to read this page
-- Endpoints are grouped by Permission and CRUD Action.
-- Endpoints marked with "any action" are accessible to any user who has the permission, regardless of which CRUD action (CREATE, READ, UPDATE, or DELETE) is assigned.
-- Some endpoints require multiple permissions; notes call out additional checks.
-- Namespace bindings apply to the namespace and all child namespaces.
-
-## Permissions reference
+- Endpoints are grouped by resource and action.
+- Each resource section gates all its endpoints: a user must have at least one valid action on the resource to reach any endpoint within it. Endpoints that do not check a specific action beyond this gate are marked "any action".
+- Some endpoints require permissions on more than one resource; notes call these out.
+- Namespace-scoped resources respect namespace bindings: access is evaluated against the namespace of the target item, and child namespaces inherit access.
:::collapse{title="FLOW"}
**Scope:** Namespace
-**CRUD meaning**
-- Create: create flows or namespace files; import flows.
-- Read: view flows, revisions, tasks, graphs, dependencies; export flows; read namespace files and triggers.
-- Update: modify flow source, tasks, enable or disable flows; move namespace files.
-- Delete: delete flows or namespace files.
+**Actions and their meaning**
+- `VIEW`: read a single flow, its graph, revisions, tasks, dependencies, and expressions context.
+- `LIST`: search or browse flows.
+- `CREATE`: create a flow. Creating a flow in a namespace that does not yet exist also requires `NAMESPACE: CREATE`.
+- `UPDATE`: modify a flow's source, tasks, or graph.
+- `DELETE`: delete a flow or its revisions.
+- `EXECUTE`: trigger an execution of a flow.
+- `DISABLE`: disable a flow.
+- `ENABLE`: enable a disabled flow.
+- `VALIDATE`: validate flow YAML. (Accessible with any `FLOW` action — no dedicated check.)
+- `EXPORT`: export flows as a ZIP archive.
+- `IMPORT`: import flows from an archive.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/flows` (YAML)
-- POST `/api/v1/{tenant}/flows` (JSON, deprecated)
-- POST `/api/v1/{tenant}/flows/{namespace}` (bulk upsert; also requires UPDATE and DELETE)
-- POST `/api/v1/{tenant}/flows/import` (imports require CREATE + UPDATE per flow)
-- POST `/api/v1/{tenant}/namespaces/{namespace}/files/directory`
-- POST `/api/v1/{tenant}/namespaces/{namespace}/files`
-
-Read
-- GET `/api/v1/{tenant}/flows/{namespace}/{id}`
-- GET `/api/v1/{tenant}/flows/{namespace}/{id}/graph`
-- POST `/api/v1/{tenant}/flows/graph` (any action; no action check)
-- GET `/api/v1/{tenant}/flows/{namespace}/{id}/revisions`
-- GET `/api/v1/{tenant}/flows/{namespace}/{id}/tasks/{taskId}`
-- GET `/api/v1/{tenant}/flows/search`
-- GET `/api/v1/{tenant}/flows/{namespace}`
-- GET `/api/v1/{tenant}/flows/source`
-- GET `/api/v1/{tenant}/flows/{namespace}/{id}/dependencies`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/dependencies`
-- GET `/api/v1/{tenant}/flows/distinct-namespaces` (any action; no action check)
-- POST `/api/v1/{tenant}/flows/validate` (any action; no action check)
-- POST `/api/v1/{tenant}/flows/validate/task` (JSON, any action; no action check)
-- POST `/api/v1/{tenant}/flows/validate/task` (YAML, any action; no action check)
-- POST `/api/v1/{tenant}/flows/validate/trigger` (any action; no action check)
-- GET `/api/v1/{tenant}/flows/export/by-query`
-- POST `/api/v1/{tenant}/flows/export/by-ids`
-- GET `/api/v1/{tenant}/flows/export/by-query/csv`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/files/search`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/files`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/files/stats`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/files/revisions`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/files/directory`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/files/export`
-- GET `/api/v1/{tenant}/triggers/search`
-- GET `/api/v1/{tenant}/triggers/{namespace}/{flowId}`
-- GET `/api/v1/{tenant}/triggers/export/by-query/csv`
-
-Update
-- PUT `/api/v1/{tenant}/flows/{namespace}/{id}` (YAML)
-- PUT `/api/v1/{tenant}/flows/{namespace}/{id}` (JSON, deprecated)
-- PATCH `/api/v1/{tenant}/flows/{namespace}/{id}/{taskId}`
-- POST `/api/v1/{tenant}/executions/{executionId}/eval/{taskRunId}`
-- POST `/api/v1/{tenant}/flows/bulk`
-- POST `/api/v1/{tenant}/flows/disable/by-query`
-- POST `/api/v1/{tenant}/flows/disable/by-ids`
-- POST `/api/v1/{tenant}/flows/enable/by-query`
-- POST `/api/v1/{tenant}/flows/enable/by-ids`
-- PUT `/api/v1/{tenant}/namespaces/{namespace}/files`
-
-Delete
-- DELETE `/api/v1/{tenant}/flows/{namespace}/{id}`
-- DELETE `/api/v1/{tenant}/flows/delete/by-query`
-- DELETE `/api/v1/{tenant}/flows/delete/by-ids`
-- DELETE `/api/v1/{tenant}/namespaces/{namespace}/files`
+VIEW
+- `GET /api/v1/{tenant}/flows/{namespace}/{id}`
+- `GET /api/v1/{tenant}/flows/{namespace}/{id}/graph`
+- `GET /api/v1/{tenant}/flows/{namespace}/{id}/revisions`
+- `GET /api/v1/{tenant}/flows/{namespace}/{id}/tasks/{taskId}`
+- `GET /api/v1/{tenant}/flows/{namespace}/{id}/dependencies`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/dependencies`
+- `POST /api/v1/{tenant}/flows/expressions`
+
+LIST
+- `GET /api/v1/{tenant}/flows/search`
+- `GET /api/v1/{tenant}/flows/{namespace}`
+- `GET /api/v1/{tenant}/flows/source`
+- `GET /api/v1/{tenant}/flows/distinct-namespaces` (any `FLOW` action)
+- `GET /api/v1/{tenant}/flows/deprecated`
+
+CREATE
+- `POST /api/v1/{tenant}/flows` (single flow)
+- `POST /api/v1/{tenant}/flows/{namespace}` (bulk upsert; also requires `UPDATE` and `DELETE`)
+
+UPDATE
+- `PUT /api/v1/{tenant}/flows/{namespace}/{id}`
+- `PATCH /api/v1/{tenant}/flows/{namespace}/{id}/{taskId}`
+- `POST /api/v1/{tenant}/flows/bulk` (also requires `CREATE` and `DELETE`)
+- `POST /api/v1/{tenant}/executions/{executionId}/eval/{taskRunId}`
+- `POST /api/v1/{tenant}/executions/{executionId}/{taskRunId}/eval`
+
+DELETE
+- `DELETE /api/v1/{tenant}/flows/{namespace}/{id}`
+- `DELETE /api/v1/{tenant}/flows/{namespace}/{id}/revisions` (specific revisions)
+- `DELETE /api/v1/{tenant}/flows/delete/by-query`
+- `DELETE /api/v1/{tenant}/flows/delete/by-ids`
+
+EXECUTE
+- `POST /api/v1/{tenant}/executions/{namespace}/{id}` (create execution)
+- `POST /api/v1/{tenant}/executions/{namespace}/{id}/validate`
+- `GET /api/v1/{tenant}/executions/namespaces/{namespace}/flows`
+
+DISABLE
+- `POST /api/v1/{tenant}/flows/disable/by-query`
+- `POST /api/v1/{tenant}/flows/disable/by-ids`
+
+ENABLE
+- `POST /api/v1/{tenant}/flows/enable/by-query`
+- `POST /api/v1/{tenant}/flows/enable/by-ids`
+
+EXPORT
+- `GET /api/v1/{tenant}/flows/export/by-query`
+- `POST /api/v1/{tenant}/flows/export/by-ids`
+- `GET /api/v1/{tenant}/flows/export/by-query/csv` (streaming)
+
+IMPORT
+- `POST /api/v1/{tenant}/flows/import`
+
+VALIDATE (any `FLOW` action — no dedicated check)
+- `POST /api/v1/{tenant}/flows/validate`
+- `POST /api/v1/{tenant}/flows/validate/task`
+- `POST /api/v1/{tenant}/flows/validate/trigger`
Notes
-- Trigger update operations require EXECUTION permissions, but trigger routes also require FLOW permission at the route level.
-- Creating a flow in a new namespace also requires NAMESPACE CREATE.
+- Webhook execution endpoints (`/executions/webhook/{namespace}/{id}/{key}`) are authorized by webhook key, not RBAC.
+- `POST /api/v1/{tenant}/flows/graph` has no action check.
:::
+---
+
:::collapse{title="EXECUTION"}
**Scope:** Namespace
-**CRUD meaning**
-- Create: trigger or create executions; replay executions (creates new executions).
-- Read: view executions, graphs, logs, metrics, files, and exports.
-- Update: change state, pause or resume, restart, replay by ids, set labels, unqueue, force-run, update task run state.
-- Delete: delete executions and logs.
+**Actions and their meaning**
+- `VIEW`: read a single execution's details, graph, or associated flow.
+- `LIST`: search or browse executions.
+- `DELETE`: delete executions.
+- `RESTART`: restart one or more executions from the beginning or a specific task.
+- `KILL`: kill a running execution.
+- `REPLAY`: replay an execution (creates a new execution from a previous one).
+- `PAUSE`: pause a running execution.
+- `RESUME`: resume a paused execution.
+- `CHANGE_LABELS`: update labels on a terminated execution.
+- `ACCESS_LOGS`: read or stream execution logs.
+- `ACCESS_OUTPUTS`: read task run outputs.
+- `ACCESS_FILES`: download or preview execution output files.
+- `FOLLOW`: stream live execution status events via SSE.
+- `EXPORT`: export execution data as CSV.
+- `UNQUEUE`: move a queued execution out of the queue.
+- `FORCE_RUN`: force a paused or queued execution to run immediately.
+- `UPDATE`: change the state of a task run within an execution.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/executions/trigger/{namespace}/{id}` (deprecated)
-- POST `/api/v1/{tenant}/executions/{namespace}/{id}`
-- POST `/api/v1/{tenant}/executions/{namespace}/{id}/validate` (any action; no action check)
-- POST `/api/v1/{tenant}/executions/{executionId}/replay`
-- POST `/api/v1/{tenant}/executions/{executionId}/replay-with-inputs`
-- POST `/api/v1/{tenant}/executions/replay/by-query` (any action; no action check)
-- GET `/api/v1/{tenant}/executions/namespaces` (requires CREATE)
-- GET `/api/v1/{tenant}/executions/namespaces/{namespace}/flows` (requires CREATE)
-
-Read
-- GET `/api/v1/{tenant}/executions/search`
-- GET `/api/v1/{tenant}/executions`
-- GET `/api/v1/{tenant}/executions/{executionId}`
-- GET `/api/v1/{tenant}/executions/{executionId}/graph`
-- GET `/api/v1/{tenant}/executions/{executionId}/flow`
-- GET `/api/v1/{tenant}/executions/flows/{namespace}/{flowId}`
-- GET `/api/v1/{tenant}/executions/{executionId}/file`
-- GET `/api/v1/{tenant}/executions/{executionId}/file/metas`
-- GET `/api/v1/{tenant}/executions/{executionId}/file/preview`
-- GET `/api/v1/{tenant}/executions/{executionId}/follow`
-- GET `/api/v1/{tenant}/executions/{executionId}/follow-dependencies`
-- POST `/api/v1/{tenant}/executions/latest` (any action; no action check)
-- GET `/api/v1/{tenant}/executions/export/by-query/csv`
-- GET `/api/v1/{tenant}/logs/search`
-- GET `/api/v1/{tenant}/logs/{executionId}`
-- GET `/api/v1/{tenant}/logs/{executionId}/download`
-- GET `/api/v1/{tenant}/logs/{executionId}/follow`
-- GET `/api/v1/{tenant}/metrics/{executionId}`
-- GET `/api/v1/{tenant}/metrics/names/{namespace}/{flowId}`
-- GET `/api/v1/{tenant}/metrics/names/{namespace}/{flowId}/{taskId}`
-- GET `/api/v1/{tenant}/metrics/tasks/{namespace}/{flowId}`
-- GET `/api/v1/{tenant}/metrics/aggregates/{namespace}/{flowId}/{metric}`
-- GET `/api/v1/{tenant}/metrics/aggregates/{namespace}/{flowId}/{taskId}/{metric}`
-
-Update
-- POST `/api/v1/{tenant}/executions/{executionId}/restart`
-- POST `/api/v1/{tenant}/executions/restart/by-ids`
-- POST `/api/v1/{tenant}/executions/restart/by-query` (any action; no action check)
-- POST `/api/v1/{tenant}/executions/{executionId}/state`
-- POST `/api/v1/{tenant}/executions/{executionId}/change-status`
-- POST `/api/v1/{tenant}/executions/change-status/by-ids`
-- POST `/api/v1/{tenant}/executions/change-status/by-query` (any action; no action check)
-- DELETE `/api/v1/{tenant}/executions/{executionId}/kill{?isOnKillCascade}`
-- DELETE `/api/v1/{tenant}/executions/kill/by-ids`
-- DELETE `/api/v1/{tenant}/executions/kill/by-query` (any action; no action check)
-- POST `/api/v1/{tenant}/executions/{executionId}/resume/validate` (any action; no action check)
-- POST `/api/v1/{tenant}/executions/{executionId}/resume`
-- POST `/api/v1/{tenant}/executions/{executionId}/resume-from-breakpoint`
-- POST `/api/v1/{tenant}/executions/resume/by-ids`
-- POST `/api/v1/{tenant}/executions/resume/by-query` (any action; no action check)
-- POST `/api/v1/{tenant}/executions/{executionId}/pause`
-- POST `/api/v1/{tenant}/executions/pause/by-ids`
-- POST `/api/v1/{tenant}/executions/pause/by-query` (any action; no action check)
-- POST `/api/v1/{tenant}/executions/{executionId}/labels`
-- POST `/api/v1/{tenant}/executions/labels/by-ids`
-- POST `/api/v1/{tenant}/executions/labels/by-query` (any action; no action check)
-- POST `/api/v1/{tenant}/executions/{executionId}/unqueue`
-- POST `/api/v1/{tenant}/executions/unqueue/by-ids`
-- POST `/api/v1/{tenant}/executions/unqueue/by-query` (any action; no action check)
-- POST `/api/v1/{tenant}/executions/{executionId}/force-run`
-- POST `/api/v1/{tenant}/executions/force-run/by-ids`
-- POST `/api/v1/{tenant}/executions/force-run/by-query` (any action; no action check)
-- POST `/api/v1/{tenant}/executions/replay/by-ids` (uses UPDATE in current implementation)
-
-Delete
-- DELETE `/api/v1/{tenant}/executions/{executionId}`
-- DELETE `/api/v1/{tenant}/executions/by-ids`
-- DELETE `/api/v1/{tenant}/executions/by-query` (any action; no action check)
-- DELETE `/api/v1/{tenant}/logs/{executionId}`
-- DELETE `/api/v1/{tenant}/logs/{namespace}/{flowId}` (any action; no action check)
+VIEW
+- `GET /api/v1/{tenant}/executions/{executionId}`
+- `GET /api/v1/{tenant}/executions/{executionId}/graph`
+- `GET /api/v1/{tenant}/executions/{executionId}/flow`
+- `GET /api/v1/{tenant}/executions/flows/{namespace}/{flowId}`
+
+LIST
+- `GET /api/v1/{tenant}/executions/search`
+- `GET /api/v1/{tenant}/executions`
+- `GET /api/v1/{tenant}/executions/flows/{namespace}/{flowId}` (search by flow)
+- `GET /api/v1/{tenant}/executions/namespaces` (distinct namespaces with executions)
+- `POST /api/v1/{tenant}/executions/latest` (any `EXECUTION` action)
+
+DELETE
+- `DELETE /api/v1/{tenant}/executions/{executionId}`
+- `DELETE /api/v1/{tenant}/executions/by-ids`
+- `DELETE /api/v1/{tenant}/executions/by-query`
+- `DELETE /api/v1/{tenant}/logs/{executionId}` (delete logs for an execution)
+- `DELETE /api/v1/{tenant}/logs/{namespace}/{flowId}` (delete logs for a flow)
+
+RESTART
+- `POST /api/v1/{tenant}/executions/{executionId}/restart`
+- `POST /api/v1/{tenant}/executions/restart/by-ids`
+- `POST /api/v1/{tenant}/executions/restart/by-query`
+
+KILL
+- `DELETE /api/v1/{tenant}/executions/{executionId}/kill`
+- `DELETE /api/v1/{tenant}/executions/kill/by-ids`
+- `DELETE /api/v1/{tenant}/executions/kill/by-query`
+
+REPLAY
+- `POST /api/v1/{tenant}/executions/{executionId}/replay`
+- `POST /api/v1/{tenant}/executions/{executionId}/replay-with-inputs`
+- `POST /api/v1/{tenant}/executions/replay/by-ids`
+- `POST /api/v1/{tenant}/executions/replay/by-query`
+
+PAUSE
+- `POST /api/v1/{tenant}/executions/{executionId}/pause`
+- `POST /api/v1/{tenant}/executions/pause/by-ids`
+- `POST /api/v1/{tenant}/executions/pause/by-query`
+
+RESUME
+- `POST /api/v1/{tenant}/executions/{executionId}/resume`
+- `POST /api/v1/{tenant}/executions/{executionId}/resume-from-breakpoint`
+- `POST /api/v1/{tenant}/executions/resume/by-ids`
+- `POST /api/v1/{tenant}/executions/resume/by-query`
+
+CHANGE_LABELS
+- `POST /api/v1/{tenant}/executions/{executionId}/labels`
+- `POST /api/v1/{tenant}/executions/labels/by-ids`
+- `POST /api/v1/{tenant}/executions/labels/by-query`
+
+ACCESS_LOGS
+- `GET /api/v1/{tenant}/logs/search`
+- `GET /api/v1/{tenant}/logs/{executionId}`
+- `GET /api/v1/{tenant}/logs/{executionId}/download`
+- `GET /api/v1/{tenant}/logs/{executionId}/follow` (SSE log stream)
+
+ACCESS_OUTPUTS
+- `GET /api/v1/{tenant}/outputs/{executionId}/{taskRunId}`
+- `GET /api/v1/{tenant}/outputs/{executionId}`
+
+ACCESS_FILES
+- `GET /api/v1/{tenant}/executions/{executionId}/file`
+- `GET /api/v1/{tenant}/executions/{executionId}/file/metas`
+- `GET /api/v1/{tenant}/executions/{executionId}/file/preview`
+
+FOLLOW
+- `GET /api/v1/{tenant}/executions/{executionId}/follow` (SSE execution state stream)
+- `GET /api/v1/{tenant}/executions/{executionId}/follow-dependencies`
+
+EXPORT
+- `GET /api/v1/{tenant}/executions/export/by-query/csv` (streaming)
+
+UNQUEUE
+- `POST /api/v1/{tenant}/executions/{executionId}/unqueue`
+- `POST /api/v1/{tenant}/executions/unqueue/by-ids`
+- `POST /api/v1/{tenant}/executions/unqueue/by-query`
+
+FORCE_RUN
+- `POST /api/v1/{tenant}/executions/{executionId}/force-run`
+- `POST /api/v1/{tenant}/executions/force-run/by-ids`
+- `POST /api/v1/{tenant}/executions/force-run/by-query`
+
+UPDATE
+- `POST /api/v1/{tenant}/executions/{executionId}/state`
+- `POST /api/v1/{tenant}/executions/change-status/by-ids`
+- `POST /api/v1/{tenant}/executions/change-status/by-query`
+- `POST /api/v1/{tenant}/executions/{executionId}/{taskRunId}/state`
Notes
-- Webhook execution endpoints (`/executions/webhook/{namespace}/{id}/{key}`) are anonymous and are authorized by webhook key, not RBAC.
-- `GET /api/v1/{tenant}/logs/search` only checks that the EXECUTION permission exists (any action).
+- Webhook execution endpoints are authorized by webhook key only; RBAC is not checked.
+- Execution creation (`POST /executions/{namespace}/{id}`) checks `FLOW: EXECUTE`, not an `EXECUTION` action.
:::
-:::collapse{title="TEMPLATE"}
+---
+
+:::collapse{title="TRIGGER"}
**Scope:** Namespace
-**CRUD meaning**
-- Create: create templates or bulk update a namespace of templates.
-- Read: view templates, search, export, validate.
-- Update: update templates or bulk update a namespace of templates.
-- Delete: delete templates, bulk delete by query or ids.
+**Actions and their meaning**
+- `LIST`: search or browse triggers.
+- `UNLOCK`: unlock a locked trigger.
+- `RESTART`: restart a trigger.
+- `DISABLE` / `ENABLE`: disable or enable triggers (both operations check the `DISABLE` action).
+- `DELETE`: delete triggers.
+- `EXPORT`: export trigger configuration.
+- `BACKFILL`: create, pause, unpause, or delete a backfill on a schedule trigger.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/templates`
-- POST `/api/v1/{tenant}/templates/{namespace}` (bulk update; also requires UPDATE and DELETE)
-- POST `/api/v1/{tenant}/templates/import` (requires FLOW CREATE + UPDATE)
+LIST
+- `GET /api/v1/{tenant}/triggers/search`
+- `GET /api/v1/{tenant}/triggers/{namespace}/{flowId}`
-Read
-- GET `/api/v1/{tenant}/templates/{namespace}/{id}`
-- GET `/api/v1/{tenant}/templates/search`
-- GET `/api/v1/{tenant}/templates/distinct-namespaces` (any action; no action check)
-- POST `/api/v1/{tenant}/templates/validate` (any action; no action check)
-- GET `/api/v1/{tenant}/templates/export/by-query`
-- POST `/api/v1/{tenant}/templates/export/by-ids`
+UNLOCK
+- `POST /api/v1/{tenant}/triggers/{namespace}/{flowId}/{triggerId}/unlock`
+- `POST /api/v1/{tenant}/triggers/unlock/by-ids`
-Update
-- PUT `/api/v1/{tenant}/templates/{namespace}/{id}`
-- POST `/api/v1/{tenant}/templates/{namespace}` (bulk update; also requires CREATE and DELETE)
+RESTART
+- `POST /api/v1/{tenant}/triggers/{namespace}/{flowId}/{triggerId}/restart`
-Delete
-- DELETE `/api/v1/{tenant}/templates/{namespace}/{id}`
-- DELETE `/api/v1/{tenant}/templates/delete/by-query`
-- DELETE `/api/v1/{tenant}/templates/delete/by-ids`
+DISABLE / ENABLE
+- `POST /api/v1/{tenant}/triggers/disable/by-ids` (also used for re-enabling)
-Notes
-- `POST /api/v1/{tenant}/templates/import` uses FLOW CREATE and UPDATE permissions in the current implementation.
+DELETE
+- `DELETE /api/v1/{tenant}/triggers/{namespace}/{flowId}/{triggerId}`
+- `DELETE /api/v1/{tenant}/triggers/by-ids`
+- `DELETE /api/v1/{tenant}/triggers/by-query`
+
+EXPORT
+- `GET /api/v1/{tenant}/triggers/export/by-query` (streaming)
+
+BACKFILL
+- `POST /api/v1/{tenant}/triggers/backfills` (create)
+- `POST /api/v1/{tenant}/triggers/backfills/pause`
+- `POST /api/v1/{tenant}/triggers/backfills/pause/by-ids`
+- `POST /api/v1/{tenant}/triggers/backfills/unpause`
+- `POST /api/v1/{tenant}/triggers/backfills/unpause/by-ids`
+- `DELETE /api/v1/{tenant}/triggers/backfills`
+- `DELETE /api/v1/{tenant}/triggers/backfills/by-ids`
:::
+---
+
:::collapse{title="NAMESPACE"}
**Scope:** Namespace
-**CRUD meaning**
-- Create: create namespaces.
-- Read: view namespaces, inherited variables, inherited plugin defaults, and export plugin defaults.
-- Update: update namespace metadata and import plugin defaults.
-- Delete: delete namespaces.
+**Actions and their meaning**
+- `VIEW`: read a namespace's details and inherited variables.
+- `LIST`: search or browse namespaces.
+- `CREATE`: create a namespace.
+- `UPDATE`: update namespace configuration.
+- `DELETE`: delete a namespace.
+- `MANAGE_FILES`: all namespace file operations (search, read, create, move, delete, export).
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/namespaces`
+VIEW
+- `GET /api/v1/{tenant}/namespaces/{id}`
+- `GET /api/v1/{tenant}/namespaces/{id}/inherited-variables`
+
+LIST
+- `GET /api/v1/{tenant}/namespaces/search`
+- `POST /api/v1/{tenant}/namespaces/autocomplete`
+
+CREATE
+- `POST /api/v1/{tenant}/namespaces`
+
+UPDATE
+- `PUT /api/v1/{tenant}/namespaces/{id}`
-Read
-- POST `/api/v1/{tenant}/namespaces/autocomplete`
-- GET `/api/v1/{tenant}/namespaces/{id}`
-- GET `/api/v1/{tenant}/namespaces/search`
-- GET `/api/v1/{tenant}/namespaces/{id}/inherited-variables`
-- GET `/api/v1/{tenant}/namespaces/{id}/inherited-plugindefaults`
-- POST `/api/v1/{tenant}/namespaces/{id}/plugindefaults/export`
+DELETE
+- `DELETE /api/v1/{tenant}/namespaces/{id}`
-Update
-- PUT `/api/v1/{tenant}/namespaces/{id}`
-- POST `/api/v1/{tenant}/namespaces/{id}/plugindefaults/import`
+MANAGE_FILES (all namespace file operations)
+- `GET /api/v1/{tenant}/namespaces/{namespace}/files/search`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/files`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/files/stats`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/files/revisions`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/files/directory`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/files/export`
+- `POST /api/v1/{tenant}/namespaces/{namespace}/files/directory` (create directory)
+- `POST /api/v1/{tenant}/namespaces/{namespace}/files` (create file)
+- `PUT /api/v1/{tenant}/namespaces/{namespace}/files` (move file or directory)
+- `DELETE /api/v1/{tenant}/namespaces/{namespace}/files` (delete file or directory)
-Delete
-- DELETE `/api/v1/{tenant}/namespaces/{id}`
:::
+
+---
+
:::collapse{title="KVSTORE"}
**Scope:** Namespace
-**CRUD meaning**
-- Create: create new KV entries.
-- Read: list or retrieve KV entries, including inherited entries.
-- Update: update existing KV entries.
-- Delete: delete KV entries.
+**Actions and their meaning**
+- `VIEW`: read a KV entry.
+- `LIST`: list or browse KV entries, including inherited entries.
+- `CREATE` / `UPDATE`: set a KV value (the same endpoint creates or updates depending on whether the key exists).
+- `DELETE`: delete KV entries.
**Endpoints**
-Create
-- PUT `/api/v1/{tenant}/namespaces/{namespace}/kv/{key}` (creates if key does not exist)
-
-Read
-- GET `/api/v1/{tenant}/kv` (any action; no action check)
-- GET `/api/v1/{tenant}/namespaces/{namespace}/kv` (deprecated)
-- GET `/api/v1/{tenant}/namespaces/{namespace}/kv/inheritance`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/kv/{key}`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/kv/{key}/detail`
+VIEW
+- `GET /api/v1/{tenant}/namespaces/{namespace}/kv/{key}`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/kv/{key}/detail`
-Update
-- PUT `/api/v1/{tenant}/namespaces/{namespace}/kv/{key}` (updates if key exists)
+LIST
+- `GET /api/v1/{tenant}/namespaces/{namespace}/kv`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/kv/inheritance`
-Delete
-- DELETE `/api/v1/{tenant}/namespaces/{namespace}/kv/{key}`
-- DELETE `/api/v1/{tenant}/namespaces/{namespace}/kv`
+CREATE / UPDATE
+- `PUT /api/v1/{tenant}/namespaces/{namespace}/kv/{key}` (creates if key does not exist, updates if it does)
-Notes
-- The PUT endpoint chooses CREATE vs UPDATE based on whether the key already exists.
+DELETE
+- `DELETE /api/v1/{tenant}/namespaces/{namespace}/kv/{key}`
+- `DELETE /api/v1/{tenant}/namespaces/{namespace}/kv` (bulk delete)
:::
+---
+
:::collapse{title="DASHBOARD"}
-**Scope:** Global (tenant)
+**Scope:** Tenant
-**CRUD meaning**
-- Create: create dashboards.
-- Read: view dashboards and charts.
-- Update: update dashboards and charts.
-- Delete: delete dashboards.
+**Actions and their meaning**
+- `VIEW` / `LIST`: read dashboards and their charts.
+- `CREATE`: create dashboards.
+- `UPDATE`: update dashboard configuration.
+- `DELETE`: delete dashboards.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/dashboards`
+VIEW / LIST (any `DASHBOARD` action — no dedicated per-action check at controller level)
+- `GET /api/v1/{tenant}/dashboards`
+- `GET /api/v1/{tenant}/dashboards/{id}`
+- `POST /api/v1/{tenant}/dashboards/{id}/charts/{chartId}`
+- `POST /api/v1/{tenant}/dashboards/charts/preview`
+- `POST /api/v1/{tenant}/dashboards/validate`
+- `POST /api/v1/{tenant}/dashboards/validate/chart`
+- `POST /api/v1/{tenant}/dashboards/{id}/charts/{chartId}/export/to-csv`
+- `POST /api/v1/{tenant}/dashboards/charts/export/to-csv`
-Read
-- GET `/api/v1/{tenant}/dashboards`
-- GET `/api/v1/{tenant}/dashboards/{id}`
-- POST `/api/v1/{tenant}/dashboards/{id}/charts/{chartId}`
-- POST `/api/v1/{tenant}/dashboards/charts/preview`
-- POST `/api/v1/{tenant}/dashboards/validate`
-- POST `/api/v1/{tenant}/dashboards/validate/chart`
-- POST `/api/v1/{tenant}/dashboards/{id}/charts/{chartId}/export/to-csv`
-- POST `/api/v1/{tenant}/dashboards/charts/export/to-csv`
+CREATE
+- `POST /api/v1/{tenant}/dashboards`
-Update
-- PUT `/api/v1/{tenant}/dashboards/{id}`
+UPDATE
+- `PUT /api/v1/{tenant}/dashboards/{id}`
-Delete
-- DELETE `/api/v1/{tenant}/dashboards/{id}`
-
-Notes
-- Read endpoints rely on repository-level permission checks (any DASHBOARD action); action-specific READ checks are not enforced at the controller level.
+DELETE
+- `DELETE /api/v1/{tenant}/dashboards/{id}`
:::
+---
+
:::collapse{title="SECRET"}
**Scope:** Namespace
-**CRUD meaning**
-- Create: create secrets (implemented via UPDATE in current API).
-- Read: list and view secret metadata.
-- Update: update secret values or metadata.
-- Delete: delete secrets.
+**Actions and their meaning**
+- `VIEW` / `LIST`: list secrets and view their metadata. Secret values are never returned by the API.
+- `UPDATE`: create or update a secret (creation is enforced via this action).
+- `DELETE`: delete a secret.
**Endpoints**
-Read
-- GET `/api/v1/{tenant}/secrets` (any action; no action check)
-- GET `/api/v1/{tenant}/namespaces/{namespace}/secrets`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/inherited-secrets`
+VIEW / LIST (any `SECRET` action)
+- `GET /api/v1/{tenant}/namespaces/{namespace}/secrets`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/inherited-secrets`
-Update
-- PUT `/api/v1/{tenant}/namespaces/{namespace}/secrets`
-- PATCH `/api/v1/{tenant}/namespaces/{namespace}/secrets/{key}`
+UPDATE
+- `PUT /api/v1/{tenant}/namespaces/{namespace}/secrets`
+- `PATCH /api/v1/{tenant}/namespaces/{namespace}/secrets/{key}`
-Delete
-- DELETE `/api/v1/{tenant}/namespaces/{namespace}/secrets/{key}`
+DELETE
+- `DELETE /api/v1/{tenant}/namespaces/{namespace}/secrets/{key}`
Notes
-- No endpoint currently checks SECRET CREATE; secret creation is enforced via UPDATE on `PUT /namespaces/{namespace}/secrets`.
+- `SECRET` has no `CREATE` action; creation uses `UPDATE`.
:::
+---
+
:::collapse{title="CREDENTIAL"}
-**Scope:** Namespace or global (tenant-level credentials)
+**Scope:** Namespace or tenant (depending on whether the credential is namespace-level or tenant-level)
-**CRUD meaning**
-- Create: create tenant or namespace credentials.
-- Read: list and view credentials.
-- Update: update credentials or test connections.
-- Delete: delete credentials.
+**Actions and their meaning**
+- `VIEW` / `LIST`: list and view credentials.
+- `CREATE`: create credentials.
+- `UPDATE`: update credentials or test a connection.
+- `DELETE`: delete credentials.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/credentials`
-- POST `/api/v1/{tenant}/namespaces/{namespace}/credentials`
-
-Read
-- GET `/api/v1/{tenant}/credentials`
-- GET `/api/v1/{tenant}/credentials/{id}`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/credentials`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/credentials/{name}`
-- GET `/api/v1/{tenant}/namespaces/{namespace}/credentials/inherited`
-
-Update
-- PUT `/api/v1/{tenant}/credentials/{id}`
-- POST `/api/v1/{tenant}/credentials/{id}/test`
-- PUT `/api/v1/{tenant}/namespaces/{namespace}/credentials/{name}`
-- POST `/api/v1/{tenant}/namespaces/{namespace}/credentials/{name}/test`
-
-Delete
-- DELETE `/api/v1/{tenant}/credentials/{id}`
-- DELETE `/api/v1/{tenant}/namespaces/{namespace}/credentials/{name}`
+VIEW / LIST
+- `GET /api/v1/{tenant}/credentials`
+- `GET /api/v1/{tenant}/credentials/{id}`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/credentials`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/credentials/{name}`
+- `GET /api/v1/{tenant}/namespaces/{namespace}/credentials/inherited`
+
+CREATE
+- `POST /api/v1/{tenant}/credentials`
+- `POST /api/v1/{tenant}/namespaces/{namespace}/credentials`
+
+UPDATE
+- `PUT /api/v1/{tenant}/credentials/{id}`
+- `POST /api/v1/{tenant}/credentials/{id}/test`
+- `PUT /api/v1/{tenant}/namespaces/{namespace}/credentials/{name}`
+- `POST /api/v1/{tenant}/namespaces/{namespace}/credentials/{name}/test`
+
+DELETE
+- `DELETE /api/v1/{tenant}/credentials/{id}`
+- `DELETE /api/v1/{tenant}/namespaces/{namespace}/credentials/{name}`
:::
+---
+
:::collapse{title="BLUEPRINT"}
-**Scope:** Global (tenant)
+**Scope:** Tenant
-**CRUD meaning**
-- Create: create custom blueprints.
-- Read: list or view custom blueprints and templates.
-- Update: update custom blueprints.
-- Delete: delete custom blueprints.
+**Actions and their meaning**
+- `VIEW` / `LIST`: view or browse custom blueprints.
+- `CREATE`: create a blueprint.
+- `UPDATE`: update a blueprint.
+- `DELETE`: delete a blueprint.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/blueprints/flows`
-- POST `/api/v1/{tenant}/blueprints/custom` (deprecated)
+VIEW / LIST
+- `GET /api/v1/{tenant}/blueprints/custom`
+- `GET /api/v1/{tenant}/blueprints/custom/{id}`
+- `GET /api/v1/{tenant}/blueprints/custom/{id}/source`
+- `GET /api/v1/{tenant}/blueprints/custom/tags`
+- `GET /api/v1/{tenant}/blueprints/flows/{id}`
+- `POST /api/v1/{tenant}/blueprints/flows/{id}/use-template`
-Read
-- GET `/api/v1/{tenant}/blueprints/custom`
-- GET `/api/v1/{tenant}/blueprints/custom/{id}`
-- GET `/api/v1/{tenant}/blueprints/custom/{id}/source`
-- GET `/api/v1/{tenant}/blueprints/custom/tags`
-- GET `/api/v1/{tenant}/blueprints/flow/{id}`
-- GET `/api/v1/{tenant}/blueprints/flows/{id}`
-- POST `/api/v1/{tenant}/blueprints/flows/{id}/use-template`
+CREATE
+- `POST /api/v1/{tenant}/blueprints/flows`
-Update
-- PUT `/api/v1/{tenant}/blueprints/flows/{id}`
-- PUT `/api/v1/{tenant}/blueprints/custom/{id}` (deprecated)
+UPDATE
+- `PUT /api/v1/{tenant}/blueprints/flows/{id}`
-Delete
-- DELETE `/api/v1/{tenant}/blueprints/flows/{id}`
-- DELETE `/api/v1/{tenant}/blueprints/custom/{id}` (deprecated)
+DELETE
+- `DELETE /api/v1/{tenant}/blueprints/flows/{id}`
Notes
-- Community blueprint endpoints under `/api/v1/{tenant}/blueprints/community/...` do not use BLUEPRINT permission.
+- Community blueprint endpoints (`/blueprints/community/...`) do not require `BLUEPRINT` permission.
:::
+---
+
:::collapse{title="APP"}
-**Scope:** Global (tenant) with namespace checks on app definitions
+**Scope:** Tenant (with namespace checks when the app definition references a namespace)
-**CRUD meaning**
-- Create: create apps and import apps.
-- Read: view app source, search, export apps.
-- Update: update apps and enable or disable apps.
-- Delete: delete apps.
+**Actions and their meaning**
+- `VIEW` / `LIST`: view app source, search, and catalog.
+- `CREATE`: create or import apps.
+- `UPDATE`: update apps, enable, or disable them.
+- `DELETE`: delete apps.
+- `EXECUTE`: dispatch actions through an app (run the app).
+- `ACCESS_FILES`: download or preview files generated by an app execution.
+- `ACCESS_LOGS`: view app execution logs.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/apps`
-- POST `/api/v1/{tenant}/apps/import`
-- POST `/api/v1/{tenant}/apps/preview` (requires global APP CREATE)
-
-Read
-- GET `/api/v1/{tenant}/apps/search`
-- GET `/api/v1/{tenant}/apps/catalog` (private apps also require APPEXECUTION READ)
-- GET `/api/v1/{tenant}/apps/tags`
-- GET `/api/v1/{tenant}/apps/{uid}`
-- POST `/api/v1/{tenant}/apps/export`
-
-Update
-- PUT `/api/v1/{tenant}/apps/{uid}`
-- POST `/api/v1/{tenant}/apps/{uid}/enable`
-- POST `/api/v1/{tenant}/apps/{uid}/disable`
-- POST `/api/v1/{tenant}/apps/enable`
-- POST `/api/v1/{tenant}/apps/disable`
-
-Delete
-- DELETE `/api/v1/{tenant}/apps/{uid}`
-- DELETE `/api/v1/{tenant}/apps`
-:::
+VIEW / LIST
+- `GET /api/v1/{tenant}/apps/search`
+- `GET /api/v1/{tenant}/apps/catalog`
+- `GET /api/v1/{tenant}/apps/tags`
+- `GET /api/v1/{tenant}/apps/{uid}`
+- `GET /api/v1/{tenant}/apps/view/{uid}` (PRIVATE apps require `APP: EXECUTE` or equivalent access level)
+
+CREATE
+- `POST /api/v1/{tenant}/apps`
+- `POST /api/v1/{tenant}/apps/import`
+- `POST /api/v1/{tenant}/apps/preview`
+- `POST /api/v1/{tenant}/apps/export`
+
+UPDATE
+- `PUT /api/v1/{tenant}/apps/{uid}`
+- `POST /api/v1/{tenant}/apps/{uid}/enable`
+- `POST /api/v1/{tenant}/apps/{uid}/disable`
+- `POST /api/v1/{tenant}/apps/enable`
+- `POST /api/v1/{tenant}/apps/disable`
+
+DELETE
+- `DELETE /api/v1/{tenant}/apps/{uid}`
+- `DELETE /api/v1/{tenant}/apps`
+
+EXECUTE
+- `POST /api/v1/{tenant}/apps/view/{id}/dispatch/{dispatch}`
+- `GET /api/v1/{tenant}/apps/view/{id}/streams/{stream}`
+
+ACCESS_FILES
+- `GET /api/v1/{tenant}/apps/view/{id}/file/preview`
+- `GET /api/v1/{tenant}/apps/view/{id}/file/meta`
+- `GET /api/v1/{tenant}/apps/view/{id}/file/download`
+
+ACCESS_LOGS
+- `GET /api/v1/{tenant}/apps/view/{uid}/logs/download`
-:::collapse{title="APPEXECUTION"}
-**Scope:** Namespace (checked when app access is PRIVATE)
+Notes
+- PUBLIC apps are accessible without authentication for view and dispatch. PRIVATE apps require appropriate `APP` actions.
+:::
-**CRUD meaning**
-- Create: not used for apps (execution happens via app dispatch).
-- Read: view apps and read execution artifacts through apps.
-- Update: dispatch app actions and stream updates.
-- Delete: not used.
+---
-**Endpoints**
+:::collapse{title="TESTSUITE"}
+**Scope:** Tenant
-Read
-- GET `/api/v1/{tenant}/apps/view/{uid}` (PRIVATE apps require APPEXECUTION READ)
-- GET `/api/v1/{tenant}/apps/view/{id}/file/preview`
-- GET `/api/v1/{tenant}/apps/view/{id}/file/meta`
-- GET `/api/v1/{tenant}/apps/view/{id}/file/download`
-- GET `/api/v1/{tenant}/apps/view/{uid}/logs/download`
+**Actions and their meaning**
+- `VIEW` / `LIST`: view tests and test results.
+- `CREATE`: create a test.
+- `UPDATE`: update or enable/disable tests.
+- `DELETE`: delete tests.
+- `EXECUTE`: run tests.
-Update
-- POST `/api/v1/{tenant}/apps/view/{id}/dispatch/{dispatch}`
-- GET `/api/v1/{tenant}/apps/view/{id}/streams/{stream}`
+**Endpoints**
-Notes
-- App view endpoints are anonymous for PUBLIC apps; PRIVATE apps require APPEXECUTION permissions and, if configured, group membership.
+VIEW / LIST
+- `GET /api/v1/{tenant}/tests/{namespace}/{id}`
+- `GET /api/v1/{tenant}/tests/search`
+- `POST /api/v1/{tenant}/tests/validate`
+- `GET /api/v1/{tenant}/tests/results/{id}`
+- `POST /api/v1/{tenant}/tests/results/search/last`
+- `GET /api/v1/{tenant}/tests/results/search`
+
+CREATE
+- `POST /api/v1/{tenant}/tests`
+
+UPDATE
+- `PUT /api/v1/{tenant}/tests/{namespace}/{id}`
+- `POST /api/v1/{tenant}/tests/disable/by-ids`
+- `POST /api/v1/{tenant}/tests/enable/by-ids`
+
+DELETE
+- `DELETE /api/v1/{tenant}/tests/{namespace}/{id}`
+- `DELETE /api/v1/{tenant}/tests/by-ids`
+
+EXECUTE
+- `POST /api/v1/{tenant}/tests/{namespace}/{id}/run`
+- `POST /api/v1/{tenant}/tests/run`
:::
+---
+
:::collapse{title="ASSET"}
-**Scope:** Global (tenant) with namespace checks when an asset has a namespace
+**Scope:** Tenant (with namespace checks when the asset has a namespace)
-**CRUD meaning**
-- Create: create assets.
-- Read: view assets, search assets, and dependency or usage graphs.
-- Update: not used (create or replace is done via POST).
-- Delete: delete assets.
+**Actions and their meaning**
+- `VIEW` / `LIST`: view assets and their dependency or usage graphs.
+- `CREATE`: create assets.
+- `DELETE`: delete assets.
+- `LOCK`: acquire a write lock on an asset (from a flow task or the UI).
+- `UNLOCK`: release a write lock on an asset (from a flow task or the UI). UI unlock is forced regardless of owner.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/assets`
+VIEW / LIST
+- `GET /api/v1/{tenant}/assets/{id}`
+- `GET /api/v1/{tenant}/assets/{id}/dependencies`
+- `GET /api/v1/{tenant}/assets/search`
+- `GET /api/v1/{tenant}/assets/usages/search`
-Read
-- GET `/api/v1/{tenant}/assets/{id}`
-- GET `/api/v1/{tenant}/assets/{id}/dependencies`
-- GET `/api/v1/{tenant}/assets/search`
-- GET `/api/v1/{tenant}/assets/usages/search`
+CREATE
+- `POST /api/v1/{tenant}/assets`
-Delete
-- DELETE `/api/v1/{tenant}/assets/{id}`
-- DELETE `/api/v1/{tenant}/assets/by-ids`
-- DELETE `/api/v1/{tenant}/assets/by-query`
+DELETE
+- `DELETE /api/v1/{tenant}/assets/{id}`
+- `DELETE /api/v1/{tenant}/assets/by-ids`
+- `DELETE /api/v1/{tenant}/assets/by-query`
+
+LOCK
+- `POST /api/v1/{tenant}/assets/{id}/lock`
+
+UNLOCK
+- `DELETE /api/v1/{tenant}/assets/{id}/lock`
:::
-:::collapse{title="TEST"}
-**Scope:** Namespace
+---
-**CRUD meaning**
-- Create: create tests or run tests.
-- Read: view tests and test results.
-- Update: update tests or enable or disable tests.
-- Delete: delete tests.
+:::collapse{title="MCP_SERVER"}
+**Scope:** Tenant
-**Endpoints**
+**Actions and their meaning**
+- `VIEW` / `LIST`: view MCP server configuration and registered tools.
+- `CREATE`: create an MCP server.
+- `UPDATE`: update an MCP server.
+- `DELETE`: delete an MCP server.
-Create
-- POST `/api/v1/{tenant}/tests`
-- POST `/api/v1/{tenant}/tests/{namespace}/{id}/run`
-- POST `/api/v1/{tenant}/tests/run`
-
-Read
-- GET `/api/v1/{tenant}/tests/{namespace}/{id}`
-- GET `/api/v1/{tenant}/tests/search`
-- POST `/api/v1/{tenant}/tests/validate`
-- GET `/api/v1/{tenant}/tests/results/{id}`
-- POST `/api/v1/{tenant}/tests/results/search/last`
-- GET `/api/v1/{tenant}/tests/results/search`
-
-Update
-- PUT `/api/v1/{tenant}/tests/{namespace}/{id}`
-- POST `/api/v1/{tenant}/tests/disable/by-ids`
-- POST `/api/v1/{tenant}/tests/enable/by-ids`
-
-Delete
-- DELETE `/api/v1/{tenant}/tests/{namespace}/{id}`
-- DELETE `/api/v1/{tenant}/tests/by-ids`
+Notes
+- Connecting an AI agent to a PRIVATE server also requires `FLOW: EXECUTE` on at least one namespace with a flow that has an `McpToolTrigger` pointing at that server.
:::
-:::collapse{title="AUDITLOG"}
-**Scope:** Global (tenant)
+---
-**CRUD meaning**
-- Read: search and export audit logs; read resource history and diffs.
+:::collapse{title="COPILOT"}
+**Scope:** Tenant
-**Endpoints**
+**Actions and their meaning**
+- `USE`: access AI flow generation and Copilot features.
-Read
-- GET `/api/v1/{tenant}/auditlogs/search`
-- POST `/api/v1/{tenant}/auditlogs/find`
-- GET `/api/v1/{tenant}/auditlogs/history/{detailId}` (requires READ on the underlying resource)
-- GET `/api/v1/{tenant}/auditlogs/{id}/diff` (requires READ on the underlying resource or AUDITLOG READ; superadmin-only for certain resources)
-- GET `/api/v1/{tenant}/auditlogs/export`
+**Endpoints**
-Notes
-- Cross-tenant audit log endpoints under `/api/v1/auditlogs/...` are superadmin-only and are not controlled by AUDITLOG permissions.
+USE (any `COPILOT` action)
+- `POST /api/v1/{tenant}/ai/generate/flow`
:::
-:::collapse{title="USER"}
-**Scope:** Global (tenant)
-
-**CRUD meaning**
-- Create, Read, Update, Delete: manage users via SCIM provisioning endpoints.
+---
-**Endpoints**
+:::collapse{title="AUDITLOG"}
+**Scope:** Tenant
-Create
-- POST `/api/v1/{tenant}/integrations/{integration}/scim/v2/Users`
+**Actions and their meaning**
+- `VIEW` / `LIST`: search audit logs and view diffs.
+- `EXPORT`: export audit logs.
-Read
-- GET `/api/v1/{tenant}/integrations/{integration}/scim/v2/Users`
-- GET `/api/v1/{tenant}/integrations/{integration}/scim/v2/Users/{id}`
+**Endpoints**
-Update
-- PUT `/api/v1/{tenant}/integrations/{integration}/scim/v2/Users/{id}`
-- PATCH `/api/v1/{tenant}/integrations/{integration}/scim/v2/Users/{id}`
+VIEW / LIST
+- `GET /api/v1/{tenant}/auditlogs/search`
+- `POST /api/v1/{tenant}/auditlogs/find`
+- `GET /api/v1/{tenant}/auditlogs/history/{detailId}`
+- `GET /api/v1/{tenant}/auditlogs/{id}/diff`
-Delete
-- DELETE `/api/v1/{tenant}/integrations/{integration}/scim/v2/Users/{id}`
+EXPORT
+- `GET /api/v1/{tenant}/auditlogs/export`
Notes
-- IAM user management endpoints under `/api/v1/users` are superadmin-only and do not use USER permissions.
+- Cross-tenant audit log endpoints under `/api/v1/auditlogs/...` (no tenant segment) are instance-owner-only and are not controlled by `AUDITLOG` permissions.
:::
-:::collapse{title="SERVICE_ACCOUNT"}
-**Scope:** Global (tenant)
+---
-**CRUD meaning**
-- Create: create service accounts.
-- Read: list or view service accounts and API tokens.
-- Update: update service accounts and create API tokens.
-- Delete: delete service accounts or API tokens.
+:::collapse{title="SYSTEM_SETTINGS"}
+**Scope:** Tenant
-**Endpoints**
+**Actions and their meaning**
+- `VIEW`: read instance-level settings.
+- `UPDATE`: modify instance-level settings.
+:::
+
+---
+
+:::collapse{title="TENANT_SETTINGS"}
+**Scope:** Tenant
-Create
-- POST `/api/v1/{tenant}/service-accounts`
+**Actions and their meaning**
+- `VIEW`: read tenant-level settings.
+- `UPDATE`: modify tenant-level settings.
+:::
-Read
-- GET `/api/v1/{tenant}/service-accounts/{id}`
-- GET `/api/v1/{tenant}/service-accounts/{id}/api-tokens`
+---
-Update
-- PUT `/api/v1/{tenant}/service-accounts/{id}`
-- POST `/api/v1/{tenant}/service-accounts/{id}/api-tokens`
+:::collapse{title="USER"}
+**Scope:** Tenant
-Delete
-- DELETE `/api/v1/{tenant}/service-accounts/{id}`
-- DELETE `/api/v1/{tenant}/service-accounts/{id}/api-tokens/{tokenId}`
+**Actions and their meaning**
+- `VIEW` / `LIST`: view user details and group membership.
+- `CREATE` / `UPDATE` / `DELETE`: manage users.
+- `MANAGE_GROUP_MEMBERSHIP`: update a user's group assignments.
+- `IMPERSONATE`: impersonate a user to test their access.
Notes
-- Superadmin-only service account endpoints under `/api/v1/service-accounts` do not use SERVICE_ACCOUNT permissions.
+- `USER` is not included in any of the standard managed roles (Viewer, Launcher, Editor, Developer). Only Admin includes it. Custom roles with `USER` actions are intended for platform administrators.
+- IAM user management endpoints under `/api/v1/users` (no tenant segment) are instance-owner-only and do not require `USER` permissions.
:::
+---
+
:::collapse{title="GROUP"}
-**Scope:** Global (tenant)
+**Scope:** Tenant
-**CRUD meaning**
-- Create, Read, Update, Delete: manage groups.
+**Actions and their meaning**
+- `VIEW` / `LIST`: view groups and their members.
+- `CREATE`: create groups.
+- `UPDATE`: update group metadata.
+- `DELETE`: delete groups.
+- `MANAGE_MEMBERS`: add or remove members from a group.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/groups`
+VIEW / LIST
+- `GET /api/v1/{tenant}/groups/{id}`
+- `GET /api/v1/{tenant}/groups/search`
+- `POST /api/v1/{tenant}/groups/autocomplete`
+- `POST /api/v1/{tenant}/groups/ids`
+- `GET /api/v1/{tenant}/groups/{id}/members`
-Read
-- GET `/api/v1/{tenant}/groups/{id}`
-- GET `/api/v1/{tenant}/groups/search`
-- POST `/api/v1/{tenant}/groups/autocomplete`
-- POST `/api/v1/{tenant}/groups/ids`
+CREATE
+- `POST /api/v1/{tenant}/groups`
-Update
-- PUT `/api/v1/{tenant}/groups/{id}`
+UPDATE
+- `PUT /api/v1/{tenant}/groups/{id}`
-Delete
-- DELETE `/api/v1/{tenant}/groups/{id}`
+DELETE
+- `DELETE /api/v1/{tenant}/groups/{id}`
+
+MANAGE_MEMBERS
+- `PUT /api/v1/{tenant}/groups/{id}/members/{userId}`
+- `PUT /api/v1/{tenant}/groups/{id}/members/membership/{userId}`
+- `DELETE /api/v1/{tenant}/groups/{id}/members/{userId}`
+- `PUT /api/v1/{tenant}/users/{id}/groups`
Notes
-- SCIM group endpoints under `/api/v1/{tenant}/integrations/{integration}/scim/v2/Groups` use GROUP permissions for CRUD.
+- SCIM group endpoints (`/integrations/{integration}/scim/v2/Groups`) also use `GROUP` permissions.
:::
-:::collapse{title="GROUP_MEMBERSHIP"}
-**Scope:** Global (tenant)
+---
-**CRUD meaning**
-- Create: add users to groups.
-- Read: list group members.
-- Update: update membership roles or replace a user's group list.
-- Delete: remove users from groups.
+:::collapse{title="ROLE"}
+**Scope:** Tenant
-**Endpoints**
+**Actions and their meaning**
+- `VIEW` / `LIST`: view roles and their permissions.
+- `CREATE`: create roles.
+- `UPDATE`: update roles.
+- `DELETE`: delete roles.
-Create
-- PUT `/api/v1/{tenant}/groups/{id}/members/{userId}`
+**Endpoints**
-Read
-- GET `/api/v1/{tenant}/groups/{id}/members`
+VIEW / LIST
+- `GET /api/v1/{tenant}/roles/{id}`
+- `GET /api/v1/{tenant}/roles/search`
+- `POST /api/v1/{tenant}/roles/autocomplete`
+- `POST /api/v1/{tenant}/roles/ids`
+- `GET /api/v1/{tenant}/acls/permissions` (any `ROLE` action)
+- `GET /api/v1/{tenant}/acls/actions` (any `ROLE` action)
-Update
-- PUT `/api/v1/{tenant}/groups/{id}/members/membership/{userId}`
-- PUT `/api/v1/{tenant}/users/{id}/groups`
+CREATE
+- `POST /api/v1/{tenant}/roles`
-Delete
-- DELETE `/api/v1/{tenant}/groups/{id}/members/{userId}`
+UPDATE
+- `PUT /api/v1/{tenant}/roles/{id}`
-Notes
-- Group owners can manage membership without GROUP_MEMBERSHIP permission; non-owners require it.
+DELETE
+- `DELETE /api/v1/{tenant}/roles/{id}`
:::
-:::collapse{title="ROLE"}
-**Scope:** Global (tenant)
+---
-**CRUD meaning**
-- Create, Read, Update, Delete: manage roles.
+:::collapse{title="BINDING"}
+**Scope:** Tenant
+
+**Actions and their meaning**
+- `VIEW` / `LIST`: view bindings.
+- `CREATE`: create a binding.
+- `DELETE`: delete a binding.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/roles`
+VIEW / LIST
+- `GET /api/v1/{tenant}/bindings/{id}`
+- `GET /api/v1/{tenant}/bindings/search`
-Read
-- GET `/api/v1/{tenant}/roles/{id}`
-- GET `/api/v1/{tenant}/roles/search`
-- POST `/api/v1/{tenant}/roles/autocomplete`
-- POST `/api/v1/{tenant}/roles/ids`
-- GET `/api/v1/{tenant}/acls/permissions` (any action; no action check)
-- GET `/api/v1/{tenant}/acls/actions` (any action; no action check)
+CREATE
+- `POST /api/v1/{tenant}/bindings`
+- `POST /api/v1/{tenant}/bindings/bulk`
-Update
-- PUT `/api/v1/{tenant}/roles/{id}`
+DELETE
+- `DELETE /api/v1/{tenant}/bindings/{id}`
-Delete
-- DELETE `/api/v1/{tenant}/roles/{id}`
+Notes
+- Bindings are immutable. To change a binding's scope or roles, delete it and create a new one.
:::
-:::collapse{title="BINDING"}
-**Scope:** Global (tenant)
+---
-**CRUD meaning**
-- Create, Read, Delete: manage bindings between users, groups, and roles.
+:::collapse{title="SERVICE_ACCOUNT"}
+**Scope:** Tenant
+
+**Actions and their meaning**
+- `VIEW` / `LIST`: view service accounts and their API tokens.
+- `CREATE`: create service accounts.
+- `UPDATE`: update service accounts and issue API tokens.
+- `DELETE`: delete service accounts or revoke API tokens.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/bindings`
-- POST `/api/v1/{tenant}/bindings/bulk`
+VIEW / LIST
+- `GET /api/v1/{tenant}/service-accounts/{id}`
+- `GET /api/v1/{tenant}/service-accounts/{id}/api-tokens`
+
+CREATE
+- `POST /api/v1/{tenant}/service-accounts`
-Read
-- GET `/api/v1/{tenant}/bindings/{id}`
-- GET `/api/v1/{tenant}/bindings/search`
+UPDATE
+- `PUT /api/v1/{tenant}/service-accounts/{id}`
+- `POST /api/v1/{tenant}/service-accounts/{id}/api-tokens`
-Delete
-- DELETE `/api/v1/{tenant}/bindings/{id}`
+DELETE
+- `DELETE /api/v1/{tenant}/service-accounts/{id}`
+- `DELETE /api/v1/{tenant}/service-accounts/{id}/api-tokens/{tokenId}`
+
+Notes
+- Instance-owner-only endpoints under `/api/v1/service-accounts` (no tenant segment) do not use `SERVICE_ACCOUNT` permissions.
:::
+---
+
:::collapse{title="INVITATION"}
-**Scope:** Global (tenant)
+**Scope:** Tenant
-**CRUD meaning**
-- Create: create invitations.
-- Read: list or view invitations.
-- Delete: delete invitations.
+**Actions and their meaning**
+- `VIEW` / `LIST`: view invitations.
+- `CREATE`: send an invitation.
+- `DELETE`: revoke an invitation.
**Endpoints**
-Create
-- POST `/api/v1/{tenant}/invitations`
-Read
-- GET `/api/v1/{tenant}/invitations/search`
-- GET `/api/v1/{tenant}/invitations/email/{email}`
-- GET `/api/v1/{tenant}/invitations/{id}`
+VIEW / LIST
+- `GET /api/v1/{tenant}/invitations/search`
+- `GET /api/v1/{tenant}/invitations/email/{email}`
+- `GET /api/v1/{tenant}/invitations/{id}`
+
+CREATE
+- `POST /api/v1/{tenant}/invitations`
-Delete
-- DELETE `/api/v1/{tenant}/invitations/{id}`
+DELETE
+- `DELETE /api/v1/{tenant}/invitations/{id}`
:::
-:::collapse{title="TENANT_ACCESS"}
-**Scope:** Global (tenant)
+:::collapse{title="POLICY"}
+**Scope:** Namespace or Tenant (a policy targets either a namespace or the full tenant, and is authorized accordingly)
-**CRUD meaning**
-- Create: grant a user access to a tenant.
-- Read: list tenant access or fetch a user's tenant access.
-- Delete: revoke tenant access.
+**Actions and their meaning**
+- `VIEW`: read a single policy.
+- `LIST`: search or browse policies.
+- `CREATE`: create a policy.
+- `UPDATE`: update a policy.
+- `DELETE`: delete a policy.
+- `EXECUTE`: dry-run evaluate a policy against every flow in its scope without persisting any changes.
+- `EXPORT`: export policies as YAML.
+- `IMPORT`: import policies from a YAML file. Each document is authorized against its own scope before anything is written — a single denial rejects the whole import.
**Endpoints**
-Create
-- PUT `/api/v1/{tenant}/tenant-access/{userId}`
-- POST `/api/v1/{tenant}/tenant-access`
+VIEW
+- `GET /api/v1/{tenant}/policies/{id}` (tenant-scope)
+- `GET /api/v1/{tenant}/namespaces/{namespace}/policies/{id}` (namespace-scope)
-Read
-- GET `/api/v1/{tenant}/tenant-access`
-- POST `/api/v1/{tenant}/tenant-access/autocomplete`
-- GET `/api/v1/{tenant}/tenant-access/{userId}`
+LIST
+- `GET /api/v1/{tenant}/policies/search` (tenant-scope)
+- `GET /api/v1/{tenant}/namespaces/{namespace}/policies/search` (namespace-scope)
-Delete
-- DELETE `/api/v1/{tenant}/tenant-access/{userId}`
+CREATE
+- `POST /api/v1/{tenant}/policies` (tenant-scope)
+- `POST /api/v1/{tenant}/namespaces/{namespace}/policies` (namespace-scope)
-Notes
-- `GET /tenant-access/{userId}` is allowed for the authenticated user without TENANT_ACCESS permission; all other access requires the permission.
-:::
-
-:::collapse{title="IMPERSONATE"}
-**Scope:** Global (tenant)
-
-**CRUD meaning**
-- Read: allow impersonation via the API header.
+UPDATE
+- `PUT /api/v1/{tenant}/policies/{id}` (tenant-scope)
+- `PUT /api/v1/{tenant}/namespaces/{namespace}/policies/{id}` (namespace-scope)
-**Endpoints**
+DELETE
+- `DELETE /api/v1/{tenant}/policies/{id}` (tenant-scope)
+- `DELETE /api/v1/{tenant}/policies/delete/by-ids` (tenant-scope)
+- `DELETE /api/v1/{tenant}/namespaces/{namespace}/policies/{id}` (namespace-scope)
+- `DELETE /api/v1/{tenant}/namespaces/{namespace}/policies/delete/by-ids` (namespace-scope)
-Read
-- Use `X-Kestra-Impersonate: user@example.com` on authenticated requests (requires IMPERSONATE READ).
+EXECUTE (dry-run evaluate)
+- `GET /api/v1/{tenant}/policies/{id}/evaluate` (tenant-scope)
+- `GET /api/v1/{tenant}/namespaces/{namespace}/policies/{id}/evaluate` (namespace-scope)
-Notes
-- The IAM endpoint `POST /api/v1/users/{id}/impersonate` is superadmin-only and does not use IMPERSONATE permission.
-:::
+EXPORT
+- `POST /api/v1/{tenant}/policies/export` (tenant-scope, all policies)
+- `POST /api/v1/{tenant}/policies/export/by-ids` (tenant-scope, by IDs)
+- `POST /api/v1/{tenant}/namespaces/{namespace}/policies/export` (namespace-scope, all policies)
+- `POST /api/v1/{tenant}/namespaces/{namespace}/policies/export/by-ids` (namespace-scope, by IDs)
-:::collapse{title="SETTING"}
-**Scope:** Global (tenant)
+IMPORT
+- `POST /api/v1/{tenant}/policies/import`
+- `POST /api/v1/{tenant}/namespaces/{namespace}/policies/import`
-**CRUD meaning**
-- Create, Read, Update, Delete: reserved for webserver settings.
+VALIDATE (any `POLICY` VIEW action — no dedicated check)
+- `POST /api/v1/{tenant}/policies/validate`
+- `POST /api/v1/{tenant}/namespaces/{namespace}/policies/validate`
-**Endpoints**
-- No API endpoints currently enforce SETTING permissions.
+Notes
+- Instance-scope policy endpoints (`/api/v1/instance/policies/...`) manage read-only static policies declared in server configuration and are instance-owner-only; they are not governed by `POLICY` RBAC permissions.
:::
-:::collapse{title="AI_COPILOT"}
-**Scope:** Global (tenant)
-
-**CRUD meaning**
-- Read: access AI flow generation.
+---
-**Endpoints**
+## Related
-Read
-- POST `/api/v1/{tenant}/ai/generate/flow` (any action; no action check)
-:::
+- [RBAC overview](../index.md) — resources, actions, and managed roles explained
+- [RBAC action model migration guide](../../../../11.migration-guide/v2.0.0/rbac-action-model/index.md) — how old CRUD permissions map to the new actions when upgrading from 1.x
+- [kestractl roles](../../../../kestra-cli/kestractl/index.md#roles) — create and manage roles from the CLI
diff --git a/src/contents/docs/07.enterprise/03.auth/rbac/superadmin_switch.png b/src/contents/docs/07.enterprise/03.auth/rbac/superadmin_switch.png
deleted file mode 100644
index c92cee1f897..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/rbac/superadmin_switch.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/authentik/index.md b/src/contents/docs/07.enterprise/03.auth/scim/authentik/index.md
index 476b4a07e4f..2e77f28cac1 100644
--- a/src/contents/docs/07.enterprise/03.auth/scim/authentik/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/scim/authentik/index.md
@@ -7,9 +7,7 @@ editions: ["EE", "Cloud"]
version: ">= 0.18.0"
---
-Sync Users and Groups from authentik to Kestra using SCIM.
-
-## authentik SCIM provisioning
+Sync users and groups from authentik to Kestra using SCIM.
## Prerequisites
@@ -29,18 +27,14 @@ Tenants are enabled by default. Please refer to the [Migration Guide](../../../.
## Kestra SCIM setup: create a new provisioning integration
-1. In the Kestra UI, navigate to the `Tenant` → `IAM` → `SCIM Provisioning` page.
-2. Click on the `Create` button in the top right corner of the page.
+1. Go to **Instance Owner**, select your tenant from the sidebar, open **IAM**, and click the **SCIM Provisioning** tab.
+2. Click **+ Create**.
3. Fill in the following fields:
- **Name**: Enter a name for the provisioning integration.
- **Description**: Provide a brief description of the integration.
- - **Provisioning Type**: Currently, only SCIM 2.0 is supported — leave the default selection and click `Save`.
-
-
+ - **Provisioning Type**: Only SCIM 2.0 is supported — leave the default selection and click **Save**.
-The above steps will generate a SCIM endpoint URL and a Secret Token that you will use to authenticate authentik with the SCIM integration in Kestra. Save those details, as they will be needed in the next steps.
-
-
+These steps generate a SCIM endpoint URL and a Secret Token. Save both — you will need them in the next steps.
The endpoint should look as follows:
@@ -52,12 +46,10 @@ The Secret Token will be a long string (approximately 200 characters) used to au
### Enable or disable SCIM integration
-Note that you can disable or completely remove the SCIM Integration at any time. When an integration is disabled, all incoming requests to that integration endpoint will be rejected.
-
-
+You can disable or remove the SCIM integration at any time. When disabled, all incoming requests to that endpoint are rejected.
:::alert{type="info"}
-At first, you can disable the integration to configure your authentik SCIM integration, and then enable it once the configuration is complete.
+You can disable the integration while configuring authentik, then enable it once setup is complete.
:::
### IAM role and service account
@@ -106,7 +98,7 @@ You can set a password for each authentik user to allow them to log in directly
-Once groups and users are created, they should be visible in the Kestra UI under the `IAM` → `Users` and `Groups` sections. It’s best to log in as the default admin user and attach the desired `Role` to each group to ensure that the users have the necessary permissions.
+Once groups and users are created, they are visible in the Kestra UI under **IAM → Users** and **Groups**. Log in as the default admin user and attach the desired role to each group to set the necessary permissions.
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/authentik/scim_authentik.png b/src/contents/docs/07.enterprise/03.auth/scim/authentik/scim_authentik.png
deleted file mode 100644
index 30d4647e2bd..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/scim/authentik/scim_authentik.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/authentik/scim_authentik2.png b/src/contents/docs/07.enterprise/03.auth/scim/authentik/scim_authentik2.png
deleted file mode 100644
index c0b4375a526..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/scim/authentik/scim_authentik2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/index.mdx b/src/contents/docs/07.enterprise/03.auth/scim/index.mdx
index ec7f0e1e4f0..c8ab1362c38 100644
--- a/src/contents/docs/07.enterprise/03.auth/scim/index.mdx
+++ b/src/contents/docs/07.enterprise/03.auth/scim/index.mdx
@@ -21,17 +21,11 @@ Sync users and groups from your Identity Provider (IdP) to Kestra using SCIM.
>
-## SCIM directory sync
-
-SCIM (System for Cross-domain Identity Management) is an open-standard protocol designed to facilitate user identity management across multiple systems.
-
-It simplifies user provisioning, de-provisioning, and group synchronization between IdPs, such as Microsoft Entra ID or Okta, and service providers (SPs) such as Kestra. In layman's terms, SCIM allows you to automatically keep your users and groups in sync between your IdP and Kestra.
-
-Kestra explicitly relies on the SCIM 2.0 protocol for directory synchronization.
+SCIM (System for Cross-domain Identity Management) is an open-standard protocol that automates user provisioning, de-provisioning, and group synchronization between identity providers (IdPs) such as Microsoft Entra ID or Okta and service providers such as Kestra. Kestra uses the SCIM 2.0 protocol.
-## Benefits of a Directory Sync with SCIM
+## Benefits of directory sync with SCIM
1. **Automated provisioning and de-provisioning**: SCIM automates the provisioning and de-provisioning of users, creating a single source of truth (SSOT) for user identity data. Instead of manually creating and managing users in Kestra, you can synchronize them from your IdP.
2. **Consistency and compliance**: With SCIM, you can ensure consistency of identity information across systems and stay compliant with security and regulatory requirements.
@@ -39,6 +33,6 @@ Kestra explicitly relies on the SCIM 2.0 protocol for directory synchronization.
## Supported identity providers
-For a detailed guide on how to set up SCIM provisioning with a specific IdP, refer to the documentation for the respective provider.
+For setup guides by provider, see the pages below.
\ No newline at end of file
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/keycloak/index.md b/src/contents/docs/07.enterprise/03.auth/scim/keycloak/index.md
index 149f3281c20..48e6746215e 100644
--- a/src/contents/docs/07.enterprise/03.auth/scim/keycloak/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/scim/keycloak/index.md
@@ -9,8 +9,6 @@ version: ">= 0.18.0"
Sync users and groups from Keycloak to Kestra using SCIM.
-## Keycloak SCIM provisioning
-
## Prerequisites
- **Keycloak Account**: An account with administrative privileges is required to configure SCIM provisioning.
@@ -29,16 +27,14 @@ Tenants are enabled by default. Please refer to the [Migration Guide](../../../.
## Kestra SCIM setup: create a new provisioning integration
-1. In the Kestra UI, navigate to the `Tenant` → `IAM` → `SCIM Provisioning` page.
-2. Click on the `Create` button in the top right corner of the page.
+1. Go to **Instance Owner**, select your tenant from the sidebar, open **IAM**, and click the **SCIM Provisioning** tab.
+2. Click **+ Create**.
3. Fill in the following fields:
- **Name**: Enter a name for the provisioning integration.
- **Description**: Provide a brief description of the integration.
- - **Provisioning Type**: currently, we only support SCIM 2.0 — leave the default selection and click `Save`.
-
-
+ - **Provisioning Type**: Only SCIM 2.0 is supported — leave the default selection and click **Save**.
-The steps above will generate a SCIM endpoint URL and a Secret Token that you will use to authenticate Keycloak with the SCIM integration in Kestra. Save those details as we will need them in the next steps.
+These steps generate a SCIM endpoint URL and a Secret Token. Save both — you will need them in the next steps.
@@ -50,18 +46,15 @@ https:///api/v1//integrations/integration_id/scim
The Secret Token is a long string (approx. 200 characters) used to authenticate requests from Keycloak to Kestra.
-### Enable or Disable SCIM Integration
-
-Note that you can disable or completely remove the SCIM Integration at any time. When an integration is disabled, all incoming requests to that integration endpoint will be rejected.
-
-
+### Enable or disable SCIM integration
+You can disable or remove the SCIM integration at any time. When disabled, all incoming requests to that endpoint are rejected.
:::alert{type="info"}
-At first, you can disable the integration to configure your Keycloak SCIM integration, and then enable it once the configuration is complete.
+You can disable the integration while configuring Keycloak, then enable it once setup is complete.
:::
-### IAM Role and Service Account
+### IAM role and service account
When creating a new Provisioning Integration, Kestra will automatically create two additional objects:
@@ -82,23 +75,24 @@ Why the `SCIMProvisioner` role doesn't have the `DELETE` permission for `USERS`?
Keycloak [does not provide](https://github.com/keycloak/keycloak/issues/13484) any built-in support for SCIM v2.0. Some [open-source solutions](https://github.com/mitodl/keycloak-scim/) support groups synchronization but not users and membership synchronization.
-However, there are paid solutions such as [SCIM for Keycloak](https://scim-for-keycloak.de/) that allow you to extend Keycloak with SCIM. The setup shown below was validated with Kestra 0.18.0 and Keyclock 25.0.2 — best if you use the same or higher versions.
+However, there are paid solutions such as [SCIM for Keycloak](https://scim-for-keycloak.de/) that allow you to extend Keycloak with SCIM. The setup shown below was validated with Kestra 0.18.0 and Keycloak 25.0.2 — best if you use the same or higher versions.
+
+1. Create an account at https://scim-for-keycloak.de/ and purchase a free license (no VAT number or credit card required).
-1. **Obtain a License**:
- - Create a new account on: https://scim-for-keycloak.de/
- - Purchase a free license (no VAT number or credit card is required for a free license).
-2. **Install the SCIM Provider Plugin**:
- - Download the plugin JAR file from the `Downloads` section in your Account (e.g. `scim-for-keycloak-kc-25-2.2.1-free.jar`).
+
+2. Download the plugin JAR file from the **Downloads** section (e.g. `scim-for-keycloak-kc-25-2.2.1-free.jar`).
+
- - Place the JAR file in the `./providers` directory of your Keycloak installation (or in the current folder if Keycloak is deployed with Docker).
- - More information: [SCIM for Keycloak Installation](https://scim-for-keycloak.de/documentation/installation/install)
+
+ Place the JAR in the `./providers` directory of your Keycloak installation. See [SCIM for Keycloak Installation](https://scim-for-keycloak.de/documentation/installation/install) for details.
+
3. **Deploy Keycloak**:
- Create a simple `docker-compose.yaml` file:
```yaml
services:
- keyclock:
- container_name: keyclock
+ keycloak:
+ container_name: keycloak
image: quay.io/keycloak/keycloak:25.0.2
ports:
- 8085:8085
@@ -115,19 +109,18 @@ However, there are paid solutions such as [SCIM for Keycloak](https://scim-for-k
```
- Run `docker compose up` to start Keycloak.
4. **Configure the SCIM for Keycloak**:
- - To synchronize Users and Groups from Keycloak to Kestra, connect to the `SCIM Administration Console` for Keycloak with SCIM.
+ - Connect to the `SCIM Administration Console` to synchronize users and groups from Keycloak to Kestra.
- - Enable SCIM for the Realm
+ - Enable SCIM for the realm.
- - Note that `Bulk` and `Password synchronization` operations are currently not supported by Kestra and must be disabled in Keycloak.
+ - Disable `Bulk` and `Password synchronization` — these operations are not supported by Kestra.
5. **Create a SCIM Client**:
- Navigate to the `Remote SCIM Provider` section
- Fill the `Base URL` field with your Kestra `SCIM Endpoint`:
- Fill the `Authentication` with your Kestra `Secret Token`:
-6. **Enable Provisioning**:
- - Now that everything is configured, you can toggle the `Enabled` field on in the Kestra Provisioning Integration to start syncing users and groups from Keycloak to Kestra.
+6. Toggle **Enabled** in the Kestra Provisioning Integration to start syncing users and groups from Keycloak to Kestra.
## Additional resources
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/keycloak/scim1_keycloak.png b/src/contents/docs/07.enterprise/03.auth/scim/keycloak/scim1_keycloak.png
deleted file mode 100644
index 16f2b8fa522..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/scim/keycloak/scim1_keycloak.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/microsoft-entra-id/index.md b/src/contents/docs/07.enterprise/03.auth/scim/microsoft-entra-id/index.md
index f60901509b0..018661dbd88 100644
--- a/src/contents/docs/07.enterprise/03.auth/scim/microsoft-entra-id/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/scim/microsoft-entra-id/index.md
@@ -9,8 +9,6 @@ version: ">= 0.18.0"
Sync users and groups from Microsoft Entra ID to Kestra using SCIM.
-## Microsoft Entra ID SCIM provisioning
-
## Prerequisites
- **Microsoft Entra ID Account**: An account with administrative privileges is required to configure SCIM provisioning.
@@ -25,16 +23,14 @@ kestra:
## Kestra SCIM setup: create a new provisioning integration
-1. In the Kestra UI, navigate to the `Tenant` → `IAM` → `SCIM Provisioning` page.
-2. Click on the `Create` button in the top right corner of the page.
+1. Go to **Instance Owner**, select your tenant from the sidebar, open **IAM**, and click the **SCIM Provisioning** tab.
+2. Click **+ Create**.
3. Fill in the following fields:
- **Name**: Enter a name for the provisioning integration.
- **Description**: Provide a brief description of the integration.
- - **Provisioning Type**: currently, we only support SCIM 2.0 — leave the default selection and click `Save`.
-
-
+ - **Provisioning Type**: Only SCIM 2.0 is supported — leave the default selection and click **Save**.
-The above steps will generate a SCIM endpoint URL and a Secret Token that you will use to authenticate Microsoft Entra ID with the SCIM integration in Kestra. Save those details as they will be needed in the next steps.
+These steps generate a SCIM endpoint URL and a Secret Token. Save both — you will need them in the next steps.
@@ -46,18 +42,15 @@ https:///api/v1//integrations/integration_id/scim
The Secret Token is a long string (approx. 200 characters) used to authenticate requests from Microsoft Entra ID to Kestra.
-### Enable or Disable SCIM Integration
-
-Note that you can disable or completely remove the SCIM Integration at any time. When an integration is disabled, all incoming requests to that integration endpoint will be rejected.
-
-
+### Enable or disable SCIM integration
+You can disable or remove the SCIM integration at any time. When disabled, all incoming requests to that endpoint are rejected.
:::alert{type="info"}
-At first, you can disable the integration to configure your Microsoft Entra ID integration in the Azure portal, and then enable it once the configuration is complete.
+You can disable the integration while configuring Entra ID, then enable it once setup is complete.
:::
-### IAM Role and Service Account
+### IAM role and service account
When creating a new Provisioning Integration, Kestra will automatically create two additional objects:
@@ -76,25 +69,29 @@ Why the `SCIMProvisioner` role doesn't have the `DELETE` permission for `USERS`?
## Microsoft Entra ID SCIM setup
-### 1. Register Kestra as an Enterprise Application:
- - Navigate to Microsoft Entra ID → Enterprise Applications.
- - Click on the `+ New application` button to create a new custom application. You can name the app "KestraSCIM" or any other relevant name.
+### 1. Register Kestra as an enterprise application
+
+- Navigate to **Microsoft Entra ID → Enterprise Applications**.
+- Click **+ New application** to create a custom application. Name it "KestraSCIM" or any relevant name.
+
-### 2. Configure SCIM Provisioning:
- - Go to the newly created Kestra application.
- - Select "Provisioning" and set the Provisioning Mode to "Automatic".
- - Enter the SCIM endpoint URL and the Secret Token provided by Kestra. Paste Kestra's SCIM endpoint URL into the Tenant URL field and the Secret Token into the Secret Token field.
- - Finally, click on `Test Connection` and on the `Save` button.
+### 2. Configure SCIM provisioning
+
+- Go to the newly created Kestra application.
+- Select **Provisioning** and set the Provisioning Mode to **Automatic**.
+- Enter the SCIM endpoint URL and Secret Token from Kestra: paste the SCIM endpoint into the **Tenant URL** field and the token into the **Secret Token** field.
+- Click **Test Connection**, then **Save**.
+
-### 3. Map User and Group Attributes:
+### 3. Map user and group attributes
-After entering and saving the **Admin Credentials** for the SCIM provisioning connection in Microsoft Entra ID (i.e., the Tenant URL and Secret Token), Azure will **enable the `Mappings` section** under the Provisioning settings.
+After entering and saving the **Admin Credentials** for the SCIM provisioning connection in Microsoft Entra ID — the Tenant URL and Secret Token — Azure will **enable the `Mappings` section** under the Provisioning settings.
The **Mappings** section allows you to define how user and group attributes should flow between Microsoft Entra ID and Kestra.
-#### SCIM Schema Support in Kestra
+#### SCIM schema support in Kestra
Kestra adheres to the [SCIM 2.0 specification (RFC 7643)](https://datatracker.ietf.org/doc/html/rfc7643#section-4), specifically supporting the following resource types:
- **User Resource**:
@@ -104,7 +101,7 @@ Kestra adheres to the [SCIM 2.0 specification (RFC 7643)](https://datatracker.ie
#### Retrieve supported schemas
-Kestra exposes SCIM resource schemas via its `/Schemas` endpoint exposed via the SCIM URL. This allows Microsoft Entra ID to discover the required attributes automatically.
+Kestra exposes SCIM resource schemas via its `/Schemas` endpoint at the SCIM URL. This allows Microsoft Entra ID to discover the required attributes automatically.
```plaintext
GET /api/v1//integrations//scim/v2/Schemas
@@ -116,7 +113,7 @@ Replace `` with your actual tenant, and `` with your act
This endpoint returns a list of supported schemas and their attributes. Use it as a reference when configuring attribute mappings in Entra ID.
-#### Configure user and group mapping
+#### Configure user and group mappings
To configure mappings:
@@ -136,15 +133,16 @@ To configure mappings:
By default, Azure will pre-populate the mapping with many Microsoft Entra ID attributes. You may need to **remove or simplify** some of these mappings if synchronization issues occur with users or groups in Kestra.
:::
-#### Test the Configuration
+#### Test the configuration
After mappings are configured:
- Trigger a **manual provisioning cycle** from the **Provisioning** tab.
- Verify that **users and groups** are correctly created or updated in Kestra.
- Review **provisioning logs** in Entra ID for any errors or warnings.
-### 4. Enable Provisioning:
- - Once everything is configured, you can enable the provisioning integration toggle in the Kestra UI to start syncing users and groups from Microsoft Entra ID to Kestra.
+### 4. Enable provisioning
+
+Enable the provisioning integration toggle in the Kestra UI to start syncing users and groups from Microsoft Entra ID.
## Additional resources
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/microsoft-entra-id/scim1.png b/src/contents/docs/07.enterprise/03.auth/scim/microsoft-entra-id/scim1.png
deleted file mode 100644
index ae3bc72c8fe..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/scim/microsoft-entra-id/scim1.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/okta/index.md b/src/contents/docs/07.enterprise/03.auth/scim/okta/index.md
index 264473fe389..d071aab1182 100644
--- a/src/contents/docs/07.enterprise/03.auth/scim/okta/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/scim/okta/index.md
@@ -9,8 +9,6 @@ version: ">= 0.18.0"
Sync users and groups from Okta to Kestra using SCIM.
-## Okta SCIM provisioning
-
## Prerequisites
- **Okta Account**: An account with administrative privileges is required to configure SCIM provisioning.
@@ -29,16 +27,14 @@ Tenants are enabled by default. Please refer to the [Migration Guide](../../../.
## Kestra SCIM setup: create a new provisioning integration
-1. In the Kestra UI, navigate to the `Tenant` → `IAM` → `SCIM Provisioning` page.
-2. Click on the `Create` button in the top right corner of the page.
+1. Go to **Instance Owner**, select your tenant from the sidebar, open **IAM**, and click the **SCIM Provisioning** tab.
+2. Click **+ Create**.
3. Fill in the following fields:
- **Name**: Enter a name for the provisioning integration.
- **Description**: Provide a brief description of the integration.
- - **Provisioning Type**: Currently, only SCIM 2.0 is supported — leave the default selection and click `Save`.
-
-
+ - **Provisioning Type**: Only SCIM 2.0 is supported — leave the default selection and click **Save**.
-The above steps will generate a SCIM endpoint URL and a Secret Token that you will use to authenticate Okta with the SCIM integration in Kestra. Save those details as we will need them in the next steps.
+These steps generate a SCIM endpoint URL and a Secret Token. Save both — you will need them in the next steps.
@@ -50,18 +46,15 @@ https:///api/v1//integrations/integration_id/scim
The Secret Token is a long string (approx. 200 characters) used to authenticate requests from Okta to Kestra.
-### Enable or Disable SCIM Integration
-
-Note that you can disable or completely remove the SCIM Integration at any time. When an integration is disabled, all incoming requests for that integration endpoint will be rejected.
-
-
+### Enable or disable SCIM integration
+You can disable or remove the SCIM integration at any time. When disabled, all incoming requests to that endpoint are rejected.
:::alert{type="info"}
-At first, you can disable the integration to configure your Okta SCIM integration, and then enable it once the configuration is complete.
+You can disable the integration while configuring Okta, then enable it once setup is complete.
:::
-### IAM Role and Service Account
+### IAM role and service account
When creating a new Provisioning Integration, Kestra will automatically create two additional objects:
@@ -78,57 +71,42 @@ When creating a new Provisioning Integration, Kestra will automatically create t
Why the `SCIMProvisioner` role doesn't have the `DELETE` permission for `USERS`? This is because you cannot delete a user through our SCIM implementation. Users are global and SCIM provisioning is per tenant. When we receive a `DELETE` query for a user, we remove their tenant access but the user itself remains in the system.
:::
----
-
## Okta SCIM setup
-1. **Create an App Integration**:
- - Navigate to Okta Admin Console → Applications → Applications.
- - Click on "Create App Integration" and then select:
- - Sign-in Method: **OIDC - OpenID Connect**
- - Application Type: Web Application
- - Then on the next page:
- - Give your application a name, e.g., `Kestra`
- - Grant Type: Client acting on behalf of itself → Client Credentials → True
- - Login
- - Sign-in redirect URIs → http:///oauth/callback/okta
- - Sign-out redirect URIs → http:///logout
- - Once application is created, select it in the Applications view and take note of the client ID and client secret.
+1. Navigate to Okta Admin Console → Applications → Applications and click **Create App Integration**:
+ - Sign-in Method: **OIDC - OpenID Connect**
+ - Application Type: Web Application
+ - Name: `Kestra`
+ - Grant Type: Client Credentials
+ - Sign-in redirect URIs: `http:///oauth/callback/okta`
+ - Sign-out redirect URIs: `http:///logout`
+
+ Note the client ID and client secret once the application is created.
+
-2. **Configure Okta in Kestra**:
- - With the above client ID and secret, add the following in your Kestra Micronaut configuration:
+2. Add the client credentials to your Kestra Micronaut configuration, and enter the SCIM endpoint URL and token provided by Kestra:
+
```yaml
- micronaut:
- security:
- oauth2:
- enabled: true
- clients:
- okta:
- client-id: "CLIENT_ID"
- client-secret: "CLIENT-SECRET"
- openid:
- issuer: "https://{okta-account}.okta.com/"
+ micronaut:
+ security:
+ oauth2:
+ enabled: true
+ clients:
+ okta:
+ client-id: “CLIENT_ID”
+ client-secret: “CLIENT-SECRET”
+ openid:
+ issuer: “https://{okta-account}.okta.com/”
```
- - Enter the SCIM endpoint URL and API token provided by Kestra.
-
-3. **Configure SCIM 2.0 in Okta**:
- - In Okta, navigate to Applications → Applications → Browse App Catalog
- - Search for SCIM 2.0
- - Select SCIM 2.0 Test App (OAuth Bearer Token)
- - in Sign-in options select Secure Web Authentication → user sets username/password
- - Click Done
- - Select the integration you have just created, then enter the `Provisioning` tab.
- - Fill in the SCIM 2.0 Base URL field with the endpoint URL you obtained from Kestra. Enter the Secret Token generated in Kestra into the `OAuth Bearer Token` field.
- - Finally, click `Test API Credentials` to verify the connection.
- 
-
-4. **Map Attributes**:
- - Select “Push Groups” and choose the Groups you wish to push to Kestra.
- - Perform a test to ensure that the mappings are correct and data is syncing properly.
-
-5. **Enable Provisioning**:
- - Enable the provisioning integration toggle in the Kestra UI to begin automatic synchronization of users and groups from Okta to Kestra.
+
+3. In Okta, navigate to **Applications → Applications → Browse App Catalog**, search for **SCIM 2.0**, and select **SCIM 2.0 Test App (OAuth Bearer Token)**. In Sign-in options, select **Secure Web Authentication**, then click **Done**. Select the new integration, open the **Provisioning** tab, and enter the Kestra SCIM endpoint URL in the Base URL field and the Secret Token in the **OAuth Bearer Token** field. Click **Test API Credentials** to verify.
+
+ 
+
+4. Select **Push Groups** and choose the groups to sync to Kestra. Run a test to verify the mappings.
+
+5. Enable the provisioning integration toggle in the Kestra UI to begin automatic synchronization.
## Additional resources
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/okta/scim1_okta.png b/src/contents/docs/07.enterprise/03.auth/scim/okta/scim1_okta.png
deleted file mode 100644
index b287c93640f..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/scim/okta/scim1_okta.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/scim/okta/scim3.png b/src/contents/docs/07.enterprise/03.auth/scim/okta/scim3.png
deleted file mode 100644
index 48cb6dc08d5..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/scim/okta/scim3.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/service-accounts/index.md b/src/contents/docs/07.enterprise/03.auth/service-accounts/index.md
index 7fa6171b580..7d4ad458560 100644
--- a/src/contents/docs/07.enterprise/03.auth/service-accounts/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/service-accounts/index.md
@@ -8,66 +8,46 @@ editions: ["EE", "Cloud"]
version: ">= 0.15.0"
---
-How to create and manage Service Accounts.
-
-## Service accounts – non-human access
+Service accounts represent applications or CI/CD systems that access Kestra programmatically, without a password or UI access. Each service account has a name, an optional description, an optional group assignment, and a list of roles that grant it permissions to specific resources.
-A Service Account represents an **application** that can access Kestra. It is not tied to a specific person and does not have personal information (such as the first name, last name, or email) attached to it. Instead, it only has a name, an optional description, an optional allocation to a group, and a list of Roles that grant it permissions to access specific resources.
-
## Service accounts vs. users
-In contrast to regular users, Service Accounts don't have a password and they do not have access to the Kestra UI — they only have a programmatic API access to Kestra. You can think of Service Accounts as bots authenticating with Kestra using an API token.
-
-## Creating a Service Account
-
-To create a new service account, go to **Service Accounts** tab on the **IAM** page under the **Tenant** section and click the **Create** button. Fill in the form with the required information, including the name and description and click **Save**:
-
-
-
-Once you have created a service account, you can add a Role that will grant it permissions to specific resources. To do this, switch to the **Access** tab and click the **Add** button and select the role you want to assign to the service account.
+Service accounts have no password and no access to the Kestra UI — they authenticate exclusively via API token. Users, by contrast, can interact with both the UI and the API using a password or an API token.
-
+## Creating a service account
-Finally, you can generate an API token for the service account by clicking the **Create API Token** button in the service account's details. This will generate a token that you can use to authenticate the service account with Kestra from external applications such as CI/CD pipelines (e.g., in Terraform provider configuration or GitHub Actions secrets).
+To create a new service account, go to **IAM** in the sidebar, open the **Service Accounts** tab, and click **Create**. Fill in the name and optional description, then click **Save**.
-:::alert{type="info"}
-**Note:** You can configure the token to expire after a certain period of time or to never expire. Also, there is a toggle called `Extended` that will automatically prolong the token's expiration date by the specified number of days (`Max Age`) if the token is actively used. That toggle is disabled by default.
-:::
+Once the service account is created, switch to the **Access** tab, click **Add**, and select the role to assign.
-Once you confirm the API token creation via the **Generate** button, the token will be generated and displayed in the UI. Make sure to copy the token and store it in a secure location as it will not be displayed again.
+To generate an API token, click **Create API Token** in the service account details. You can configure the token to expire after a set period or never expire. The **Extended** toggle (disabled by default) automatically resets the expiry each time the token is used. Click **Generate**, then copy the token immediately — it is shown only once.
-
+## Users, service accounts, and API tokens
-## Users vs. Service Accounts vs. API Tokens
+You can create an API token for a regular user as well. While service accounts are recommended for programmatic access from CI/CD or external applications, user API tokens are useful when you want programmatic actions to be tracked and audited against a specific person.
-You can create an **API token** for a regular user as well. While Service Accounts are recommended for programmatic API access to Kestra from CI/CD or other external applications, it's often useful to create an API token for a regular user, so that programmatic actions performed by that user can be tracked and audited.
-
-
-
-Therefore, the difference between a service account and a user is that a service account is designed for programmatic access and doesn't have a password or personal information attached to it. Instead, it is authenticated exclusively using an API token. A user, on the other hand, can interact with both the Kestra UI and the API, and can be authenticated using a password or an API token.
-
-## The purpose of service accounts
-
-Service Accounts are intended for programmatic access to Kestra from any other application, such as CI/CD pipelines or your own custom APIs. For example, you can use the token **to authenticate with Kestra Terraform provider or Kestra's GitHub Actions CI/CD pipeline**.
+The key difference: a service account has no password or personal information and is authenticated exclusively with an API token. A user can interact with both the UI and the API, using either a password or an API token.
## Allocating service accounts to groups
-Each Service Account can be attached to one or more Groups such as a group called “Bots” that centrally governs programmatic access for CI/CD across multiple projects with just one Role. This is useful to manage programmatic access used by Terraform, GitHub Actions, or other external applications, in one place by attaching a single Role to that Group.
+Each service account can be attached to one or more groups, such as a “Bots” group that centrally governs programmatic access for CI/CD across multiple projects with a single role. This simplifies managing Terraform, GitHub Actions, or other external application access in one place.
+
+## CLI authentication
-Speaking of CI/CD, note that Kestra currently supports authenticating with either a basic authentication user or an API token:
+When using the Kestra CLI, you can authenticate with either an API token or a username and password:
-1. Use the `--api-token=mytoken` CLI property to allow authenticating with a service account token:
+1. Use `--api-token` to authenticate with a service account token:
```bash
./kestra namespace files update prod scripts . \
--server=https://demo.kestra.io --api-token yourtoken
```
-2. Use the `--user user_email:password` flag to the CLI to allow authenticating with a Basic Authentication access:
+2. Use `--user` to authenticate with Basic Auth credentials:
```bash
./kestra namespace files update prod scripts . \
@@ -76,17 +56,17 @@ Speaking of CI/CD, note that Kestra currently supports authenticating with eithe
## Service account name convention
-When creating a new Service Account, make sure to follow the DNS naming convention. Specifically, the `name` property needs to:
+Follow the DNS naming convention when naming service accounts. The `name` property must:
- contain at most 63 characters
-- contain only lowercase alphanumeric characters or hyphens (i.e., the `-` character)
+- contain only lowercase alphanumeric characters or hyphens (`-`)
- start with an alphanumeric character
- end with an alphanumeric character.
-Some examples to make that clear:
+Examples:
- ✅ `my-service-account` is a valid name
- ✅ `my-service-account-1` is a valid name
- ❌ `MY_SERVICE_ACCOUNT` is not a valid name because it contains uppercase characters and underscores
- ❌ `myServiceAccount` is not a valid name because it contains uppercase characters and camel case
- ❌ `my-service-account-` is not a valid name because it ends with a hyphen.
-**Why do we follow such a restrictive convention?** We follow the standard DNS-style pattern to be ready for potential future use cases where we could, for example, forward the service account name to a Kubernetes pod's labels. This way, we ensure that the service account name can be used in a variety of contexts without any issues.
+Kestra uses the DNS-style naming convention so that service account names remain valid across contexts where they may be forwarded, such as Kubernetes pod labels.
diff --git a/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_create.png b/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_create.png
deleted file mode 100644
index 4da669987ac..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_create.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_create_2.png b/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_create_2.png
deleted file mode 100644
index 2a184d9f5aa..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_create_2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_create_3.png b/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_create_3.png
deleted file mode 100644
index 15bcc34388b..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_create_3.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_role.png b/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_role.png
deleted file mode 100644
index ec6894fb291..00000000000
Binary files a/src/contents/docs/07.enterprise/03.auth/service-accounts/service_account_role.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/03.auth/sso/authentik/index.md b/src/contents/docs/07.enterprise/03.auth/sso/authentik/index.md
index bc2816bc613..3d23423fec3 100644
--- a/src/contents/docs/07.enterprise/03.auth/sso/authentik/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/sso/authentik/index.md
@@ -6,25 +6,22 @@ sidebarTitle: authentik SSO
icon: /src/contents/docs/icons/tutorial.svg
editions: ["EE", "Cloud"]
---
-Set up authentik SSO to manage authentication for users.
-## Configure authentik SSO
+Set up authentik as an OIDC provider for Kestra authentication. In conjunction with SSO, see the [authentik SCIM provisioning guide](../../scim/authentik/index.md).
-In conjunction with SSO, check out the [authentik SCIM provisioning guide](../../scim/authentik/index.md).
+## Install authentik
-### Install authentik
-
-Authentik provides a simple docker-compose installer for testing purposes. Follow [the instructions](https://docs.goauthentik.io/docs/installation/docker-compose) and click on the [initial setup URL](http://docker.for.mac.localhost:9000/if/flow/initial-setup/) to create your first user.
+authentik provides a simple docker-compose installer for testing purposes. Follow [the instructions](https://docs.goauthentik.io/docs/installation/docker-compose) and click on the [initial setup URL](http://docker.for.mac.localhost:9000/if/flow/initial-setup/) to create your first user.
-### Create Application and SSO Provider in authentik
+## Create application and SSO provider in authentik
On the left-hand side, select **Applications → Applications**. For simplicity, we’ll use the **Create with Wizard** button, as this will create both an application and a provider.
-On the **Application Details** screen, fill in the application `name` and `slug`. Set both here to `kestra` and click `Next`.
+On the **Application Details** screen, fill in the application **Name** and **Slug**. Set both to `kestra` and click **Next**.
@@ -36,14 +33,14 @@ On the **Provider Configuration** screen:
1. In the **Authentication flow** field, select “default-authentication-flow (Welcome to authentik!)”.
2. In the **Authorization flow** field, select “default-provider-authorization-explicit-consent (Authorize Application)”.
-3. Keep the Client type as **Confidential**. Under the **Redirect URIs/Origins (RegEx)**, enter your Kestra host's `/oauth/callback/authentik` endpoint in the format `http://:/oauth/callback/authentik` (e.g., http://localhost:8080/oauth/callback/authentik) and then `Submit` the Application.
+3. Keep the Client type as **Confidential**. Under the **Redirect URIs/Origins (RegEx)**, enter your Kestra host's `/oauth/callback/authentik` endpoint in the format `http://:/oauth/callback/authentik` (e.g., http://localhost:8080/oauth/callback/authentik) and click **Submit**.
-Note the `Client ID` and `Client Secret` as you will need these to configure Kestra in the next step.
+Note the `Client ID` and `Client Secret` for the next step.
-### Configure Authentik SSO in Kestra Settings
+## Configure authentik SSO in Kestra
-With the above Client ID and Secret, add the following in the `micronaut` configuration section:
+Add the following to your `micronaut` configuration:
```yaml
micronaut:
@@ -58,11 +55,11 @@ With the above Client ID and Secret, add the following in the `micronaut` config
issuer: "http://localhost:9000/application/o/kestra/"
```
-You may need to adjust the above `issuer` URL if you named your application something other than `kestra`. Make sure to update that URL to match your application name `http://localhost:9000/application/o//`.
+You may need to adjust the above `issuer` URL if you named your application something other than `kestra`. Update that URL to match your application name: `http://localhost:9000/application/o//`.
-### Configure a Default Role for your SSO users in Kestra Settings
+## Configure a default role for SSO users
-To ensure that your SSO users have initial permissions within the Kestra UI, set up a default role for them. Achieve this by adding the following configuration under the `kestra.security` section:
+SSO users need a default role for initial access in Kestra. Add the following to `kestra.security`:
```yaml
kestra:
@@ -71,17 +68,53 @@ kestra:
name: default_admin_role
description: "Default Admin Role"
permissions:
- NAMESPACE: ["CREATE", "READ", "UPDATE", "DELETE"]
- ROLE: ["CREATE", "READ", "UPDATE", "DELETE"]
- GROUP: ["CREATE", "READ", "UPDATE", "DELETE"]
- EXECUTION: ["CREATE", "READ", "UPDATE", "DELETE"]
- AUDITLOG: ["CREATE", "READ", "UPDATE", "DELETE"]
- USER: ["CREATE", "READ", "UPDATE", "DELETE"]
- BINDING: ["CREATE", "READ", "UPDATE", "DELETE"]
- FLOW: ["CREATE", "READ", "UPDATE", "DELETE"]
- SECRET: ["CREATE", "READ", "UPDATE", "DELETE"]
- BLUEPRINT: ["CREATE", "READ", "UPDATE", "DELETE"]
- KVSTORE: ["CREATE", "READ", "UPDATE", "DELETE"]
+ FLOW:
+ - VIEW
+ - LIST
+ - CREATE
+ - UPDATE
+ - DELETE
+ - EXECUTE
+ - DISABLE
+ - ENABLE
+ - VALIDATE
+ - EXPORT
+ - IMPORT
+ EXECUTION:
+ - VIEW
+ - LIST
+ - UPDATE
+ - DELETE
+ - RESTART
+ - KILL
+ - REPLAY
+ - PAUSE
+ - RESUME
+ - CHANGE_LABELS
+ - ACCESS_LOGS
+ - ACCESS_OUTPUTS
+ - ACCESS_FILES
+ - EXPORT
+ - UNQUEUE
+ - FORCE_RUN
+ - FOLLOW
+ NAMESPACE:
+ - VIEW
+ - LIST
+ - CREATE
+ - UPDATE
+ - DELETE
+ - MANAGE_FILES
+ - EXPORT_PLUGIN_DEFAULTS
+ - IMPORT_PLUGIN_DEFAULTS
+ SECRET: ["VIEW", "LIST", "UPDATE", "DELETE"]
+ KVSTORE: ["VIEW", "LIST", "CREATE", "UPDATE", "DELETE"]
+ BLUEPRINT: ["VIEW", "LIST", "CREATE", "UPDATE", "DELETE"]
+ ROLE: ["VIEW", "LIST", "CREATE", "UPDATE", "DELETE"]
+ GROUP: ["VIEW", "LIST", "CREATE", "UPDATE", "DELETE", "MANAGE_MEMBERS"]
+ USER: ["VIEW", "LIST", "CREATE", "UPDATE", "DELETE", "MANAGE_GROUP_MEMBERSHIP"]
+ BINDING: ["VIEW", "LIST", "CREATE", "DELETE"]
+ AUDITLOG: ["VIEW", "LIST", "EXPORT"]
ee:
tenants:
enabled: true
@@ -89,5 +122,5 @@ kestra:
```
:::alert{type="info"}
-⚠️ Make sure that your `defaultRole` is added under the `kestra.security` section, not under `micronaut.security`. Also, ensure that the `defaultRole` has the necessary permissions for your users to interact with Kestra. The above configuration is just an example and you might want to restrict the permissions boundaries for production use.
+Place `defaultRole` under `kestra.security`, not under `micronaut.security`. The example above grants broad access — adjust the action lists to match the permissions your users actually need in production.
:::
diff --git a/src/contents/docs/07.enterprise/03.auth/sso/google-oidc/index.md b/src/contents/docs/07.enterprise/03.auth/sso/google-oidc/index.md
index 3e30d71d292..fba182ad4e1 100644
--- a/src/contents/docs/07.enterprise/03.auth/sso/google-oidc/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/sso/google-oidc/index.md
@@ -7,42 +7,28 @@ icon: /src/contents/docs/icons/tutorial.svg
editions: ["EE", "Cloud"]
---
-## Set up Google OIDC SSO
-
-This guide provides step-by-step instructions to configure **OpenID Connect (OIDC) authentication using Google Identity Platform** and link it to [**Kestra Enterprise**](../../../index.mdx) for [Single Sign-On (SSO)](../index.md).
+Authenticate Kestra users with their Google accounts using Google Identity Platform and OIDC.
## Prerequisites
-- **Google Cloud Project**: Ensure you have a Google Cloud project with billing enabled.
-- **Administrator Access**: You need sufficient permissions to configure Identity Platform and manage identity providers.
-- **Kestra Enterprise Edition**: Kestra SSO is available only in the Enterprise Edition.
-
-Refer to the [Google OIDC setup documentation](https://cloud.google.com/identity-platform/docs/web/oidc) for more details.
+- A Google Cloud project with billing enabled.
+- Sufficient permissions to configure Identity Platform and manage identity providers.
----
+See the [Google OIDC setup documentation](https://cloud.google.com/identity-platform/docs/web/oidc) for reference.
## Step 1: Enable Identity Platform in Google Cloud
-1. **Navigate to the Identity Platform**:
- - Go to the [Identity Platform page](https://console.cloud.google.com/identity) in the Google Cloud Console.
-
-2. **Confirm your project**:
- - Make sure that you have the correct project selected to add an identity provider to.
-
----
+1. Go to the [Identity Platform page](https://console.cloud.google.com/identity) in the Google Cloud Console.
+2. Confirm the correct project is selected.
## Step 2: Add an OIDC Provider in Google Cloud
-1. **Access Identity Providers**:
- - In the Identity Platform menu, select **Providers**.
-
-2. **Add a New Provider**:
- - Click on **Add a Provider**.
- - From the list, choose **OpenID Connect**.
+1. In the Identity Platform menu, select **Providers**.
+2. Click **Add a Provider** and choose **OpenID Connect**.
-3. **Configure the OIDC Provider**:
+3. Configure the OIDC Provider:
- **Grant type**: Select the Code Flow grant type.
- **Provider Name**: Enter a display name for the OIDC provider.
- **Client ID**: Enter the **Client ID** obtained from Google.
@@ -52,20 +38,11 @@ Refer to the [Google OIDC setup documentation](https://cloud.google.com/identity
-4. **Save the Configuration**:
- - Click **"Save"** to add the OIDC provider to your Identity Platform configuration.
-
----
-
-## Step 3: Configure Kestra to Use Google as an OIDC SSO Provider
-
-Now that Google is set up as an OIDC provider, we need to link it to Kestra.
+4. Click **Save** to add the provider.
-1. **Navigate to the Kestra Configuration File**:
- - Locate the [Kestra Security and Secrets configuration](../../../../configuration/05.security-and-secrets/index.md) file.
+## Step 3: Configure Kestra
-2. **Add the OIDC Settings**:
- - Add the following configuration to enable Google as an OIDC provider for Kestra:
+Add the following to your [Kestra Security and Secrets configuration](../../../../configuration/05.security-and-secrets/index.md):
```yaml
micronaut:
@@ -79,13 +56,9 @@ Now that Google is set up as an OIDC provider, we need to link it to Kestra.
openid:
issuer: 'https://accounts.google.com'
```
-- Replace `clientId` and `clientSecret` with the values from the Google Identity Platform.
-- Update the `redirectUri` with your Kestra instance URL.
-- Restart Kestra to apply the changes.
+Replace `clientId` and `clientSecret` with the values from the Google Identity Platform, then restart Kestra.
-## Additional Resources
+## Additional resources
- [Managing SAML and OIDC Providers Programmatically](https://cloud.google.com/identity-platform/docs/managing-providers-programmatically)
- [Identity Platform Documentation](https://cloud.google.com/identity-platform/docs)
-
-By following these steps, you can successfully set up OIDC authentication using Google Identity Platform, allowing users to sign in with their existing credentials via your chosen OIDC provider.
diff --git a/src/contents/docs/07.enterprise/03.auth/sso/index.md b/src/contents/docs/07.enterprise/03.auth/sso/index.md
index 5538dce74bf..35ad3000b18 100644
--- a/src/contents/docs/07.enterprise/03.auth/sso/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/sso/index.md
@@ -7,19 +7,15 @@ icon: /src/contents/docs/icons/admin.svg
editions: ["EE", "Cloud"]
---
-How to enable and set up SSO in your Kestra Enterprise instance.
-
-## Configure single sign-on
-
-Single Sign-On (SSO) is an authentication process that allows users to access multiple applications with a single set of login credentials (e.g., "Sign in with Google"). Kestra supports SSO using the OpenID Connect (OIDC) protocol, which is a simple identity layer built on top of the OAuth 2.0 protocol.
+Single Sign-On (SSO) lets users authenticate to Kestra using an external identity provider such as Google, Microsoft, Okta, or Keycloak.
-## Configuring single sign-on with OpenID Connect (OIDC)
+## Configure OIDC
-To implement OIDC SSO, you'll need to configure the Micronaut framework that Kestra uses under the hood. Start by enabling OIDC in your `yaml` configuration file as follows:
+Kestra supports SSO using the OpenID Connect (OIDC) protocol, a simple identity layer built on top of OAuth 2.0. Enable OIDC in your configuration file:
```yaml
micronaut:
@@ -40,9 +36,10 @@ For more configuration details, refer to the [Micronaut OIDC configuration guide
## Provider guides
-Check out our guides for specific SSO providers:
+Provider-specific setup guides:
- [Google](/docs/enterprise/auth/sso/google-oidc)
- [Microsoft](/docs/enterprise/auth/sso/microsoft-oidc)
- [Keycloak](/docs/enterprise/auth/sso/keycloak)
- [Okta](/docs/enterprise/auth/sso/okta)
- [authentik](/docs/enterprise/auth/sso/authentik)
+- [LDAP](/docs/enterprise/auth/sso/ldap)
diff --git a/src/contents/docs/15.how-to-guides/keycloak/01-groups_create_client_scope.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/01-groups_create_client_scope.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/01-groups_create_client_scope.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/01-groups_create_client_scope.png
diff --git a/src/contents/docs/15.how-to-guides/keycloak/02-add-mappers.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/02-add-mappers.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/02-add-mappers.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/02-add-mappers.png
diff --git a/src/contents/docs/15.how-to-guides/keycloak/03-configure-mappers.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/03-configure-mappers.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/03-configure-mappers.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/03-configure-mappers.png
diff --git a/src/contents/docs/15.how-to-guides/keycloak/04-mapper-details.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/04-mapper-details.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/04-mapper-details.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/04-mapper-details.png
diff --git a/src/contents/docs/15.how-to-guides/keycloak/05-add_client_scope.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/05-add_client_scope.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/05-add_client_scope.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/05-add_client_scope.png
diff --git a/src/contents/docs/15.how-to-guides/keycloak/client1.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/client1.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/client1.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/client1.png
diff --git a/src/contents/docs/15.how-to-guides/keycloak/client2.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/client2.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/client2.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/client2.png
diff --git a/src/contents/docs/15.how-to-guides/keycloak/clientId.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/clientId.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/clientId.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/clientId.png
diff --git a/src/contents/docs/15.how-to-guides/keycloak/clientSecret.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/clientSecret.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/clientSecret.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/clientSecret.png
diff --git a/src/contents/docs/07.enterprise/03.auth/sso/keycloak/index.md b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/index.md
index 529755e9cf6..560cd3d37b8 100644
--- a/src/contents/docs/07.enterprise/03.auth/sso/keycloak/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/index.md
@@ -6,11 +6,8 @@ sidebarTitle: Keycloak SSO
icon: /src/contents/docs/icons/tutorial.svg
editions: ["EE", "Cloud"]
---
-Set up Keycloak SSO to manage authentication for users.
-## Configure Keycloak SSO
-
-In conjunction with SSO, check out the [Keycloak SCIM provisioning guide](../../scim/keycloak/index.md).
+Set up Keycloak as an OIDC provider for Kestra authentication. In conjunction with SSO, see the [Keycloak SCIM provisioning guide](../../scim/keycloak/index.md).
## Start a Keycloak service
@@ -22,12 +19,12 @@ You can follow the steps described in the [Keycloak tutorial documentation](http
Once in Keycloak, create a new client:
-
-
+
+
Set `https://{{ yourKestraInstanceURL }}/oauth/callback/keycloak` as the valid redirect URI and `https://{{ yourKestraInstanceURL }}/logout` as the valid post-logout redirect URI.
-
+
## Kestra Configuration
@@ -47,12 +44,12 @@ micronaut:
get-allowed: true
```
-You can retrieve the `clientId` and `clientSecret` via Keycloak user interface
+You can retrieve the `clientId` and `clientSecret` via the Keycloak user interface.
-
-
+
+
-Don't forget to set a default role in your [Kestra Security and Secrets configuration](../../../../configuration/05.security-and-secrets/index.md) to streamline the process of onboarding new users.
+Set a default role in your [Kestra Security and Secrets configuration](../../../../configuration/05.security-and-secrets/index.md) to assign initial permissions to new SSO users.
```yaml
kestra:
@@ -61,16 +58,40 @@ kestra:
name: Editor
description: Default Editor role
permissions:
- FLOW: ["CREATE", "READ", "UPDATE", "DELETE"]
- EXECUTION:
+ FLOW:
+ - VIEW
+ - LIST
- CREATE
- - READ
- UPDATE
- DELETE
+ - EXECUTE
+ - DISABLE
+ - ENABLE
+ - VALIDATE
+ - EXPORT
+ - IMPORT
+ EXECUTION:
+ - VIEW
+ - LIST
+ - UPDATE
+ - DELETE
+ - RESTART
+ - KILL
+ - REPLAY
+ - PAUSE
+ - RESUME
+ - CHANGE_LABELS
+ - ACCESS_LOGS
+ - ACCESS_OUTPUTS
+ - ACCESS_FILES
+ - EXPORT
+ - UNQUEUE
+ - FORCE_RUN
+ - FOLLOW
```
:::alert{type="info"}
-Note: depending on the Keycloak configuration, you might want to tune the issuer URL.
+Depending on your Keycloak configuration, you may need to adjust the issuer URL.
:::
For more configuration details, refer to the [Keycloak OIDC configuration guide](https://guides.micronaut.io/latest/micronaut-oauth2-keycloak-gradle-java.html).
@@ -115,25 +136,25 @@ micronaut:
These two settings are independent: `end-session` controls what happens on logout, while `prompt=login` controls what happens on the next login attempt.
-## Manage Groups via OIDC Claims
+## Manage groups via OIDC claims
If you are unable to use [SCIM with Keycloak](../../scim/keycloak/index.md), you can configure Kestra to source user groups from OIDC claims. In this setup, Keycloak acts as the single source of truth for user group membership. This method requires creating a `groups` client scope that exposes group membership via a claim in the ID Token.
-### Create a Groups Client Scope
+### Create a groups client scope
In Keycloak, go to **Client Scopes** and click **Create Client Scope**. Name it `groups`, set Type to **Default**, and keep Protocol as **OpenID Connect**.
-
+
-### Add a Group Membership Mapper
+### Add a group membership mapper
In the newly created `groups` scope, go to the **Mappers** tab and click **Configure a new mapper**.
-
+
Select **Group Membership** from the list of available mapper types.
-
+
Configure the mapper with the following settings:
- **Name**: `groups`
@@ -141,13 +162,13 @@ Configure the mapper with the following settings:
- **Full group path**: Off
- **Add to ID token**: On
-
+
-### Add the Client Scope to Your Client
+### Add the client scope to your client
Go to **Clients**, select your Kestra client, and add the `groups` client scope.
-
+
### Configure Kestra
diff --git a/src/contents/docs/15.how-to-guides/keycloak/redirect-uri.png b/src/contents/docs/07.enterprise/03.auth/sso/keycloak/redirect-uri.png
similarity index 100%
rename from src/contents/docs/15.how-to-guides/keycloak/redirect-uri.png
rename to src/contents/docs/07.enterprise/03.auth/sso/keycloak/redirect-uri.png
diff --git a/src/contents/docs/07.enterprise/03.auth/sso/ldap/index.md b/src/contents/docs/07.enterprise/03.auth/sso/ldap/index.md
index 5cc2e82d8a4..a859388f936 100644
--- a/src/contents/docs/07.enterprise/03.auth/sso/ldap/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/sso/ldap/index.md
@@ -1,19 +1,25 @@
---
-title: "LDAP Authentication in Kestra: Directory Login"
+title: "LDAP Authentication in Kestra: Directory Login and Group Sync"
h1: Connect Your LDAP Directory for User Login and Group Sync
-description: Enable LDAP authentication in Kestra. Connect your existing LDAP directory to manage user login and group synchronization securely.
+description: Enable LDAP authentication in Kestra. Use your LDAP directory for user login, group synchronization, or both — including alongside an existing SSO provider.
sidebarTitle: LDAP
icon: /src/contents/docs/icons/admin.svg
-editions: ["EE"]
+editions: ["EE", "Cloud"]
version: "0.22.0"
---
-Enable LDAP authentication in Kestra to authenticate users against your existing directory and sync group memberships automatically.
+Enable LDAP authentication to authenticate users against your existing directory, sync group memberships, or both. You can also use LDAP solely for group sync while keeping an existing SSO provider for login.
+
+## Configure LDAP authentication
+:::alert{type="warning"}
+LDAP is a licensed feature. If `micronaut.security.ldap.default` is configured but your license does not include LDAP, Kestra will refuse to start with the error: `LDAP is not supported by your license`. Contact your Kestra account team to enable it.
+:::
+
## What is LDAP
Lightweight directory access protocol (LDAP) allows applications to quickly query user information. Organizations use directories to store usernames, passwords, email addresses, and other static data. LDAP is an open, vendor-neutral protocol for accessing and managing that data.
@@ -22,11 +28,19 @@ With Kestra, you can use an existing LDAP directory to authenticate users and sy
## Configuration
-LDAP is configured under the security context of your [Kestra Security and Secrets configuration](../../../../configuration/05.security-and-secrets/index.md) file.
+LDAP is configured under the security context of your [Kestra Security and Secrets configuration](../../../../configuration/05.security-and-secrets/index.md) file.
-[LDAP with Micronaut](https://micronaut-projects.github.io/micronaut-security/4.11.3/guide/#ldap) supports `context`, `search`, and `groups` as core configuration properties supported out of the box. These properties define the connection context, user attribute mapping, and group filtering needed to synchronize users and their group memberships with Kestra.
+[LDAP with Micronaut](https://micronaut-projects.github.io/micronaut-security/4.11.3/guide/#ldap) supports `context`, `search`, and `groups` as core configuration properties. These define the connection context, user attribute mapping, and group filtering needed to synchronize users and their group memberships with Kestra.
-The `user-attributes` section maps LDAP attributes such as `givenName`, `sn`, and `mail` to the corresponding Kestra user properties (first name, last name, and email).
+The `user-attributes` section maps LDAP attributes such as `givenName`, `sn`, and `mail` to Kestra user properties (first name, last name, and email).
+
+The `mode` property controls how Kestra uses the LDAP connection:
+
+| Mode | Description |
+|---|---|
+| `AUTHENTICATION` | LDAP handles user login only. No group sync. **This is the default.** |
+| `AUTHENTICATION_AND_GROUP_SYNC` | LDAP handles both user login and group membership sync. |
+| `GROUP_SYNC_ONLY` | LDAP is used only to resolve group memberships. Users log in via an existing SSO provider. |
The examples below extend the base Micronaut LDAP configuration with these Kestra-specific mappings.
@@ -37,6 +51,7 @@ micronaut:
security:
ldap:
default:
+ mode: AUTHENTICATION_AND_GROUP_SYNC # or AUTHENTICATION to skip group sync
user-attributes:
firstName: givenName
lastName: sn
@@ -58,6 +73,7 @@ micronaut:
base: "ou=groups,dc=example,dc=org"
filter: "{&(objectClass=posixGroup)(memberUid={0})}"
filter-attribute: uid
+ attribute: cn
```
### Windows configuration
@@ -68,6 +84,7 @@ micronaut:
ldap:
default:
enabled: true
+ mode: AUTHENTICATION_AND_GROUP_SYNC # or AUTHENTICATION to skip group sync
user-attributes:
firstName: givenName
lastName: sn
@@ -89,6 +106,7 @@ micronaut:
base: "DC=domain,DC=local"
filter: "(&(objectClass=group)(member={0}))"
filter-attribute: dn
+ attribute: cn
```
Key points for Windows Active Directory:
@@ -144,16 +162,62 @@ Get-ADGroupMember -Identity "CN=Auto,OU=Distro,OU=Groups,DC=kestra,DC=local" | S
Replace the identity string with the DN of your target group.
+### Group sync with SSO (GROUP_SYNC_ONLY)
+
+If your users already authenticate via SSO, Basic auth, or Passwordless, you can use LDAP solely to resolve group memberships without changing how users log in. Set `mode: GROUP_SYNC_ONLY` and configure the `groups` block. No `user-attributes` mapping is required.
+
+```yaml
+micronaut:
+ security:
+ ldap:
+ default:
+ mode: GROUP_SYNC_ONLY
+ context:
+ server: "ldap://localhost:389"
+ manager-dn: "cn=admin,dc=kestra,dc=io"
+ manager-password: "LDAP_ADMIN_PASSWORD"
+ search:
+ base: "ou=users,dc=kestra,dc=io"
+ filter: "(mail={0})"
+ groups:
+ enabled: true
+ base: "ou=groups,dc=kestra,dc=io"
+ filter: "(member={0})"
+ attribute: cn
+```
+
+With this configuration:
+- Users log in using their SSO provider. LDAP credentials are never checked.
+- At each login, Kestra queries the LDAP directory for the user's group memberships and merges them with any groups sourced from OIDC claims.
+- Groups found in LDAP are synced to Kestra using the same rules as standard LDAP group sync — new groups are created automatically, and membership is updated on login.
+
+Two `groups` properties control how Kestra reads group entries from the directory:
+- `filter`: the LDAP search filter used to find groups for a user. `{0}` is replaced with the user's distinguished name (DN).
+- `attribute`: the attribute on the group entry whose value becomes the Kestra group name. Defaults to `cn`.
+- `filter-attribute`: the user entry attribute substituted into `{0}` in the group filter. Use `dn` for directories that store full DNs in group membership attributes (common in Active Directory). Use `uid` for POSIX-style directories.
+
+:::alert{type="info"}
+`GROUP_SYNC_ONLY` mode requires that the user already exists in Kestra (created on first login). LDAP group sync fires on every subsequent login.
+:::
+
+:::alert{type="warning"}
+If the LDAP server is unreachable or misconfigured, group sync fails silently — the user logs in successfully but receives no LDAP-sourced groups. Check server connectivity and `groups` configuration if group assignments are not appearing after login.
+:::
+
## LDAP users in Kestra
-Once LDAP is configured, when a user logs into Kestra for the first time, their credentials are validated against the LDAP directory, and a corresponding user is created in Kestra. If a matching account already exists in Kestra, the user is authenticated using their LDAP credentials.
+On first login with LDAP, Kestra validates the user's credentials against the directory and creates a corresponding user account. If an account already exists, the user authenticates with their LDAP credentials.
-If they are a part of any groups specified in the directory, those groups will be added to Kestra. If the group already exists in Kestra, they will be automatically added. If a user is added to a group after their initial login, they must log out and log back in for the new group assignment to sync, as synchronization occurs only at login. Any user authenticated via LDAP will show `LDAP` as their Authentication method in the **IAM - Users** tab in Kestra.
+If the user belongs to any groups in the directory, those groups are created in Kestra if they don't exist, and the user is added to each. Group assignments sync only at login — if a user is added to a group in LDAP after their first login, they must log out and back in to pick up the new membership. Any user authenticated via LDAP shows `LDAP` in the **Login & API Tokens** column on the **IAM → Users** page.
Any updates to a user and their group access on the LDAP server will update in Kestra at the next synchronization (typically at the next login).
+:::alert{type="info"}
+Users who log in via SSO with `GROUP_SYNC_ONLY` mode show their SSO provider in the **Login & API Tokens** column on the **IAM → Users** page, not `LDAP`. The LDAP connection is used only to resolve group memberships in the background.
+:::
+
:::alert{type="warning"}
If a user is deleted from the LDAP server, they will lose access to Kestra at the next synchronization or login attempt.
:::
diff --git a/src/contents/docs/07.enterprise/03.auth/sso/microsoft-oidc/index.md b/src/contents/docs/07.enterprise/03.auth/sso/microsoft-oidc/index.md
index 8d91d532b78..f48f2778662 100644
--- a/src/contents/docs/07.enterprise/03.auth/sso/microsoft-oidc/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/sso/microsoft-oidc/index.md
@@ -7,9 +7,11 @@ icon: /src/contents/docs/icons/admin.svg
editions: ["EE", "Cloud"]
---
-## Set up Microsoft OIDC SSO
+Authenticate Kestra users with their Microsoft Entra ID credentials using OIDC.
-To configure Microsoft authentication, follow these steps:
+## Configure Microsoft OIDC
+
+To authenticate with Microsoft, add the following to your Micronaut configuration:
```yaml
micronaut:
@@ -26,7 +28,7 @@ micronaut:
To get your `client-id` and `client-secret`, refer to the [Microsoft Documentation](https://learn.microsoft.com/en-us/entra/identity-platform/v2-protocols-oidc).
-## Using Microsoft Entra ID as an OIDC SSO provider
+## Using Microsoft Entra ID as an OIDC provider
### Create an Enterprise Application
@@ -34,9 +36,9 @@ To get your `client-id` and `client-secret`, refer to the [Microsoft Documentati
2. Select **Microsoft Entra ID**.
3. Navigate to **App registrations**.
4. Click on **New registration** and provide the necessary details:
-- Enter a name for your application.
-- Set **Supported account types** (e.g., "Default Directory only - Single tenant").
-- Under **Redirect URI**, select *Web* and enter `https://{{ url }}/oauth/callback/microsoft`. Be sure to use `https` and the actual URL of your webserver.
+ - Enter a name for your application.
+ - Set **Supported account types** (e.g., "Default Directory only - Single tenant").
+ - Under **Redirect URI**, select *Web* and enter `https://{{ url }}/oauth/callback/microsoft`. Use `https` and your actual webserver URL.
### Generate client secret
@@ -47,11 +49,11 @@ To get your `client-id` and `client-secret`, refer to the [Microsoft Documentati
### Kestra configuration
- Copy the **Application (client) ID** from the **Overview** section and use it as your `{{ clientId }}`.
-- In the **Endpoints** section, locate the **OpenID Connect metadata document** URL. Remove the `.well-known/openid-configuration` suffix, and use the remainining base URL as your `{{ issuerUrl }}`.
+- In the **Endpoints** section, locate the **OpenID Connect metadata document** URL. Remove the `.well-known/openid-configuration` suffix, and use the remaining base URL as your `{{ issuerUrl }}`.
The final URL should look like `https://login.microsoftonline.com/{{ directory }}/v2.0/`.
-Here's an example Microsoft OIDC configuration:
+Example configuration:
```yaml
micronaut:
@@ -66,4 +68,4 @@ micronaut:
issuer: '{{ issuerUrl }}'
```
-With these settings, Kestra is now configured to use OIDC for SSO with your chosen providers. Ensure that all placeholders are replaced with the actual values obtained during the provider's setup process.
+Replace all placeholders with the values obtained from Entra ID.
diff --git a/src/contents/docs/07.enterprise/03.auth/sso/okta/index.md b/src/contents/docs/07.enterprise/03.auth/sso/okta/index.md
index 058461ede05..2dd8e8f2dae 100644
--- a/src/contents/docs/07.enterprise/03.auth/sso/okta/index.md
+++ b/src/contents/docs/07.enterprise/03.auth/sso/okta/index.md
@@ -7,17 +7,11 @@ icon: /src/contents/docs/icons/admin.svg
editions: ["EE", "Cloud"]
---
-## Set up Okta OIDC SSO
-
-This guide provides step-by-step instructions to configure **OpenID Connect (OIDC) authentication using Okta** and link it to [**Kestra Enterprise**](../../../01.overview/index.mdx) for [Single Sign-On (SSO)](./index.md).
-
## Prerequisites
-- **Okta Developer Account**: Ensure you have an Okta Developer Account or Organization.
-- **Administrator Access**: You need sufficient permissions to configure Identity Platform and manage identity providers.
-- **Kestra Enterprise Edition**: Kestra SSO is available only in the Enterprise Edition.
+- An Okta Developer Account or Organization with administrative access.
-This guide covers setup with Okta from a high level, refer to the [Okta OIDC setup documentation](https://help.okta.com/oie/en-us/content/topics/apps/apps_app_integration_wizard_oidc.htm) for more details.
+For more detail, refer to the [Okta OIDC setup documentation](https://help.okta.com/oie/en-us/content/topics/apps/apps_app_integration_wizard_oidc.htm).
## Step 1: Create an App Integration
@@ -25,7 +19,7 @@ Log in to your Okta account and select **Applications** from the left side menu.
-Next, select **Create App Integration**, select **OIDC - OpenID Connect** as the sign-in method and **Web Application** as the application type. Select **Next**, and you will be taken to configure the general settings of the new web app integration.
+Next, select **Create App Integration**, select **OIDC - OpenID Connect** as the sign-in method and **Web Application** as the application type. Select **Next** to configure the general settings of the new web app integration.
@@ -38,13 +32,13 @@ In the General Settings, give your App integration a name and set your grant typ
Here, you also set the **Sign-in redirect URIs** and **Sign-out redirect URIs** for your App integration. For this example connecting to Kestra, we set a Sign-in redirect URI as `http://localhost:8080/oauth/callback/okta` and sign-out as `http://localhost:8080/logout`, but you can customize this to your environment.
Further down the page, you can configure optional **Trusted Origins**, and then choose the **Assignments** and the access settings for the App integration.
-We'll set the access to everyone in the organization, but you can set stricter access to only certain selected groups or skip for now. Lastly, we uncheck the setting to enable immediate access with Federation Broker Mode because we will give manual app access for this basic example. Finally, hit **Save**.
+We'll set the access to everyone in the organization, but you can set stricter access to only certain selected groups or skip for now. Lastly, we uncheck the setting to enable immediate access with Federation Broker Mode because we will give manual app access for this basic example. Click **Save**.
## Step 3: Add test user to Okta app integration
-To create a test user in your Okta Directory to test your app integration, in your Okta Admin Dashboard, navigate to **Directory > People**. Select **Add Person**.
+To create a test user in your Okta Directory to test your app integration, in your Okta Admin Dashboard, navigate to **Directory → People**. Select **Add Person**.
@@ -54,23 +48,19 @@ In the **Directory**, select the new user, and navigate to the **Applications**
-Select the Kestra application name you created and enter the added details for the user and hit **Save**.
+Select the Kestra application, enter the required details for the user, and click **Save**.
## Step 4: Connect to Kestra
-Now that Okta is set up as an OIDC provider, we need to link it to Kestra. After saving your settings in the previous step, Okta will automatically redirect you to your integration. Here, you can collect your client credentials to connect to Kestra, **Client ID** and **Client Secret**.
+After saving, Okta redirects you to your integration, where you can find your **Client ID** and **Client Secret**.
-After copying your **Client ID** and **Client Secret**, switch from the **General** tab to the **Sign On** tab. Here, you can configure your **OpenID Connect ID Token**. For this example, we will edit the issuer from Dynamic to our Okta URL. Click **Save** and copy the URL to be used in our [Kestra Security and Secrets configuration](../../../../configuration/05.security-and-secrets/index.md) along with the Client ID and Client Secret.
+After copying your **Client ID** and **Client Secret**, switch to the **Sign On** tab. Under **OpenID Connect ID Token**, change the issuer from Dynamic to your Okta URL. Click **Save** and copy the URL for use in your [Kestra Security and Secrets configuration](../../../../configuration/05.security-and-secrets/index.md).
-1. **Navigate to the Kestra Configuration File**:
- - Locate the [Kestra Security and Secrets configuration](../../../../configuration/05.security-and-secrets/index.md) file.
-
-2. **Add the OIDC Settings**:
- - Add the following configuration to enable Okta as an OIDC provider for Kestra:
+Add the following configuration to enable Okta as an OIDC provider:
```yaml
micronaut:
@@ -85,13 +75,13 @@ After copying your **Client ID** and **Client Secret**, switch from the **Genera
issuer: 'https://.okta.com'
```
- Replace `clientId` and `clientSecret` with the values copied from the Okta App integration.
-- Replace `issuer` with your issuer URL from Application's sign on settings from before.
+- Replace `issuer` with your issuer URL from the application's sign-on settings.
- Restart Kestra to apply the changes and log in.
-On restart, you will now see Okta as an available login method.
+On restart, Okta appears as an available login method.
-After logging in with the created user, navigate to the **Administration > IAM** tab, and you can see in the **Users** tab that the user can sign in with basic authentication as well as Okta.
+After logging in, go to **IAM → Users** to confirm the user shows both login methods in the **Login & API Tokens** column.
diff --git a/src/contents/docs/07.enterprise/04.scalability/apps/app-editor-views.png b/src/contents/docs/07.enterprise/04.scalability/apps/app-editor-views.png
index c2795831d34..d690afd25eb 100644
Binary files a/src/contents/docs/07.enterprise/04.scalability/apps/app-editor-views.png and b/src/contents/docs/07.enterprise/04.scalability/apps/app-editor-views.png differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/apps/app-no-code.png b/src/contents/docs/07.enterprise/04.scalability/apps/app-no-code.png
deleted file mode 100644
index b30da974c0a..00000000000
Binary files a/src/contents/docs/07.enterprise/04.scalability/apps/app-no-code.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/apps/apps-catalog-customization.png b/src/contents/docs/07.enterprise/04.scalability/apps/apps-catalog-customization.png
index 1c00ffc1b8d..f5b2bacdc7d 100644
Binary files a/src/contents/docs/07.enterprise/04.scalability/apps/apps-catalog-customization.png and b/src/contents/docs/07.enterprise/04.scalability/apps/apps-catalog-customization.png differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/apps/apps_catalog.png b/src/contents/docs/07.enterprise/04.scalability/apps/apps_catalog.png
index 9b4ef72e425..0515fa47caa 100644
Binary files a/src/contents/docs/07.enterprise/04.scalability/apps/apps_catalog.png and b/src/contents/docs/07.enterprise/04.scalability/apps/apps_catalog.png differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/apps/customized-catalog.png b/src/contents/docs/07.enterprise/04.scalability/apps/customized-catalog.png
index 856c9e196ff..18ea72f5c42 100644
Binary files a/src/contents/docs/07.enterprise/04.scalability/apps/customized-catalog.png and b/src/contents/docs/07.enterprise/04.scalability/apps/customized-catalog.png differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/apps/index.md b/src/contents/docs/07.enterprise/04.scalability/apps/index.md
index 8741fbeb427..c5f32358f65 100644
--- a/src/contents/docs/07.enterprise/04.scalability/apps/index.md
+++ b/src/contents/docs/07.enterprise/04.scalability/apps/index.md
@@ -15,41 +15,42 @@ Build custom UIs to interact with Kestra from the outside world.
-## Apps – build frontends for Flows
+## What are Apps
-Apps let you use your Kestra workflows as the backend for custom applications. Within each app, you can specify custom frontend blocks, such as forms for data entry, output displays, approval buttons, or markdown blocks.
+Apps let you wrap any Kestra flow in a custom UI — forms for data entry, output displays, approval buttons, progress indicators, and markdown blocks — without writing any frontend code. Building and serving a frontend, connecting it to Kestra's API, validating user inputs, handling responses, managing workflow outputs, and handling authentication and authorization — Apps take care of all of that.
-**Flows** act as the **backend**, processing data and executing tasks, while **Apps** serve as the **frontend**, allowing anyone to interact with your workflows regardless of their technical background.
+Flows act as the backend, processing data and executing tasks. Apps act as the frontend, giving anyone — including business users without Kestra access — a clean interface to trigger executions, submit approvals, and view results.
-Business users can trigger new workflow executions, manually approve workflows that are paused, submit data to automated processes using simple forms, and view the execution results.
-
-You can think of Apps as **custom UIs for flows**. They are useful both for external-facing forms and for internal workflows such as approvals, requests, and guided operations.
+Apps work well for external-facing forms and for internal workflows such as approvals, requests, and guided operations.
---
-## Common App use cases
+## Common App patterns
-Most Apps fall into one of these two patterns:
-- **Execution forms**: users submit a form that starts a new execution with input parameters. For example, a requester can specify resources that need to be provisioned, and those inputs feed directly into a flow.
-- **Approval or resume interfaces**: users review a paused execution and approve, reject, or resume it. For example, a platform team can validate a provisioning request before the flow continues.
+Start with one of these patterns when designing your own App:
-## App benefits
+- **FTP upload portal**: give users a simple upload form while Kestra handles the backend credentials and transfer logic. See the [business user Apps blog example](../../../../blogs/use-case-apps/index.md#requests--review).
+- **Self-serve analytics request**: let users choose a dimension and time range, run a query and chart generation flow, and return the generated output on `SUCCESS`. See the [dynamic self-serve example](../../../../blogs/use-case-apps/index.md#dynamic-self-serve).
+- **AI-assisted intake or user research assistant**: collect free-form context from a sales, product, or support team member, run an LLM-backed flow, and display the suggested answer or categorization back in the App. See the [everyday automation example](../../../../blogs/use-case-apps/index.md#simple-interfaces-for-everyday-automation).
+- **VM or infrastructure request**: collect the requested environment, size, region, and justification on `OPEN`, show validation progress on `RUNNING`, pause for approval on `PAUSE`, then display the created VM details on `SUCCESS`. This pattern also fits the infrastructure workflows described in the [infrastructure automation blog](../../../../blogs/infra-automation/index.md).
+- **Human-in-the-loop review**: display task outputs, logs, or model results, then let an approver accept or reject the execution from the same screen.
-Apps offer custom UIs on top of your Kestra workflows. Often, workflows are designed for non-technical users, and creating custom frontends for each of these workflows can be a lot of work. Imagine having to build and serve a frontend, connect it to Kestra’s API, validate user inputs, handle responses, manage workflow outputs, and deal with authentication and authorization — all from scratch. Apps generate a custom UI for any flow without custom frontend development.
+Start by mapping the user journey:
-Here are some common scenarios where a custom UI is useful:
+1. What should the user submit?
+2. What should they see while the flow is running?
+3. Does the flow need approval or review?
+4. What is the final outcome you want to show back in the App?
-- **Manual Approval**: workflows that need manual approval, such as provisioning resources, granting access to services, deploying apps, validating data results, or reviewing AI-generated outputs.
-- **Report Generation**: workflows where business users request data and receive a downloadable CSV or Excel file.
-- **IT Helpdesk**: workflows that accept bug reports, feature requests, or other tickets, and automatically forward the ticket to the relevant team.
-- **User Feedback & Signups**: workflows that collect feedback or allow users to sign up for events or email lists.
-- **Data Entry**: workflows where business users enter data that is processed and either sent back to them or stored in a database.
+Those answers determine which blocks to choose for each stage.
-Apps let non-technical users interact with workflows without editing YAML or flow configuration.
+For more patterns, browse the Apps-focused posts in the [blog section](../../../../blogs/introducing-apps/index.md) and [solutions content](../../../../blogs/use-case-apps/index.md).
+
+---
## How App stages map to execution progress
-Apps render different blocks based on the current execution state. This is useful when you want the page to guide users through the full lifecycle of a request, from submission to approval to delivery.
+The stage-based model lets a single app guide users through the full request lifecycle — from submission through approval to delivery — by rendering different blocks based on the current execution state.
| App stage | What the user sees | What usually happens in the flow |
|-----------|--------------------|----------------------------------|
@@ -67,44 +68,25 @@ This stage-based layout is what makes Apps easier for non-technical users: they
---
-## Common App patterns
-
-The examples below are a good starting point when designing your own App:
-
-- **FTP upload portal**: give users a simple upload form while Kestra handles the backend credentials and transfer logic. See the [business user Apps blog example](../../../../blogs/use-case-apps/index.md#requests--review).
-- **Self-serve analytics request**: let users choose a dimension and time range, run a query and chart generation flow, and return the generated output on `SUCCESS`. See the [dynamic self-serve example](../../../../blogs/use-case-apps/index.md#dynamic-self-serve).
-- **AI-assisted intake or user research assistant**: collect free-form context from a sales, product, or support team member, run an LLM-backed flow, and display the suggested answer or categorization back in the App. See the [everyday automation example](../../../../blogs/use-case-apps/index.md#simple-interfaces-for-everyday-automation).
-- **VM or infrastructure request**: collect the requested environment, size, region, and justification on `OPEN`, show validation progress on `RUNNING`, pause for approval on `PAUSE`, then display the created VM details on `SUCCESS`. This pattern also fits the infrastructure workflows described in the [infrastructure automation blog](../../../../blogs/infra-automation/index.md).
-- **Human-in-the-loop review**: display task outputs, logs, or model results, then let an approver accept or reject the execution from the same screen.
-
-When in doubt, start by mapping the user journey first:
-
-1. What should the user submit?
-2. What should they see while the flow is running?
-3. Does the flow need approval or review?
-4. What is the final outcome you want to show back in the App?
-
-Once you know those answers, it becomes much easier to choose the right blocks for each stage.
-
-If you want inspiration beyond the examples on this page, browse the Apps-focused posts in the [blog section](../../../../blogs/introducing-apps/index.md) and [solutions content](../../../../blogs/use-case-apps/index.md).
-
----
-
## Creating Apps in code
-To create a new app, go to the **Apps** page in the main UI and click **+ Create**. Add your app configuration as YAML and click **Save**. Like flows, apps have multiple editor views — you can configure the app while viewing documentation, previewing the layout, or searching the blueprint repository.
+If you already have a flow, building an App is a single YAML file: point it at the flow's `namespace` and `flowId`, define what users see at each execution stage with layout blocks, and save. No additional backend code is required — your flow is already the backend.
+
+To create a new app, go to **Apps** and click **+ Create**. Add your app configuration as YAML and click **Save**. Like flows, apps have multiple editor views — you can configure the app while viewing documentation, previewing the layout, or searching the blueprint repository.
-You can set `disabled: true` in the YAML to create an app in an inactive state. A disabled app does not appear in the catalog and cannot be opened via its URL until you enable it. This is useful for staging an app before you are ready to release it.
+Set `disabled: true` to stage an app before release. A disabled app does not appear in the catalog and cannot be opened via its URL until you enable it.
+
+For the full list of available blocks and their properties, see [App layout blocks](#app-layout-blocks) below.
### App to run a Hello World flow
-Apps serve as custom UIs for workflows, so you need to first create a flow. Here is a simple configuration for a parameterized flow that logs a message when triggered:
+Every app is backed by a flow. The following flow accepts a name input and logs a message when triggered:
```yaml
id: myflow
@@ -154,53 +136,316 @@ layout:
You can find a related example in the [enterprise-edition-examples repository](https://github.com/kestra-io/enterprise-edition-examples/blob/main/apps/06_hello_world_app.yaml).
-This app is `PUBLIC`, so anyone with the URL can access it without requiring login. Alternatively, you can set the `access` type to `PRIVATE` to restrict the app only to specific users.
-
-This app is perfect for building **public forms** that anyone in the world can access.
+This app is `PUBLIC`, so anyone with the URL can access it without logging in. Set `access.type` to `PRIVATE` to restrict it to authenticated users.
### App to request and download data
-Let's create a flow that fetches the relevant dataset based on user input: [flow source code](https://github.com/kestra-io/enterprise-edition-examples/blob/main/flows/company.team.get_data.yaml).
+A common pattern is to let business users request a filtered dataset and download it as a file — no Kestra access required. The flow fetches data from an external API, filters and sorts it based on user inputs, writes a CSV, and surfaces a summary string and the file as typed flow outputs. The app wraps that flow in a form, shows a loading indicator while it runs, and presents the summary and a download button on success.
+
+**Flow:**
+
+```yaml
+id: product-report
+namespace: company.ops
+
+inputs:
+ - id: requested_by
+ type: STRING
+ displayName: Requested By
+
+ - id: category
+ type: SELECT
+ displayName: Product Category
+ values:
+ - smartphones
+ - laptops
+ - fragrances
+ - skincare
+ - groceries
+ - furniture
+
+ - id: min_rating
+ type: FLOAT
+ displayName: Minimum Rating
+ defaults: 4.0
+
+ - id: sort_by
+ type: SELECT
+ displayName: Sort By
+ values:
+ - rating
+ - price
+ - stock
+
+tasks:
+ - id: fetch_products
+ type: io.kestra.plugin.core.http.Download
+ uri: "https://dummyjson.com/products/category/{{ inputs.category }}?limit=100"
+
+ - id: build_report
+ type: io.kestra.plugin.scripts.python.Script
+ containerImage: python:3.12-slim
+ inputFiles:
+ data.json: "{{ outputs.fetch_products.uri }}"
+ script: |
+ import csv
+ import json
+
+ with open("data.json") as f:
+ products = json.load(f)["products"]
+
+ filtered = [p for p in products if p["rating"] >= {{ inputs.min_rating }}]
+ filtered.sort(key=lambda p: p["{{ inputs.sort_by }}"], reverse=True)
+
+ fields = ["title", "brand", "category", "price", "rating", "stock", "discountPercentage"]
+ with open("report.csv", "w", newline="") as f:
+ writer = csv.DictWriter(f, fieldnames=fields, extrasaction="ignore")
+ writer.writeheader()
+ writer.writerows(filtered)
+
+ avg_price = sum(p["price"] for p in filtered) / len(filtered) if filtered else 0
+ avg_rating = sum(p["rating"] for p in filtered) / len(filtered) if filtered else 0
+
+ summary = (
+ f"Report for {{ inputs.requested_by }}: "
+ f"{len(filtered)} {{'{{ inputs.category }}'}} products rated ≥ {{ inputs.min_rating }}. "
+ f"Avg price ${avg_price:.2f} · Avg rating {avg_rating:.2f}/5 · "
+ f"Sorted by {{ inputs.sort_by }}."
+ )
+
+ print("::" + json.dumps({"outputs": {"summary": summary, "count": len(filtered)}}) + "::")
+ outputFiles:
+ - report.csv
+
+outputs:
+ - id: report_summary
+ type: STRING
+ value: "{{ outputs.build_report.vars.summary }}"
+
+ - id: report_file
+ type: FILE
+ value: "{{ outputs.build_report.outputFiles['report.csv'] }}"
+```
+
+**App:**
+
+```yaml
+id: product_report
+type: io.kestra.plugin.ee.apps.Execution
+namespace: company.ops
+displayName: Product Report
+flowId: product-report
+access:
+ type: PRIVATE
+ catalog: true
+
+layout:
+ - on: OPEN
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Markdown
+ content: "## Submit a request"
+ - type: io.kestra.plugin.ee.apps.execution.blocks.CreateExecutionForm
+ - type: io.kestra.plugin.ee.apps.execution.blocks.CreateExecutionButton
+ text: Submit
+
+ - on: RUNNING
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Markdown
+ content: "## Keep this page open to see the results"
+ - type: io.kestra.plugin.ee.apps.core.blocks.Loading
+ - type: io.kestra.plugin.ee.apps.execution.blocks.Logs
+ - type: io.kestra.plugin.ee.apps.execution.blocks.CancelExecutionButton
+ text: Cancel request
+ style: WARNING
+
+ - on: SUCCESS
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Alert
+ style: SUCCESS
+ showIcon: true
+ content: Your request has completed.
+ - type: io.kestra.plugin.ee.apps.execution.blocks.Outputs
+ - type: io.kestra.plugin.ee.apps.execution.blocks.Logs
+ - type: io.kestra.plugin.ee.apps.core.blocks.Button
+ text: Submit a new request
+ url: "{{ app.url }}"
+ style: DEFAULT
-Now, from the Apps page, you can create a new app that allows users to select the data they want to download: [app source code](https://github.com/kestra-io/enterprise-edition-examples/blob/main/apps/05_request_data_form.yaml).
+ - on: FAILURE
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Alert
+ style: ERROR
+ showIcon: true
+ content: The request failed. The logs below show what went wrong.
+ - type: io.kestra.plugin.ee.apps.execution.blocks.Logs
+ filter:
+ logLevel: ERROR
+```
+
+#### Interactive demo
-This app is perfect for reporting and analytics use cases where users can request data and download the results.
+
+
+This pattern works for any reporting or analytics use case where users need to request and download data without accessing Kestra directly.
### App to request a VM and get it approved
-One common enterprise use case is a self-service infrastructure request. A requester fills out a form with the VM size, environment, and justification. Kestra validates the request, pauses for approval, and resumes the flow only after the request is approved.
+Self-service infrastructure requests are one of the most common approval-based App patterns. A requester fills out a form with the VM size, environment, region, and business justification. The flow logs and validates the request, then pauses for review. A platform engineer opens the same app in its `PAUSE` state, sees the original request details, adds reviewer notes, and either approves (resumes the execution) or rejects (cancels it). On approval, the flow continues to provisioning.
-Add a flow simulating a request for compute resources that needs manual approval: [flow source code](https://github.com/kestra-io/enterprise-edition-examples/blob/main/flows/company.team.request_resources.yaml).
+This example demonstrates two distinct user roles interacting with the same app at different stages.
-Then, add your app configuration to create a form that requests the VM and routes it through the approval process: [app source code](https://github.com/kestra-io/enterprise-edition-examples/blob/main/apps/03_compute_resources_approval.yaml).
+**Flow:**
-In practice, that app often uses the following stages:
+```yaml
+id: vm-request
+namespace: company.ops
-- `OPEN`: request form with VM size, environment, owner, and business justification.
-- `RUNNING`: validation of the request, available quotas, tags, or naming conventions.
-- `PAUSE`: approval screen for the platform, security, or operations team.
-- `RESUME` or `SUCCESS`: confirmation that the request was approved and the VM is being created or is ready to use.
+inputs:
+ - id: requested_by
+ type: STRING
+ displayName: Requested By
+
+ - id: vm_size
+ type: SELECT
+ displayName: VM Size
+ values:
+ - "small (2 vCPU / 4 GB)"
+ - "medium (4 vCPU / 8 GB)"
+ - "large (8 vCPU / 16 GB)"
+
+ - id: environment
+ type: SELECT
+ displayName: Environment
+ values:
+ - dev
+ - staging
+ - production
+
+ - id: region
+ type: SELECT
+ displayName: Region
+ values:
+ - us-east-1
+ - eu-west-1
+ - ap-southeast-1
+
+ - id: justification
+ type: STRING
+ displayName: Business Justification
+
+tasks:
+ - id: validate_request
+ type: io.kestra.plugin.core.log.Log
+ message: |
+ New VM request from {{ inputs.requested_by }}.
+ Size: {{ inputs.vm_size }}, Environment: {{ inputs.environment }}, Region: {{ inputs.region }}.
+ Justification: {{ inputs.justification }}
+
+ - id: await_approval
+ type: io.kestra.plugin.core.flow.Pause
+ onResume:
+ - id: notes
+ type: STRING
+ displayName: Reviewer Notes
+ required: false
+
+ - id: provision_vm
+ type: io.kestra.plugin.core.log.Log
+ message: |
+ Provisioning {{ inputs.vm_size }} in {{ inputs.environment }}/{{ inputs.region }} for {{ inputs.requested_by }}.
+ Reviewer notes: {{ resume.notes }}
+```
-This pattern also works for adjacent use cases such as database access requests, sandbox environment creation, firewall rule approvals, or SaaS account provisioning.
+The `Pause` task's `onResume` inputs define what the reviewer submits when approving — in this case, optional notes. Those values are available to downstream tasks as `{{ resume. }}`.
+**App:**
+
+```yaml
+id: vm_request
+type: io.kestra.plugin.ee.apps.Execution
+displayName: VM Request
+namespace: company.ops
+flowId: vm-request
+access:
+ type: PRIVATE
+ catalog: true
+
+layout:
+ - on: OPEN
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Markdown
+ content: |
+ ## Request a Virtual Machine
+ Fill in the details below. Your request will be validated and sent for approval before provisioning begins.
+ - type: io.kestra.plugin.ee.apps.execution.blocks.CreateExecutionForm
+ - type: io.kestra.plugin.ee.apps.execution.blocks.CreateExecutionButton
+ text: Submit Request
+ style: SUCCESS
+
+ - on: RUNNING
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Markdown
+ content: "## Validating your request…"
+ - type: io.kestra.plugin.ee.apps.core.blocks.Loading
+ - type: io.kestra.plugin.ee.apps.execution.blocks.Logs
+
+ - on: PAUSE
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Markdown
+ content: |
+ ## Review this VM request
+ Inspect the request details below, then approve or reject.
+ - type: io.kestra.plugin.ee.apps.execution.blocks.Inputs
+ - type: io.kestra.plugin.ee.apps.execution.blocks.ResumeExecutionForm
+ - type: io.kestra.plugin.ee.apps.execution.blocks.ResumeExecutionButton
+ text: Approve
+ style: SUCCESS
+ - type: io.kestra.plugin.ee.apps.execution.blocks.CancelExecutionButton
+ text: Reject
+ style: DANGER
+
+ - on: SUCCESS
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Alert
+ style: SUCCESS
+ showIcon: true
+ content: Your VM has been provisioned.
+ - type: io.kestra.plugin.ee.apps.execution.blocks.Logs
+ - type: io.kestra.plugin.ee.apps.core.blocks.Button
+ text: Submit a new request
+ url: "{{ app.url }}"
+ style: DEFAULT
+
+ - on: FAILURE
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Alert
+ style: ERROR
+ showIcon: true
+ content: The request could not be processed. Review the logs for details.
+ - type: io.kestra.plugin.ee.apps.execution.blocks.Logs
+ filter:
+ logLevel: ERROR
+```
+
+The `PAUSE` stage shows the original request inputs (`ExecutionInputs`), the `onResume` form for reviewer notes (`ResumeExecutionForm`), and the approve/reject buttons. Clicking **Approve** resumes the execution; clicking **Reject** cancels it.
+
+This pattern applies directly to database access requests, sandbox environment creation, firewall rule approvals, SaaS account provisioning, and any other workflow that needs a human gate before it continues.
---
## Creating Apps without code
-Like flows, Apps can also be created using the no-code editor. Every element available in code — such as blocks, properties, and configuration options — is fully supported in the no-code interface. When you build or update an App in the no-code editor, those changes are immediately reflected in the code view, preserving the declarative YAML definition behind the scenes. This ensures consistency between visual and code-first approaches, allowing teams to switch seamlessly between them without losing control, readability, or versioning.
-
-
+Apps can also be built using the no-code editor. Every block, property, and configuration option available in YAML is fully supported visually. Changes made in the no-code editor are immediately reflected in the code view, and vice versa — teams can switch between approaches at any time without losing the underlying YAML definition.
---
## App catalog
-The App Catalog is where users can find available apps. You can filter apps by name, type, namespace, or tags. From this page, you can also create new apps, edit existing ones, enable or disable individual apps, or delete them.
+The App Catalog lists all available apps in a tenant. Filter by name, type, namespace, or tags. From this page you can also create new apps, edit existing ones, enable or disable individual apps, or delete them.
-Kestra provides a direct access URL to the Apps Catalog in the format `http://your_host/ui/your_tenant/apps/catalog`. Any Kestra user with at least `APP`-Read and `APPEXECUTION`-Read permissions in that tenant can reach this URL (adding all `APPEXECUTION` permissions is recommended).
+Kestra provides a direct access URL to the Apps Catalog in the format `http://your_host/ui/your_tenant/apps/catalog`. Any Kestra user with at least `APP: VIEW` permission in that tenant can reach this URL.
The catalog page requires authentication, so it is never publicly accessible. Users see only the apps they are permitted to see based on their RBAC permissions. You can limit visibility to specific groups by setting the `groups` property in the `access` block:
@@ -230,27 +475,87 @@ You can also export a selection of apps as a ZIP archive (`kestra-{tenant}-apps.
### Customize the Apps Catalog
-You can customize your Apps Catalog to align with organization branding by navigating to the **Tenant** tab and then **Apps Catalog**.
+Customize the catalog's title, colors, and banner image to match your organization's look and feel. Go to **Instance Owner**, then in the left sidebar under your tenant, click **Apps Catalog**.
-Here, you can give your catalog a display title, set a primary banner display color, and upload an image for banner (typically an organization logo).
+The configuration panel lets you set a **Title**, **Title Color**, **Primary Color** (used for buttons), **Background Color**, **Tile Color**, **Button Text Color**, and a **Banner** image displayed at the top of the catalog page.
:::alert{type="info"}
-Currently, the uploaded banner display image must be an `.svg` file.
+The banner image must be an `.svg` file.
:::
-Once saved, navigate to the Apps Catalog, and see your branding:
+Once saved, open the **Apps Catalog** to see your branding:
-From the Apps Catalog, you can also access the customization settings directly at any time by clicking on the **gear icon**.
+Access these settings any time from the **gear icon** in the catalog.
+
+### App tags
+
+Add custom tags to organize and filter apps in the catalog. For example: `DevOps`, `data-team`, `project-x`.
---
-## App tags
+## App URL
+
+Each app has a unique URL — share it with team members, customers, or partners, or embed it in internal tools. Anyone with the link (and appropriate access) can open the app directly.
-You can add custom tags to organize and filter apps in the App Catalog. For example, you might tag apps with `DevOps`, `data-team`, `project-x`. You can then filter apps by tags to quickly find the apps you are looking for.
+The URL format is: `https://yourHost/ui/tenantId/apps/appUid`, for example `http://localhost:8080/ui/release/apps/5CS8qsm7YTif4PWuAUWHQ5`.
+
+Copy the URL from the **Apps Catalog**.
+
+:::alert{type="info"}
+App URL generation relies on the `kestra.url` server configuration property. If this property is not set, generated links may be broken or missing. Set it to the externally reachable base URL of your Kestra instance, for example `kestra.url: https://kestra.example.com`.
+:::
+
+### App expressions
+
+From within flows, you can generate app URLs using the Enterprise-only `appLink` expression. See [Workflow Functions](../../../expressions/04.functions/04.workflow/index.mdx) for parameters and examples.
+
+---
+
+## App access and RBAC permissions
+
+Each app has an `access` block that controls who can open and submit it.
+
+### Public access
+
+When an app is set to `PUBLIC`, anyone with the URL can open the form and submit requests without logging in. This is suitable for public-facing forms, surveys, or intake pages you share via email or embed on a website.
+
+:::alert{type="info"}
+For `PUBLIC` apps, execution IDs exposed through file download or log links are encrypted so that anonymous users cannot reference executions outside the app.
+:::
+
+### Private access for using apps
+
+When an app is set to `PRIVATE`, only users with `APP: EXECUTE` permission on the app's namespace can open or submit it. `PRIVATE` apps are also only visible in the App Catalog to users with `APP: EXECUTE` — they do not appear for users who only hold `APP: LIST` or `APP: VIEW`. You can further narrow access to specific IAM groups using the `groups` field:
+
+```yaml
+access:
+ type: PRIVATE
+ groups:
+ - DataOps
+ - Finance
+```
+
+Group membership is checked at runtime on every request. Users who belong to at least one listed group are granted access; users outside those groups are denied even if they have `APP: EXECUTE` permission on the namespace. If `groups` is omitted, any authenticated user with `APP: EXECUTE` permission on the namespace can use the app.
+
+`APP: EXECUTE` is namespace-scoped. A user with `APP: EXECUTE` on `company.team` cannot dispatch an app in `company.other`, even if both apps appear in the same catalog view.
+
+This makes the `PRIVATE` + `groups` combination useful when you want to allow a specific group of business stakeholders or external partners to use an app without giving them access to the broader Kestra UI.
+
+### Private access for building apps
+
+The `APP` resource controls who can create, view, update, or delete apps within a tenant. It can be scoped to specific namespaces. `APP: EXECUTE`, `APP: ACCESS_FILES`, and `APP: ACCESS_LOGS` govern the ability to submit requests through an app and access its artifacts; the remaining `APP` actions govern the ability to build and manage apps.
+
+---
+
+## App executions
+
+Each time a user creates an execution by submitting a form in the app, a new execution is generated with the system label `system.app` and a value of `yourAppId`. For example, to filter all executions created by the `computeResourcesForm` app, you can search for `system.app:computeResourcesForm` in the label filter.
+
+For every execution, you can track the user inputs, see the current state, view logs, and check the outputs — all from the Kestra UI. This lets you observe, troubleshoot, and manage app executions just as you would any other workflow execution.
---
@@ -290,11 +595,11 @@ Expiration is evaluated against the server clock at the moment a user opens or s
## App thumbnails
-Design Apps with thumbnails to clearly display their intended use case or function to catalog users. To add a thumbnail to your app, upload an image file as a [namespace file](../../../06.concepts/02.namespace-files/index.md) to the same namespace as the App's connected flow. For example, add an `.svg` (it can also be `.jpg`, `.png`, or other image file extension) to the `company.team` namespace. The example below adds `kestra-icon.svg`.
+Add a thumbnail to give catalog users a visual indicator of what the app does. Upload an image file as a [namespace file](../../../06.concepts/02.namespace-files/index.md) to the same namespace as the app's connected flow. Supported formats include `.svg`, `.jpg`, and `.png`. The example below adds `kestra-icon.svg`.
-In your app code, add the `thumbnail` string property and point it towards the correct namespace file using `nsfiles:///`. For example:
+In your app code, add the `thumbnail` string property and point it toward the correct namespace file using `nsfiles:///`. For example:
```yaml
id: request_data_form
@@ -310,83 +615,19 @@ tags:
- Analytics
```
-Once added, navigate to the Apps Catalog, and a new thumbnail will display on the connected app to help designate its use case:
+Once added, open the **Apps Catalog** — a new thumbnail displays on the connected app:
---
-## App URL
-
-Each app has a unique URL that you can share with others. When someone opens the URL, they see the app and can submit requests. You can share the URL with team members, customers, or partners.
-
-The URL format is: `https://yourHost/ui/tenantId/apps/appUid`, for example `http://localhost:8080/ui/release/apps/5CS8qsm7YTif4PWuAUWHQ5`.
-
-You can copy the URL from the Apps Catalog page in the Kestra UI.
-
-:::alert{type="info"}
-App URL generation relies on the `kestra.url` server configuration property. If this property is not set, generated links may be broken or missing. Set it to the externally reachable base URL of your Kestra instance, for example `kestra.url: https://kestra.example.com`.
-:::
-
-### App expressions
-
-From within flows, you can generate app URLs using the Enterprise-only `appLink` expression. See [Workflow Functions](../../../expressions/04.functions/04.workflow/index.mdx) for parameters and examples.
-
----
-
-## App access and RBAC permissions
-
-Each app has an `access` block that controls who can open and submit it.
-
-### Public access
-
-When an app is set to `PUBLIC`, anyone with the URL can open the form and submit requests without logging in. This is suitable for public-facing forms, surveys, or intake pages you share via email or embed on a website.
-
-:::alert{type="info"}
-For `PUBLIC` apps, execution IDs exposed through file download or log links are encrypted so that anonymous users cannot reference executions outside the app.
-:::
-
-### Private access for using apps
-
-When an app is set to `PRIVATE`, only authenticated users with the `APPEXECUTION` permission on the app’s namespace can open or submit it. You can further narrow access to specific IAM groups using the `groups` field:
-
-```yaml
-access:
- type: PRIVATE
- groups:
- - DataOps
- - Finance
-```
-
-Group membership is checked at runtime on every request. Users who belong to at least one listed group are granted access; users outside those groups are denied even if they have `APPEXECUTION` permission on the namespace. If `groups` is omitted, any authenticated user with `APPEXECUTION` permission on the namespace can use the app.
-
-The `APPEXECUTION` permission is also namespace-scoped. A user with `APPEXECUTION` on `company.team` cannot dispatch an app in `company.other`, even if both apps appear in the same catalog view.
-
-This makes the `PRIVATE` + `groups` combination useful when you want to allow a specific group of business stakeholders or external partners to use an app without giving them access to the broader Kestra UI.
-
-### Private access for building apps
-
-The `APP` permission controls who can create, read, update, or delete apps within a tenant. Like `APPEXECUTION`, it can be scoped to specific namespaces. Unlike `APPEXECUTION`, which governs the ability to submit requests through an app, `APP` governs the ability to build and manage apps.
-
----
-
-## App executions
-
-Each time a user creates an execution by submitting a form in the app, a new execution is generated with the system label `system.app` and a value of `yourAppId`. For example, to filter all executions created by the `computeResourcesForm` app, you can search for `system.app:computeResourcesForm` in the label filter.
-
-For every execution, you can track the user inputs, see the current state, view logs, and check the outputs — all from the Kestra UI. This lets you observe, troubleshoot and manage issues with your apps just as you would with any other workflow execution in Kestra.
-
----
-
## App layout blocks
-Each app is made up of blocks that define the layout and content of the app. You can add blocks for markdown text, forms, buttons, logs, inputs, outputs, and more. The blocks are displayed in a specific order based on the app’s state (e.g. on `OPEN`, `RUNNING`, `SUCCESS`, `FAILURE`, `PAUSE`, `RESUME`).
-
-By combining different blocks, you can create a custom UI that guides users through the app’s workflow. For example, you could start with a markdown block that explains the purpose of the app, followed by a form block for users to enter their inputs, and a button block to submit the request. You can also add blocks to display execution logs, outputs, and buttons for approving or rejecting paused workflows.
+Each app layout is a list of stage-specific blocks. Each block renders when the execution reaches the matching stage (`OPEN`, `RUNNING`, `SUCCESS`, `FAILURE`, `PAUSE`, `RESUME`). Combine blocks to guide users through the full lifecycle of a request — form on `OPEN`, progress indicator on `RUNNING`, download link and logs on `SUCCESS`.
| Block type | Available on | Properties | Example |
|--------------------------|--------------------------------------------------------------------------|---------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| `Markdown` | OPEN, CREATED, RUNNING, PAUSE, RESUME, SUCCESS, FAILURE, FALLBACK | - `content` | `- type: io.kestra.plugin.ee.apps.core.blocks.Markdown` `content: "## Please validate the request. Inspect the logs and outputs below. Then, approve or reject the request."` |
+| `Markdown` | OPEN, CREATED, RUNNING, PAUSE, RESUME, SUCCESS, FAILURE, FALLBACK | - `content` (Pebble template) | `- type: io.kestra.plugin.ee.apps.core.blocks.Markdown` `content: "## Please validate the request. Inspect the logs and outputs below. Then, approve or reject the request."` |
| `RedirectTo` | OPEN, CREATED, RUNNING, PAUSE, RESUME, SUCCESS, FAILURE, ERROR, FALLBACK | - `url`: redirect URL - `delay`: delay in seconds | `- type: io.kestra.plugin.ee.apps.core.blocks.RedirectTo` `url: "https://kestra.io/docs"` `delay: "PT60S"` |
| `CreateExecutionForm` | OPEN | None | `- type: io.kestra.plugin.ee.apps.execution.blocks.CreateExecutionForm` |
| `ResumeExecutionForm` | PAUSE | None | `- type: io.kestra.plugin.ee.apps.execution.blocks.ResumeExecutionForm` |
@@ -397,11 +638,42 @@ By combining different blocks, you can create a custom UI that guides users thro
| `ExecutionOutputs` | PAUSE, RESUME, SUCCESS, FAILURE | - `filter`: include, exclude | `- type: io.kestra.plugin.ee.apps.execution.blocks.Outputs` `filter:` `include: []` `exclude: []` |
| `ExecutionLogs` | PAUSE, RESUME, SUCCESS, FAILURE, FALLBACK | - `filter`: logLevel, taskIds | `- type: io.kestra.plugin.ee.apps.execution.blocks.Logs` `filter:` `logLevel: "INFO"` `taskIds: []` |
| `Loading` | RUNNING | None | `- type: io.kestra.plugin.ee.apps.core.blocks.Loading` |
-| `Alert` | FAILURE | - `style`: SUCCESS, WARNING, ERROR, INFO - `showIcon`: true, false | `- type: io.kestra.plugin.ee.apps.core.blocks.Alert` `style: "WARNING"` `showIcon: true` `content: "An error occurred!"` |
+| `Alert` | OPEN, CREATED, RUNNING, PAUSE, RESUME, SUCCESS, FAILURE, FALLBACK | - `style`: SUCCESS, WARNING, ERROR, INFO - `showIcon`: true, false | `- type: io.kestra.plugin.ee.apps.core.blocks.Alert` `style: "WARNING"` `showIcon: true` `content: "An error occurred!"` |
| `Button` | SUCCESS, FAILURE | - `text` - `url` - `style`: DEFAULT, SUCCESS, DANGER, INFO | `- type: io.kestra.plugin.ee.apps.core.blocks.Button` `text: "More examples"` `url: "https://github.com/kestra-io/examples"` `style: "INFO"` |
| `TaskOutputs` | RUNNING, PAUSE, RESUME, SUCCESS | - `outputs`: list of outputs with `displayName`, `value`, and `type` | `- type: io.kestra.plugin.ee.apps.execution.blocks.TaskOutputs` `outputs:` `- displayName: My Task Output` `value: "{{ outputs.test.value }}"` `type: FILE` |
-Everything is customizable, from the text and style of buttons to the messages displayed before and after submissions.
+### Dynamic content in the OPEN state
+
+The `Markdown` block's `content` property is a Pebble template. When the layout includes a `CreateExecutionForm`, `{{ inputs. }}` is available in the render context and updates live as the user interacts with the form — useful for context-sensitive descriptions that react to dropdown changes.
+
+For example, given a flow with a `product` SELECT input, the app below renders the current selection inline:
+
+```yaml
+id: software_request_form
+type: io.kestra.plugin.ee.apps.Execution
+displayName: Software Request Form
+namespace: company.team
+flowId: software_request
+access:
+ type: PRIVATE
+
+layout:
+ - on: OPEN
+ blocks:
+ - type: io.kestra.plugin.ee.apps.execution.blocks.CreateExecutionForm
+ - type: io.kestra.plugin.ee.apps.core.blocks.Markdown
+ content: "You are requesting access to: **{{ inputs.product }}**"
+ - type: io.kestra.plugin.ee.apps.execution.blocks.CreateExecutionButton
+ text: Submit
+
+ - on: SUCCESS
+ blocks:
+ - type: io.kestra.plugin.ee.apps.core.blocks.Alert
+ style: SUCCESS
+ content: Your request has been submitted.
+```
+
+Before the user interacts with the form, `{{ inputs.* }}` resolves to each input's default value. In other states (RUNNING, SUCCESS, FAILURE), `{{ inputs.* }}` references the execution's submitted values.
:::alert{type="info"}
When the flow uses [`FORM` inputs](../../../05.workflow-components/05.inputs/index.md#form-inputs), `CreateExecutionForm` renders a multi-step Next/Back wizard — one step per FORM group, a step for ungrouped inputs, then a recap. No additional App configuration is required; the wizard is driven entirely by the flow's input definition.
diff --git a/src/contents/docs/07.enterprise/04.scalability/index.mdx b/src/contents/docs/07.enterprise/04.scalability/index.mdx
index dd20dd580ba..f4954ab8f66 100644
--- a/src/contents/docs/07.enterprise/04.scalability/index.mdx
+++ b/src/contents/docs/07.enterprise/04.scalability/index.mdx
@@ -9,8 +9,6 @@ editions: ["EE", "Cloud"]
import ChildCard from "~/components/docs/ChildCard.astro"
-The following topics describe Kestra features that help scale and enhance the productivity of your orchestration workflows such as Apps and Worker Groups.
-
-## Scale Kestra Enterprise – worker groups, task runners, apps
+Use Worker Groups and Apps to scale your Kestra Enterprise deployment and give non-technical users a way to interact with your workflows. For offloading compute-intensive scripting tasks to remote environments, see [Task Runners](../../task-runners/index.mdx).
\ No newline at end of file
diff --git a/src/contents/docs/07.enterprise/04.scalability/task-runners/index.md b/src/contents/docs/07.enterprise/04.scalability/task-runners/index.md
deleted file mode 100644
index ffae9f2ab8e..00000000000
--- a/src/contents/docs/07.enterprise/04.scalability/task-runners/index.md
+++ /dev/null
@@ -1,40 +0,0 @@
----
-title: "Task Runners in Kestra Enterprise: Offload Compute"
-h1: Offload Tasks to Docker, Kubernetes, AWS Batch, and More
-description: Optimize compute with Kestra Task Runners. Offload intensive tasks to Docker, Kubernetes, AWS Batch, and other remote environments for scalability.
-sidebarTitle: Task Runners
-icon: /src/contents/docs/icons/admin.svg
-editions: ["EE", "Cloud"]
-version: ">= 0.18.0"
----
-
-Task Runner capabilities and supported plugins.
-
-## Task runners – offload and isolate compute
-
-[Task Runners](../../../task-runners/index.mdx) offer a powerful way to offload compute-intensive tasks to remote environments.
-
-
-
-
-
-## Task runner types
-
-There are a number of task runner types. The [Docker](../../../task-runners/04.types/02.docker-task-runner/index.md) and [Process](../../../task-runners/04.types/01.process-task-runner/index.md) task runners are included in the Open Source edition. All other types require an [Enterprise Edition](./index.md) license or a [Kestra Cloud](/cloud) account.
-
-Enterprise Edition Task Runners:
-- [Kubernetes](../../../task-runners/04.types/03.kubernetes-task-runner/index.md)
-- [AWS Batch](../../../task-runners/04.types/04.aws-batch-task-runner/index.md)
-- [AWS EC2](../../../task-runners/04.types/05.aws-ec2-task-runner/index.md)
-- [Azure Batch](../../../task-runners/04.types/06.azure-batch-task-runner/index.md)
-- [Azure Virtual Machine](../../../task-runners/04.types/07.azure-virtualmachine-task-runner/index.md)
-- [Google Batch](../../../task-runners/04.types/08.google-batch-task-runner/index.md)
-- [Google Cloud Run](../../../task-runners/04.types/09.google-cloudrun-task-runner/index.md)
-
-## Task runners vs Worker Groups
-
-[Task Runners](../../../task-runners/index.mdx) and [Worker Groups](../worker-group/index.md) both **offload compute-intensive tasks to dedicated workers**. However, **worker groups have a broader scope**, applying to **all tasks** in Kestra, whereas **task runners** are limited to **scripting tasks** (Python, R, JavaScript, Shell, dbt, etc. — see the full list in the [Task Runner Overview](../../../task-runners/index.mdx)). Worker groups can be used with any plugins.
-
-For instance, if you need to query an on-premise SQL Server database running on a different server than Kestra, your SQL Server Query task can target a worker with access to that server. Additionally, worker groups can fulfill the same use case as task runners by distributing the load of scripting tasks to dedicated workers with the necessary resources and dependencies (_incl. hardware, region, network, operating system_).
-
-You can read more about the differences on the [dedicated Task Runners vs. Worker Groups page](../../../task-runners/03.task-runners-vs-worker-groups/index.md).
diff --git a/src/contents/docs/07.enterprise/04.scalability/worker-group/create-worker-group.png b/src/contents/docs/07.enterprise/04.scalability/worker-group/create-worker-group.png
deleted file mode 100644
index 011c03b783f..00000000000
Binary files a/src/contents/docs/07.enterprise/04.scalability/worker-group/create-worker-group.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/worker-group/data-isolation.png b/src/contents/docs/07.enterprise/04.scalability/worker-group/data-isolation.png
deleted file mode 100644
index 7ec845efbb2..00000000000
Binary files a/src/contents/docs/07.enterprise/04.scalability/worker-group/data-isolation.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/worker-group/distant-worker.png b/src/contents/docs/07.enterprise/04.scalability/worker-group/distant-worker.png
deleted file mode 100644
index 63b3c625832..00000000000
Binary files a/src/contents/docs/07.enterprise/04.scalability/worker-group/distant-worker.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/worker-group/index.md b/src/contents/docs/07.enterprise/04.scalability/worker-group/index.md
index aebfa6d767f..43907f778b0 100644
--- a/src/contents/docs/07.enterprise/04.scalability/worker-group/index.md
+++ b/src/contents/docs/07.enterprise/04.scalability/worker-group/index.md
@@ -5,240 +5,633 @@ description: Manage workloads with Kestra Worker Groups. Target specific workers
sidebarTitle: Worker Group
icon: /src/contents/docs/icons/admin.svg
editions: ["EE"]
-version: ">= 0.10.0"
+version: ">= 2.0.0"
---
-How to configure Worker Groups in Kestra Enterprise Edition.
+Worker Groups route tasks to the right machines in your fleet. A Worker Group is a named, token-authenticated pool of workers that subscribes to one or more Worker Queues — tag-based routing lanes that tasks declare requirements against. The result is flexible many-to-many routing: GPU machines and spot instances can serve the same queue, or a single group can cover multiple queues with per-queue capacity guarantees.
-## Worker groups – configure targeted workers
+Worker Groups are an Enterprise Edition feature. In the open-source edition, all work runs in a single implicit default pool.
-A Worker Group is a set of workers that can be explicitly targeted for task execution or polling trigger evaluation. For example, tasks that require heavy resources can be isolated to a Worker Group designed to handle that load, and tasks that perform best on a specific Operating System can be optimized to run on a Worker Group designed for them.
+## Getting started
-:::alert{type="info"}
-Please note that Worker Groups are not yet available in Kestra Cloud, only in Kestra Enterprise Edition.
-:::
-
-
-
-
+To set up Worker Groups end-to-end:
-## Creating Worker Groups from the UI
+1. [Create a Worker Queue](#worker-queues) — define a routing lane with tags
+2. [Create a Worker Group](#creating-and-managing-worker-groups) — create a pool and subscribe it to queues
+3. [Generate a registration token](#generating-a-registration-token) — authenticate workers to the group
+4. [Start a worker](#starting-a-worker-for-a-group) — connect with the token and controller endpoint
+5. [Route tasks](#using-workerselector-in-tasks) — add `workerSelector` to any task
-:::badge{version=">=0.19" editions="EE"}
-:::
+For IaC and Helm deployments, see [Declarative configuration](#declarative-configuration) to provision the full topology at startup without runtime API calls.
-To create a new Worker Group, navigate to the **Instance** page, go to the **Worker Groups** tab, and click on the `+ Add Worker Group` button. Then, set a **Key**, a **Description**, and optionally **Allowed Tenants** for that worker group. You can also accomplish this via API, CLI, or Terraform.
+## How Worker Groups work
-
+Three building blocks define the routing model:
-## Starting workers for a Worker Group
+| Building block | Role |
+|---|---|
+| **Worker** | A process that runs tasks; joins a group by presenting a registration token and locally enforces its capacity allocation |
+| **Worker Group** | A named pool of workers that subscribes to queues and holds capacity reservation settings |
+| **Worker Queue** | A routing lane identified by a tag set; tasks declare `workerSelector.tags` to target a queue |
-Once a worker group key is created, you can start a worker with the `kestra server worker --worker-group {workerGroupKey}` flag to assign it to that worker group. You can also assign a default worker group at the namespace and tenant level.
+The routing path flows from task requirements down to infrastructure:
-
+1. A task declares `workerSelector.tags: [gpu, eu]`
+2. Kestra finds the Worker Queue whose tags match
+3. Kestra checks which Worker Groups subscribe to that queue
+4. A worker from one of those groups picks up the task
-The Worker Groups UI tracks the health of worker groups, showing how many workers are polling for tasks within each worker group. This gives you visibility into which worker groups are active and the number of active workers.
+**Developer perspective**: declare what a task needs using tags. No machine names, no group names.
-
+**Operator perspective**: create queues with meaningful tags, subscribe groups to those queues, and set capacity guarantees per subscription.
-:::alert{type="info"}
-In order to run the command at startup, you need to run each component independently and use the command for the worker component startup. To set this up, read more about running [Kestra with separated server components](../../../kestra-cli/kestra-server/index.md#kestra-with-server-components-in-different-services).
-:::
+## Using workerSelector in tasks
-## Using Worker Groups
+Add `workerSelector` to any task to route it to a matching Worker Queue. The `workerSelector` object has three properties:
-To assign a worker group, add the `workerGroup.key` property to the task or the polling trigger. A default worker group can also be configured at the `namespace` or `tenant` level.
+| Property | Description | Default |
+|---|---|---|
+| `tags` | List of RFC 1123 labels (max 20) identifying the required Worker Queue | — |
+| `match` | `ALL`: queue tags must include all selector tags. `ANY`: queue tags must include at least one selector tag | `ALL` |
+| `fallback` | Behavior when no worker is available for the matched queue: `FAIL`, `WAIT`, `CANCEL`, or `IGNORE` | `FAIL` |
-Worker groups can be defined at the flow level, and the flow editor validates worker group keys when creating flows from the UI. If the provided key doesn’t exist, the syntax validation will prevent the flow from being saved.
-
-Below is an example flow configuration with a worker group:
+:::alert{type="warning"}
+The default `fallback` in 2.0 is `FAIL`. If you upgraded from an earlier version where tasks waited by default, set `fallback: WAIT` directly on each task, or use a [Policy](../../02.governance/policies/index.md) with an `Add` rule to apply it across a namespace.
+:::
```yaml
-id: worker_group
+id: process_sensitive_data
namespace: company.team
tasks:
- - id: wait
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.core.runner.Process
+ - id: process
+ type: io.kestra.plugin.scripts.python.Commands
+ workerSelector:
+ tags: [sensitive, eu]
+ fallback: WAIT
commands:
- - sleep 10
- workerGroup:
- key: gpu
+ - python process.py
```
-If the `workerGroup.key` property is not provided, all tasks and polling triggers are executed on the default worker group. That default worker group doesn't have a dedicated key.
+Kestra routes the task to a Worker Queue whose tag set includes all declared tags (or any, when `match: ANY`). Any Worker Group subscribed to that queue may execute the task.
-A `workerGroup.key` can also be assigned dynamically using `inputs` like in the following example:
+If `workerSelector` is absent or all tags resolve to null, the task routes to the default queue.
-```yaml
-id: worker_group_dynamic
-namespace: company.team
+### Fallback options
+
+| Value | Behavior |
+|---|---|
+| `FAIL` | Fail the task run immediately if no worker is available (default) |
+| `WAIT` | Hold the task in `CREATED` state until a worker becomes available |
+| `CANCEL` | Cancel the task gracefully; the execution is marked `KILLED` |
+| `IGNORE` | Drop the tag requirement and route to the default Worker Queue instead |
+
+`IGNORE` is useful when the target infrastructure is optional — the task proceeds on any available worker rather than failing when the specialized pool is unavailable.
+`fallback` can only be set when `tags` is non-empty.
+
+:::alert{type="warning"}
+`fallback` only applies when a matching Worker Queue exists but has no live worker right now. If no queue matches the tags at all, the task fails immediately regardless of the `fallback` value — that is a configuration error, not a capacity gap.
+:::
+
+### Dynamic routing
+
+Use Pebble expressions to set tags at runtime:
+
+```yaml
inputs:
- - id: my_worker_group
+ - id: region
type: STRING
+ defaults: eu
tasks:
- - id: workerGroup
- type: io.kestra.plugin.core.debug.Return
- format: "{{ taskrun.startDate }}"
- workerGroup:
- key: "{{ inputs.my_worker_group }}"
+ - id: process
+ type: io.kestra.plugin.scripts.python.Commands
+ workerSelector:
+ tags:
+ - "{{ inputs.region }}"
+ - sensitive
+ fallback: WAIT
+ commands:
+ - python process.py
```
-If the expression resolves to `null` or a blank string, the task is routed to the default worker group — the same behavior as omitting `workerGroup` entirely. This makes `null` a useful sentinel for conditional routing:
+When an expression resolves to null or a blank string, that tag is omitted from the selector. If all tags resolve to null, the task routes to the default queue.
+
+### Namespace and tenant-level routing defaults
+
+Instead of adding `workerSelector` to every task, set a default selector at the namespace or tenant level. Kestra resolves the selector most-specific-first — task, then flow, then the nearest namespace ancestor, then the tenant — and stops at the first level that declares one.
+
+Set a namespace-level default in the namespace settings:
```yaml
-id: worker_group_conditional
-namespace: company.team
+workerSelector:
+ tags: [eu]
+ fallback: WAIT
+```
-inputs:
- - id: use_gpu
- type: BOOLEAN
- defaults: false
+Every task in that namespace (and its child namespaces, unless overridden closer) inherits this selector automatically. Any selector closer to the task — on the task itself or the flow — wins over the namespace or tenant default.
+
+This is the recommended approach when an entire namespace or team should always run on a specific fleet — it keeps flow YAML clean and makes routing changes a single admin update rather than a find-and-replace across all flows.
+
+### Applying workerSelector with Policies
+
+Use a [Policy](../../02.governance/policies/index.md) to route all tasks of a given plugin type to a specific Worker Queue without modifying each task individually:
+
+```yaml
+id: gpu-worker-routing
+description: "Route all Python tasks to GPU workers."
+enforcement: ACTIVE
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.scripts.python
+ values:
+ workerSelector:
+ tags: [gpu]
+ fallback: WAIT
+```
+
+With this Policy applied to the namespace, flows need no per-task configuration:
+
+```yaml
+id: ml_pipeline
+namespace: company.team
tasks:
- id: train
- type: io.kestra.plugin.core.debug.Return
- format: "{{ taskrun.startDate }}"
- workerGroup:
- key: "{{ inputs.use_gpu ? 'gpu' : null }}"
+ type: io.kestra.plugin.scripts.python.Commands
+ commands:
+ - python train.py
+
+ - id: evaluate
+ type: io.kestra.plugin.scripts.python.Commands
+ commands:
+ - python eval.py
```
-When `inputs.use_gpu` is `false`, the key resolves to `null` and the task runs on the default worker group. When `true`, it targets the `gpu` worker group.
+## Worker Queues
+
+A Worker Queue is a routing lane with a stable id and a set of tags. Multiple Worker Groups may subscribe to the same queue. Removing a group's subscription never deletes the queue — queues exist independently.
+
+Two ids are reserved and never created manually:
+- `default` — the global default queue; receives tasks with no `workerSelector`
+- `system` — the in-process system worker
+
+Worker Queue ids must follow RFC 1123 label format: lowercase alphanumerics and hyphens, starting and ending with an alphanumeric character, max 63 characters.
+
+### Creating Worker Queues
+
+Navigate to **Instance Owner → Infrastructure → Worker Queues** and click **Create**. You can also create Worker Queues via the API or Terraform.
+
+**Tenant scoping**: a Worker Queue can restrict which tenants may route tasks through it. An empty tenant list means unrestricted.
+
+## Creating and managing Worker Groups
+
+A Worker Group is identified by a stable id (RFC 1123 label), has a display name, and holds a list of queue subscriptions and registration tokens.
+
+### Creating a Worker Group
+
+Navigate to **Instance Owner → Infrastructure → Worker Groups** and click **Add Worker Group**. Set an id, display name, and optional description. You can add queue subscriptions and generate registration tokens immediately, or configure them after creation.
-## Worker Group fallback behavior
+Worker Group ids must follow RFC 1123 label format.
-:::badge{version=">=0.20" editions="EE"}
+### The default group
+
+One group always exists and cannot be deleted: the `default` group. It subscribes to the `default` queue and receives all tasks that have no `workerSelector`. Workers that start without a registration token join the default group automatically.
+
+:::alert{type="warning"}
+Keep at least one worker running in the default group to ensure tasks without a `workerSelector` always have somewhere to execute.
:::
-By default, a task configured to run on a given worker will wait for the worker to be available (i.e., `workerGroup.fallback: WAIT`). If you prefer to fail the task when the worker is not available, set `workerGroup.fallback: FAIL`.
+### Queue subscriptions
+
+A subscription connects a Worker Group to a Worker Queue. Each subscription specifies:
+
+- **Target queue id** — which Worker Queue this group's workers will serve
+- **Reserved capacity percentage** (optional) — a per-worker floor guarantee, 1–100
+- **Interaction mode** — `STRICT` or `ELASTIC` (see [Capacity reservation](#capacity-reservation))
+
+A group may subscribe to multiple queues. The sum of reserved percentages across a worker's subscriptions must not exceed 100.
+
+Manage subscriptions through the UI or the subscriptions API:
+
+| Operation | Endpoint |
+|---|---|
+| Add subscription | `POST /api/v1/instance/worker-groups/{id}/subscriptions` |
+| Update reservation | `PATCH /api/v1/instance/worker-groups/{id}/subscriptions/{workerQueueId}` |
+| Remove subscription | `DELETE /api/v1/instance/worker-groups/{id}/subscriptions/{workerQueueId}` |
+
+## Capacity reservation
+
+Reserved capacity is a per-worker floor guarantee, not a fleet-wide quota. Remaining slots beyond reserved percentages form a shared pool available to all of that worker's subscriptions.
+
+**Example**: a worker with 16 slots subscribing to two queues at 50% and 25% reserves 8 slots for queue A and 4 slots for queue B, with 4 slots in the shared pool.
+
+### Interaction modes
+
+- **STRICT** — idle reserved slots remain exclusive to this subscription and are never lent to other work
+- **ELASTIC** — idle reserved slots may be lent to other `ELASTIC` subscriptions on the same worker when the subscription has spare capacity
+
+In both modes, tasks also draw from the shared pool once reserved slots are busy. Lent slots are not preempted — a busy lender may temporarily dip below its floor until borrowed work completes.
+
+Capacity reservations are live-configurable: updating a subscription's reserved percentage takes effect within seconds with no worker restarts required.
+
+## Worker authentication
+
+:::alert{type="info"}
+Worker authentication must be enabled server-side before registration tokens have any effect. A worker configured with a token but connecting to an instance where auth is disabled will join the default group instead.
+:::
+
+Workers join a group by presenting a registration token generated for that group. The token is stored as a hash and shown only once at creation — copy it immediately.
+
+On first connect, the worker exchanges the registration token for a short-lived access token and a rotating refresh token. The access token is refreshed automatically before it expires. Revoking or deleting a token immediately invalidates credentials for any workers that registered with it; those workers fail closed once their current access token expires.
+
+### Generating a registration token
+
+In the Worker Groups UI, select a group and generate a token from the **Tokens** tab. Alternatively, use the API:
+
+| Operation | Endpoint |
+|---|---|
+| Generate token | `POST /api/v1/instance/worker-groups/{id}/tokens` |
+| Revoke token | `POST /api/v1/instance/worker-groups/{id}/tokens/{tokenId}/revoke` |
+| Delete token | `DELETE /api/v1/instance/worker-groups/{id}/tokens/{tokenId}` |
+
+### Server-side configuration
+
+Enable worker authentication on your webserver or standalone Kestra instance:
```yaml
-id: worker_group
-namespace: company.team
+kestra:
+ ee:
+ worker:
+ auth:
+ enabled: true
+ jwt-signing-key: "{{ a strong shared secret, >= 32 bytes }}"
+ access-token-lifetime: PT5M # optional, default PT5M
+ refresh-token-lifetime: P7D # optional, default P7D
+```
-tasks:
- - id: wait
- type: io.kestra.plugin.core.flow.Sleep
- duration: PT0S
- workerGroup:
- key: gpu
- fallback: FAIL
+### Worker-side configuration
+
+Each worker needs two things to join a group: a registration token that identifies the group, and a controller endpoint that tells the worker where to connect. Both are required — a worker started with only the token will try `localhost` and fail.
+
+```yaml
+kestra:
+ worker:
+ name: gpu-pool-1 # optional display name
+ auth:
+ registration-token: "{{ token generated for the target group }}"
+ credentials-path: /var/kestra/worker/.auth/worker-credentials.json # default: /tmp/kestra/worker/.auth/...
+ refresh-buffer: PT60S # how early to refresh the access token before it expires
+ controllers:
+ type: STATIC
+ static:
+ endpoints:
+ - host: kestra-controller.internal
+ port: 50051
```
-Possible values for `workerGroup.fallback` are `WAIT` (default), `FAIL`, or `CANCEL`:
-- `WAIT`: The task will wait for the worker to be available and will remain in a `CREATED` state until the worker picks it up.
-- `FAIL`: The task run will be terminated immediately if the worker is not available, and the execution will be marked as `FAILED`.
-- `CANCEL`: The task run will be gracefully terminated, and the execution will be marked as `KILLED` without an error.
+:::alert{type="warning"}
+The endpoint `host` and `port` must be separate YAML keys. A single `host:port` string fails with `Static configuration requires at least one endpoint`.
+:::
-You can set a custom `workerGroup.key` and `workerGroup.fallback` per plugin type and/or per namespace using `pluginDefaults`.
+#### Controller discovery strategies
-When Fallback behavior is set in multiple places, Kestra resolves which action to take by following this priority order:
-1. **Flow-Level**: Uses the behavior specified in the `fallback` property of the Flow task.
-2. **Namespace-Level**: Uses the behavior set in the the Namespace settings.
-3. **Tenant-Level**: Uses the behavior set in the the Tenant settings.
+`type: STATIC` is the default and suitable for most bare-metal and Docker deployments. Two other strategies are available:
-### Fallback behavior at the namespace level
+| Type | When to use |
+|---|---|
+| `STATIC` | Fixed controller addresses — explicit `host`/`port` list |
+| `DNS` | Kubernetes or any environment where controllers are reachable by a stable DNS name; resolves SRV or A records on an interval |
+| `STORAGE` | Dynamic, cross-cloud deployments; controllers self-register in internal storage and workers list the registry |
-Namespaces can be configured to have a default `fallback` behavior. It can be configured by creating a namespace manaully or modifying in the **Edit** tab of the namespace.
+For Kubernetes, use `type: DNS` with a service hostname:
-
+```yaml
+kestra:
+ worker:
+ controllers:
+ type: DNS
+ dns:
+ hostname: kestra-controller.kestra.svc.cluster.local
+ record-type: SRV # or A if no SRV records
+ default-port: 50051 # used with A records only
+ refresh-interval: PT30S
+```
-### Fallback behavior at the tenant level
+For Helm deployments, controller discovery is preconfigured — see the [Helm gRPC and Worker-Controller migration guide](../../../11.migration-guide/v2.0.0/helm-grpc-worker-controller/index.md). For bare-metal or Docker with components on separate hosts, see [running Kestra with separated server components](../../../kestra-cli/kestra-server/index.md#kestra-with-server-components-in-different-services).
-Tenants can be configured to have a default `fallback` behavior. It can be configured when creating a tenant on in the tenant's properties.
+### Starting a worker for a group
-
+With both the registration token and controller endpoint configured, start the worker normally:
-## When to use Worker Groups
+```bash
+kestra server worker
+```
-Here are common use cases in which Worker Groups can be beneficial:
-- Execute tasks and polling triggers on specific compute instances (e.g., a VM with a GPU and preconfigured CUDA drivers).
-- Execute tasks and polling triggers on a worker with a specific Operating System (e.g., a Windows server).
-- Restrict backend access to a set of workers (firewall rules, private networks, etc.).
-- Execute tasks and polling triggers close to a remote backend (region selection).
+No additional CLI flags are needed. The registration token in `kestra.worker.auth.registration-token` identifies which group the worker joins at connection time.
-You can configure plugin groups to use a specific worker group. In this example, all [script tasks](../../../16.scripts/index.mdx) are set to run on the `gpu` worker group:
+:::alert{type="info"}
+Workers do not connect to the database. Any `datasources` or `kestra.repository.type` keys in the config are ignored; a startup warning lists which keys were skipped.
+:::
-```yaml
-id: worker_group
-namespace: company.team
+## Declarative configuration
-tasks:
- - id: wait
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.core.runner.Process
- commands:
- - sleep 10
+You can declare the entire worker topology — queues, groups, subscriptions, and registration tokens — in `application.yml` under `kestra.ee.setup`. Kestra applies this configuration at startup, which enables a fully automated single-pass deployment with no runtime API calls.
- - id: python_gpu
- type: io.kestra.plugin.scripts.python.Commands
- namespaceFiles:
+```yaml
+kestra:
+ ee:
+ setup:
enabled: true
- commands:
- - python ml_on_gpu.py
-pluginDefaults:
- - forced: false
- type: io.kestra.plugin.scripts
- values:
- workerGroup:
- key: gpu
+ worker-queues:
+ - id: gpu
+ tags: [gpu, linux]
+ allowed-tenants: [acme] # optional; empty = unrestricted
+ - id: etl
+ tags: [etl]
+
+ worker-groups:
+ - id: gpu-workers
+ name: GPU workers
+ registration-tokens:
+ - name: bootstrap
+ token-file: /var/run/secrets/kestra/gpu-workers-token
+ subscriptions:
+ - worker-queue-id: gpu
+ reserved-percent: 70
+ - worker-queue-id: etl
```
-### Distant workers
+Workers already retry registration until their token is known to the controller, so all services can start concurrently. Workers converge as soon as the webserver has applied the configuration.
-You can use a Worker Group to designate a worker to execute **any** task on a remote resource. Additionally, you may want to have an **always-on** worker that stays available for execution-intensive workloads.
+### Secret handling
-The Distant Worker use case requires a connection to the Kestra metastore, and it solves for scenarios of always-on, intensive workloads and workloads that need to execute workloads on an external environment.
+Registration tokens must not appear as plaintext in a committed configuration file. Two options are available per token entry:
-
+- **`token-file`** — path to a file containing the pre-generated token. Preferred in Kubernetes environments where Secrets mount as files. The file must exist and be non-empty at startup.
+- **`token: "${ENV_VAR}"`** — environment variable placeholder resolved at startup. Simpler outside Kubernetes, but environment variables are readable from `/proc//environ` and may appear in crash dumps.
-### Task runners
+Exactly one of the two is required. Use `kestra workers registration-tokens generate` to mint a token offline before deployment.
-If you are using scripting tasks, you can set up Worker Group of Task Runners to leverage **on-demand** cloud resources to execute intensive workloads. For example, you can have a Worker Group dedicated to executing on AWS Batch or Kubernetes.
+### The default group
-This is particularly useful for script task workloads that have bursts in resource demand.
+The default group can be declared under its reserved id `default`:
-
+```yaml
+worker-groups:
+ - id: default
+ name: Shared workers
+ registration-tokens:
+ - name: bootstrap
+ token-file: /var/run/secrets/kestra/default-workers-token
+```
-### Data isolation
+The default group always subscribes to the default queue — Kestra adds that subscription automatically even when `subscriptions` is omitted or does not include the default queue. Any subscriptions you declare are added alongside it.
-Worker Groups strongly fits **Data Isolation** use cases. Multi-tenancy requirements may demand that you have strict isolation of remote resources such as key vaults. Worker groups enable you to split out dedicated workers per tenant.
+### Semantics
-In the below architecture, it is not possible to execute tasks on worker 1 from tenant 3.
+`kestra.ee.setup` is a seed, not a desired state:
-
+- Each declared entity is created only when no entity with the same id already exists in the database.
+- An existing entity is skipped as a whole — no subscriptions are changed, no tokens are added or revoked.
+- Re-applying a changed configuration against an existing entity is a no-op. The database remains the source of truth once an entity exists; editing a live topology stays an API or UI operation.
-:::alert{type="warning"}
-Even if you are using worker groups, we strongly recommend having at least one worker in the default worker group.
+A rogue instance cannot self-authorize by changing configuration: declarative setup can only add what is absent, never replace or revoke what the authenticated API created.
+
+### Which server roles apply it
+
+Only the webserver and standalone server roles apply `kestra.ee.setup` at startup. Worker processes never apply it — a worker must not be able to create the group or the token it authenticates against.
+
+### Validation
+
+The entire declaration is validated before anything is written. An invalid configuration fails startup with an actionable message identifying the offending path:
+
+```
+The subscription declared at 'kestra.ee.setup.worker-groups[0].subscriptions[1]'
+references the unknown Worker Queue 'etl'. Declare it under
+'kestra.ee.setup.worker-queues' or create it first.
+```
+
+Validation rejects: missing or duplicate ids, non-RFC-1123 ids, the reserved worker queue ids `default` and `system`, empty tag sets, tag collisions with existing queues, unknown `worker-queue-id` references, `reserved-percent` out of range or summing above 100, unreadable or empty token files, tokens already registered on another group, and malformed registration tokens.
+
+Validation is all-or-nothing, but writes are not atomic. A failure mid-apply leaves already-created entities in place. The next startup resumes from where it stopped because existing entities are skipped.
+
+### Observability
+
+Kestra logs one line per created entity and one line per skipped entity. A summary line follows after the setup phase completes. Each created entity also produces a regular audit log entry. Token values are never logged.
+
+## Transport security (TLS)
+
+By default, gRPC traffic between workers and the controller is unencrypted. For production deployments, enable TLS on both sides.
+
+### Server-side TLS
+
+Add TLS config to the controller (or standalone) instance:
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ key-store:
+ path: /etc/kestra/tls/controller-keystore.p12
+ password: "{{ secret('TLS_KEYSTORE_PASSWORD') }}"
+ # Required when client-auth is OPTIONAL or REQUIRE
+ trust-store:
+ path: /etc/kestra/tls/ca-truststore.p12
+ password: "{{ secret('TLS_TRUSTSTORE_PASSWORD') }}"
+ client-auth: NONE # NONE | OPTIONAL | REQUIRE (mTLS)
+```
+
+### Worker-side TLS
+
+Add matching TLS config to each worker:
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ # Required only for mTLS (client-auth: REQUIRE on the server)
+ key-store:
+ path: /etc/kestra/tls/worker-keystore.p12
+ password: "{{ secret('TLS_KEYSTORE_PASSWORD') }}"
+ # Optional — falls back to the system trust store
+ trust-store:
+ path: /etc/kestra/tls/ca-truststore.p12
+ password: "{{ secret('TLS_TRUSTSTORE_PASSWORD') }}"
+```
+
+:::alert{type="info"}
+When using `type: STATIC` discovery with TLS, the synthetic gRPC authority derived from the endpoint list may not match the certificate's SANs. Add `authority-override` to the worker config to specify the hostname the certificate was issued for:
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ authority-override: kestra-controller
+```
+
+Under `type: DNS` discovery, the authority is derived from the DNS hostname automatically and no override is needed.
:::
-## Load balancing
+## Use cases
+
+### Hardware affinity
+
+Dedicate workers with GPUs, high-memory configurations, or OS-specific environments to tasks that need them. Developers declare the requirement via tags; operators manage the physical mapping independently.
+
+```yaml
+workerSelector:
+ tags: [gpu, cuda-12]
+```
+
+### Multi-tenant isolation
+
+Give each tenant a dedicated Worker Queue with a reserved capacity percentage to prevent noisy-neighbor effects. An additional ELASTIC subscription to a shared burst queue lets idle capacity absorb traffic spikes while the per-tenant floor stays guaranteed.
+
+### Regulated and air-gapped environments
+
+Workers in restricted networks connect outbound-only, presenting a registration token to authenticate. No inbound firewall rules are required. Revoking a token immediately stops those workers from receiving new work, giving operators a fast, clean isolation path.
+
+### Spiky workloads
-Whether you leverage worker groups or not, Kestra will balance the load across all available workers. The primary difference is that with worker groups, you can target **specific** workers for task execution or polling trigger evaluation.
+Use a fixed worker pool with STRICT reservations to handle baseline load, and a spot pool with ELASTIC subscriptions that claims shared-pool capacity during spikes. The ELASTIC pool scales out and in without changing the baseline pool's guarantees.
+
+### Priority lanes
+
+Split capacity across multiple queues with reserved percentages to guarantee throughput for high-priority work:
+
+```yaml
+# Three priority queues — critical: 50%, standard: 25%, batch: 25%
+workerSelector:
+ tags: [critical] # or [standard], or [batch]
+```
-A worker is part of a worker group if it is started with the `--worker-group workerGroupKey` argument.
+Critical work always has guaranteed slots regardless of the volume of batch jobs in the queue.
-There's a slight difference between Kafka and JDBC architectures in terms of load balancing:
-- The Kafka architecture relies on Kafka consumer group protocol — each worker group will use a different consumer group protocol, therefore each worker group will balance the load independently.
-- For JDBC, each worker within a group will poll the `queues` database table using the same poll query. All workers within the same worker group will poll for task runs and polling triggers in a FIFO manner.
+### Day/night capacity shifting
-### Central queue to distribute task runs and polling triggers
+Reserved percentages are live-configurable via the API. Changes propagate to all workers within seconds, with no restarts required. Shift capacity toward batch workloads during off-peak hours and back to interactive workloads during business hours without touching any worker process.
-In both JDBC and Kafka architectures, we leverage a Central Queue to ensure that tasks and polling triggers are executed only once and in the right order.
+### Zero-downtime worker upgrades
-Here's how it works:
-- Jobs (task runs and polling triggers) are submitted to a centralized queue. The queue acts as a holding area for all incoming jobs.
-- Workers periodically poll the central queue to check for available jobs. When a worker becomes free, it requests the next job from the queue.
-- Kestra backend keeps track of assignment of jobs to workers to ensure reliable execution and prevent duplicate processing.
+Run two Worker Groups subscribed to the same queues simultaneously. Reduce the old group's reservation to 0% to drain it of new work, bring up the new group, verify it is healthy, then delete the old group. At no point does the queue go unserved.
-### What if multiple workers from the same Worker Group poll for jobs from the central queue?
+For guidance on when to use Worker Groups versus Task Runners for compute-intensive scripting workloads, see [Task Runners vs Worker Groups](../../../task-runners/03.task-runners-vs-worker-groups/index.md).
-Whether the jobs (task runs and polling triggers) are evenly distributed among workers depends on several factors:
-1. The order in which workers poll the queue will affect distribution — workers that poll the queue first will get jobs first (FIFO).
-2. Variations in worker compute capabilities (and their processing speeds) can cause uneven job distribution. Faster workers will complete jobs and return to poll the queue more quickly than slower workers.
+## Worker shutdown and task continuity
+
+When a worker process stops — whether from a deployment, a crash, or a manual restart — any tasks it was running may be interrupted. The `worker-task-restart-strategy` setting controls what happens to those tasks cluster-wide:
+
+| Strategy | Behavior |
+|---|---|
+| `AFTER_TERMINATION_GRACE_PERIOD` | The worker stops accepting new work and waits up to the grace period for in-flight tasks to finish; any tasks still running at that point are re-dispatched to another worker (default) |
+| `IMMEDIATELY` | Interrupted tasks are re-dispatched immediately to another worker without waiting |
+| `NEVER` | Interrupted tasks fail permanently and are not re-dispatched |
+
+Configure these in `application.yml` on each worker:
+
+```yaml
+kestra:
+ server:
+ termination-grace-period: 5m
+ worker-task-restart-strategy: AFTER_TERMINATION_GRACE_PERIOD
+```
+
+During the grace period, the worker stops accepting new jobs but lets running tasks finish. If the grace period elapses before all tasks complete, the worker force-terminates and the restart strategy decides the outcome for the remaining tasks.
+
+:::alert{type="info"}
+`AFTER_TERMINATION_GRACE_PERIOD` is the recommended setting for production deployments. It gives tasks time to finish cleanly while still guaranteeing that a stalled worker does not block the cluster indefinitely.
+:::
+
+## Monitoring
+
+Metrics scoped to a group carry a `worker_group` tag; metrics scoped to a queue also carry a `worker_queue` tag. The configurable metrics prefix (default `kestra`) is prepended before export.
+
+### Controller metrics
+
+Published by the controller process — the server-side view of fleet capacity and dispatch activity:
+
+| Metric | Type | Tags | Description |
+|---|---|---|---|
+| `controller.worker.active` | gauge | `worker_group`, `worker_queue` | Workers currently subscribed to a queue |
+| `controller.worker.active.all` | gauge | — | Total workers connected to this controller |
+| `controller.permits.available` | gauge | `worker_group`, `worker_queue` | Remaining advertised capacity across subscribed workers |
+| `controller.permits.available.all` | gauge | — | Remaining capacity across all connected workers |
+| `controller.job.inflight` | gauge | `worker_queue` | In-flight jobs for a queue |
+| `controller.worker.group.job.inflight` | gauge | `worker_group` | In-flight jobs across workers in a group |
+| `controller.capacity.subscription.allocated` | gauge | `worker_group`, `worker_queue` | Reserved slots allocated to a queue subscription |
+| `controller.capacity.subscription.used` | gauge | `worker_group`, `worker_queue` | Reserved slots currently in use |
+| `controller.capacity.shared.allocated` | gauge | `worker_group` | Shared (unreserved) slots allocated |
+| `controller.capacity.shared.used` | gauge | `worker_group` | Shared slots currently in use |
+| `controller.job.dispatched.total` | counter | `worker_queue` | Total jobs dispatched to workers |
+| `controller.job.requeued.total` | counter | `worker_queue` | Jobs re-queued because no worker had capacity |
+| `controller.job.killed.total` | counter | `worker_queue` | Jobs short-circuited by the pre-dispatch kill check |
+| `controller.job.dispatch.failed.total` | counter | `worker_queue` | Dispatch attempts that failed on send |
+| `controller.worker.registered.total` | counter | — | Worker-queue subscription registrations |
+| `controller.worker.unregistered.total` | counter | — | Worker-queue subscription removals |
+| `controller.subscription.paused.total` | counter | — | Queue subscription pause transitions |
+| `controller.subscription.resumed.total` | counter | — | Queue subscription resume transitions |
+
+### Worker metrics
+
+Published by each worker process — the worker-side view of capacity and throughput:
+
+| Metric | Type | Description |
+|---|---|---|
+| `worker.job.thread` | gauge | Configured thread count (maximum concurrent jobs) |
+| `worker.max.concurrency` | gauge | Maximum in-flight capacity: threads + buffered jobs |
+| `worker.running.count` | gauge | Tasks currently executing |
+| `worker.pending.count` | gauge | Tasks waiting for a free thread slot |
+| `worker.queue.size` | gauge | Items currently held in a buffer (job, result, log, or metric) |
+| `worker.queue.remaining.capacity` | gauge | Free slots in the inbound job buffer — equals the worker's advertised permit count |
+| `worker.queued.duration` | timer | Time a task spent waiting before a thread was available |
+| `worker.started.count` | counter | Total tasks started |
+| `worker.ended.count` | counter | Total tasks completed (any terminal state) |
+| `worker.ended.duration` | timer | Task run duration as measured by the worker |
+| `worker.timeout.count` | counter | Tasks that exceeded their configured timeout |
+| `worker.killed.count` | counter | Kill events received from the controller |
+| `worker.queue.enqueued` | counter | Total items put into a buffer |
+| `worker.queue.dequeued` | counter | Total items drained from a buffer |
+| `worker.trigger.running.count` | gauge | Trigger evaluations currently in progress |
+| `worker.trigger.started.count` | counter | Total trigger evaluations started |
+| `worker.trigger.ended.count` | counter | Total trigger evaluations completed |
+| `worker.trigger.error.count` | counter | Trigger evaluations that failed |
+| `worker.trigger.execution.count` | counter | Executions produced by triggers on this worker |
+| `worker.trigger.duration` | timer | Trigger evaluation duration |
+
+When `worker.running.count` consistently equals `worker.job.thread` and `worker.pending.count` is non-zero, that worker is fully saturated — scale by adding more workers to the group or increasing the thread count. When `worker.queue.remaining.capacity` on the `job` buffer approaches zero, the worker's local inbound buffer is full.
+
+The live capacity snapshot is also available via the API:
+
+```
+GET /api/v1/instance/worker-groups/{id}/capacity
+GET /api/v1/instance/worker-groups/{id}/workers
+```
+
+## Migrating from earlier versions
+
+In Kestra 2.0, the task-level routing property changed from targeting a group by name to declaring requirements via tags:
+
+| Before 2.0 | 2.0+ |
+|---|---|
+| `workerGroup.key: gpu` | `workerSelector.tags: [gpu]` |
+| Routes directly to a named group | Routes to a Worker Queue by tags; any subscribed group may serve the task |
+| `workerGroup.fallback` (defaults to `WAIT`) | `workerSelector.fallback` (defaults to `FAIL`) |
+| No `match` strategy | `workerSelector.match: ALL` or `ANY` |
+| No capacity control per queue | Reserved percentage per subscription, STRICT or ELASTIC mode |
+| No worker authentication | Registration token-based authentication with rotating credentials |
+
+`workerGroup` is not recognized in 2.0. Flows using it will fail validation and cannot be saved. Update your flows to replace `workerGroup.key` with `workerSelector.tags`. The group name in the old property corresponds to a tag on a Worker Queue in the new model.
+
+:::alert{type="warning"}
+The fallback default changed from `WAIT` to `FAIL`. Tasks that previously waited for an unavailable worker will now fail immediately unless you explicitly set `workerSelector.fallback: WAIT`.
+:::
diff --git a/src/contents/docs/07.enterprise/04.scalability/worker-group/task-runners.png b/src/contents/docs/07.enterprise/04.scalability/worker-group/task-runners.png
deleted file mode 100644
index a0011c66e15..00000000000
Binary files a/src/contents/docs/07.enterprise/04.scalability/worker-group/task-runners.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-details.png b/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-details.png
deleted file mode 100644
index 5df6b6cd126..00000000000
Binary files a/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-details.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-namespace.png b/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-namespace.png
deleted file mode 100644
index e8dd1f5956a..00000000000
Binary files a/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-namespace.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-tenant.png b/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-tenant.png
deleted file mode 100644
index 9ecc63882aa..00000000000
Binary files a/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-tenant.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-ui.png b/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-ui.png
deleted file mode 100644
index 4f8504d79c5..00000000000
Binary files a/src/contents/docs/07.enterprise/04.scalability/worker-group/worker-group-ui.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/00.instance-owner/index.md b/src/contents/docs/07.enterprise/05.instance/00.instance-owner/index.md
new file mode 100644
index 00000000000..a5ff123eaae
--- /dev/null
+++ b/src/contents/docs/07.enterprise/05.instance/00.instance-owner/index.md
@@ -0,0 +1,105 @@
+---
+title: Instance Owner Console in Kestra Enterprise
+h1: Manage Your Instance with the Instance Owner Console
+description: Use the Instance Owner console in Kestra Enterprise to manage tenants, instance-wide IAM, infrastructure, and governance from a single interface.
+sidebarTitle: Instance Owner
+icon: /src/contents/docs/icons/admin.svg
+editions: ["EE", "Cloud"]
+---
+
+The Instance Owner console provides instance-wide administration for tenants, IAM, infrastructure, and governance — separate from the tenant workspace you work in day to day.
+
+:::alert{type="warning"}
+Instance Owner operations are instance-wide. Changes to tenants, users, worker groups, and instance-level policies affect the entire Kestra instance, not just the tenant you are currently logged into.
+:::
+
+## Entering Instance Owner
+
+In the sidebar, click **Instance Owner**. The entry is only visible to users with the Instance Owner privilege.
+
+The UI switches to the Instance Owner console and shows a banner confirming you are administering the whole instance. Your regular tenant workspace is unaffected — you can return to it at any time.
+
+## The Instance Owner console
+
+The console organizes instance-wide administration into five sections:
+
+
+
+| Section | What you can manage |
+| :--- | :--- |
+| **Instance** | Overview, System Overview, Services, Kill Switch, Announcements |
+| **Tenants** | Create, edit, and delete tenants; configure dedicated storage and secrets backends per tenant |
+| **Instance IAM** | Users and Service Accounts that exist at the instance level, independently of any tenant |
+| **Infrastructure** | Worker Groups, Worker Queues, Versioned Plugins, MCP Servers |
+| **Governance** | Instance-level Policies and Audit Logs across all tenants |
+
+## Exiting Instance Owner
+
+Click **Exit Instance Owner** at the top of the left sidebar to return to your tenant workspace.
+
+## Who can access Instance Owner
+
+Only users with the Instance Owner privilege can enter the Instance Owner console.
+
+## Creating an Instance Owner user
+
+### Through the setup wizard
+
+When you launch Kestra for the first time, the [setup wizard](../../01.overview/02.setup/index.md) invites you to create the first user, which is automatically assigned the Instance Owner privilege.
+
+### Through the CLI
+
+To create a new user with the Instance Owner privilege:
+
+```bash
+kestra auths users create admin@kestra.io TopSecret42 --instance-owner
+
+# with tenant scoping:
+kestra auths users create \
+--tenant= --instance-owner
+```
+
+:::alert{type="info"}
+`--superadmin` is a deprecated alias for `--instance-owner` and still works.
+:::
+
+### Through configuration
+
+An Instance Owner can also be defined in the configuration file:
+
+```yaml
+kestra:
+ security:
+ instance-owner:
+ username:
+ password:
+ tenant-admin-access:
+ -
+```
+
+:::alert{type="info"}
+`kestra.security.super-admin` is a deprecated alias for `kestra.security.instance-owner` and still works.
+:::
+
+For the full list of security configuration options, see [Security and Secrets configuration](../../../configuration/05.security-and-secrets/index.md).
+
+## Granting and revoking Instance Owner access
+
+You must be an Instance Owner yourself to grant or revoke the privilege.
+
+### Through the UI
+
+Open the user's detail page and toggle the Instance Owner switch:
+
+
+
+### Through the CLI
+
+```bash
+kestra auths users set-instance-owner admin@kestra.io true # grant
+kestra auths users set-instance-owner admin@kestra.io false # revoke
+```
+
+:::alert{type="info"}
+`set-superadmin` is a deprecated alias for `set-instance-owner` and still works.
+:::
diff --git a/src/contents/docs/07.enterprise/05.instance/00.instance-owner/super-admin-console.png b/src/contents/docs/07.enterprise/05.instance/00.instance-owner/super-admin-console.png
new file mode 100644
index 00000000000..1ed2890f063
Binary files /dev/null and b/src/contents/docs/07.enterprise/05.instance/00.instance-owner/super-admin-console.png differ
diff --git a/src/contents/docs/07.enterprise/05.instance/00.instance-owner/superadmin_switch.png b/src/contents/docs/07.enterprise/05.instance/00.instance-owner/superadmin_switch.png
new file mode 100644
index 00000000000..9d846d149d0
Binary files /dev/null and b/src/contents/docs/07.enterprise/05.instance/00.instance-owner/superadmin_switch.png differ
diff --git a/src/contents/docs/07.enterprise/05.instance/announcements/create-announcement.png b/src/contents/docs/07.enterprise/05.instance/announcements/create-announcement.png
deleted file mode 100644
index 9e016353aaa..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/announcements/create-announcement.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/announcements/display-announcement.png b/src/contents/docs/07.enterprise/05.instance/announcements/display-announcement.png
index 357f1e8be5d..0298cff5a3c 100644
Binary files a/src/contents/docs/07.enterprise/05.instance/announcements/display-announcement.png and b/src/contents/docs/07.enterprise/05.instance/announcements/display-announcement.png differ
diff --git a/src/contents/docs/07.enterprise/05.instance/announcements/index.md b/src/contents/docs/07.enterprise/05.instance/announcements/index.md
index 4d32a751775..56d8c5290ab 100644
--- a/src/contents/docs/07.enterprise/05.instance/announcements/index.md
+++ b/src/contents/docs/07.enterprise/05.instance/announcements/index.md
@@ -14,23 +14,14 @@ Communicate planned maintenance or incidents with in-app banners
-## Announcements – in-app banners
-
-Announcements allow you to notify your users about any important events such as planned maintenance downtime.
-
## How to create an announcement
-To add a custom in-app banner, go to the **Instance → Announcements** tab.
-
-
+To add a custom in-app banner, go to **Instance Owner → Instance → Announcements**.
-As a user with an Admin role, you can configure the following within each announcement:
+As an Instance Owner, you can configure the following within each announcement:
- **Message**: the text to display in the banner
-- **Type**: the type of banner to display (**INFO, WARNING, ERROR**)
-- The **START** and **END** date during which the announcement should be displayed.
-
-
-
+- **Type**: the severity of the banner — `INFO`, `WARNING`, or `ERROR`
+- **Start** and **End** date during which the announcement is displayed.
diff --git a/src/contents/docs/07.enterprise/05.instance/announcements/instance-announcements.png b/src/contents/docs/07.enterprise/05.instance/announcements/instance-announcements.png
deleted file mode 100644
index c7f24fd956a..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/announcements/instance-announcements.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/index.mdx b/src/contents/docs/07.enterprise/05.instance/index.mdx
index 739d184b66b..1224cec1543 100644
--- a/src/contents/docs/07.enterprise/05.instance/index.mdx
+++ b/src/contents/docs/07.enterprise/05.instance/index.mdx
@@ -9,7 +9,7 @@ editions: ["EE", "Cloud"]
import ChildCard from "~/components/docs/ChildCard.astro"
-The Instance menu gives you a centralized view of your Kestra deployment's health along with controls for upgrades, maintenance, and user notifications.
+The [Instance Owner console](./00.instance-owner/index.md) gives you a centralized view of your Kestra deployment's health along with controls for upgrades, maintenance, and user notifications.
-## Overview
+## System overview
-The **Instance** menu provides centralized control over your platform’s infrastructure so administrators can monitor service health, manage configurations, and communicate announcements (like planned maintenance downtime) to users without relying on additional observability tools. The **System Overview** tab gives a high-level snapshot of your instance’s operational status. Here, you can:
-
-- View **License Details**: validity, usage statistics, and installed secrets and storage plugins.
-
-
+Inside the Instance Owner console, the **Instance** section covers service health, kill switches, announcements, maintenance mode, and plugin versioning. The **System Overview** tab gives a high-level snapshot of your instance’s operational status, including license validity, usage statistics, and installed secrets and storage plugins.
## Services
@@ -38,16 +34,7 @@ Kestra tracks the health of critical components, including:
- **Executors**: Manage task execution.
- **Webservers**: Host the UI and API.
-
-
-Each service displays:
-
-- **State**: Whether the service is active (`RUNNING`) or unresponsive.
-- **Host Name**: Identifier of the server/pod (e.g., `kafka-ee-preview-79fb7755f8-zhlhq`).
-- **Server Type**: For example, STANDALONE.
-- **Version**
-- **Start Date**
-- **Health Check Date**
+Each service displays its state, hostname, server type, version, start date, and last health check date. Click a service to see per-instance details for debugging: session timeout, heartbeat interval, last heartbeat, and termination grace period.
### Service states
@@ -58,66 +45,18 @@ Each service displays:
- **TERMINATED_GRACEFULLY / TERMINATED_FORCED** — shutdown completed (clean or forced), moves to `NOT_RUNNING`.
- **NOT_RUNNING / INACTIVE** — post-shutdown, final states reported in the UI.
-States come from the service lifecycle. Typical flow:
-- **CREATED** → **RUNNING** after a clean start; **MAINTENANCE** can be set directly for planned work.
-- **DISCONNECTED** signals a lost dependency (DB/queue) and may recover to **RUNNING** or proceed to shutdown.
-- **TERMINATING** attempts graceful stop; it ends in **TERMINATED_GRACEFULLY** or **TERMINATED_FORCED**, then **NOT_RUNNING** → **INACTIVE**.
-
-### Server information and liveness
-
-Each service instance provides technical details for debugging when clicked on:
-
-- **Hostname**: Identifier of the server/pod (e.g., `kafka-ee-preview-79fb7755f8-zhlhq`).
-- **Session Timeout**: Time before an unresponsive service is marked offline (e.g., `60 seconds`).
-- **Heartbeat Interval**: The expected time between heartbeats.
-- **Last Heartbeat**: Timestamp of the latest health check.
-- **Termination Grace Period**: The expected time for this service to complete all its tasks before initiating a graceful shutdown.
-
-
-
-Additional tabs include **Configuration** to display port configuration, **Metrics** such as CPU Usage and Executor Thread Count:
-
-
-
-And an **Events Timeline** to give an overview of the service's lifecycle:
-
-
+Additional tabs include **Configuration** for port details, **Metrics** such as CPU usage and executor thread count, and an **Events Timeline** showing the service’s full lifecycle.
## Announcements
-Notify users about planned maintenance or updates:
+[Announcements](./announcements/index.md) let you broadcast in-app banners to notify users about planned maintenance, incidents, or other events. Each announcement has a message, a severity type (`INFO`, `WARNING`, or `ERROR`), and a start and end date.
-1. **Create Announcements**: Specify a title, message, and date range.
-2. **Choose Type**: Define severity of the announcement (e.g., `info`, `warning`, `error`).
+## Maintenance mode
-[Announcements](./announcements/index.md) appear in the UI during the selected period, ensuring users stay informed.
-
-## Maintenance Mode
-
-From the **Instance - Services** tab, enter [Maintenance Mode](../05.instance/maintenance-mode/index.md) to temporarily pause all workflows and services for upgrades:
-
-- Services enter a paused state, and new executions are blocked.
-- Combine it with the Announcements feature so users see a maintenance banner while running workflows gracefully terminate.
-
-## Worker Groups
-
-Create [Worker Groups](../04.scalability/worker-group/index.md) to isolate workloads or delegate tasks to specific workers:
-
-- **Add Worker Groups**: Define groups with specific resource limits or labels.
-- **Assign Tasks**: Route workflows to designated groups via worker group key within a task or trigger.
-
-## Audit Logs
-
-View [Audit Logs](../02.governance/06.audit-logs/index.md) at a glance to monitor actions on all resource types taken by users in the instance.
-
-
+[Maintenance mode](./maintenance-mode/index.md) pauses the platform broadly: new executions are queued and running ones are allowed to finish before services stop. Combine it with an Announcement so users see a banner while the platform drains.
## Versioned Plugins
-View all installed [Versioned Plugins](../05.instance/versioned-plugins/index.md) on the instance and upgrade, install, or uninstall as needed.
-
-
-
-## Instance management – services and maintenance
+[Versioned Plugins](./versioned-plugins/index.md) lists all plugins installed on the instance. Upgrade, install, or uninstall plugins without restarting Kestra.
diff --git a/src/contents/docs/07.enterprise/05.instance/instance-audit-logs.png b/src/contents/docs/07.enterprise/05.instance/instance-audit-logs.png
deleted file mode 100644
index 7c3e4b76c90..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/instance-audit-logs.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/instance-system-overview.png b/src/contents/docs/07.enterprise/05.instance/instance-system-overview.png
deleted file mode 100644
index 9c6f701a567..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/instance-system-overview.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/instance-table.png b/src/contents/docs/07.enterprise/05.instance/instance-table.png
deleted file mode 100644
index 295bb33f55c..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/instance-table.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/instance-versioned-plugins.png b/src/contents/docs/07.enterprise/05.instance/instance-versioned-plugins.png
deleted file mode 100644
index 4d6a64af1fe..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/instance-versioned-plugins.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/kill-switch/create-kill-switch.png b/src/contents/docs/07.enterprise/05.instance/kill-switch/create-kill-switch.png
deleted file mode 100644
index bb8f77d06cc..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/kill-switch/create-kill-switch.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/kill-switch/index.md b/src/contents/docs/07.enterprise/05.instance/kill-switch/index.md
index 72b214200a8..92e812fe45f 100644
--- a/src/contents/docs/07.enterprise/05.instance/kill-switch/index.md
+++ b/src/contents/docs/07.enterprise/05.instance/kill-switch/index.md
@@ -25,9 +25,7 @@ Kill Switch replaces the CLI-only `--skip-executions` and `--skip-flows` command
## Configure a Kill Switch
-To configure a Kill Switch, navigate to your **Instance → Kill Switch** section in Kestra. From there, name the Kill Switch (e.g., `Kill Switch – Payments Namespace Outage (TEMP)` ) and configure the switch's specifications.
-
-
+To configure a Kill Switch, go to **Instance Owner → Instance → Kill Switch**. Name the switch (e.g., `Kill Switch – Payments Namespace Outage (TEMP)`) and configure its specifications.
### Kill Switch types
diff --git a/src/contents/docs/07.enterprise/05.instance/maintenance-mode/index.md b/src/contents/docs/07.enterprise/05.instance/maintenance-mode/index.md
index 1275839ac83..602dcf9924e 100644
--- a/src/contents/docs/07.enterprise/05.instance/maintenance-mode/index.md
+++ b/src/contents/docs/07.enterprise/05.instance/maintenance-mode/index.md
@@ -10,25 +10,18 @@ version: "0.21.0"
Prepare your Kestra instance for maintenance or migration.
-Maintenance Mode is an enterprise feature designed to transition your Kestra instance into a paused state to conduct maintenance operations such as platform updates.
-
-## Maintenance mode – pause for upgrades
-
-Maintenance Mode addresses a common challenge faced by organizations running numerous workflows: finding the right moment to perform platform updates without disrupting ongoing operations. When activated, Maintenance Mode introduces a controlled state where:
-
-- The [executor](../../../08.architecture/02.server-components/index.md#executor) stops processing new executions and automatically queues new flow executions.
-- Existing executions are allowed to be completed gracefully ([workers](../../../08.architecture/02.server-components/index.md#worker) complete their current tasks without picking up new ones).
-- The platform continues to accept and schedule new executions, storing them for later processing ([web server](../../../08.architecture/02.server-components/index.md#webserver) and [scheduler](../../../08.architecture/02.server-components/index.md#scheduler) components remain active, ensuring no requests are lost).
-- New executions are queued for processing after maintenance concludes
+## How maintenance mode works
-## Access maintenance mode
+When activated:
-Maintenance Mode is accessible via the **Instance** menu section of the Kestra UI. You can switch to maintenance mode in the **Services** tab by clicking the **enter maintenance mode** button. This triggers a confirmation prompt and displays information regarding the transition into maintenance mode.
+- The [executor](../../../08.architecture/02.server-components/index.md#executor) stops processing new executions and queues them automatically.
+- [Workers](../../../08.architecture/02.server-components/index.md#worker) finish their current tasks without picking up new ones.
+- The [web server](../../../08.architecture/02.server-components/index.md#webserver) and [scheduler](../../../08.architecture/02.server-components/index.md#scheduler) remain active — no requests are lost.
-
+## Enter maintenance mode
-After completing all maintenance operations, you can exit maintenance mode with the same button and confirm that you want to switch back to a live state of your Kestra instance.
+Go to **Instance Owner → Instance → Services** and click **Enter maintenance mode** in the top-right corner. A confirmation prompt explains what will happen before the transition begins. To exit, click the same button and confirm you want to return to a live state.
diff --git a/src/contents/docs/07.enterprise/05.instance/maintenance-mode/maintenance-mode.png b/src/contents/docs/07.enterprise/05.instance/maintenance-mode/maintenance-mode.png
deleted file mode 100644
index 5ad71d1609a..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/maintenance-mode/maintenance-mode.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/service-events.png b/src/contents/docs/07.enterprise/05.instance/service-events.png
deleted file mode 100644
index 61c3aca771c..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/service-events.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/service-metrics.png b/src/contents/docs/07.enterprise/05.instance/service-metrics.png
deleted file mode 100644
index 9af933624ce..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/service-metrics.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/services-overview.png b/src/contents/docs/07.enterprise/05.instance/services-overview.png
deleted file mode 100644
index 8e5f9757f48..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/services-overview.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/index.md b/src/contents/docs/07.enterprise/05.instance/versioned-plugins/index.md
index 29a7f84fa0a..555a54a5d77 100644
--- a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/index.md
+++ b/src/contents/docs/07.enterprise/05.instance/versioned-plugins/index.md
@@ -100,32 +100,20 @@ Below is a video demonstration walking through each step from installation to ap
-Both official and custom plugins can be installed from the UI. Navigate to the **Instance > Versioned Plugins** section. You can click **+ Install** and open up the full library of available plugins.
+Both official and custom plugins can be installed from the UI. Go to **Instance Owner → Infrastructure → Versioned Plugins** and click **+ Install** to browse the full plugin library. Search for the plugin, select the version, and confirm.
-
+After installing, the full list of versioned plugins is displayed. Kestra alerts you when a newer version is available. Upgrading installs the new version separately — the previous version stays active in existing flows.
-From the list, search and select the plugin to install and select the version.
+For a custom plugin, click **+ Install**, switch to **Custom plugin**, and provide:
-
+- **Group ID**: The group identifier of the plugin.
+- **Artifact ID**: The artifact identifier of the plugin.
-After installing plugins, the full list of versioned plugins is displayed. Kestra alerts you that a newer version of your plugin is available and allows you to upgrade by installing the latest version. When upgrading, the previous version of the plugin is preserved, and a separate, fresh installation of the latest version is added.
-
-
-
-For a custom plugin, after clicking **+ Install**, switch from Official plugin to Custom plugin. You need to specify two identifiers for each custom plugin installation:
-
-- Group ID: The group identifier of the plugin to be installed.
-- Artifact ID: The artifact identifier of the plugin to be installed.
-
-
-
-Instead of installing a new plugin, you can **Upload** a plugin by choosing a valid Java archive file (`.jar`).
-
-
+To upload a custom build, click **Upload** and select a valid `.jar` file.
### From the API
-Only Super Admin users can install versioned plugins with the API. To install a versioned plugin, you can use the API POST request with your username and password with `-u` or an [API token](../../03.auth/api-tokens/index.md).
+Only Instance Owner users can install versioned plugins with the API. To install a versioned plugin, you can use the API POST request with your username and password with `-u` or an [API token](../../03.auth/api-tokens/index.md).
With Kestra username and password:
@@ -221,7 +209,7 @@ When there are multiple versions of a plugin available, Kestra resolves the vers
4. **Instance-Level**: Using the value set in `kestra.plugins.management.defaultVersion` (default: `LATEST`).
- This property can be configured to `NONE` to enforce that a version is always explicitly defined.
-**Note**: By default, Kestra defaults to `LATEST` for core plugins if no version can be resolved. For other plugins, if no version can be resolved, the Flow will be considered invalid.
+By default, Kestra uses `LATEST` for core plugins when no version can be resolved. For other plugins, an unresolvable version renders the flow invalid.
:::alert{type="info"}
The version is resolved both at flow creation time and execution time to ensure the correct plugin version is used during both stages. This means that a Task/Trigger can only be deserialized after ensuring that all default versions are properly resolved.
diff --git a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-1.png b/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-1.png
deleted file mode 100644
index e7759d5b3cb..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-1.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-2.png b/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-2.png
deleted file mode 100644
index 17dcbdfef80..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-2.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-3.png b/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-3.png
deleted file mode 100644
index 6095cded714..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-3.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-4.png b/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-4.png
deleted file mode 100644
index 93280abe387..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-4.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-5.png b/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-5.png
deleted file mode 100644
index 1598ebea60d..00000000000
Binary files a/src/contents/docs/07.enterprise/05.instance/versioned-plugins/versioned-plugins-5.png and /dev/null differ
diff --git a/src/contents/docs/07.enterprise/06.ee-faq/index.md b/src/contents/docs/07.enterprise/06.ee-faq/index.md
index 9d0c88a3ad8..ed1526eecae 100644
--- a/src/contents/docs/07.enterprise/06.ee-faq/index.md
+++ b/src/contents/docs/07.enterprise/06.ee-faq/index.md
@@ -9,11 +9,9 @@ editions: ["EE", "Cloud"]
Frequently asked questions about the Cloud and Enterprise Edition of Kestra.
-## Kestra Cloud & Enterprise FAQ – common questions
-
## My session expires too quickly. Is there a way to change the session expiration time?
-Yes, there is! Add the following Micronaut setting to your [Observability and Networking configuration](../../configuration/03.observability-and-networking/index.md) to change the session expiration time to 10 hours:
+Add the following Micronaut setting to your [Observability and Networking configuration](../../configuration/03.observability-and-networking/index.md) to set the session expiration time to 10 hours:
```yaml
environment:
@@ -28,13 +26,13 @@ Yes, there is! Add the following Micronaut setting to your [Observability and Ne
cookie-max-age: 10h
```
-In Cloud, you might need to ask our support team to change this setting for you.
+On Kestra Cloud, contact support to change this setting.
## How do I configure Kestra with my license details?
-To use Kestra Enterprise Edition, you will need a valid license configured under the `kestra.ee.license` configuration. The license is unique to your organization. If you need a license, please reach out to our Sales team at [sales@kestra.io](mailto:sales@kestra.io).
+Kestra Enterprise Edition requires a valid license configured under `kestra.ee.license`. The license is unique to your organization. To get a license, contact the Sales team at [sales@kestra.io](mailto:sales@kestra.io).
-The license is set up using three configuration properties: `id`, `fingerprint`, and `key`.
+The license uses three configuration properties: `id`, `fingerprint`, and `key`.
- `kestra.ee.license.id`: license identifier.
- `kestra.ee.license.fingerprint`: license authentication.
@@ -50,7 +48,7 @@ kestra:
```
-When you launch Kestra Enterprise Edition, it will check the license and display the validation step in the log.
+Kestra validates the license on startup and logs the validation result.
## When should I use Secrets vs Credentials?
diff --git a/src/contents/docs/07.enterprise/07.cloud-getting-started/index.md b/src/contents/docs/07.enterprise/07.cloud-getting-started/index.md
new file mode 100644
index 00000000000..ba3cf7a55e9
--- /dev/null
+++ b/src/contents/docs/07.enterprise/07.cloud-getting-started/index.md
@@ -0,0 +1,125 @@
+---
+title: "Kestra Cloud: Getting Started"
+h1: Getting Started with Kestra Cloud
+description: Set up your Kestra Cloud instance, run your first flow, add secrets, invite your team, and connect to production.
+sidebarTitle: Cloud Getting Started
+icon: /src/contents/docs/icons/admin.svg
+editions: ["Cloud"]
+---
+
+Kestra Cloud is a fully managed Kestra instance run by the Kestra team. You build the pipelines; Kestra handles hosting, scaling, upgrades, SSO, and secrets.
+
+## Two places you'll work
+
+Kestra Cloud consists of two platforms:
+
+- **[Console](https://console.kestra.io/ui/login):** where you create, monitor, and manage your instances and billing.
+- **Your Kestra instance:** where you and your team build and run flows.
+
+| | Console | Your Kestra instance |
+|---|---|---|
+| For | Managing instances and billing | Building and running flows |
+| What | View instances, usage, and invoices; set up SSO; add Console users | Write flows, run executions, add secrets, invite teammates, configure IAM/RBAC, build Apps |
+
+:::alert{type="info"}
+Console users and instance users are separate. Adding someone in Console does not give them access to the instance, and vice versa. Invite people where they'll actually work (usually the instance). SSO is configured in Console per instance.
+:::
+
+## Step 1: Log in
+
+Go to your instance URL, or sign in to **[Console](https://console.kestra.io/ui/login)** to find it. Sign-in is passwordless: enter the login code emailed to you.
+
+If the code doesn't arrive, your email security filter is likely blocking it. Allowlist **noreply@kestra.io**, check spam, then request a new code.
+
+Whoever creates the instance becomes its **Instance Owner**: the role that manages instances, users, and IAM across the whole instance.
+
+:::alert{type="info"}
+Instance Owner does not grant access to namespaces or data by default. Working with flows also requires a tenant role (see [Step 5](#step-5-invite-your-team)).
+:::
+
+## Step 2: Run your first flow from a blueprint
+
+Three getting-started blueprints run successfully on the first execution with public data and no secrets required. Pick one based on your use case:
+
+| Blueprint | Use for | What it does |
+|---|---|---|
+| [ELT pipeline](https://kestra.io/blueprints/getting-started-elt-pipeline) | Data pipelines | Public CSV → aggregate in DuckDB → log top products |
+| [AI enrichment](https://kestra.io/blueprints/getting-started-ai-enrichment) | AI / LLM work | Fetch records → log a preview; add a key to summarize and classify |
+| [Automation](https://kestra.io/blueprints/getting-started-automation) | Alerts / ops | Call an API → log the response; add a Slack alert and a trigger |
+
+In your instance: go to **Flows → Create → Blueprints**, pick one, **Save**, then **Execute**. Open the run and review the **Logs**, the **Gantt** timeline, and each task's **Outputs**. Edit, execute, review the run: that's the core loop.
+
+Hundreds more are available in the [blueprints library](https://kestra.io/blueprints). The [AI Copilot](../../ai-tools/01.ai-copilot/index.md) can also write a flow from a plain-English description.
+
+Building with AI tools? The [Kestra MCP server](../../ai-tools/03.mcp-server/index.md) gives Claude, Cursor, and other tools live access to Kestra docs, plugins, and blueprints. Add [kestractl](https://github.com/kestra-io/kestractl) to generate and deploy flows from the CLI.
+
+Prefer your editor? The [VS Code extension](../../version-control-cicd/05.vscode/index.md) edits, validates, and runs flows locally with instance-aware autocomplete and a topology preview.
+
+## Step 3: Namespaces
+
+Every flow belongs to a [namespace](../../05.workflow-components/02.namespace/index.md) (for example, `mycompany`): a folder that holds flows, secrets, files, and variables. Three rules to follow:
+
+- **Create your own** namespace (for example, `dev`) and build there. Leave `system` alone; it holds Kestra's own maintenance flows.
+- **Nest with `.` to share config.** `marketing.dev` and `marketing.prod` are two environments under `marketing`. Both inherit the parent's secrets, variables, and files, so shared config lives in one place.
+- **Split `dev` from `prod`.** When flows matter, use separate namespaces and promote between them (see [Step 6](#step-6-go-to-production)) so only reviewed work reaches production.
+
+## Step 4: Add your first secret
+
+Store credentials as [secrets](../../06.concepts/04.secret/index.md) and reference them with `{{ secret('NAME') }}`. Never paste credentials directly into a flow.
+
+To add a Slack webhook (used by all three getting-started blueprints):
+
+1. In your instance: go to **Namespaces → your namespace → Secrets → Add**. Name the key `SLACK_WEBHOOK`.
+2. In the blueprint's `notify` task, remove `disabled: true` and run it. You'll receive a Slack message.
+
+Use the same pattern for a `DB_PASSWORD` to point the ELT blueprint at your own warehouse.
+
+Secrets are namespace-scoped but readable across namespaces in the same tenant, so shared tokens can live in one place:
+
+```yaml
+message: "{{ secret('SHARED_TOKEN', namespace='shared') }}"
+```
+
+Cross-namespace reads are allowed by default. Restrict them with `allowedNamespaces` on the owning namespace.
+
+[Policies](../02.governance/policies/index.md) set plugin values (host, credentials, region) once per namespace and apply them to every matching flow. Use them to avoid repeating auth config on every task.
+
+## Step 5: Invite your team
+
+Go to **Tenant → IAM → Invitations → Create**: enter their email and assign a role. For people building flows, **Developer** is the best fit (**Editor** and **Admin** also work). Avoid Instance Owner — that role manages the instance, not the data.
+
+Leave the namespace field empty to apply the role to all namespaces, or name specific namespaces to limit access.
+
+Related:
+- [Invite colleagues](../03.auth/invitations/index.md)
+- [Set up SSO](../03.auth/sso/index.md) (configured in [Console](https://console.kestra.io/ui/login))
+
+## Step 6: Go to production
+
+Once flows matter, stop editing production by hand. Build in `dev`, commit to Git, and let Git be the source of truth for `prod`. Common patterns — combinable:
+
+| Method | Best for |
+|---|---|
+| [Git Sync](../../version-control-cicd/04.git/index.md) | Flows and namespace files: Kestra pulls `main` on a schedule or push |
+| [Terraform](../../13.terraform/index.mdx) | Static config: secrets, variables, namespaces, roles |
+| GitHub Actions / CI/CD | Validate on PR, deploy on merge: [validate flows](https://github.com/kestra-io/validate-flows-action-v2), [deploy flows](https://github.com/kestra-io/deploy-flows-action-v2), [deploy namespace files](https://github.com/kestra-io/deploy-namespace-files-action) |
+
+No Git? [Promote](../02.governance/promote/index.md) copies a flow from one instance to another directly from the UI with a diff review before it lands. It moves the flow YAML only, so configure secrets and files per environment separately. See the [dev-to-prod guide](../../14.best-practices/1.from-dev-to-prod/index.md) for the full picture.
+
+## What's next
+
+In the order most active teams adopt them:
+
+1. **Put it on a [schedule or trigger](../../05.workflow-components/07.triggers/index.mdx)** so it runs unattended.
+2. **Wire a Slack alert** (Step 4) so you know when something breaks.
+3. **Connect a second system**: your warehouse, dbt, an API.
+4. **Sync from Git** (Step 6).
+5. **Create a Kestra App.** An [App](../04.scalability/apps/index.md) gives stakeholders a dedicated URL to submit or download data and approve runs without touching flows directly.
+
+Common first-week blockers:
+
+- Credentials not reaching a task: [Secrets](../../06.concepts/04.secret/index.md)
+- A worker that can't reach your network: [Worker Groups](../04.scalability/worker-group/index.md)
+- Cloud auth or OIDC trust issues: [Credentials](../03.auth/credentials/index.md)
+
+For help, reach the team at support@kestra.io, in the [Slack community](https://kestra.io/slack), or by replying to your onboarding email.
diff --git a/src/contents/docs/07.enterprise/index.mdx b/src/contents/docs/07.enterprise/index.mdx
index a7ad0b54694..96d8badc9c3 100644
--- a/src/contents/docs/07.enterprise/index.mdx
+++ b/src/contents/docs/07.enterprise/index.mdx
@@ -14,7 +14,7 @@ How to configure Kestra Enterprise Edition and Kestra Cloud.
[Enterprise Edition](/enterprise) is a self-hosted version of Kestra deployed to your private infrastructure. It offers security and governance features including Multi-tenancy, Authentication, SSO, RBAC, Namespace-level management, distributed Worker Groups, Worker isolation, Secrets Manager integrations, Audit Logs, and more.
-[Kestra Cloud](/cloud) is a fully managed version of Kestra Enterprise Edition, hosted and maintained by the Kestra team. It provides most of the features of the Enterprise Edition, plus the additional benefits of automatic updates, backups, and infrastructure monitoring.
+[Kestra Cloud](/cloud) is a fully managed version of Kestra Enterprise Edition, hosted and maintained by the Kestra team. It provides most of the features of the Enterprise Edition, plus the additional benefits of automatic updates, backups, and infrastructure monitoring. New to Kestra Cloud? See the [Cloud getting started guide](./07.cloud-getting-started/index.md).
## Key differences between Kestra Enterprise and Kestra Cloud
@@ -23,14 +23,14 @@ While Kestra Cloud is fully managed, it differs from Kestra Enterprise in severa
| Feature / Area | Kestra Cloud | Kestra Enterprise Edition |
| ------------------------------------- | -------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------- |
| **Infrastructure Control** | Fully managed by Kestra for simplicity and reliability | Full control and customization |
-| **Backend Technology** | PostgreSQL JDBC only | Customizable Kafka, PostgreSQL, MySQL, H2 (testing) |
+| **Backend Technology** | PostgreSQL JDBC only | Customizable: PostgreSQL, MySQL, Kafka, Redis, AMQP, GCP Pub/Sub, H2 (testing) |
| **Workers** | Managed worker pools sized for stability | Remote [Worker Groups](/docs/enterprise/scalability/worker-group), autoscaling |
| **Custom Internal Storage & Secrets** | No instance-level control, **tenant/namespace-level only** backends | Fully customizable backends |
| **Network Configuration** | Secure access over public Internet | Private networking (self-hosted, VPC peering, etc.) |
| **Backup Access** | Automatic backups handled by Kestra | Customer-controlled backups |
| **Plugins** | Curated plugin environment | Full plugin customization |
| **Identity Providers (IdP)** | Built-in Google, Microsoft, or Basic Authentication | Custom SSO/SCIM supported |
-| **Log Retention** | Automatic retention protocol managed by Kestra | Unlimited (based on customer setup) |
+| **Log Retention** | Automatic retention protocol managed by Kestra | Unlimited (based on customer setup); route logs to a separate JDBC or Elasticsearch store with the [External Log Data Store](../10.administrator-guide/log-data-store/index.md) |
| **Deployment Regions** | US & EU (Belgium) on GCP | Any cloud, any region |
| **Task Runners** | Compatible with most, [Process Task Runner](/docs/task-runners/types/process-task-runner) excluded | Compatible with all task runners |
diff --git a/src/contents/docs/08.architecture/01.main-components/index.md b/src/contents/docs/08.architecture/01.main-components/index.md
index ad160167473..9537ccb1abf 100644
--- a/src/contents/docs/08.architecture/01.main-components/index.md
+++ b/src/contents/docs/08.architecture/01.main-components/index.md
@@ -6,13 +6,11 @@ sidebarTitle: Main components
icon: /src/contents/docs/icons/architecture.svg
---
-Technical overview of Kestra’s main components: internal storage, queue, repository, and plugins.
-
Kestra relies on the following internal components:
- **Internal storage**: stores flow data such as task outputs and flow inputs.
- **Queue**: enables internal communication between Kestra server components.
-- **Repository**: persists flows, templates, executions, logs, and all other internal objects.
+- **Repository**: persists flows, executions, logs, and all other internal objects.
- **Plugins**: extend Kestra’s core with additional task and trigger types, storage implementations, and data transformations.
Each component has multiple implementations depending on deployment architecture. Some require additional plugins.
@@ -28,22 +26,22 @@ Internal storage is used to:
- Automatically persist [flow inputs](../../05.workflow-components/05.inputs/index.md) of type `FILE`.
- Provide download links for stored files in the **Outputs** tab of an execution.
-Files can be retrieved in the execution context using `{{ outputs.task_id.output_attribute }}` (often the `uri` property). Kestra fetches the file automatically when referenced.
+Files are addressed by stable `kestra://` URIs that the engine resolves on demand. A worker on one host and the Webserver on another can both reach the same artifact through the same URI. Files can be retrieved in the execution context using `{{ outputs.task_id.output_attribute }}` (often the `uri` property). Kestra fetches the file automatically when referenced.
Execution metadata — including storage file paths — is recorded in the **repository**.
### Storage types
-By default, Kestra uses **local storage**, which stores files on the host filesystem. This option is simple but not scalable and is usually not recommended for production (unless for standalone deployments).
+By default, Kestra uses **local storage**, which stores files on the host filesystem. Local storage is not recommended for production distributed deployments — use cloud object storage or a self-hosted alternative instead.
:::alert{type="warning"}
Local storage behavior differs between standalone and distributed deployments:
- ✅ **Standalone**: Local storage with persistent volumes is OK
-- ❌ **Distributed with ReadWriteOnce**: NOT recommended for distributed services
-- ✅ **Distributed with ReadWriteMany**: OK for distributed services (rarely available)
+- ❌ **Distributed on single-writer storage**: NOT recommended — storage that only one instance can mount read-write at a time (in Kubernetes, `ReadWriteOnce`) cannot back a distributed service
+- ✅ **Distributed on shared storage**: OK when all instances can mount it read-write at once (in Kubernetes, `ReadWriteMany`), though this is rarely available
- ❌ **Host storage sharing**: NOT recommended — difficult to achieve reliably
-When `ReadWriteMany` is unavailable, use cloud storage (S3, GCS, Azure) or self-hosted S3-compatible object storage (Ceph, SeaweedFS, Garage, MinIO).
+When shared read-write storage (`ReadWriteMany`) is unavailable, use cloud storage (S3, GCS, Azure) or self-hosted S3-compatible object storage (Ceph, SeaweedFS, Garage, MinIO).
:::
Scalable alternatives are available as plugins:
@@ -58,21 +56,31 @@ For details, see [Runtime and Storage](../../configuration/02.runtime-and-storag
## Queue
-The **queue** is used internally for communication between Kestra’s server components. Each repository type has a matching queue implementation:
+The **queue** is the internal communication channel between Kestra’s server components. Server roles emit typed messages onto named queues and subscribe to the queues they consume — no role calls another directly. The full queue surface is defined once as an abstract contract, satisfied by one chosen backend:
+
+- **Database queue** (default) — backed by PostgreSQL or MySQL. Available in all editions.
+- **In-memory queue** — for testing and ephemeral use only.
+- **Kafka queue** — Enterprise Edition. Higher throughput; pairs with the Elasticsearch repository.
+- **Redis queue** — Enterprise Edition.
+- **AMQP queue** — Enterprise Edition.
+- **GCP Pub/Sub queue** — Enterprise Edition.
+
+The queue surface covers four delivery families:
+
+- **Dispatch** — point-to-point; exactly one subscriber processes each message. Used for executions, execution events and commands, worker task results, logs, and metrics.
+- **Keyed dispatch** — point-to-point partitioned by a routing key so a subscriber receives only messages for its key. Used for worker job routing: each Worker Queue is a key, and a worker subscribes only to the queues its group covers.
+- **VNode dispatch** — sharded across a fixed set of virtual nodes so a scaled component can divide a single logical stream deterministically. Used for trigger evaluation across a Scheduler fleet.
+- **Broadcast** — fan-out; every active subscriber receives every message. Used for kill signals, flow and metadata change notifications, follow-execution streams, and cluster-wide events.
-- **In-memory queue** — must be used with the in-memory repository.
-- **Database queue** — must be used with the database repository.
-- **Kafka queue** — must be used with the Elasticsearch repository.
- **Only available in the [Enterprise Edition](../../07.enterprise/01.overview/01.enterprise-edition/index.md).**
+Messages above a configurable size limit are rejected before reaching the backend, protecting it from oversized payloads. Terminal execution states are always allowed through regardless of size. Message protection is enabled by default with a 1 MB limit and can be adjusted under `kestra.queue.message-protection`.
## Repository
-The **repository** persists all internal objects, including flows, executions, logs, and templates. Each repository type must be paired with its corresponding queue:
+The **repository** persists all domain entities, including flows, executions, logs, and triggers. The backend is chosen alongside the queue:
-- **In-memory repository** — must be used with the in-memory queue.
-- **Database repository** — must be used with the database queue.
-- **Elasticsearch repository** — must be used with the Kafka queue.
- **Only available in the [Enterprise Edition](../../07.enterprise/01.overview/01.enterprise-edition/index.md).**
+- **Database repository** (default) — backed by PostgreSQL, MySQL, or H2. Available in all editions.
+- **In-memory repository** — for testing only.
+- **Elasticsearch repository** — Enterprise Edition. Backs the high-volume search and read model; requires the Kafka queue and the Indexer server role to keep it in sync.
## Plugins
diff --git a/src/contents/docs/08.architecture/02.server-components/index.md b/src/contents/docs/08.architecture/02.server-components/index.md
index 1d383d9adc6..2a859cc7f14 100644
--- a/src/contents/docs/08.architecture/02.server-components/index.md
+++ b/src/contents/docs/08.architecture/02.server-components/index.md
@@ -6,36 +6,65 @@ sidebarTitle: Server components
icon: /src/contents/docs/icons/architecture.svg
---
-Detailed breakdown of the server components behind Kestra.
+Kestra runs six server roles that can be deployed as a single process or as independent, separately scaled services. Every role communicates exclusively through the [Queue](../01.main-components/index.md#queue) and reads shared state from the [Repository](../01.main-components/index.md#repository). Workers are the only role that accesses [Internal Storage](../data-components/index.md#internal-storage) and user infrastructure directly.
-Kestra consists of multiple server components that can be scaled independently.
-
-Each server component interacts with internal components ([Internal Storage](../data-components/index.md#internal-storage), [Queue](../01.main-components/index.md#queue), and [Repository](../01.main-components/index.md#repository)).
+The Webserver is the only role reachable from outside the cluster. The orchestration roles — Executor, Worker Controller, Scheduler, Indexer, and Workers — cannot be reached from the network; they only produce and consume queue messages. All external actors reach the platform through the Webserver's authenticated API.
## Executor
-The **Executor** is a lightweight server component responsible for processing all executions and orchestrating the next tasks to run. It does not perform heavy computations itself, instead deferring actual task execution to [Workers](#worker).
+The **Executor** is a lightweight server component responsible for driving the execution state machine. Given a flow and an execution, it decides which task runs next, what state the execution is in, what to dispatch, what to retry, and when an execution terminates. It does not perform heavy computation itself — runnable tasks are dispatched to [Workers](#worker) via the [Worker Controller](#worker-controller).
-The Executor plays a central role in coordinating workflows based on the information it receives from the [Scheduler](#scheduler) and the [Queue](../01.main-components/index.md#queue). It handles specific types of tasks, such as:
+The Executor subscribes to the queue and handles:
- [Flowable Tasks](../../05.workflow-components/01.tasks/00.flowable-tasks/index.md)
- [Flow Triggers](../../05.workflow-components/07.triggers/02.flow-trigger/index.md)
-- Templates *(deprecated)*
-- Listeners *(deprecated)*
+- Subflow and loop coordination
+- Concurrency limits, retries, SLA monitoring, and kill signals
+
+Before dispatching a task, the Executor consults a worker-queue resolver that returns one of four decisions: `DISPATCH` (send immediately), `WAIT_AND_DISPATCH` (enqueue for a worker not yet connected), `FAIL` (fail the task run), or `CANCEL` (cancel it). The resolver — not the Executor — performs the worker availability check.
+
+A one-second delay loop re-injects executions when their scheduled wakeup arrives, driving task retries, paused-flow resumption, and `LoopUntil` iterations.
+
+The Executor also runs the cluster-wide **service liveness coordinator**: it periodically reviews every registered service instance — workers, schedulers, and peer executors — drives state transitions on missed heartbeats, and releases orphaned worker jobs back to the queue for reprocessing. See [Cluster liveness model](#cluster-liveness-model).
+
+The Executor never interacts directly with user data or infrastructure. Because of its low resource usage, it rarely needs to be scaled. In deployments with very high execution volume, Executors can scale horizontally.
+
+## Worker Controller
+
+The **Worker Controller** is the sole communication point between the cluster and its workers. Workers never subscribe to the job queue or access the database directly — all job dispatch and result intake are funnelled through the Worker Controller.
+
+Each worker opens a persistent bidirectional gRPC stream to a Worker Controller and uses that stream for its lifetime:
+
+- The Worker Controller dispatches jobs from the queue onto the stream.
+- Workers return results, logs, and metrics over the same stream.
+- Kill signals and metadata changes are broadcast to all connected workers.
+
+Dispatch is partitioned by **Worker Queue** — a stable identifier derived from the tag set a task or trigger declares through its worker selector. Two queues are always reserved: the **default queue** carries untagged work; the **system queue** carries platform-internal tasks and is served exclusively by the [system worker](#worker). A worker connects under a **worker group** that maps it to one or more Worker Queue subscriptions. Each subscription carries a reserved-capacity percentage so a busy queue cannot starve one to which the group has committed capacity. In the open-source build, there is a single implicit default group subscribed only to the default queue; in Enterprise Edition, groups are persisted entities with their own authorization tokens.
-Although the Executor oversees all executions, it never interacts directly with your data.
+Before dispatching a job, the Worker Controller writes it to a durable **running state store**. If the controller crashes after persisting but before the worker receives the job, the Executor recovers and re-dispatches from that store. The capacity slot reserved for a dispatched job is held for its entire lifetime — not just until delivery — so the permit count accurately reflects in-flight load.
-Because of its low resource usage, the Executor rarely needs to be scaled. However, in deployments with a very large number of executions, you can scale Executors horizontally to meet demand.
+Multiple Worker Controller instances can run in parallel. Workers discover available controllers through static endpoint lists, DNS, or self-registration in internal storage. The controller periodically recycles long-lived streams so that newly deployed controller instances pick up traffic without requiring worker restarts.
+
+gRPC transport is available in all editions. TLS and mTLS secure the connection in all editions; JWT-based worker authentication is an Enterprise Edition feature.
## Worker
-The **Worker** is a server component responsible for executing all [runnable tasks](../../05.workflow-components/01.tasks/01.runnable-tasks/index.md) and [Polling Triggers](../../05.workflow-components/07.triggers/04.polling-trigger/index.md). These are received from the [Executor](#executor) and the [Scheduler](#scheduler), respectively.
+The **Worker** is the server component responsible for executing all [runnable tasks](../../05.workflow-components/01.tasks/01.runnable-tasks/index.md) and [Polling Triggers](../../05.workflow-components/07.triggers/04.polling-trigger/index.md). Workers are the only roles that load user plugins, access user infrastructure, and consume CPU on user code.
+
+Workers come in two shapes:
+
+- **Worker agent** — a dedicated process that connects to a Worker Controller over gRPC. This is the standard deployment unit.
+- **System worker** — an in-process variant embedded in the Executor (or the standalone server). It serves the reserved system queue for platform-internal tasks and starts automatically; it does not require separate deployment.
+
+Internally, each worker agent runs as a configurable thread pool. Between the gRPC stream fetcher and the thread pool sits a bounded in-memory **buffer queue**. When the buffer fills, the worker stops pulling new jobs from the stream, letting the distributed queue's lag metric reflect a saturated worker rather than silently overloading it. Set the thread count based on your workload — more threads for I/O-bound tasks, fewer for memory-intensive ones.
+
+A task can declare a **worker selector**: a set of tags, a match strategy (`all` tags must match, or `any`), and a fallback policy for when no matching worker is available (`fail`, `wait`, `cancel`, or fall back to the default queue). The Worker Controller routes the task to the Worker Queue whose tags match the selector.
-Workers are highly configurable and designed to handle a wide range of workloads — from simple API calls to heavy computational tasks. Internally, each Worker functions as a configurable thread pool, allowing you to define the number of threads per instance based on your workload requirements.
+Workers optionally support a **task output cache**: task outputs are stored in internal storage keyed by a hash of the task definition and its inputs. On a cache hit, the worker emits the cached outputs without running the task.
-You can deploy multiple Worker instances across different servers to scale horizontally. This flexibility enables efficient handling of parallel executions, especially in high-throughput environments.
+Worker shutdown is two-phase: the worker stops fetching new jobs and waits up to the configured grace period for in-flight jobs to finish, then drains the outbound result senders before exiting.
-Because Workers directly execute tasks and triggers, they are the **only** server components that require access to external systems — such as databases, REST APIs, message brokers, and any other services your flows interact with.
+Deploy multiple worker agent instances to scale horizontally. Each handles its assigned tasks independently, so adding workers increases throughput without coordination overhead.
:::alert{type="info"}
Looking for runtime status? The **Instance – Services** view shows live health for each component. See [Instance – services](../../07.enterprise/05.instance/index.mdx#services).
@@ -43,9 +72,11 @@ Looking for runtime status? The **Instance – Services** view shows live health
## Worker Group (EE)
-In the [Enterprise Edition](../../07.enterprise/01.overview/01.enterprise-edition/index.md), [Worker Groups](../../07.enterprise/04.scalability/worker-group/index.md) allow tasks and [Polling Triggers](../../05.workflow-components/07.triggers/04.polling-trigger/index.md) to be executed on specific worker sets. They can be beneficial in various scenarios, such as using compute instances with GPUs, executing tasks on a specific OS, restricting backend access, and region-specific execution. A default worker group is recommended per [tenant](../10.multi-tenancy/index.md) or namespace.
+In the [Enterprise Edition](../../07.enterprise/01.overview/01.enterprise-edition/index.md), [Worker Groups](../../07.enterprise/04.scalability/worker-group/index.md) are persisted entities that define which Worker Queues a connected worker serves and how much of its capacity each queue is guaranteed. Each subscription in a group pairs a Worker Queue with a reserved-capacity percentage and a reservation mode — **strict** (reserved slots are exclusive to that queue) or **elastic** (reserved slots can be borrowed by other subscriptions when idle).
-To specify a worker group for a task, use the `workerGroup.key` property in the task definition to point the task to a specific worker group key. If no worker group is specified, the task will be executed on the default worker group.
+Worker Groups enable scenarios such as GPU workloads, OS-specific execution, infrastructure access restrictions, and region-based routing. Every worker agent connects under a group; the group determines its Worker Queue subscriptions and carries its own authorization tokens.
+
+To route a task to a specific queue, set `workerSelector.tags` on the task definition with the tags matching the target Worker Queue. Tasks without a `workerSelector` run on the default queue.
:::alert{type="info"}
Worker Groups are available in Kestra Enterprise Edition only, not in Kestra Cloud.
@@ -71,11 +102,9 @@ By default, Kestra handles all date and time values using your system's timezone
## Indexer
-The **Indexer** is responsible for reading content from Kafka topics — such as flows and executions — and indexing it into Elasticsearch. This component enables [low-latency querying](../../11.migration-guide/v0.20.0/elasticsearch-indexer/index.md) when using Kafka and Elasticsearch together.
-
-By default, the Indexer runs as part of the [Web Server](#webserver). However, you can choose to run the Web Server independently without the Indexer by using the `server webserver --no-indexer` CLI option.
+The **Indexer** reads content from the queue — flows, executions, logs, metrics, and audit logs — and writes it to the repository backend. It is required in all deployments.
-The Indexer is required for deployments that rely on Kafka and Elasticsearch, particularly in **Kestra Enterprise Edition** and **Kestra Cloud**.
+By default, the Indexer runs embedded in the [Webserver](#webserver). You can disable it there with the `server webserver --no-indexer` CLI option and run it as a standalone process — useful when the Webserver and Indexer need access to different infrastructure, or when you want to isolate the Webserver behind a bastion host.
## Webserver
@@ -91,3 +120,9 @@ The Webserver primarily interacts with the [Repository](../01.main-components/in
:::alert{type="info"}
As long as the [Queue](../01.main-components/index.md#queue) is operational, most server components — including the Webserver — will continue to function. While the Repository is essential for rendering the UI, workloads can still be processed even if the Repository is temporarily unavailable.
:::
+
+## Cluster liveness model
+
+Every running server registers itself as a service and sends heartbeats at a fixed interval. The Executor runs the cluster-wide liveness coordinator: on a scheduled tick it reviews every registered service instance, drives state transitions when heartbeats are missed (`running → disconnected → not-running`), and releases any work orphaned by a vanished server — such as a worker's in-flight jobs — back onto the queue so a healthy server picks it up.
+
+The same model coordinates **maintenance mode**: on maintenance entry, every server pauses its queue subscribers while in-flight work drains, then resumes when maintenance exits. Old service rows are purged on a schedule to keep the registry bounded.
diff --git a/src/contents/docs/08.architecture/03.deployment-architecture/index.md b/src/contents/docs/08.architecture/03.deployment-architecture/index.md
index 03438434f3c..92ee1e273de 100644
--- a/src/contents/docs/08.architecture/03.deployment-architecture/index.md
+++ b/src/contents/docs/08.architecture/03.deployment-architecture/index.md
@@ -6,24 +6,23 @@ sidebarTitle: Deployment architecture
icon: /src/contents/docs/icons/architecture.svg
---
-Examples of deployment architectures, depending on your needs.
-
Kestra is a Java application distributed as an executable. It supports multiple deployment options:
- [Docker](../../02.installation/02.docker/index.md)
- [Kubernetes](../../02.installation/03.kubernetes/index.md)
- Manual deployment
-Kestra’s plugin system allows you to choose the dependency types that best match your requirements.
-Below are three common deployment architectures.
+Kestra's plugin system allows you to choose the dependency types that best match your requirements.
## Small-sized deployment
-For small-scale deployments, you can use the Kestra **standalone server**, which runs all server components in a single process. This architecture has no scaling capability.
+For small-scale deployments, you can use the Kestra **standalone server**, which runs all server components as threads inside a single process. This architecture has no scaling capability but behaves identically to a distributed cluster — the same components run, collocated in one JVM.
+
+For quick local experimentation, Kestra also offers a **local mode** (`server local`) that reduces the footprint further: it uses an embedded H2 database with no external dependencies, requiring no infrastructure setup.
-In this setup, a database is the only dependency, minimizing the stack to maintain. Supported databases include:
+In standalone mode, a database is the only dependency. Supported databases include:
- PostgreSQL
- MySQL
@@ -33,18 +32,20 @@ In this setup, a database is the only dependency, minimizing the stack to mainta
-For medium-scale deployments, where high availability is not required, Kestra can be run with a relational database (Postgres or MySQL) as the only dependency. H2 is not recommended in distributed setups.
+For medium-scale deployments where high availability is not required, Kestra can be run with a relational database (PostgreSQL or MySQL) as the only dependency. H2 is not recommended in distributed setups.
- Supported databases: PostgreSQL and MySQL
-- All server components communicate through the database
+- All server components communicate through the database queue
+- Each server role runs as its own process and can be scaled independently
+- Workers communicate with the Worker Controller via gRPC; they never access the queue or database directly
-In this mode, if components are distributed across multiple hosts, you must use a shared [internal storage](../data-components/index.md#internal-storage) implementation such as [Google Cloud Storage](../../02.installation/09.gcp-vm/index.md), [AWS S3](../../02.installation/08.aws-ec2/index.md), or [Azure Blob Storage](../../02.installation/10.azure-vm/index.md).
+If components are distributed across multiple hosts, use a shared [internal storage](../data-components/index.md#internal-storage) implementation such as [Google Cloud Storage](../../02.installation/09.gcp-vm/index.md), [AWS S3](../../02.installation/08.aws-ec2/index.md), or [Azure Blob Storage](../../02.installation/10.azure-vm/index.md).
## High-availability deployment
-For high throughput and full horizontal and vertical scaling, the database is replaced with Kafka and Elasticsearch. This architecture removes single points of failure and enables scaling of all server components.
+For high throughput and full horizontal and vertical scaling, replace the database queue with Kafka and Elasticsearch. This architecture removes single points of failure and enables scaling of all server components.
- Dependencies: Kafka and Elasticsearch
- Available only in the [Enterprise Edition](../../07.enterprise/01.overview/01.enterprise-edition/index.md)
@@ -53,30 +54,14 @@ As with medium deployments, a distributed [internal storage](../data-components/
### Kafka
-[Kafka](https://kafka.apache.org/) is the backbone of high availability mode, powering communication and scalability.
-
-#### Kafka executor
-
-The [executor](../02.server-components/index.md#executor) runs as a [Kafka Streams](https://kafka.apache.org/documentation/streams/) application. It:
-
-- Processes all events from Kafka in order
-- Maintains the internal state of executions
-- Merges task run results from [workers](../02.server-components/index.md#worker)
-- Detects failed workers and resubmits their tasks
+[Kafka](https://kafka.apache.org/) is the queue backbone of the high-availability deployment. The Executor, Scheduler, Worker Controller, Webserver, and Indexer emit to and subscribe from named Kafka topics — no two roles call each other directly.
-Executors scale horizontally within the limits of Kafka partitions. Since executors perform lightweight operations, they typically require minimal resources unless handling very high execution volumes.
+Workers do not subscribe to Kafka topics. They connect to the Worker Controller via gRPC, and all job dispatch, result intake, and broadcast events travel over that stream.
-#### Kafka worker
-
-The [worker](../02.server-components/index.md#worker) runs as a [Kafka consumer](https://kafka.apache.org/documentation/#consumerapi). It:
-
-- Processes tasks assigned by executors
-- Runs tasks in an internal thread pool
-- Scales horizontally, with multiple instances across servers
-
-If a worker fails, the executor detects it and resubmits the tasks to another available worker.
+Executors scale horizontally — each instance subscribes to the queue and processes the executions assigned to it. Because the executor performs lightweight orchestration work (state transitions, dispatch decisions), it typically requires minimal resources.
### Elasticsearch
-[Elasticsearch](https://www.elastic.co/elasticsearch) acts as the database for Kestra’s [webserver](../02.server-components/index.md#webserver), providing fast search, aggregation, and retrieval of flows, executions, and logs.
-It is only required in high availability mode and is used exclusively by the [API and UI](../../09.ui/index.mdx).
+[Elasticsearch](https://www.elastic.co/elasticsearch) acts as the search and read backend for Kestra's webserver, providing fast retrieval and aggregation of flows, executions, and logs. It is used exclusively by the API and UI.
+
+The Indexer subscribes to Kafka topics and writes to Elasticsearch, keeping the search index in sync. Because the queue and search index are separate, executions continue processing even if Elasticsearch is temporarily unavailable.
diff --git a/src/contents/docs/08.architecture/10.multi-tenancy/index.md b/src/contents/docs/08.architecture/10.multi-tenancy/index.md
index 9cbbb1f1f2d..5c876ca345e 100644
--- a/src/contents/docs/08.architecture/10.multi-tenancy/index.md
+++ b/src/contents/docs/08.architecture/10.multi-tenancy/index.md
@@ -16,18 +16,25 @@ This feature requires the [Enterprise Edition](../../07.enterprise/index.mdx).
## How does multi-tenancy work in Kestra
-Multi-tenancy is enabled by default and required. All resources (such as flows, triggers, executions, RBAC, and more) are isolated by the tenant. This means that you can have a flow with the same identifier and the same namespace in multiple tenants at the same time.
+Every resource in Kestra belongs to exactly one tenant. The following are fully isolated per tenant:
-Data stored inside the [Internal Storage](../data-components/index.md#internal-storage) is also isolated by tenants.
+| Resource | Description |
+|---|---|
+| [Flows](../../05.workflow-components/01.flow/index.md), [triggers](../../05.workflow-components/07.triggers/index.mdx), [executions](../../05.workflow-components/03.execution/index.md) | Core workflow resources — the same flow ID and namespace can exist independently in multiple tenants |
+| [Namespaces](../../07.enterprise/02.governance/07.namespace-management/index.md) | Namespace hierarchy, variables, KV store, namespace files, and task defaults |
+| [RBAC](../../07.enterprise/03.auth/rbac/index.md) — roles, users, groups, service accounts | Access control is fully scoped to the tenant |
+| [Secrets](../../07.enterprise/02.governance/secrets-manager/index.md) | Secret keys and values are never shared across tenants |
+| [Policies](../../07.enterprise/02.governance/policies/index.md) | Governance rules (injection, validation, enforcement) are scoped to tenant and namespace |
+| [Worker Queues](../../07.enterprise/04.scalability/worker-group/index.md) | Task routing rules are tenant-scoped |
+| [Audit logs](../../07.enterprise/02.governance/06.audit-logs/index.md) | Activity logs are isolated and queryable per tenant |
+| [Internal storage](../data-components/index.md#internal-storage) | Execution outputs and task data are stored in tenant-specific paths |
-End-users can use the tenant selection dropdown menu from the [UI](../../09.ui/index.mdx) to see tenants they have access to. Users can switch between tenants from this dropdown. Each UI page also includes the tenant ID in the URL (e.g., `https://demo.kestra.io/ui/yourTenantId/executions/namespace/flow/executionId`.)
+Instance-level resources — configuration, license, static policies, and instance owner banners — sit above the tenant layer and require Instance Owner access.
-
+Users switch between tenants using the tenant dropdown in the bottom-left corner of the UI. The dropdown lists every tenant the user has access to; the active tenant is indicated with a checkmark. Each UI page also includes the tenant ID in the URL (e.g., `https://demo.kestra.io/ui/yourTenantId/executions/namespace/flow/executionId`).
-Most [API](../../api-reference/index.mdx) endpoints also include the tenant identifier. The exception is instance-level endpoints such as `/configs`, `/license-info`, or `/banners`, which require Superadmin access.
+
-For example, the URL of the API operation to list flows of the `products` namespace is `/api/v1/{your_tenant_id}/flows/products`. See the [Enterprise Edition API Guide](../../api-reference/01.enterprise/index.mdx) for details.
+Tenants are created and managed through the **Instance Owner console** (**Instance Owner → Tenants**) — only users with the Instance Owner privilege can create, edit, or delete tenants. Users must be granted access to a tenant before they can switch to it. See [Tenants](../../07.enterprise/02.governance/tenants/index.md) for configuration details.
-:::alert{type="warning"}
-Tenants must be created upfront, and a user needs to be granted access to use a specific tenant.
-:::
+Most [API](../../api-reference/index.mdx) endpoints are scoped to a tenant and include the tenant identifier in the path — for example, `/api/v1/{tenant_id}/flows/products` to list flows in the `products` namespace. Instance-level endpoints such as `/api/v1/configs` or `/api/v1/license-info` have no tenant segment. See the [Enterprise Edition API Guide](../../api-reference/01.enterprise/index.mdx) for the full reference.
diff --git a/src/contents/docs/08.architecture/10.multi-tenancy/tenants-select.png b/src/contents/docs/08.architecture/10.multi-tenancy/tenants-select.png
index bbe03d3eaa0..1fab9f9f8f3 100644
Binary files a/src/contents/docs/08.architecture/10.multi-tenancy/tenants-select.png and b/src/contents/docs/08.architecture/10.multi-tenancy/tenants-select.png differ
diff --git a/src/contents/docs/08.architecture/architecture-data-planes.png b/src/contents/docs/08.architecture/architecture-data-planes.png
new file mode 100644
index 00000000000..24a555188ab
Binary files /dev/null and b/src/contents/docs/08.architecture/architecture-data-planes.png differ
diff --git a/src/contents/docs/08.architecture/architecture-overview.png b/src/contents/docs/08.architecture/architecture-overview.png
new file mode 100644
index 00000000000..b7616f55b36
Binary files /dev/null and b/src/contents/docs/08.architecture/architecture-overview.png differ
diff --git a/src/contents/docs/08.architecture/data-components/index.md b/src/contents/docs/08.architecture/data-components/index.md
index 6c302044240..e5f31b30da6 100644
--- a/src/contents/docs/08.architecture/data-components/index.md
+++ b/src/contents/docs/08.architecture/data-components/index.md
@@ -23,7 +23,7 @@ The table below outlines key data components, where they are stored, and their p
| Data component | Storage location | Description |
|--------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------| --- |
| **Flows & definitions** | Repository | Stores flows, tasks, and their configurations. |
-| **Namespaces** | Repository | Organizes workflows and manages secrets, plugin defaults, and variables. |
+| **Namespaces** | Repository | Organizes workflows and manages secrets, variables, and KV store entries. |
| **Namespace files** | Internal storage | Stores code and configuration files in Kestra’s storage backend. |
| **Executions & metadata** | Repository | Stores execution details including status, timestamps, and metadata. |
| **Input values** (non-FILE types) | Repository (executions table) | Scalar input values stored in the executions table. Non-sensitive types (STRING, INTEGER, etc.) are stored as plaintext; `SECRET` type inputs are stored encrypted. |
@@ -31,10 +31,10 @@ The table below outlines key data components, where they are stored, and their p
| **Output values** | Repository (task_outputs table) | Scalar task outputs stored in a dedicated task_outputs table. In Enterprise Edition, values emitted via `encryptedOutputs` are stored as encrypted strings rather than plaintext. |
| **Output files** | Internal storage | Generated files available for download and reuse in downstream tasks. |
| **Key-value pairs** | Internal storage & repository (metadata only) | KV store holds data in key-value format. Metadata is recorded in the repository. |
-| **Logs & [audit logs](../../07.enterprise/02.governance/06.audit-logs/index.md) (Enterprise Edition)** | Repository | Stores logs generated by tasks. |
+| **Logs & [audit logs](../../07.enterprise/02.governance/06.audit-logs/index.md) (Enterprise Edition)** | Repository or [external log data store](../../10.administrator-guide/log-data-store/index.md) | Stores logs generated by tasks. Audit logs always use the repository. |
| **Task state & variables** | Repository | Stores dynamic variables and task states during executions. |
| **Secrets** | Repository or external [secret manager](../../07.enterprise/02.governance/secrets-manager/index.md) | Stores secrets internally or integrates with services like AWS Secrets Manager, Vault, or Google Secret Manager. |
-| **Queues** | Repository or Kafka | Handles internal communication between Kestra components. |
+| **Queues** | Database (default), or Kafka / Redis / AMQP / GCP Pub/Sub (Enterprise Edition) | Handles internal communication between Kestra components. |
| **Triggers** | Repository | Stores definitions of event-based triggers. |
| **User administration** | Repository | Stores RBAC, user management, and related metadata. |
@@ -81,15 +81,14 @@ For full details, see [internal storage configuration](../../configuration/02.ru
### Logs
-- **Open source**: Logs are stored in the database.
-- **Enterprise Edition**: Supports Elasticsearch as a log backend, in addition to the database.
- - Audit logs are stored in the repository.
-- Logs can be accessed through the API, UI, or external logging integrations such as the [log shipper](../../07.enterprise/02.governance/logshipper/index.md).
+- By default, logs are stored in the repository. In Kestra 2.0+ (EE), you can route them to a separate JDBC database or Elasticsearch using the [external log data store](../../10.administrator-guide/log-data-store/index.md).
+ - Audit logs are always stored in the repository.
+- Logs can be accessed through the API, UI, or external logging integrations such as the [Log Shipper](../../07.enterprise/02.governance/logshipper/index.md).
### Queues
-- **Open source**: Stored in the database.
-- **Enterprise Edition**: Can use Kafka for inter-component messaging.
+- **Open source**: Backed by the relational database (PostgreSQL or MySQL).
+- **Enterprise Edition**: Can use Kafka, Redis, AMQP, or GCP Pub/Sub for higher-throughput inter-component messaging.
### Secrets management
@@ -129,10 +128,10 @@ Understanding where data is persisted is critical when flows process personally
#### Encrypting sensitive task outputs
:::alert{type="info"}
-`encryptedOutputs` is an Enterprise Edition feature.
+`encryptedOutputs` is an Enterprise Edition and Cloud feature.
:::
-In Enterprise Edition, script tasks support an `encryptedOutputs` key in the `::{}::` output protocol. Values written this way are wrapped in an `EncryptedString` and stored encrypted in the task_outputs table rather than as plaintext. They are merged into the same outputs map as regular outputs and are decrypted by Kestra at evaluation time.
+In Enterprise Edition and Cloud, script tasks support an `encryptedOutputs` key in the `::{}::` output protocol. Values written this way are wrapped in an `EncryptedString` and stored encrypted in the task_outputs table rather than as plaintext. They are merged into the same outputs map as regular outputs and are decrypted by Kestra at evaluation time.
```yaml
id: sensitive_data_flow
diff --git a/src/contents/docs/08.architecture/index.mdx b/src/contents/docs/08.architecture/index.mdx
index af112698cb9..5e74f21cc4e 100644
--- a/src/contents/docs/08.architecture/index.mdx
+++ b/src/contents/docs/08.architecture/index.mdx
@@ -6,103 +6,86 @@ sidebarTitle: Architecture
icon: /src/contents/docs/icons/architecture.svg
---
-Kestra's architecture is designed to be scalable and fault-tolerant. Depending on your needs, you can choose between two different architectures: **JDBC** and **Kafka**.
+Kestra is built on a single unified, pluggable backend. One set of queue and persistence contracts is satisfied by your chosen backend — a relational database by default, or a broker-backed alternative for higher throughput. All server roles are stateless and communicate only through the queue layer; no role calls another directly.
-## Choose the right Kestra architecture
+
-The following diagram shows the main components of Kestra using the JDBC backend.
+## Control plane and data plane
-
+Kestra's server roles divide into two planes:
-Here are the components and their interactions:
+- **Control plane** — the stateless coordination roles: Executor, Worker Controller, Scheduler, Webserver, and Indexer. These roles never execute user code and have no access to user infrastructure. They communicate exclusively through the queue.
+- **Data plane** — the Worker. Workers execute runnable tasks and polling triggers, and are the only roles that access user infrastructure and internal storage. A Worker connects to the control plane's Worker Controller over a bidirectional gRPC stream and can run in cloud, on-premises, or hybrid environments independently of the rest of the deployment.
-1. **JDBC Backend**: the data storage layer used for orchestration metadata
+This separation enables hybrid deployments where the control plane is managed by Kestra while Workers run inside your own infrastructure, or both are self-hosted.
-2. **Server**: the central part of the system, composed of:
- - [**Webserver**](./02.server-components/index.md#webserver): serves both the [API](../api-reference/index.mdx) and the [User Interface](../09.ui/index.mdx)
- - [**Scheduler**](./02.server-components/index.md#scheduler): schedules [workflows](../05.workflow-components/01.flow/index.md) and handles all [triggers](../05.workflow-components/07.triggers/index.mdx) except for the flow triggers (see below)
- - [**Executor**](./02.server-components/index.md#executor): responsible for the orchestration logic including [flow triggers](../05.workflow-components/07.triggers/02.flow-trigger/index.md)
- - [**Worker**](./02.server-components/index.md#worker): one or multiple processes that carry out the heavy computation of [runnable tasks](../05.workflow-components/01.tasks/01.runnable-tasks/index.md) and [Polling Triggers](../05.workflow-components/07.triggers/04.polling-trigger/index.md). For privacy reasons, workers are the only components that interact with the user's infrastructure, including the [Internal Storage](./data-components/index.md#internal-storage) and external services.
+## Server roles
-3. **User**: interacts with the system via [UI](../09.ui/index.mdx) and [API](../api-reference/index.mdx)
+Kestra has six server roles. In a standalone deployment, all six run as threads inside a single process. In a distributed deployment, each role runs as its own independently scaled process.
-4. **User's Infrastructure**: private infrastructure components that are part of the user's environment, which Kestra interacts with:
- - [**Internal Storage**](./data-components/index.md#internal-storage): object storage system within the user's infrastructure (e.g. AWS S3, Google Cloud Storage, or Azure Blob Storage)
- - **External Services**: third-party APIs or services outside of Kestra which Workers might interact with to process data within a given task
+| Role | Responsibility |
+|------|---------------|
+| **Executor** | Drives the execution state machine. Consumes execution events and worker results, determines the next task to run, and dispatches work. Runs no user code. |
+| **Worker Controller** | The sole communication point for workers. Dispatches jobs to workers over a bidirectional gRPC stream; accepts results, logs, and metrics back. Workers never touch the queue or database directly. |
+| **Worker** | Executes runnable tasks and polling triggers. Connects to the Worker Controller via gRPC. The only role that accesses user infrastructure and [internal storage](./data-components/index.md#internal-storage). |
+| **Scheduler** | Evaluates trigger conditions (except flow triggers, which the Executor handles) and submits executions to the queue. |
+| **Webserver** | Serves the [REST API](../api-reference/index.mdx) and [UI](../09.ui/index.mdx). |
+| **Indexer** | Reads from the queue and writes indexed content — flows, executions, logs, metrics, and audit logs — to the repository backend. Required in all deployments. |
-The arrows indicate the direction of communication. The JDBC Backend connects to the Server, which in turn interacts with the User's Infrastructure. The User interacts with the system through the API and UI.
+## Queue and repository
-For either database backend, the respective [PostgreSQL JDBC Driver](https://jdbc.postgresql.org/documentation/ssl/#configuring-the-client) can provide an encrypted connection with some configuration.
+The queue layer is the only channel through which server roles communicate. A server emits a typed message onto a named queue; another server consumes it. One backend implementation satisfies all queue contracts:
-### Scalability with JDBC
+- **JDBC** (default) — backed by any supported relational database (PostgreSQL, MySQL). Available in all editions.
+- **Kafka** — Enterprise Edition. Pairs with Elasticsearch for the search and read model.
+- **Redis**, **AMQP**, **GCP Pub/Sub** — Enterprise Edition. Additional broker-backed options for higher throughput.
-The scalable design of the architecture allows you to run multiple instances of the [Webserver](./02.server-components/index.md#webserver), [Executor](./02.server-components/index.md#executor), [Worker](./02.server-components/index.md#worker), and [Scheduler](./02.server-components/index.md#scheduler) to handle increased load. As your workload increases, more instances of the required components can be added to the system to distribute the load and maintain performance.
+The repository stores all domain entities: flows, executions, logs, triggers, and secrets. In a JDBC deployment, the relational database handles both the queue and repository. In a Kafka deployment, Elasticsearch backs the high-volume read model; the Indexer keeps it in sync.
-The JDBC Backend can be scaled too, either through clustering or sharding, to handle larger volumes of data and a higher number of requests from the [Server components](./02.server-components/index.md). Most cloud providers offer managed database services that can be scaled up and down as needed.
+## Worker communication
-## Architecture with Kafka and Elasticsearch backend
+Workers do not subscribe to the job queue directly. Each worker opens a persistent bidirectional gRPC stream to the Worker Controller and uses that stream for the lifetime of its connection:
-The following diagram shows the main components of Kestra using the [Kafka](https://kafka.apache.org/) and [Elasticsearch](https://www.elastic.co/elasticsearch) backend.
+- The Worker Controller dispatches jobs from the queue onto the stream.
+- Workers return results, logs, and metrics over the same stream.
+- Kill signals and metadata changes are broadcast to all connected workers.
-
+gRPC transport is available in all editions. TLS and mTLS secure the connection in all editions; JWT-based worker authentication is an Enterprise Edition feature.
-:::alert{type="info"}
-This architecture is only available in the [Enterprise Edition](../07.enterprise/01.overview/01.enterprise-edition/index.md) of Kestra.
-See [Open Source vs Enterprise](../oss-vs-paid/index.md) for a comparison of editions.
-:::
+
-This architecture provides enhanced scalability, high availability, and fault tolerance for large-scale deployments.
+## How an execution runs
-1. **Kafka**: serves as the messaging backend, which communicates between different components of the system and provides scalability and fault tolerance
+1. A user or client defines a flow through the Webserver's REST API. The flow is validated and stored in the repository under its namespace.
+2. A trigger fires — a schedule comes due, an external event matches, or an API call requests a run. The Scheduler (for schedule and polling triggers) or the Webserver (for manual runs) emits a new execution onto the queue.
+3. The Executor picks up the execution and runs its state machine to decide the next task.
+4. For each task, the Executor asks the worker-queue resolver for a routing decision, then dispatches a worker task onto the queue.
+5. The Worker Controller routes the job to a matching worker. The worker loads the task plugin, reads any inputs from internal storage, executes the task code, writes outputs back to internal storage, and emits the result.
+6. The Executor joins the result back into the execution, advances the state machine, and either dispatches the next task or terminates the execution.
+7. The Webserver serves the evolving execution state and logs to the UI by reading the repository. The Indexer keeps the read model in sync.
-2. **Microservices**: This layer includes several services:
- - [**Webserver**](./02.server-components/index.md#webserver): serves the [API](../api-reference/index.mdx) and the [User Interface](../09.ui/index.mdx) for interaction with the system
- - [**Scheduler**](./02.server-components/index.md#scheduler): schedules [workflows](../05.workflow-components/01.flow/index.md) and processes all [triggers](../05.workflow-components/07.triggers/index.mdx) except for the flow triggers
- - [**Executor**](./02.server-components/index.md#executor): handles the orchestration logic, including [flow triggers](../05.workflow-components/07.triggers/02.flow-trigger/index.md)
- - [**Indexer**](./02.server-components/index.md#indexer): indexes data from Kafka to Elasticsearch for quick retrieval and search
- (optional component since [Kestra v0.20](../11.migration-guide/v0.20.0/elasticsearch-indexer/index.md))
- - [**Worker**](./02.server-components/index.md#worker): runs [runnable tasks](../05.workflow-components/01.tasks/01.runnable-tasks/index.md) and interacts with the user's infrastructure
+## Execution context
-3. **User**: engages with the system through the Webserver's [UI](../09.ui/index.mdx) and [API](../api-reference/index.mdx)
+Task outputs are stored separately from the execution row and fetched on demand. This keeps the execution record small as flows grow in complexity and improves performance at high concurrency.
-4. **Elasticsearch**: acts as a search and UI backend, storing [logs](./data-components/index.md#logs), execution history, and enabling fast data retrieval
+## Enterprise Edition
-5. **User's Infrastructure**: private infrastructure components that are part of the user's environment, which Kestra interacts with:
- - [**Internal Storage**](./data-components/index.md#internal-storage): object storage system where user's data is stored (e.g. AWS S3, Google Cloud Storage, or Azure Blob Storage)
- - **External Services**: APIs or services that Workers might interact with during task processing
+Enterprise Edition is an additive overlay on the open-source core — not a fork. It adds multi-tenancy, action-based RBAC, identity-provider integration, audit logging, and additional queue backends by supplying alternative implementations of the same core contracts the open-source engine depends on. All open-source behavior is preserved.
-### Scalability with Kafka and Elasticsearch
+## Monitoring
-Kafka's messaging backend handles large volumes of data and scales horizontally. You can run multiple instances of Workers, Schedulers, Webservers, and Executors to distribute load, ensure fault tolerance, and maintain performance as demand increases.
+Every server role exposes a Prometheus scrape endpoint and OpenTelemetry traces. Three platform-wide metrics serve as primary health signals:
-Elasticsearch contributes to scalability by providing a horizontally scalable UI backend that can efficiently search across large amounts of data.
+| Metric | Description |
+|--------|-------------|
+| `kestra.queue.message.lag.count` | Backlog of unprocessed messages, tagged by worker queue. A steadily rising lag means consumers cannot keep up with producers. |
+| `kestra.worker.job.pending` | Worker jobs waiting for a free worker thread across the cluster. |
+| `kestra.worker.job.running` | Worker jobs currently executing across the cluster. |
-## Comparison between JDBC and Kafka architectures
-
-The main difference between the **JDBC** and **Kafka** architectures is the data layer (_JDBC Database vs. Kafka and Elasticsearch_).
-
-:::alert{type="info"}
-You can use the [Enterprise Edition](../07.enterprise/01.overview/01.enterprise-edition/index.md) with a JDBC database backend for smaller deployments. It's often more practical to start with JDBC and migrate to Kafka and Elasticsearch as your deployment grows.
-:::
-
-The **Worker** is the only component communicating with your private data sources to extract and transform data. The Worker also interacts with [**Internal Storage**](./data-components/index.md#internal-storage) to persist intermediary results and store the final task run outputs.
-
-All components of the **application layer** (including the Worker, Executor, and Scheduler) are decoupled and stateless, communicating with each other through the [**Queue**](./01.main-components/index.md#queue) (Kafka/JDBC). You can deploy and scale them independently.
-
-The **Webserver** communicates with the (Elasticsearch/JDBC) [Repository](./01.main-components/index.md#repository) to serve data for Kestra UI and API.
-
-The **data layer** is decoupled from the application layer and provides a separation between:
-
-- storing your private data processing artifacts — **Internal Storage** is used to store outputs of your executions; you can think of Internal Storage as your own private AWS S3 bucket
-- storing execution metadata — (Kafka/JDBC) [**Queue**](./01.main-components/index.md#queue) is used as the orchestration backend
-- storing logs and user-facing data — the (Elasticsearch/JDBC) [**Repository**](./01.main-components/index.md#repository) is used to store data needed to serve Kestra UI and API.
-
-The Indexer, available only in the [Enterprise Edition](../07.enterprise/01.overview/01.enterprise-edition/index.md), indexes content from Kafka topics (_such as the flows and executions topics_) to the Elasticsearch repositories. Because the Queue and Repository are separate in the Kafka architecture, executions continue even if Elasticsearch experiences downtime.
+Per-role metrics are documented on each component's page.
## Components in detail
-The following sections provide more details about the components of the architecture.
-
import ChildCard from "~/components/docs/ChildCard.astro"
-
\ No newline at end of file
+
diff --git a/src/contents/docs/09.ui/00.dashboard/create-dashboard.png b/src/contents/docs/09.ui/00.dashboard/create-dashboard.png
deleted file mode 100644
index 53bedef2178..00000000000
Binary files a/src/contents/docs/09.ui/00.dashboard/create-dashboard.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/00.dashboard/dashboard-table-export-2-0.png b/src/contents/docs/09.ui/00.dashboard/dashboard-table-export-2-0.png
new file mode 100644
index 00000000000..da7580a16ff
Binary files /dev/null and b/src/contents/docs/09.ui/00.dashboard/dashboard-table-export-2-0.png differ
diff --git a/src/contents/docs/09.ui/00.dashboard/dashboard-table-export.png b/src/contents/docs/09.ui/00.dashboard/dashboard-table-export.png
deleted file mode 100644
index 93d18cb22b1..00000000000
Binary files a/src/contents/docs/09.ui/00.dashboard/dashboard-table-export.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/00.dashboard/documentation-view.png b/src/contents/docs/09.ui/00.dashboard/documentation-view.png
deleted file mode 100644
index e2175e6c006..00000000000
Binary files a/src/contents/docs/09.ui/00.dashboard/documentation-view.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/00.dashboard/index.md b/src/contents/docs/09.ui/00.dashboard/index.md
index ccf76de0de8..4f87e206437 100644
--- a/src/contents/docs/09.ui/00.dashboard/index.md
+++ b/src/contents/docs/09.ui/00.dashboard/index.md
@@ -6,96 +6,31 @@ sidebarTitle: Dashboards
icon: /src/contents/docs/icons/ui.svg
---
-Get insights into your workflows with Dashboards.
-
-The first time you access the main **Dashboard**, you'll see the **Welcome Page** and you can click **Create my first flow** to launch a Guided Tour.
-
-Once you have executed a flow, you will see your flow executions in the dashboard.
-
-## Dashboard page
-
-The Dashboard page displays both the **default dashboard** and any **custom dashboards** you've created. To switch between dashboards, use the hamburger menu. If you have over 10 dashboards, type the dashboard name in the search bar to quickly find it. The same menu also lets you edit or delete existing dashboards. From your dashboard, you can apply and save filters, refresh data, and set an automatic periodic refresh.
+The **Dashboards** page displays the default dashboard and any custom dashboards you've created. Switch between them using the hamburger menu — if you have more than 10, use the search bar. The same menu lets you edit or delete dashboards. From any dashboard you can apply and save filters, refresh data, and set an automatic periodic refresh.
-Dashboards display the following data:
-- Executions over time
-- Execution Status for Today, Yesterday as well as Last 30 days
-- Executions per namespace
-- Execution errors per namespace
-- List of failed Executions
-- List of error logs
-- A ratio of execution successes to total executions
-
## Custom dashboards
-Dashboards let you define custom queries and charts to visualize data on your executions, logs, and metrics. Rather than relying only on the default dashboard on Kestra's home screen, you can create a custom dashboard with charts that answer specific questions and track key metrics. Each chart's configuration can be modified individually using the pencil icon in the dashboard view.
+Custom dashboards let you define queries and charts to visualize execution, log, and metric data. Edit each chart individually using the pencil icon in the dashboard view.
-You can build custom dashboards using the **No Code editor** (form-based, no YAML required) or by **writing YAML directly**. Both produce the same schema-validated output.
+Build dashboards using the **No Code editor** (no YAML required) or by **writing YAML directly**. Both produce the same schema-validated output.
### No Code editor
-The No Code editor lets you design dashboards directly in the UI using structured forms. As you configure charts and data sources, the editor generates YAML in real time and stays synchronized with the live preview — switch between views at any time.
-
-- **Speed & accessibility**: Build dashboards without writing YAML — ideal for analysts, operators, or anyone new to Kestra.
-- **Visual clarity**: Live previews let you see your dashboard evolve as you edit.
-- **No ceiling**: Switch to YAML when you need filters, dynamic queries, or advanced layout logic — all in the same editor.
+The No Code editor lets you design dashboards using structured forms. As you configure charts and data sources, YAML is generated in real time and the live preview stays in sync. Switch to YAML at any point for filters, dynamic queries, or advanced layout logic — both views reflect the same underlying definition.
#### Interactive demo
-
-
-#### Quick start
-
-Navigate to the **Dashboards** tab, click **Default Dashboard**, and select **+ Create Dashboard**.
-
-
-
-In the Dashboard YAML editor, select the **No Code** tab. It appears alongside the YAML editor so you can view both as you work.
-
-
-
-#### Example: build a KPI success ratio chart
-
-Give your dashboard an ID, title, description, and time window. Changes in No Code forms immediately reflect in the YAML editor.
-
-
-
-Click **+ Add** in the **charts** block to create your first chart. Choose **KPI Chart** as the chart type. Each type has its own options — see the [Chart Plugin documentation](/plugins/core/chart) for the full list. Open the **Documentation** tab at any time to view chart-specific guidance without leaving the editor.
-
-
-
-Give the chart an ID and set the data type to **Executions**. Set `field` to `ID` and `agg` to `COUNT` to capture all executions. Optionally add a display name.
-
-
+
-Click **+ Add** under the numerator section to add a filter. Choose `IN` for `type`, add `SUCCESS` as a value, and set `field` to `STATE`. This scopes the numerator to successful executions only.
-
-
-
-Return to the `charts` No Code tab and open **Optional Properties**. Set `displayName`, change `numberType` to `PERCENTAGE`, and set `width` to `3`.
-
-
-
-Open the **Preview** tab to review the chart. Click **Save** when satisfied. To create a failure ratio chart, copy the generated YAML, paste it into the YAML editor, and replace `SUCCESS` with `FAILED`.
-
-
-
-#### Best practices
-
-**Organize by purpose.** Group related charts into dashboards with a clear goal — for example, separate dashboards for system health, execution performance, and user activity.
-
-**Use consistent naming.** A pattern like `team_metric_type` (e.g., `dataops_executions_latency`) makes dashboards easier to find, version, and export.
-
-**Use YAML for reuse.** When charts share the same structure with small differences in filters or fields, copy-paste the YAML and modify — faster than rebuilding forms.
-
-**Preview before saving.** Catch mismatched fields and aggregation errors early before they make it into a published dashboard.
+For a step-by-step walkthrough of building your first KPI chart using the No Code editor, see [Build a KPI success ratio chart](../../15.how-to-guides/dashboard-kpi-chart/index.md).
### Chart types
Dashboards support six chart types: **Bar**, **Pie**, **TimeSeries**, **Table**, **KPI**, and **Markdown**. Each data chart type is composed of `chartOptions` and `data`.
-A chart's `chartOptions` property is where you customize display names and descriptions, and choose whether to add legends and tooltips to complement the visualization. A chart's `data` property is where you specify which Kestra data to use as a column, how you want the data displayed (e.g., an aggregate count or an `ORDER BY`), and add any [filters](#querying-data) you might want applied to the chart (e.g., REGEX match, greater or less than, or not Null).
+A chart's `chartOptions` property controls display names, descriptions, legends, and tooltips. The `data` property specifies the data source, which columns to display (including aggregations and sort order), and any [filters](#querying-data) to apply.
Each chart's options are listed in the [Chart Plugin Documentation](/plugins/core/chart) where you can dive further into the properties of each type.
@@ -350,13 +285,9 @@ charts:
flowId: my_flow
```
-:::alert{type="info"}
-The `content` shorthand (used in earlier examples) sets plain text content directly. The `source` property gives you access to the `FlowDescription` type to pull dynamic content from a flow's description field.
-:::
-
-## Create a new custom dashboard as code
+## Dashboard as code
-Clicking on the `+ Create new dashboard` button opens a code editor where you can define the dashboard layout and data sources as code.
+The **+ Create new dashboard** button opens the editor. Select the YAML view to define the dashboard layout and charts directly as code.
The top-level dashboard properties are:
@@ -368,7 +299,7 @@ The top-level dashboard properties are:
| `timeWindow.max` | Maximum selectable time range (e.g., `P365D`) |
| `charts` | List of chart definitions |
-Below is an example of a dashboard definition that displays executions over time, flow execution success ratio, a table that uses metrics to display the sum of sales per namespace, a table that shows the log count by level per namespace, and a Markdown insights panel:
+The example below defines a dashboard with a TimeSeries execution chart, a success ratio KPI, two data tables, and a Markdown panel:
:::collapse{title="Expand for an example dashboard definition"}
```yaml
@@ -487,15 +418,13 @@ charts:
```
:::
-:::alert{type="info"}
-To see all available properties to configure a custom dashboard as code, see examples provided in the [Enterprise Edition Examples](https://github.com/kestra-io/enterprise-edition-examples) repository.
-:::
+More dashboard examples are available in the [Enterprise Edition Examples](https://github.com/kestra-io/enterprise-edition-examples) repository.
## Exporting data
-Table data can be exported as a CSV file by hovering over the top-right corner and clicking the download icon. This enables dashboard users to build custom queries in Dashboards and to export data with one click without having to worry about pagination.
+Export table data as CSV using the download icon in the top-right corner of a table chart — no pagination required.
-
+
## Querying data
@@ -516,6 +445,7 @@ Dashboards can query data from these source `types`:
| `io.kestra.plugin.core.dashboard.data.Metrics` | Metrics emitted by your plugins |
| `io.kestra.plugin.core.dashboard.data.MetricsKPI` | Metrics data for KPI charts (supports `numerator`) |
| `io.kestra.plugin.core.dashboard.data.Triggers` | Trigger state and scheduling data |
+| `io.kestra.plugin.ee.dashboard.data.Assets` | Asset inventory data (EE and Cloud only). Not filtered by the dashboard time range — charts always reflect the current inventory. |
### Available fields by data source
@@ -586,13 +516,27 @@ After defining the data source, specify the columns to display in the chart. Eac
| `NEXT_EXECUTION_DATE` | Scheduled next execution date |
| `WORKER_ID` | Worker handling the trigger |
+#### Assets (EE and Cloud only)
+
+| Field | Description |
+| --- | --- |
+| `ID` | Asset identifier |
+| `TYPE` | Asset type (e.g., `io.kestra.plugin.ee.assets.VM`) |
+| `NAMESPACE` | Asset namespace |
+| `DISPLAY_NAME` | Asset display name |
+| `METADATA` | Asset metadata map. Use `metadataKey` in the column definition to group or filter by a specific metadata key. |
+| `CREATED` | Asset creation timestamp |
+| `UPDATED` | Asset last updated timestamp |
+
+Asset charts are not filtered by the dashboard time range — they always reflect the current inventory. `Assets` is compatible with `Bar`, `Pie`, and `Table` chart types. Use `NAMESPACE` in a `where` clause to scope results to a specific namespace.
+
### Column properties
Each entry in `data.columns` supports the following properties:
| Property | Description |
| --- | --- |
-| `field` | The only required property; specifies which field from the data source to use |
+| `field` | Required. The field from the data source to display. |
| `displayName` | Sets the label displayed in the chart |
| `agg` | Aggregation function: `AVG`, `COUNT`, `MAX`, `MIN`, or `SUM` |
| `graphStyle` | Series render style for TimeSeries charts: `LINES`, `BARS`, or `POINTS` (defaults to `LINES` when `agg` is set) |
@@ -619,5 +563,6 @@ Available filter types:
- `NOT_EQUAL_TO`
- `NOT_IN`
- `OR`
+- `PREFIX`
- `REGEX`
- `STARTS_WITH`
diff --git a/src/contents/docs/09.ui/00.dashboard/main_page.png b/src/contents/docs/09.ui/00.dashboard/main_page.png
index ad116b35eff..ca4bff078ef 100644
Binary files a/src/contents/docs/09.ui/00.dashboard/main_page.png and b/src/contents/docs/09.ui/00.dashboard/main_page.png differ
diff --git a/src/contents/docs/09.ui/00.dashboard/no-code-dashboards.png b/src/contents/docs/09.ui/00.dashboard/no-code-dashboards.png
deleted file mode 100644
index 7cb04824298..00000000000
Binary files a/src/contents/docs/09.ui/00.dashboard/no-code-dashboards.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/01-Welcome.png b/src/contents/docs/09.ui/01-Welcome.png
index f9931fb9f59..00c29ea7f1c 100644
Binary files a/src/contents/docs/09.ui/01-Welcome.png and b/src/contents/docs/09.ui/01-Welcome.png differ
diff --git a/src/contents/docs/09.ui/01.flows/04-Flows.png b/src/contents/docs/09.ui/01.flows/04-Flows.png
index 770c97c3182..eb8c7001930 100644
Binary files a/src/contents/docs/09.ui/01.flows/04-Flows.png and b/src/contents/docs/09.ui/01.flows/04-Flows.png differ
diff --git a/src/contents/docs/09.ui/01.flows/05-Flows-Flow.png b/src/contents/docs/09.ui/01.flows/05-Flows-Flow.png
index b2b6460671d..ac58463c91c 100644
Binary files a/src/contents/docs/09.ui/01.flows/05-Flows-Flow.png and b/src/contents/docs/09.ui/01.flows/05-Flows-Flow.png differ
diff --git a/src/contents/docs/09.ui/01.flows/blueprints-editor.png b/src/contents/docs/09.ui/01.flows/blueprints-editor.png
deleted file mode 100644
index 0e30edf6a17..00000000000
Binary files a/src/contents/docs/09.ui/01.flows/blueprints-editor.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/01.flows/brave.png b/src/contents/docs/09.ui/01.flows/brave.png
deleted file mode 100644
index 9dac6a21c06..00000000000
Binary files a/src/contents/docs/09.ui/01.flows/brave.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/01.flows/docs-editor.png b/src/contents/docs/09.ui/01.flows/docs-editor.png
deleted file mode 100644
index 6ac6b036aaa..00000000000
Binary files a/src/contents/docs/09.ui/01.flows/docs-editor.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/01.flows/files-editor.png b/src/contents/docs/09.ui/01.flows/files-editor.png
deleted file mode 100644
index 21e7f665902..00000000000
Binary files a/src/contents/docs/09.ui/01.flows/files-editor.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/01.flows/flow-dependencies-1-0.png b/src/contents/docs/09.ui/01.flows/flow-dependencies-1-0.png
deleted file mode 100644
index 4f73219b034..00000000000
Binary files a/src/contents/docs/09.ui/01.flows/flow-dependencies-1-0.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/01.flows/flow-editor.png b/src/contents/docs/09.ui/01.flows/flow-editor.png
index 9a15c9dacaf..af33d761f98 100644
Binary files a/src/contents/docs/09.ui/01.flows/flow-editor.png and b/src/contents/docs/09.ui/01.flows/flow-editor.png differ
diff --git a/src/contents/docs/09.ui/01.flows/index.md b/src/contents/docs/09.ui/01.flows/index.md
index bdbc67d80de..0d17ee3ba22 100644
--- a/src/contents/docs/09.ui/01.flows/index.md
+++ b/src/contents/docs/09.ui/01.flows/index.md
@@ -9,13 +9,11 @@ icon: /src/contents/docs/icons/ui.svg
Manage your flows in one place.
-On the **Flows** page, you see a list of flows that you can edit and execute. You can also create a new flow in the top-right corner.
-
-Click a flow ID or the eye icon to open a flow.
+The **Flows** page lists all flows. Click a flow ID to open it, or create a new flow from the top-right corner.
-A **Flow** page has multiple tabs that allow you to: see the flow topology, all flow executions, edit the flow, view its revisions, logs, metrics, and dependencies. You can also edit namespace files in the Flow editor.
+A **Flow** page has tabs for Overview, Executions, Edit, Revisions, Triggers, Logs, Metrics, Dependencies, and more.
@@ -23,125 +21,93 @@ A **Flow** page has multiple tabs that allow you to: see the flow topology, all
From the main Flows page, you can filter the displayed flows on fields like namespace, scope, labels, and open text. The filters are key based with comma-separated OR-conditions and spaced-separated AND-conditions. The following video demonstrates the filters in action:
-
+
## Edit
-The Edit interface provides a rich view of your workflow, as well as Namespace Files. The editor allows you to add multiple panels:
-- Flow code
-- No Code
-- Topology
-- Documentation
-- Files
-- Blueprints
+The **Edit** tab is the main authoring environment. Open panels from the tab bar and arrange them side by side:
+- **Flow Code** — YAML editor with autocomplete
+- **No-code** — visual flow builder with task cards and a structured outline
+- **Topology** — visual DAG of the flow
+- **Docs** — plugin documentation, updates as you move the cursor
+- **Files** — namespace files editor
+- **Blueprints** — ready-to-use flow examples
+- **Context** — namespace variables, KV pairs, and secrets (Enterprise)
-Additionally, from the **Actions** menu, you can export your flow as a YAML file, delete, or copy your flow.
+From the top-right of the editor, you can access **Revisions**, **Dependencies**, and any validation **Errors**. Use **Export flow** to download the flow as a YAML file.
-### Flow code view
+### Flow Code view
-The **Flow** code view allows you to edit your workflows with YAML. Autocomplete is available as you write. As new tasks are added, they automatically appear in the No-code and topology views.
+The **Flow Code** view is a YAML editor with autocomplete. Tasks added here appear immediately in the No-code and Topology views.
### No Code view
-The **No Code** view lets you edit workflows directly in the UI using structured forms. As you modify your flow, YAML is generated in real time in the code view and you can switch between both views at any time.
-
-- **Speed & onboarding**: Build flows without writing YAML first; switch to code view whenever you need advanced control.
-- **Consistency**: UI-driven forms align with plugin schemas and validation, reducing drift.
-- **No ceiling**: When you outgrow forms, switch to YAML, add files/scripts, and keep everything in one place.
-
-#### Build a flow in No Code
-
-1. **Create a flow** from **Flows → + Create**; confirm namespace and identifiers.
-2. **Open No Code view** from the editor panel. Browse or search the plugin catalog and select a plugin to reveal its form fields.
-
-
+The **No Code** view is a canvas-based flow editor. The canvas displays each flow section — Triggers, Tasks, Errors, Finally, and After Execution — as a group of visual blocks. Selecting a block opens its configuration form in a third panel alongside the canvas.
-You can close, open, and reposition panels at any time. In the example below, the Slack plugin documentation is open alongside the No Code editor with the YAML view closed.
+
-
+Click any block to open its form. The form has two tabs: **Form** (guided fields with inline documentation) and **Source** (raw YAML for that block). You can switch to **Source** to write or paste YAML directly — the flow YAML editor on the left stays in sync instantly.
-3. **Configure inputs** by clicking **+ Add** in the inputs section. Each input opens a configuration tab. If the YAML view is open, you'll see it update in real time as you add inputs.
+
-
+To add a block, click **+ Add task** or **+ Add trigger** in the relevant section, or press `/` anywhere on the canvas to search and insert a block at the cursor position. Use the keyboard shortcuts shown in the bottom bar to navigate (`↑ ↓`), open a selected block (`⇧`), or insert after the current selection.
-4. **Configure task properties** via forms. Each task opens a No Code tab and generates YAML as you select properties. Fields can autocomplete expressions from inputs you've already configured.
+Click **Configure** at the top of the canvas to edit flow-level properties (namespace, description, inputs, outputs, variables, and more).
-
+#### Focused view
-5. **Add flow logic** — If, Switch, ForEach, and Subflow tasks — to control execution paths.
-6. **Add a trigger** (schedule, file event, webhook) to automate runs.
+Opening a block expands it into a focused modal by default. The modal has two panels:
-
+- **Left — Inputs**: lists every value you can reference in this task's properties. **Upstream Outputs** shows the output keys of all tasks that run before this one; **Execution Context** lists all built-in variables available at runtime (`flow.id`, `execution.id`, `taskrun.id`, `trigger.date`, and so on). Use these as a reference when writing Pebble expressions in the form fields.
+- **Right — Form / Source**: the task configuration form. Switch to **Source** to edit raw YAML for the block. An **Output** panel on the right edge shows the task's output schema.
-7. **Add additional flow components** such as [outputs](../../05.workflow-components/06.outputs/index.md), [retry](../../05.workflow-components/12.retries/index.md), [SLA](../../05.workflow-components/18.sla/index.md), [afterExecution](../../05.workflow-components/20.afterexecution/index.md), and [Plugin Defaults](../../05.workflow-components/09.plugin-defaults/index.md). Everything possible in YAML is available in No Code.
+
-
+Tasks open as a modal by default. To open blocks as tabs in the editor instead, change the default in **Settings**.
-8. **Save and run**: execute from the UI to see logs and results.
-
-Edits in No Code forms update YAML instantly, and edits in YAML reflect back in No Code. For complex expressions, advanced plugin fields, or bulk edits, switch to the YAML view — then switch back. Use the **Actions** menu to export or copy the flow at any time.
+You can open multiple panels simultaneously — for example, keep **Docs** open alongside the canvas to reference plugin documentation while configuring a task. Use the **Actions** menu to export or copy the flow at any time.
:::alert{type="info"}
-You can also skip YAML with the [AI Copilot](../../ai-tools/ai-copilot/index.md), which generates a flow from a plain-language description.
+Flow Code, No-code, and [AI Copilot](../../ai-tools/01.ai-copilot/index.md) all stay in sync. Start in any mode — write YAML, describe your flow to the Copilot, or build visually on the canvas — and switch freely at any point. Every change is reflected across all three views instantly.
:::
### Topology view
-The **Topology** view allows you to visualize the structure of your flow. This is especially useful when you have complex flows with multiple branches of logic. From the bottom left corner of the Topology view, you can zoom in, zoom out, and export your flow topology as a `.png` file.
+The **Topology** view shows a visual DAG of the flow — useful for complex flows with multiple branches. Zoom controls and a `.png` export are in the bottom-left corner.
### Documentation view
-The **Documentation** view displays Kestra's documentation directly inside the editor. As you move your cursor around the editor, the documentation panel updates to reflect the specific task type documentation.
-
-
+The **Documentation** view displays plugin docs directly inside the editor. The panel updates as you move your cursor to reflect the task type at the current position.
:::alert{type="warning"}
-If you use the [Brave browser](https://brave.com/), you may need to disable Brave Shields to make the editor work as expected. To view task documentation, set the `Block cookies` option to `Disabled` in Shields settings: `brave://settings/shields`.
-
-
+If you use the [Brave browser](https://brave.com/), you may need to disable Brave Shields to make the editor work as expected. To view task documentation, set the **Block cookies** option to **Disabled** in Shields settings at `brave://settings/shields`.
:::
-## Files view
-
-The **Files** view allows you to create, edit and delete [Namespaces Files](../../06.concepts/02.namespace-files/index.md). Multiple files can be opened at the same time, as well as displayed side by side using multiple panels.
+### Files view
-
+The **Files** view lets you create, edit, and delete [Namespace Files](../../06.concepts/02.namespace-files/index.md). Open multiple files side by side using the multi-panel layout.
### Blueprints view
-The **Blueprints** view gives you example flows to copy directly into your flow. Blueprints are especially useful when working with a new plugin, since you can start from a working example.
+The **Blueprints** view gives you example flows to copy directly into the editor — useful when working with a new plugin.
-
+### Context panel (Enterprise)
-### Namespace context (Enterprise)
-
-In the **Namespace Context** view, you can directly access your Variables, KV pairs, and Secrets managed at the namespace level. You can also render expressions that fall within those categories.
-
-
+The **Context** panel gives you direct access to namespace Variables, KV pairs, and Secrets from within the editor. You can also render expressions against those values inline.
## Revisions
You can view the history of your flow code changes under the **Revisions** tab. For more details, see [Revisions](../../06.concepts/03.revision/index.md).
-
-
## Dependencies
-
-
-The **Dependencies** page shows the relationship dependencies between other flows and the selected flow, and lets you navigate between them.
-
-
-
-:::alert{type="info"}
-The **Dependencies View** on the **Namespaces** page shows all the flows in the namespace and how they each relate to one another, if at all, whereas the Flow Dependencies view is only for the selected flow.
-:::
+The **Dependencies** tab shows the relationship between this flow and other flows, and lets you navigate between them. The **Dependencies View** on the **Namespaces** page shows all flows in the namespace and how they relate to one another — the flow-level Dependencies view is scoped to the selected flow only.
diff --git a/src/contents/docs/09.ui/01.flows/namespace-context.png b/src/contents/docs/09.ui/01.flows/namespace-context.png
deleted file mode 100644
index a4012322d0c..00000000000
Binary files a/src/contents/docs/09.ui/01.flows/namespace-context.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/01.flows/no-code-canvas.png b/src/contents/docs/09.ui/01.flows/no-code-canvas.png
new file mode 100644
index 00000000000..16a5c282288
Binary files /dev/null and b/src/contents/docs/09.ui/01.flows/no-code-canvas.png differ
diff --git a/src/contents/docs/09.ui/01.flows/no-code-errors.png b/src/contents/docs/09.ui/01.flows/no-code-errors.png
new file mode 100644
index 00000000000..a270db1a9e8
Binary files /dev/null and b/src/contents/docs/09.ui/01.flows/no-code-errors.png differ
diff --git a/src/contents/docs/09.ui/01.flows/no-code-task-modal.png b/src/contents/docs/09.ui/01.flows/no-code-task-modal.png
new file mode 100644
index 00000000000..f89fc9a0c3e
Binary files /dev/null and b/src/contents/docs/09.ui/01.flows/no-code-task-modal.png differ
diff --git a/src/contents/docs/09.ui/01.flows/revisions.png b/src/contents/docs/09.ui/01.flows/revisions.png
deleted file mode 100644
index f338bdd27fe..00000000000
Binary files a/src/contents/docs/09.ui/01.flows/revisions.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/01.flows/topology-editor.png b/src/contents/docs/09.ui/01.flows/topology-editor.png
index adf9e5cbee8..cfb21be364e 100644
Binary files a/src/contents/docs/09.ui/01.flows/topology-editor.png and b/src/contents/docs/09.ui/01.flows/topology-editor.png differ
diff --git a/src/contents/docs/09.ui/02.executions/execution-debug-expression.png b/src/contents/docs/09.ui/02.executions/execution-debug-expression.png
index caba5b04d2e..89b450e7633 100644
Binary files a/src/contents/docs/09.ui/02.executions/execution-debug-expression.png and b/src/contents/docs/09.ui/02.executions/execution-debug-expression.png differ
diff --git a/src/contents/docs/09.ui/02.executions/execution-gantt-view.png b/src/contents/docs/09.ui/02.executions/execution-gantt-view.png
index 10c4f8cf4b0..508cf27fef3 100644
Binary files a/src/contents/docs/09.ui/02.executions/execution-gantt-view.png and b/src/contents/docs/09.ui/02.executions/execution-gantt-view.png differ
diff --git a/src/contents/docs/09.ui/02.executions/execution-logs-view.png b/src/contents/docs/09.ui/02.executions/execution-logs-view.png
index e56521c382b..b614b46f758 100644
Binary files a/src/contents/docs/09.ui/02.executions/execution-logs-view.png and b/src/contents/docs/09.ui/02.executions/execution-logs-view.png differ
diff --git a/src/contents/docs/09.ui/02.executions/execution-metrics-view.png b/src/contents/docs/09.ui/02.executions/execution-metrics-view.png
index fe5bcf470ce..0c0a4a69f72 100644
Binary files a/src/contents/docs/09.ui/02.executions/execution-metrics-view.png and b/src/contents/docs/09.ui/02.executions/execution-metrics-view.png differ
diff --git a/src/contents/docs/09.ui/02.executions/execution-results-overview.png b/src/contents/docs/09.ui/02.executions/execution-results-overview.png
index 60cde2c2958..65a8cbfddc0 100644
Binary files a/src/contents/docs/09.ui/02.executions/execution-results-overview.png and b/src/contents/docs/09.ui/02.executions/execution-results-overview.png differ
diff --git a/src/contents/docs/09.ui/02.executions/execution-topology-view.png b/src/contents/docs/09.ui/02.executions/execution-topology-view.png
deleted file mode 100644
index 0a4b99789ce..00000000000
Binary files a/src/contents/docs/09.ui/02.executions/execution-topology-view.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/02.executions/executions-dependencies-1-0.png b/src/contents/docs/09.ui/02.executions/executions-dependencies-1-0.png
index d20633e0481..e92e782b9ea 100644
Binary files a/src/contents/docs/09.ui/02.executions/executions-dependencies-1-0.png and b/src/contents/docs/09.ui/02.executions/executions-dependencies-1-0.png differ
diff --git a/src/contents/docs/09.ui/02.executions/executions-overview.png b/src/contents/docs/09.ui/02.executions/executions-overview.png
index 300fa12e0fc..186820dcee1 100644
Binary files a/src/contents/docs/09.ui/02.executions/executions-overview.png and b/src/contents/docs/09.ui/02.executions/executions-overview.png differ
diff --git a/src/contents/docs/09.ui/02.executions/fix-with-ai-logs.png b/src/contents/docs/09.ui/02.executions/fix-with-ai-logs.png
index 9f455e32ff9..4532540ffcf 100644
Binary files a/src/contents/docs/09.ui/02.executions/fix-with-ai-logs.png and b/src/contents/docs/09.ui/02.executions/fix-with-ai-logs.png differ
diff --git a/src/contents/docs/09.ui/02.executions/index.md b/src/contents/docs/09.ui/02.executions/index.md
index 122a687e0c3..58483cf1615 100644
--- a/src/contents/docs/09.ui/02.executions/index.md
+++ b/src/contents/docs/09.ui/02.executions/index.md
@@ -8,75 +8,36 @@ icon: /src/contents/docs/icons/ui.svg
Inspect and manage flow executions.
-On the **Executions** page, you see a list of all your completed flow executions.
-
-You can select multiple checkboxes to choose executions for bulk actions, such as Restart, Kill, Pause, or Force Run. Alternatively, you can click an execution ID or the magnifying glass icon to open an execution for further examination.
+The **Executions** page lists all flow executions. Select one or more to perform bulk actions (Restart, Kill, Pause, Force Run), or click an execution ID to open it.
## Overview
-An **Execution's Overview** page displays the details of a flow execution, organized into the following sections. For reference, below is an example flow and its **Execution Overview**.
-
-```yaml
-id: conditionallyReturnOutputs
-namespace: company.team
-
-labels:
- - key: environment
- value: dev
- - key: owner
- value: data-team
-variables:
- description: This is a demo flow
- version: 1.0.0
-
-inputs:
- - id: runTask
- type: BOOL
- defaults: true
-
-tasks:
- - id: taskA
- runIf: "{{ inputs.runTask }}"
- type: io.kestra.plugin.core.debug.Return
- format: Hello World!
-
- - id: taskB
- type: io.kestra.plugin.core.debug.Return
- format: Fallback output
-
-outputs:
- - id: flowOutput
- type: STRING
- value: "{{ tasks.taskA.state != 'SKIPPED' ? outputs.taskA.value : outputs.taskB.value }}"
-
-triggers:
- - id: every_minute_schedule
- type: io.kestra.plugin.core.trigger.Schedule
- cron: "* * * * *"
-```
+The **Overview** tab displays the details of a flow execution.
From the **Overview** tab, you can:
-- Set Labels: give a label to the execution for tracking or filtering.
-- Change State: change the execution state.
-- Force Run: forces the execution to run. This may create duplicate task executions — use with caution.
+- **Set Labels** — add a label to the execution for tracking or filtering.
+- **Change State** — manually update the execution state.
+- **Force Run** — force the execution to run; may create duplicate task executions, so use with caution.
-The **Previous and Next Execution** buttons navigate you through past and future (if there's a trigger) flow executions.
+The **Previous** and **Next Execution** buttons step through past and scheduled future executions.
- Execution **state** is displayed along with a timestamped state history from `CREATED` to `RUNNING` to `SUCCESS` (or any other possible state).
- Flow [Variables](../../05.workflow-components/04.variables/index.md) and [Inputs](../../05.workflow-components/05.inputs/index.md) are clearly listed along with execution details including dates and the corresponding namespace and flow.
- Flow outputs and trigger data are captured with expression rendering.
-From the **Overview** page, you can also take actions such as [**Replay**](../../06.concepts/10.replay/index.md) or **Pause**, and view executions over time to compare previous runs.
+From the **Overview** tab, you can also take actions such as [**Replay**](../../06.concepts/10.replay/index.md) or **Pause**, and view executions over time to compare previous runs.
+
+The Overview tab also embeds a **topology diagram** showing each task's state in the execution graph — green for `SUCCESS`, red for `FAILED`. Click any task node to access its logs, replay it, or change its status.
## Filters
-From the main Executions page, you can filter the displayed executions on fields like namespace, flowId, labels, state, startDate, open text, and more. You can save applied filters and export the data all from the UI. The following video demonstrates the filters in action:
+Filter executions by namespace, flow ID, labels, state, start date, or free text. Save applied filters or export results. The following video demonstrates the filters in action:
-
+
## Gantt
@@ -84,9 +45,7 @@ The **Gantt** tab visualizes each task's duration. From this interface, you can
-The **Gantt** view displays all successful and failed tasks in the execution. For failed tasks, you can open the task and click the three dots to **"Fix with AI"**. This option reopens the flow editor with the [AI Copilot](../../ai-tools/ai-copilot/index.md) prompted to help resolve any issues with the task.
-
-
+The **Gantt** view displays all successful and failed tasks in the execution. For failed tasks, use **Fix with AI** from the task menu to open the flow editor with [AI Copilot](../../ai-tools/01.ai-copilot/index.md) pre-loaded with the error context.
## Logs
@@ -94,21 +53,13 @@ The **Logs** tab gives access to a task's logs. You can filter by log level, cop
-For failed tasks, click the three dots to **"Fix with AI"**. This option reopens the flow editor with the [AI Copilot](../../ai-tools/ai-copilot/index.md) prompted to help resolve any issues with the task.
+For failed tasks, use **Fix with AI** from the task menu to open the flow editor with [AI Copilot](../../ai-tools/01.ai-copilot/index.md) pre-loaded with the error context.
-## Topology
-
-Similar to the Editor view, you can see your execution's topology. **Topology** provides a graphical view to access specific task logs, replay certain tasks, or change task status. Tasks' state progression is shown and updated as the status changes. For example, green indicates a task has reached **SUCCESS** while red indicates **FAILED**.
-
-
+## Input/Output
-From a **FAILED** task, click the magnifying glass icon to open the logs and read the error message, investigate, and **"Fix with AI"** if you have [AI Copilot](../../ai-tools/ai-copilot/index.md) configured.
-
-## Outputs
-
-The **Outputs** tab presents the execution's generated outputs. All tasks and their corresponding outputs are accessible from this page for examination and debugging. Outputs could be results or variables to pass onto downstream tasks, or files to download or pass downstream as a URI for processing. The example below downloads an outputted file generated from a SQL query.
+The **Input/Output** tab shows execution inputs and all task outputs — variables to pass downstream or files to download and inspect. The example below downloads a file generated from a SQL query.
@@ -116,18 +67,14 @@ The **Debug Expression** button lets you evaluate [expressions](../../expression
-Use **Debug Expression** to inspect task outputs and test expressions interactively.
-
## Metrics
-The Metrics tab shows every metric exposed by tasks after execution. For example, a [BigQuery load task](/plugins/plugin-gcp/google-cloud-bigquery/io.kestra.plugin.gcp.bigquery.load) might show the amount of files inputted, rows inserted, and how long the operation took to complete. Another example, a flow using an AI plugin shows token usage as a metric for the task.
+The **Metrics** tab shows every metric exposed by tasks after execution. For example, an HTTP Request task emits `request.bytes`, `response.bytes`, and `response.length` per request. A BigQuery load task might expose rows inserted and bytes processed. An AI plugin task might show token usage.
## Dependencies
-
-
-The Dependencies tab shows the relationship dependencies between other flows and the selected execution. It also displays extra execution metadata such as state.
+The **Dependencies** tab shows the relationship between other flows and the selected execution, including extra execution metadata such as state.
diff --git a/src/contents/docs/09.ui/04.namespaces/dependencies-namespaces.png b/src/contents/docs/09.ui/04.namespaces/dependencies-namespaces.png
deleted file mode 100644
index 96bd461b9c3..00000000000
Binary files a/src/contents/docs/09.ui/04.namespaces/dependencies-namespaces.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/04.namespaces/flows-namespaces.png b/src/contents/docs/09.ui/04.namespaces/flows-namespaces.png
index e154d43f02b..3a8f87f32ba 100644
Binary files a/src/contents/docs/09.ui/04.namespaces/flows-namespaces.png and b/src/contents/docs/09.ui/04.namespaces/flows-namespaces.png differ
diff --git a/src/contents/docs/09.ui/04.namespaces/index.md b/src/contents/docs/09.ui/04.namespaces/index.md
index 896c0f880ae..1096ff9b46c 100644
--- a/src/contents/docs/09.ui/04.namespaces/index.md
+++ b/src/contents/docs/09.ui/04.namespaces/index.md
@@ -8,18 +8,12 @@ icon: /src/contents/docs/icons/ui.svg
Manage all resources associated with a Namespace in one place.
-The **Namespaces** tab in the UI for Open Source users displays all Namespaces associated with different flows in your Kestra instance.
+The **Namespaces** page lists all namespaces in your Kestra instance.
-## Interactive demo
-
-Explore the Namespace UI through this interactive demo:
-
-
-
## Overview
The **Overview** tab is the default landing page of a Namespace. It displays dashboards and summaries of flow executions within that Namespace.
@@ -28,7 +22,7 @@ The **Overview** tab is the default landing page of a Namespace. It displays das
## Flows
-The **Flows** tab lists all flows within the Namespace. It displays key information such as the flow ID, labels, last execution date and status, and execution statistics. Selecting the **details** button on a flow opens its detailed page.
+The **Flows** tab lists all flows within the namespace, showing flow ID, labels, last execution date and status, and execution statistics.
@@ -38,14 +32,9 @@ The **Dependencies** tab visualizes relationships between flows, showing which f
This view is similar to the **Dependencies** page in the Flow Editor but focuses on inter-flow relationships within a single Namespace — even if some flows are independent.
-
-
## KV store
-The **KV Store** tab lets you manage key-value pairs associated with a Namespace.
-For more information, see the [KV Store concept guide](../../06.concepts/05.kv-store/index.md).
-
-
+The **KV Store** tab lets you manage key-value pairs scoped to the namespace. For more information, see the [KV Store concept guide](../../06.concepts/05.kv-store/index.md).
@@ -55,8 +44,8 @@ For more information, see the [KV Store concept guide](../../06.concepts/05.kv-s
The **Files** tab lets you create, edit, and manage Namespace Files used in your flows — from custom Python scripts to images. Learn more in [Namespace Files](../../06.concepts/02.namespace-files/index.md).
-
+## Additional tabs
-## Additional enterprise pages
+Each namespace also has tabs for **Executions**, **Variables**, and **Reusable Inputs**, as well as a **Revisions** history for namespace-level configuration changes.
-In the [Enterprise Edition](../../07.enterprise/01.overview/01.enterprise-edition/index.md), additional Namespace pages provide deeper insights and management capabilities. Learn more on the [Enterprise Namespace Management page](../../07.enterprise/02.governance/07.namespace-management/index.md).
+[Kestra Enterprise Edition](../../07.enterprise/01.overview/01.enterprise-edition/index.md) adds **Secrets**, **Credentials**, **Assets**, **Policies**, and **Audit Logs** tabs. Learn more on the [Enterprise Namespace Management page](../../07.enterprise/02.governance/07.namespace-management/index.md).
diff --git a/src/contents/docs/09.ui/04.namespaces/kvstore-namespaces.png b/src/contents/docs/09.ui/04.namespaces/kvstore-namespaces.png
deleted file mode 100644
index 832659ce79b..00000000000
Binary files a/src/contents/docs/09.ui/04.namespaces/kvstore-namespaces.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/04.namespaces/namespace-files-tab.png b/src/contents/docs/09.ui/04.namespaces/namespace-files-tab.png
deleted file mode 100644
index dfb8131bfbb..00000000000
Binary files a/src/contents/docs/09.ui/04.namespaces/namespace-files-tab.png and /dev/null differ
diff --git a/src/contents/docs/09.ui/04.namespaces/overview-namespaces.png b/src/contents/docs/09.ui/04.namespaces/overview-namespaces.png
index 0d509d9621a..7b0837bf3a6 100644
Binary files a/src/contents/docs/09.ui/04.namespaces/overview-namespaces.png and b/src/contents/docs/09.ui/04.namespaces/overview-namespaces.png differ
diff --git a/src/contents/docs/09.ui/06.settings/index.md b/src/contents/docs/09.ui/06.settings/index.md
index 5a9f1e10719..07de30410d5 100644
--- a/src/contents/docs/09.ui/06.settings/index.md
+++ b/src/contents/docs/09.ui/06.settings/index.md
@@ -7,25 +7,36 @@ icon: /src/contents/docs/icons/ui.svg
docId: settings
---
-Configure Settings for Kestra.
-
-**Settings** are accessible from the bottom left environment menu. All configuration options are per-user.
+Configure per-user preferences, profile details, and API tokens from the **Settings** page, accessible via the bottom-left environment menu.
+## Preferences
+
+Options you can configure under **Preferences** include:
+- **Default Namespace**: Pre-selects a namespace when creating a new flow and filters the Flows and Executions pages to that namespace by default.
+- **Default Log Level**: Minimum log level shown in execution logs (e.g., `INFO`, `TRACE`).
+- **Default Log Display**: How logs are presented when opening an execution — expand all, collapse all, or expand only failed tasks.
+- **Default Editor Type**: Editor shown when opening a flow for the first time — YAML Editor or No Code Editor.
+- **Execute the Flow**: Where execution results open after triggering a run — in the same tab or a new tab.
+- **Default Execution Tab**: Tab selected when navigating to an execution (e.g., Gantt, Logs, Outputs).
+- **Default Flow Tab**: Tab selected when navigating to a flow (e.g., Overview, Topology, Edit).
+- **Default Triggers Tab**: Tab selected when navigating to the triggers page.
+- **Auto Refresh Interval**: Seconds between automatic data refreshes on list pages.
+- **Playground**: Toggle to enable or disable the editor playground, which lets you run tasks individually.
+- **Customize Sidebar**: Drag and drop items to reorder the left navigation sidebar, or remove items you don't need.
+
+## Profile
-## Main configuration
+Access **Profile** from the Settings left-hand menu to manage your personal account details.
-Options you can configure under **Main Configuration** include:
-- **Default Namespace**: e.g., `company.team` - by default, this is empty. Once set, this will be the default namespace when creating a new flow (otherwise `company.team` is used as a placeholder). Also, when navigating to the Flows or Executions pages, it will filter for this default namespace.
-- **Default Editor Type**: e.g., "YAML Editor" or "No Code Editor"
-- **Default Log Level**: e.g., `TRACE`
-- **Default Log Display**: Expand all, Collapse all, or Expand only failed tasks
-- **Execute the Flow**: In the same tab or in a new tab
-- **Default Execution Tab**: Sets which Execution tab you are directed to (e.g., Gantt, Logs, Outputs, etc.) after executing a flow.
-- **Default Flow Tab**: Sets which flow tab opens by default when you click a flow (e.g., Overview, Topology, Edit, etc.)
+The following fields are editable directly — no separate save step is required:
+- **First Name** and **Last Name**
+- **Password** — enter and confirm a new password to update your login credentials.
-## Theme preferences
+Your avatar displays your initials derived from your first and last name. If you have pending workspace invitations, they appear in a table at the bottom of the Profile page.
+
+## Appearance
Kestra supports both Light and Dark mode.
@@ -33,11 +44,7 @@ You can also configure the Editor independently in Light or Dark mode. In additi
There's also the option to change the environment name and color to help you identify if you have multiple Kestra instances, for example a `dev` and `prod` environment.
-
-
-
-
-Below is a detailed list of the Theme Preferences you can configure:
+Below is a detailed list of the Appearance options you can configure:
- **Theme Mode**: Dark or Light
- **Chart Color Scheme**: Classic (red-green) or Kestra (pink-purple)
@@ -71,6 +78,3 @@ Below is a detailed list of the Theme Preferences you can configure:
This setting only affects the UI display. It does not affect [Schedule triggers](../../05.workflow-components/07.triggers/01.schedule-trigger/index.md) or flow execution times, which run on UTC by default.
:::
-## Export
-
-You can also export all of your flows as a `.zip` file. This allows you to back up your flows or migrate them to another instance of Kestra.
diff --git a/src/contents/docs/09.ui/06.settings/settings.png b/src/contents/docs/09.ui/06.settings/settings.png
index 75172315124..11ba074b3c4 100644
Binary files a/src/contents/docs/09.ui/06.settings/settings.png and b/src/contents/docs/09.ui/06.settings/settings.png differ
diff --git a/src/contents/docs/09.ui/10.playground/index.md b/src/contents/docs/09.ui/10.playground/index.md
index 23ecbd4ed79..3adb92503bd 100644
--- a/src/contents/docs/09.ui/10.playground/index.md
+++ b/src/contents/docs/09.ui/10.playground/index.md
@@ -5,7 +5,6 @@ description: Experiment in the Kestra Playground. Build and test tasks iterative
sidebarTitle: Playground
icon: /src/contents/docs/icons/ui.svg
editions: ["OSS", "EE"]
-version: "0.24.0"
---
Iteratively build and test flows task by task without running the entire workflow.
@@ -14,9 +13,7 @@ Iteratively build and test flows task by task without running the entire workflo
-## Playground
-
-The **Playground mode** in Kestra allows you to build workflows iteratively, one task at a time. This feature is especially useful when building data processing flows, where you typically start with a task extracting data, and you need to inspect the output before knowing what kind of transformation might be required. Then, you can work on that transformation task without rerunning the extraction task.
+**Playground mode** lets you build workflows iteratively, one task at a time. This feature is especially useful when building data processing flows, where you typically start with a task extracting data, and you need to inspect the output before knowing what kind of transformation might be required. Then, you can work on that transformation task without rerunning the extraction task.
If you've ever worked with a [Jupyter](https://jupyter.org/) notebook, you might be familiar with this pattern: you run the first cell to extract data, then you run the second cell to transform that data, and you can rerun the second cell multiple times to test different transformations without having to rerun the first cell again. Kestra's Playground mode allows you to do the same within your flows.
@@ -33,8 +30,37 @@ To use Playground mode:
Kestra tracks up to 10 recent playground runs, so you can go back to inspect the outputs of previously executed tasks. Older runs are purged automatically. Playground runs won't appear in the regular execution list to avoid confusion with production executions.
-Playground mode requires a DAG (Directed Acyclic Graph) structure, so you cannot run a task before its upstream tasks have been played. If you change flow-level `inputs`, `variables`, `pluginDefaults`, or `outputs` properties while in Playground mode, existing task runs are automatically reset and must be rerun. Kestra resets them to ensure that task outputs remain consistent with the flow-level properties.
+Playground mode requires a DAG (Directed Acyclic Graph) structure, so you cannot run a task before its upstream tasks have been played. If you change flow-level `inputs`, `variables`, or `outputs` properties while in Playground mode, existing task runs are automatically reset and must be rerun. Kestra resets them to ensure that task outputs remain consistent with the flow-level properties.
To see Playground in action, check out the demo below.
-
+
+
+```yaml
+id: playground_data_pipeline
+namespace: company.team
+description: |
+ A step-by-step data pipeline designed for the Playground feature.
+ Run each task individually to inspect outputs before proceeding to the next stage.
+labels:
+ team: engineering
+ type: demo
+
+tasks:
+ - id: fetch_products
+ type: io.kestra.plugin.core.http.Request
+ uri: https://jsonplaceholder.typicode.com/todos?_limit=10
+ method: GET
+
+ - id: extract_titles
+ type: io.kestra.plugin.core.debug.Return
+ format: "{{ outputs.fetch_products.body | jq('map(select(.completed == false)) | map(.title)') }}"
+
+ - id: count_pending
+ type: io.kestra.plugin.core.debug.Return
+ format: "{{ outputs.fetch_products.body | jq('[.[] | select(.completed == false)] | length') }}"
+
+ - id: build_report
+ type: io.kestra.plugin.core.debug.Return
+ format: "Pending tasks: {{ outputs.count_pending.value }} | First item: {{ outputs.extract_titles.value | jq('.[0]') }}"
+```
diff --git a/src/contents/docs/09.ui/customize-sidebar.png b/src/contents/docs/09.ui/customize-sidebar.png
new file mode 100644
index 00000000000..b0ddb1acce2
Binary files /dev/null and b/src/contents/docs/09.ui/customize-sidebar.png differ
diff --git a/src/contents/docs/09.ui/index.mdx b/src/contents/docs/09.ui/index.mdx
index 1b42df9deaa..2fd04ab5be8 100644
--- a/src/contents/docs/09.ui/index.mdx
+++ b/src/contents/docs/09.ui/index.mdx
@@ -10,32 +10,45 @@ import ChildCard from "~/components/docs/ChildCard.astro"
Kestra's web UI is available by default on port 8080.
-When you first navigate to the Kestra UI, you see the **Welcome** page.
+When you first open the UI, a **Welcome to Kestra** tour popup appears on the AI Copilot page.
-Click **Start Product Tour** to open the Kestra **Guided Tour**, which guides you through creating and executing your first flow step by step.
+Click **Start the tour** to walk through creating and executing your first flow.
-The UI also includes a **No Code editor** for building flows and dashboards visually without writing YAML — available from the Flows and Dashboard pages.
+The UI includes a **No Code editor** for building flows and dashboards visually without writing YAML — available from the Flows and Dashboard pages.
-The left menu includes the following pages:
+The left sidebar is organized into three sections:
-- The **Dashboards** page shows visualizations of flow execution data and metrics.
-- The **Flows** page lists all your flows, where you can create, edit, and execute them.
-- The **Executions** page lets you inspect and manage previous executions.
-- The **Logs** page shows all task logs from previous executions.
-- The **Namespaces** page lists all namespaces and lets you set namespace-level configurations.
-- The **Blueprints** page provides a catalog of ready-to-use flow examples.
-- The **Plugins** page provides a catalog of plugins you can use inside your flows.
-- The **Tenant** page provides a system overview, the full KV Store, triggers, and concurrency limits.
+**Workspace** — day-to-day work:
+- **Dashboards** — visualizations of flow execution data and metrics.
+- **AI Copilot** — generate and edit flows from a plain-language description.
+- **Flows** — browse, create, edit, and execute flows.
+- **Executions** — inspect and manage previous executions.
+- **Logs** — task logs from previous executions.
-[Kestra Enterprise Edition](../oss-vs-paid/index.md) adds the following pages to the UI:
+**Resources** — reusable assets and plugins:
+- **Namespaces** — namespace-level configuration, files, secrets, and variables.
+- **Plugins** — catalog of plugins available in your flows.
+- **Blueprints** — ready-to-use flow examples.
-- The **Apps** page lists your Apps and lets you create new ones.
-- The **Tests** page shows your flow unit tests where you can view, edit, and create assertions without creating executions.
-- The **Assets** page lets you manage reusable assets available to your flows and apps.
-- The **Tenant** page provides a system overview, KV Store, secrets and credentials, triggers, audit logs, concurrency limits, the Apps Catalog, and IAM.
-- The **Instance** page includes sections for Services, Versioned Plugins, tenant management, Worker Groups, Kill Switch, and Announcements.
+**Tenant** — instance-level configuration:
+- **System Overview** — instance health and usage metrics.
+- **KV Store** — key-value pairs available across flows.
+- **Triggers** — all active triggers across the tenant.
+- **MCP Servers** — registered Model Context Protocol servers.
+
+[Kestra Enterprise Edition](../oss-vs-paid/index.md) adds the following to the UI:
+
+- **Apps** (Workspace) — list and create Apps that expose flows as user-facing forms.
+- **Tests** (Workspace) — run flow unit tests and view assertions without creating executions.
+- **Assets** (Resources) — manage reusable assets available to flows and apps.
+- The **Tenant** section gains: **Policies**, **Secrets**, **Credentials**, **Audit Logs**, and **IAM**.
+- **Instance Owner** (sidebar footer, visible to Instance Owner users only) — instance-wide administration covering services, tenants, Worker Groups, Worker Queues, Kill Switch, Announcements, Versioned Plugins, and instance-level IAM.
+
+The sidebar is customizable. Go to **Settings → Preferences → Customize Sidebar** to reorder items or hide the ones you don't need.
+
+
+
+
+:::alert{type="warning"}
+**Start from Kestra 1.3.** This guide assumes you are already running the latest **1.3.x** release. If you are on an older version, complete the required metadata migrations before upgrading to 2.0.0:
+
+- [KV Store and Secrets metadata migration](../v1.1.0/kv-secrets-metadata-migration) (introduced in 1.1)
+- [Namespace Files metadata migration](../v1.2.0/namespace-file-migration) (introduced in 1.2)
+
+If you are upgrading directly from 1.0, the [LTS migration guide (1.0 → 1.3)](../v1.3.0/lts-migration) consolidates all required steps in one pass.
+:::
+
+## Before you start
+
+### Back up your database
+
+Take a full database backup before upgrading. Several 2.0 migrations are irreversible: in particular, the BasicAuth password rehash prevents rollback to 1.x.
+
+### Run database migrations (EE)
+
+**Enterprise Edition users must run database migrations manually before starting Kestra 2.0.** By default, Kestra EE refuses to start if pending migrations exist. Stop your 1.3 instance, then run the following using the 2.0 binary:
+
+```bash
+kestra migrate plan # preview what will run
+kestra migrate run # apply migrations
+```
+
+Open-source users do not need to do this manually; migrations run automatically on startup.
+
+See [Database Migrations](./database-migrations) for the full upgrade sequence, Docker Compose and Kubernetes examples, and configuration options.
+
+### Inventory your flows
+
+Before upgrading, scan your flow sources for constructs that are removed in 2.0:
+
+```bash
+grep -rl "pluginDefaults:" flows/
+grep -rl "ForEach\|ForEachItem" flows/
+grep -rl "conditions:" flows/
+grep -rl "preconditions:" flows/
+grep -rl "json(" flows/
+grep -rl "workerGroup.key" flows/
+```
+
+Fix each match using the guides below before restarting Kestra.
+
+## Migration tooling
+
+Two tools are available to reduce manual effort during the upgrade.
+
+### kestra-migrate CLI
+
+[`kestra-migrate`](https://github.com/kestra-io/kestra2-flow-migration) is a command-line tool that automatically rewrites v1.3 flow YAML to the v2.0 format. It covers 16 migration rules (type renames, property renames, auth restructuring, and more) and flags flows that use removed types requiring manual rewriting.
+
+**Install on macOS or Linux:**
+
+```bash
+curl -fsSL https://raw.githubusercontent.com/kestra-io/kestra2-flow-migration/main/install-scripts/install.sh | bash
+```
+
+**Check which flows need migration:**
+
+```bash
+kestra-migrate --check ./flows/
+```
+
+**Migrate flows to an output directory:**
+
+```bash
+kestra-migrate -o v2-flows/ ./flows/
+```
+
+Run `--check` first to scope the work. After `kestra-migrate` runs, use the guides below for the two patterns the CLI cannot handle automatically: the `json()` function and `pluginDefaults` replacement.
+
+### Claude Code migration skill
+
+The [`migrate-kestra-2`](https://github.com/kestra-io/agent-skills) Claude Code skill guides the full upgrade journey: pre-flight access audit, server upgrade (Docker Compose and Helm paths), CLI-first flow migration using `kestra-migrate`, and guided rewrites for patterns the CLI does not cover.
+
+**Install:**
+
+```bash
+git clone git@github.com:kestra-io/agent-skills.git
+ln -s $(pwd)/agent-skills/skills/migrate-kestra-2 ~/.claude/skills/migrate-kestra-2
+```
+
+Then open any Claude Code session and say: "migrate my Kestra instance to 2.0".
+
+## What changed in 2.0.0
+
+The changes below are grouped by the area they affect. Work through the categories relevant to your setup; not every change applies to every installation.
+
+### Flows
+
+Changes that affect how flows are written or behave at runtime. Review all of these before upgrading, regardless of edition.
+
+| Guide | Editions | What changed |
+|---|---|---|
+| [ForEach replaced by Loop](./foreach-loop) | OSS, EE | `ForEach` and `ForEachItem` are removed. Migrate all uses to the `Loop` task before upgrading; flows that reference either task fail to parse in 2.0. |
+| [Trigger conditions redesign](./trigger-conditions-redesign) | OSS, EE | The `conditions` list and `preconditions` block on triggers are removed. All trigger types now use a top-level `when` Pebble expression. |
+| [condition → when (Checks)](./checks-condition-renamed-when) | OSS, EE | The `condition` property on flow-level checks is renamed to `when`. |
+| [json() → fromJson()](./json-function-removed) | OSS, EE | The `json()` Pebble function is removed. Replace all calls with `fromJson()`; the signature is identical. |
+| [local.Delete recursive default](./local-delete-recursive-default) | OSS, EE | `io.kestra.plugin.fs.local.Delete` now defaults `recursive` to `false`. Flows deleting directories without setting `recursive` explicitly will stop removing subdirectory contents. |
+
+Two additional removals have no dedicated guide. The four tasks `io.kestra.plugin.core.execution.Count`, `Resume`, `trigger.Toggle`, and `log.Fetch` are removed from core; replace them with the equivalent tasks from `plugin-kestra`. The `CANCELED` execution state alias is also removed; replace with `CANCELLED` (double-L) in flow expressions, API clients, and any tooling that reads execution state.
+
+### Governance and plugins
+
+Changes to flow configuration, permissions, and plugin defaults. The SDK auth and `pluginDefaults` changes apply to all editions; RBAC is EE only.
+
+| Guide | Editions | What changed |
+|---|---|---|
+| [SDK authentication required for internal tasks](./sdk-authentication) | OSS, EE | Tasks that call the Kestra API internally (git sync tasks, and others) now require explicit credentials. Add an `auth` block to the task, configure a namespace/tenant default service account, or set `kestra.tasks.sdk.authentication` in your server config. |
+| [pluginDefaults removed](./plugin-defaults-removed) | OSS, EE | The `pluginDefaults` keyword is removed at all scopes. EE users migrate to Policies; OSS users inline task values manually. |
+| [pluginDefaults.forced removed from flows](./plugin-defaults-forced-removed) | OSS, EE | The `forced` property on flow-level `pluginDefaults` is removed. Use `Add` rules with `override: true` in a Policy (EE) or inline the values (OSS). |
+| [RBAC action model](./rbac-action-model) | EE | The CRUD-based permission model is replaced by a resource and action model. Existing roles are migrated automatically; CLI role definitions require updates. |
+
+### Data and storage
+
+Changes to execution data formats, API responses, and storage configuration.
+
+| Guide | Editions | What changed |
+|---|---|---|
+| [Execution API response](./execution-api-response) | OSS, EE | The execution endpoint response no longer includes task run outputs. Update any API consumers that read `taskRunList[*].outputs`. |
+| [ION binary format](./ion-binary-format) | OSS, EE | ION task output files are now written in binary format. Expressions that call `read()` on ION output and then perform string operations will fail; wrap with `fromIon()`. |
+| [Execution data in internal storage removed](./execution-data-internal-storage) | EE | The configuration option that stored task run outputs in internal storage is removed. Remove the affected keys from your configuration. |
+
+### Infrastructure
+
+Changes to deployment, Helm charts, and server configuration. Relevant primarily for operators and platform teams.
+
+| Guide | Editions | What changed |
+|---|---|---|
+| [Database migrations](./database-migrations) | OSS, EE | How to run the 2.0 schema migrations, including the upgrade sequence, Docker Compose and Kubernetes examples, and key scripts. |
+| [Helm gRPC worker-controller](./helm-grpc-worker-controller) | EE | Port 50051 is now exposed by default on all pods, and a dedicated controller deployment is added to the Helm chart. The `workerGroup.key` property is removed; migrate to `workerSelector.tags`. The `fallback` default changes from `WAIT` to `FAIL`. Review before upgrading. |
+| [Management endpoint hardening](./management-endpoint-hardening) | OSS, EE | Several management endpoints that were open without authentication in 1.x now require explicit opt-in. Review before upgrading in production. |
+
+### Terraform provider
+
+Changes to the official Terraform provider. Relevant for teams managing Kestra resources as Infrastructure as Code.
+
+| Guide | Editions | What changed |
+|---|---|---|
+| [Terraform provider](./terraform-provider) | EE, Cloud | The `permissions` block on `kestra_role` is renamed to `resources` with `actions` instead of CRUD verbs; `plugin_defaults` and `worker_group` are removed from `kestra_namespace`; `kestra_template` resources must be deleted. Pin to `~> 2.0`. |
+| [Super Admin renamed to Instance Owner](./superadmin-renamed-instance-owner) | EE, Cloud | The Super Admin privilege is renamed to Instance Owner. Deprecated CLI, config, and request-body aliases are retained. HTTP API responses emit `instanceOwner` instead of `superAdmin`; update API consumers that read this field. |
+
+## Completing the migration
+
+Migration is complete when the 1.3 instance has no flows remaining. Migrate them one at a time using this sequence:
+
+1. **Migrate dependencies first.** A flow will break at runtime if any of its dependencies are missing on the 2.0 instance. Before migrating a flow, ensure the following are in place:
+ - **Subflows** — migrate and validate any subflows the flow calls before migrating the parent. Removing a subflow from 1.3 while a parent flow still runs there will break in-flight executions.
+ - **Namespace files** — sync any namespace files the flow reads.
+ - **KV store** — migrate KV entries the flow reads at runtime.
+ - **Secrets** — confirm secrets are available in the 2.0 secret backend.
+2. **Validate on 2.0.** Trigger at least one test execution on the 2.0 instance and confirm it completes successfully.
+3. **Drain before removing.** Wait for any active executions of the flow on 1.3 to finish before deleting it. Removing a flow with running executions leaves those executions in an unresolvable state.
+4. **Activate triggers on 2.0 first.** If the flow has a schedule or polling trigger, confirm the trigger is active on 2.0 before removing the flow from 1.3; otherwise scheduled runs are silently dropped during the gap.
+5. **Delete from 1.3.** Once validated, drained, and triggers are live, delete the flow from the 1.3 instance.
\ No newline at end of file
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/ion-binary-format/index.md b/src/contents/docs/11.migration-guide/v2.0.0/ion-binary-format/index.md
new file mode 100644
index 00000000000..35a8b64a86b
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/ion-binary-format/index.md
@@ -0,0 +1,115 @@
+---
+title: ION Output Files Are Now Binary
+sidebarTitle: ION Binary Format
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["OSS", "EE"]
+description: Kestra 2.0 writes ION task output files in binary format. Expressions that call read() on an ION output and then perform string operations will fail. Wrap with fromIon() to fix.
+---
+
+Kestra 2.0 stores ION task output files in **binary format** instead of text format, reducing storage by ~20–40% and improving serialization speed. The `read()` Pebble function now returns `byte[]` instead of a String when it reads a binary ION file. Expressions that perform string operations directly on the result of `read()` — such as `contains`, string embedding in log messages, or raw text comparison — will fail or display binary data.
+
+:::alert{type="warning"}
+Any expression that calls `read()` on an ION task output URI and then performs string operations must be updated before upgrading to 2.0.0. Affected expressions silently produce incorrect results or throw type errors at runtime.
+:::
+
+## What changed
+
+Before 2.0, Kestra stored ION data as human-readable text. The `read()` function decoded every file to a UTF-8 `String`, so `{{ read(outputs.task.uri) contains 'value' }}` worked as expected.
+
+In 2.0, ION data is stored in binary format. The `read()` function detects binary ION by inspecting the file header and returns `byte[]` instead of `String`. String operations on `byte[]` either throw a type error or compare against raw bytes — neither produces the expected result.
+
+`fromIon()` now accepts both `String` and `byte[]`, so `{{ fromIon(read(outputs.task.uri)) }}` works correctly across both versions.
+
+## Affected expressions
+
+| Expression | 1.x | 2.0 | Action |
+|---|---|---|---|
+| `{{ read(outputs.x.uri) contains 'value' }}` | ✅ works | ❌ fails | Wrap with `fromIon()` |
+| `{{ read(outputs.x.uri) }}` in a log/message | ✅ human-readable text | ❌ binary bytes | Wrap with `fromIon()` |
+| `{{ fromIon(read(outputs.x.uri)) }}` | ✅ works | ✅ works | No change needed |
+| `{{ fromIon(read(outputs.x.uri), allRows=true) }}` | ✅ works | ✅ works | No change needed |
+| `{{ read(outputs.x.outputFiles['file.csv']) }}` | ✅ works | ✅ works | Not ION — no change |
+| `{{ read(outputs.x.outputFiles['file.json']) }}` | ✅ works | ✅ works | Not ION — no change |
+
+Only URIs pointing to ION-format files are affected. Files in CSV, JSON, XML, YAML, or any other non-ION format continue to be returned as `String` by `read()`, unchanged.
+
+Tasks that produce ION output URIs include: `FileTransform` (Nashorn, Groovy, Python, and other script runtimes), query tasks with `fetchType: FETCH` or `fetchType: FETCH_ONE`, `io.kestra.plugin.core.storage.Write` with `.ion` extension, `Split`, and `Concat` when producing `.ion` output.
+
+## Patterns and fixes
+
+### Assert conditions
+
+**Before**
+
+```yaml
+- id: assert
+ type: io.kestra.plugin.core.execution.Assert
+ conditions:
+ - "{{ read(outputs.transform.uri) contains 'jane' }}"
+```
+
+**After**
+
+```yaml
+- id: assert
+ type: io.kestra.plugin.core.execution.Assert
+ conditions:
+ - "{{ fromIon(read(outputs.transform.uri), allRows=true) | length == 1 }}"
+ - "{{ fromIon(read(outputs.transform.uri), allRows=true) | first | values | first contains 'jane' }}"
+```
+
+For simpler single-field checks, access the field directly on the deserialized row:
+
+```yaml
+conditions:
+ - "{{ fromIon(read(outputs.transform.uri)).name == 'jane' }}"
+```
+
+### Log messages with ION outputs
+
+**Before**
+
+```yaml
+- id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Row content: {{ read(outputs.transform.uri) }}"
+```
+
+**After**
+
+```yaml
+- id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Row content: {{ fromIon(read(outputs.transform.uri), allRows=true) }}"
+```
+
+### Checking row count
+
+**Before**
+
+```yaml
+condition: "{{ read(outputs.query.uri) | length > 0 }}"
+```
+
+**After**
+
+```yaml
+condition: "{{ fromIon(read(outputs.query.uri), allRows=true) | length > 0 }}"
+```
+
+## Backup compatibility
+
+Backups created by Kestra 2.0 are stored in binary ION format. Kestra 2.0 can restore backups created by 1.x (auto-detection handles the format difference). Kestra 1.x **cannot** restore backups created by 2.0. If you need to roll back to 1.x after upgrading, use a backup taken before the upgrade.
+
+## Existing stored files
+
+ION files already stored in internal storage from a 1.x instance are in text format. Kestra 2.0 reads both formats automatically — no data migration is needed for existing files.
+
+## Migration steps
+
+1. Search all flows for expressions matching `read(outputs.` that perform string operations: `contains`, direct string embedding in messages, or string comparison.
+2. For each match, confirm the task referenced by the output URI produces ION format (FileTransform, query FETCH tasks, Write with `.ion` extension).
+3. Replace `read(outputs.x.uri)` with `fromIon(read(outputs.x.uri))` to get the first row deserialized, or `fromIon(read(outputs.x.uri), allRows=true)` to get all rows as a list.
+4. Update Assert conditions, log messages, and any downstream expressions that operated on the raw string content.
+5. Validate updated flows in a 2.0 environment before promoting to production.
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/json-function-removed/index.md b/src/contents/docs/11.migration-guide/v2.0.0/json-function-removed/index.md
new file mode 100644
index 00000000000..1b7c2f66f86
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/json-function-removed/index.md
@@ -0,0 +1,28 @@
+---
+title: json() Function Removed
+sidebarTitle: json() → fromJson()
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["OSS", "EE"]
+description: The json() Pebble function has been removed in Kestra 2.0.0. Replace all calls to json() with fromJson() — the signature is identical.
+---
+
+The `json()` Pebble function has been removed in Kestra 2.0.0. Replace every call to `json(...)` with `fromJson(...)`. The function signature and behavior are identical.
+
+## Before
+
+```twig
+{{ json(outputs.request.body).products[0].id }}
+{{ json(kv('my_json_key')).field }}
+```
+
+## After
+
+```twig
+{{ fromJson(outputs.request.body).products[0].id }}
+{{ fromJson(kv('my_json_key')).field }}
+```
+
+## What to update
+
+Search your flows and templates for `json(` and replace each occurrence with `fromJson(`. The `json` Pebble test (`{% if x is json %}`) is unrelated and still works — only the function call form changes.
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/local-delete-recursive-default/index.md b/src/contents/docs/11.migration-guide/v2.0.0/local-delete-recursive-default/index.md
new file mode 100644
index 00000000000..8128b2e88e5
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/local-delete-recursive-default/index.md
@@ -0,0 +1,41 @@
+---
+title: local.Delete recursive Default Changed to false
+sidebarTitle: local.Delete recursive Default
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["OSS", "EE"]
+description: The recursive property of io.kestra.plugin.fs.local.Delete now defaults to false. Flows that delete a directory without setting recursive explicitly will stop removing subdirectory contents after upgrading.
+---
+
+The `recursive` property of `io.kestra.plugin.fs.local.Delete` now defaults to `false` instead of `true`.
+
+:::alert{type="warning"}
+Flows that call `io.kestra.plugin.fs.local.Delete` on a directory without setting `recursive` explicitly will silently stop deleting subdirectory contents after upgrading to 2.0.0. No error is raised — the task will succeed but leave nested files in place.
+:::
+
+## Why the change
+
+The previous default of `true` made directory deletions recursive without any explicit opt-in. A misconfigured `from` path could wipe an entire directory tree. The new default of `false` matches the behavior of every other `Delete` task in `plugin-fs` (SFTP, FTP, NFS, SMB) and requires an explicit opt-in for recursive deletion.
+
+## Migration steps
+
+1. Search all flows for tasks of type `io.kestra.plugin.fs.local.Delete`.
+2. For each task where `from` points to a directory and `recursive` is not set, add `recursive: true` to preserve the previous behavior.
+3. For tasks where `from` points to a single file, no change is needed — `recursive` has no effect on file targets.
+
+**Before** (recursive deletion happened implicitly)
+
+```yaml
+- id: cleanup
+ type: io.kestra.plugin.fs.local.Delete
+ from: /data/uploads/processed/
+```
+
+**After** (opt in to keep the same behavior)
+
+```yaml
+- id: cleanup
+ type: io.kestra.plugin.fs.local.Delete
+ from: /data/uploads/processed/
+ recursive: true
+```
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/management-endpoint-hardening/index.md b/src/contents/docs/11.migration-guide/v2.0.0/management-endpoint-hardening/index.md
new file mode 100644
index 00000000000..09094ee5e0e
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/management-endpoint-hardening/index.md
@@ -0,0 +1,65 @@
+---
+title: Management Endpoint Hardening
+sidebarTitle: Management Endpoint Hardening
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["OSS", "EE"]
+description: Kestra 2.0 tightens the default management endpoint configuration. Several endpoints that were open or writable without authentication in 1.x now require explicit opt-in.
+---
+
+Kestra 2.0 hardens the default configuration of the management port (`8081`). Several settings that were permissive in 1.x have been removed or reversed.
+
+## What changed
+
+| Setting | 1.x default | 2.0 default |
+|---|---|---|
+| `endpoints.all.sensitive` | `false` (all endpoints open) | Micronaut default (`true`) |
+| `endpoints.env.enabled` | `true` | `false` |
+| `endpoints.health.details-visible` | `ANONYMOUS` | `AUTHENTICATED` |
+| `endpoints.loggers.write-sensitive` | `false` (unauthenticated writes) | `true` |
+| `/worker` endpoint sensitivity | `false` (open) | `true` (sensitive) |
+| `/scheduler` endpoint sensitivity | `false` (open) | `true` (sensitive) |
+| docker-compose port mapping | `8081:8081` exposed | commented out |
+
+## Who is affected
+
+You are affected if any of the following apply:
+
+- You query `/env` on the management port (e.g., from a monitoring agent or automation script).
+- You call `POST /loggers` to change log levels at runtime without any authentication.
+- You poll `/worker` or `/scheduler` on the management port from tooling that does not send credentials.
+- You rely on anonymous health details at `/health`.
+- You use the `docker-compose.yml` bundled with Kestra to expose the management port.
+
+## How to restore the previous behavior
+
+If your environment relies on the old defaults, you can opt back into each setting individually in your `application.yml`:
+
+```yaml
+endpoints:
+ all:
+ sensitive: false # re-opens all endpoints for unauthenticated access
+ env:
+ enabled: true # re-enables the /env endpoint
+ health:
+ details-visible: ANONYMOUS # shows health details to unauthenticated requests
+ loggers:
+ write-sensitive: false # allows unauthenticated logger level changes
+ worker:
+ sensitive: false # re-opens the /worker endpoint
+ scheduler:
+ sensitive: false # re-opens the /scheduler endpoint
+```
+
+For the docker-compose setup, uncomment the management port mapping in `docker-compose.yml`:
+
+```yaml
+ports:
+ - "8081:8081"
+```
+
+:::alert{type="warning"}
+Re-opening these endpoints restores the previous behavior but also restores the associated security risks — unauthenticated access to environment variables, the ability to change log levels, and exposure of running task and trigger details. Only opt back in if you understand the implications and can restrict access at the network layer.
+:::
+
+See [Management endpoint access](../../../10.administrator-guide/security-hardening/index.md#management-endpoint-access) for the recommended hardening approach.
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/plugin-defaults-forced-removed/index.md b/src/contents/docs/11.migration-guide/v2.0.0/plugin-defaults-forced-removed/index.md
new file mode 100644
index 00000000000..a2a58bfc288
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/plugin-defaults-forced-removed/index.md
@@ -0,0 +1,55 @@
+---
+title: pluginDefaults.forced Removed from Flows
+sidebarTitle: pluginDefaults.forced Removed from Flows
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["OSS", "EE"]
+description: The forced property is removed from flow-level pluginDefaults in Kestra 2.0. In 2.0, pluginDefaults is removed entirely — migrate to Policies (EE) or inline task values (OSS).
+---
+
+The `forced` property is removed from flow-level `pluginDefaults` in Kestra 2.0.
+
+:::alert{type="warning"}
+Flows that include `forced: true` inside a `pluginDefaults` block fail to parse after upgrading to 2.0.0. Remove this property before upgrading.
+:::
+
+:::alert{type="info"}
+**`pluginDefaults` is removed entirely in 2.0.0** — not just `forced: true`. After removing `forced: true`, migrate the remaining `pluginDefaults` entries. See the full [pluginDefaults Removed migration guide](../plugin-defaults-removed/index.md).
+:::
+
+## Why the change
+
+`forced: true` in a flow's `pluginDefaults` let a flow author override any value a task explicitly set. This created a security problem: a regular user editing a flow could use `forced: true` to override plugin defaults that a platform administrator had configured at the namespace or tenant level.
+
+In 2.0, `pluginDefaults` is removed in favor of [Policies](../../../07.enterprise/02.governance/policies/index.md) (EE), which give platform administrators centralized, enforceable control over plugin configuration.
+
+## Migration steps
+
+1. Search all flows for `pluginDefaults` blocks that include `forced: true`:
+
+```bash
+grep -rl "forced:" flows/
+```
+
+2. Remove the `forced: true` line from each flow.
+
+**Before:**
+
+```yaml
+pluginDefaults:
+ - type: io.kestra.plugin.scripts.runner.docker.Docker
+ forced: true
+ values:
+ pullPolicy: NEVER
+```
+
+**After (as an interim step):**
+
+```yaml
+pluginDefaults:
+ - type: io.kestra.plugin.scripts.runner.docker.Docker
+ values:
+ pullPolicy: NEVER
+```
+
+3. Migrate the remaining `pluginDefaults` entries. See [pluginDefaults Removed](../plugin-defaults-removed/index.md) for the complete guide covering all scopes (flow-level, namespace-level, and global configuration) and the equivalent Policy DSL for Enterprise Edition.
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/plugin-defaults-removed/index.md b/src/contents/docs/11.migration-guide/v2.0.0/plugin-defaults-removed/index.md
new file mode 100644
index 00000000000..b7126f95b39
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/plugin-defaults-removed/index.md
@@ -0,0 +1,312 @@
+---
+title: pluginDefaults Removed
+sidebarTitle: pluginDefaults Removed
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["OSS", "EE"]
+description: The pluginDefaults keyword is removed in Kestra 2.0 at all scopes. Replace flow-level and namespace-level defaults with Policies in Enterprise Edition, or inline task values in OSS.
+---
+
+The `pluginDefaults` keyword is removed in Kestra 2.0 in all editions and at all scopes: flow level, namespace level (EE), and global server configuration. The older `taskDefaults` alias (used in Kestra versions before `pluginDefaults` was introduced) is also removed.
+
+:::alert{type="warning"}
+Flows that contain a `pluginDefaults` block fail to parse after upgrading to 2.0.0. Remove or migrate all `pluginDefaults` entries before upgrading.
+:::
+
+## What changed
+
+In Kestra 1.x, plugin defaults could be defined at three levels:
+
+| Before (1.x) | After (2.x) |
+|---|---|
+| Flow-level `pluginDefaults:` block | Values inlined onto tasks (OSS), or a `REFERENCE` Policy attached via `policyRefs:` (EE) |
+| Namespace-level Plugin Defaults (EE) | Namespace-scoped Policy (UI / API); existing namespace-level defaults are migrated automatically |
+| `kestra.plugins.defaults` in server config | Static policy under `kestra.policies` in server config |
+| `forced: false` (fill only when unset) | `Add` rule with `override: false` (the default) |
+| `forced: true` (policy value always wins) | `Add` rule with `override: true` |
+
+Tenant-scoped governance did not exist in 1.x. Kestra 2.0 introduces tenant-scoped Policies as a new capability — you can now apply defaults and enforcement rules across an entire tenant without touching individual namespaces.
+
+All `pluginDefaults` constructs are removed in 2.0.0. In **Enterprise Edition**, they are replaced by [Policies](../../../07.enterprise/02.governance/policies/index.md) — a governance layer that covers injection, enforcement, and validation in one place. Policies also extend `pluginDefaults` with the ability to target **flow-level properties** (`retry`, `concurrency`, `labels`) in addition to plugin properties. In **OSS**, there is no direct replacement and no automated migration tooling — move default values inline onto each task or use flow-level variables.
+
+Precedence is preserved. Policies apply along a scope chain `STATIC → INSTANCE → TENANT → NAMESPACE`:
+
+- **Non-overriding** (`override: false`) values fill properties the flow author left unset; the innermost scope wins — the same "task > flow > namespace > global" precedence as 1.x non-forced defaults.
+- **Overriding** (`override: true`) values replace the author's value; the outermost scope wins — the same "global beats namespace" precedence as 1.x forced defaults.
+
+## Before you migrate — build your inventory
+
+Collect every place plugin defaults are defined **before** upgrading. The 1.x UI for namespace and tenant defaults does not exist in 2.x.
+
+1. **Server configuration**: every entry under `kestra.plugins.defaults` in your `application.yml` or Helm values.
+2. **Namespace defaults** (EE): export them from each namespace's **Plugin Defaults** settings page, or via the 1.x API.
+3. **Flow-level blocks**: search your flow sources for `pluginDefaults:`:
+
+```bash
+grep -rl "pluginDefaults:" flows/
+```
+
+## Migration — Enterprise Edition
+
+Each `pluginDefaults` entry maps to an `Add` rule inside a Policy.
+
+### Step 1 — global server config → static policy
+
+Entries from `kestra.plugins.defaults` map to a static policy declared under `kestra.policies` in server configuration — not a tenant-scoped or namespace-scoped Policy. Static policies are cross-tenant, form the outermost scope, and are read-only through the API.
+
+**Before (`application.yml`):**
+
+```yaml
+kestra:
+ plugins:
+ defaults:
+ - type: io.kestra.plugin.scripts.shell.Commands
+ values:
+ containerImage: ubuntu:24.04
+ - type: io.kestra.plugin.aws
+ forced: true
+ values:
+ region: eu-west-1
+```
+
+**After (`application.yml`):**
+
+```yaml
+kestra:
+ policies:
+ instance-plugin-defaults:
+ description: "Migrated from kestra.plugins.defaults."
+ rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: EQUAL_TO
+ value: io.kestra.plugin.scripts.shell.Commands
+ values:
+ containerImage: ubuntu:24.04
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ override: true
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.aws
+ values:
+ region: eu-west-1
+```
+
+- `kestra.policies` is a map. The key (`instance-plugin-defaults` above) is the policy identity — do not add an `id` field inside the body.
+- A malformed static policy prevents server startup. Validate in staging first.
+
+### Step 2 — namespace-level Plugin Defaults → namespace Policy
+
+Recreate each entry as an `Add` rule in a namespace-scoped Policy, via the **Policies** UI or the API:
+
+```
+POST /api/v1/{tenant}/namespaces/{namespace}/policies
+```
+
+Namespace policies apply to the namespace and all its child namespaces, matching 1.x namespace-default inheritance.
+
+:::alert{type="info"}
+Existing namespace-level Plugin Defaults are migrated to Policies automatically during the 2.0 upgrade. Review the migrated Policies and adjust if needed.
+:::
+
+**Before (1.x namespace Plugin Default):**
+
+```yaml
+- type: io.kestra.plugin.aws
+ values:
+ accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
+ secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
+ region: "us-east-1"
+```
+
+**After (2.0 namespace-scoped Policy):**
+
+```yaml
+id: aws-credentials
+description: "Central AWS credentials for all AWS plugin tasks."
+enforcement: ACTIVE
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.aws
+ values:
+ accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
+ secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
+ region: "us-east-1"
+```
+
+Scope, tenant, and namespace come from the URL — not from the policy body.
+
+If multiple namespaces had identical Plugin Defaults, you can consolidate them into a single tenant-scoped Policy with a `target.namespaces` list instead of maintaining one policy per namespace:
+
+```yaml
+id: aws-credentials
+description: "Central AWS credentials — applies to analytics and data namespaces."
+enforcement: ACTIVE
+target:
+ namespaces:
+ - analytics
+ - data
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.aws
+ values:
+ accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
+ secretKeyId: "{{ secret('AWS_SECRET_ACCESS_KEY') }}"
+ region: "us-east-1"
+```
+
+Create this via `POST /api/v1/{tenant}/policies` (tenant level). The `target.namespaces` list uses ancestor-chain matching — listing `analytics` covers `analytics` and all its children. See [Policies](../../../07.enterprise/02.governance/policies/index.md#policy-scope-and-inheritance) for the full `target` reference.
+
+### Step 3 — flow-level pluginDefaults
+
+Flow-level `pluginDefaults` never supported `forced` in 1.x (it was stripped with a warning). Two migration options:
+
+**Option A — inline the values (recommended for one or two flows).** Copy each default's `values` onto the matching tasks and delete the `pluginDefaults:` block.
+
+**Option B — reference Policy (recommended when many flows share the same block).** Create a Policy with `enforcement: REFERENCE` at the namespace or tenant scope. Each flow opts in with `policyRefs:`.
+
+**Before (1.x flow):**
+
+```yaml
+id: daily-report
+namespace: company.team
+
+pluginDefaults:
+ - type: io.kestra.plugin.scripts.python.Script
+ values:
+ containerImage: ghcr.io/kestra-io/pydata:latest
+
+tasks:
+ - id: transform
+ type: io.kestra.plugin.scripts.python.Script
+ script: ...
+```
+
+**After — the shared reference Policy (created once, on `company.team`):**
+
+```yaml
+id: pydata-defaults
+description: "Python data science container image default."
+enforcement: REFERENCE
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: EQUAL_TO
+ value: io.kestra.plugin.scripts.python.Script
+ values:
+ containerImage: ghcr.io/kestra-io/pydata:latest
+```
+
+**After — the flow:**
+
+```yaml
+id: daily-report
+namespace: company.team
+
+policyRefs:
+ - pydata-defaults
+
+tasks:
+ - id: transform
+ type: io.kestra.plugin.scripts.python.Script
+ script: ...
+```
+
+A `REFERENCE` Policy applies only to flows that list it in `policyRefs`. `ACTIVE` Policies apply automatically to every flow in scope.
+
+### Forced defaults
+
+`forced: true` on a `pluginDefaults` entry becomes `override: true` on the corresponding `Add` rule.
+
+**Before:**
+
+```yaml
+pluginDefaults:
+ - type: io.kestra.plugin.scripts.runner.docker.Docker
+ forced: true
+ values:
+ pullPolicy: NEVER
+```
+
+**After:**
+
+```yaml
+id: docker-pull-policy
+description: "Force Docker pull policy to NEVER across all script tasks."
+enforcement: ACTIVE
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: EQUAL_TO
+ value: io.kestra.plugin.scripts.runner.docker.Docker
+ override: true
+ values:
+ pullPolicy: NEVER
+```
+
+## Migration — OSS
+
+OSS does not include Policies. There is no centralized replacement for `pluginDefaults` in OSS.
+
+Your options are:
+
+- **Inline values** — move the default values directly onto each task that used them.
+- **Flow variables** — define a variable at the flow level and reference it in each task with `{{ vars.myVariable }}`.
+- **Upgrade to Enterprise Edition** — use Policies for centralized, enforced defaults across namespaces.
+
+## Behavioral differences
+
+A few behaviors differ from 1.x `pluginDefaults`:
+
+- **Lists are replaced, not merged.** Map-valued properties deep-merge as before, but if a Policy injects a list (for example, a list of environment variables), it replaces the author's list entirely when `override: true`. Check any default whose `values` contain lists.
+- **Plugin aliases are not resolved.** 1.x resolved deprecated plugin aliases through the plugin registry. Policy conditions match the `type` string literally. If your flows use an alias, cover both the alias and the canonical name — or migrate all flows to the canonical name first.
+- **`EVALUATE` mode does not inject values.** Only `ACTIVE` and attached `REFERENCE` Policies mutate flows. Do not leave a migrated Policy in `EVALUATE` in production — tasks run without their former defaults.
+- **`Add` + `Delete` conflicts are rejected at save time.** 1.x had no `Delete` concept, so this only matters if you also adopt new validation rules. If one Policy injects a property and another removes it, saving any affected flow fails with an error citing both policies.
+
+## Verify the migration
+
+**Preview the effective policy chain for a flow:**
+
+```
+POST /api/v1/{tenant}/flows/policies/preview
+```
+
+Send the flow source; the response shows the mutated source with per-property attribution of which Policy injected what.
+
+**Dry-run a single policy against existing flows:**
+
+```
+GET /api/v1/{tenant}/policies/{id}/evaluate
+GET /api/v1/{tenant}/namespaces/{namespace}/policies/{id}/evaluate
+```
+
+**Smoke test**: run one execution per critical flow and inspect the task configuration used (container image, region, task runner) in the execution view.
+
+## Migration checklist
+
+1. Inventory all plugin defaults before upgrading — server config, namespace/tenant UI, and flow YAML files.
+2. Remove `kestra.plugins.defaults` from server config and add equivalent static policies under `kestra.policies`.
+3. For each namespace that had Plugin Defaults, review the auto-migrated Policies and adjust if needed.
+4. For each flow with a `pluginDefaults:` block, either inline the values onto tasks or create a reference Policy and add `policyRefs:` to the flow.
+5. Verify flows parse correctly: `kestra flow validate /path/to/flow.yml`
+6. Grant teams that managed namespace defaults the `POLICY` permission (`VIEW`, `CREATE`, `UPDATE`, `DELETE`) — it is separate from namespace edit rights.
+
+:::alert{type="info"}
+See [Policies](../../../07.enterprise/02.governance/policies/index.md) for the full Policy DSL reference, enforcement modes, inheritance behavior, and examples.
+:::
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/rbac-action-model/index.md b/src/contents/docs/11.migration-guide/v2.0.0/rbac-action-model/index.md
new file mode 100644
index 00000000000..fac8aabacd2
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/rbac-action-model/index.md
@@ -0,0 +1,152 @@
+---
+title: RBAC Permissions Replaced by Resource and Action Model
+sidebarTitle: RBAC Action Model
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["EE", "Cloud"]
+description: Kestra 2.0 replaces the CRUD-based permission model with a resource and action model. Existing roles are migrated automatically; this guide covers what changed, how migration maps old values, and how to update CLI role definitions.
+---
+
+Kestra 2.0 replaces the CRUD-based permission model (`READ`, `CREATE`, `UPDATE`, `DELETE`) with a resource and action model where each resource has its own set of specific actions.
+
+## What changed
+
+Previously, every resource shared the same four CRUD actions. Kestra 2.0 removes this uniform set and gives each resource its own actions that match what users actually do with it. For example:
+
+- `EXECUTION: READ` no longer exists. Instead, `EXECUTION` now has `VIEW`, `LIST`, `ACCESS_LOGS`, `ACCESS_OUTPUTS`, `ACCESS_FILES`, and `FOLLOW` as separate actions.
+- `FLOW: UPDATE` no longer covers enabling, disabling, or executing flows. Those are now `FLOW: ENABLE`, `FLOW: DISABLE`, and `FLOW: EXECUTE`.
+- Namespace files moved from `FLOW: READ/UPDATE` to `NAMESPACE: MANAGE_FILES`.
+- Triggers moved from `FLOW: READ` to their own `TRIGGER` resource.
+
+### Resources that were renamed or consolidated
+
+| Old resource | New resource | Notes |
+|---|---|---|
+| `SETTING` | `SYSTEM_SETTINGS` + `TENANT_SETTINGS` | Split into two resources |
+| `AI_COPILOT` | `COPILOT` | Renamed |
+| `APPEXECUTION` | `APP` | Merged; app execution actions moved onto `APP` |
+| `TENANT_ACCESS` | `USER` | Merged; tenant access actions moved onto `USER` |
+| `GROUP_MEMBERSHIP` | `GROUP` | Merged; membership actions (`MANAGE_MEMBERS`) moved onto `GROUP` |
+| `IMPERSONATE` | `USER: IMPERSONATE` | Converted from a standalone resource to an action on `USER` |
+
+### Resources that were removed
+
+| Removed resource | Reason |
+|---|---|
+| `TEMPLATE` | Templates were deprecated in an earlier release and are no longer permission-controlled |
+
+## Automatic migration
+
+Existing role permissions in the database are migrated automatically when you upgrade. You do not need to manually recreate roles. The database migration (`V2_1RolePermissionMigration`) runs on startup and expands each old `(resource, CRUD action)` pair into the equivalent new actions.
+
+:::alert{type="warning"}
+`TEMPLATE` permissions are silently dropped during migration; there is no equivalent resource in 2.0. If any of your roles had `TEMPLATE` permissions, review those roles after upgrading.
+
+`IMPERSONATE` permissions are also dropped; there is no direct migration path. Grant `USER: IMPERSONATE` explicitly on affected roles after upgrading.
+:::
+
+## CRUD → action migration mapping
+
+The table below shows exactly how the migration expands each old CRUD action. Actions not listed for a combination are not granted.
+
+| Resource (old) | CRUD action | New resource | New actions granted |
+|---|---|---|---|
+| `FLOW` | `READ` | `FLOW` | `VIEW`, `LIST`, `EXPORT` |
+| `FLOW` | `CREATE` | `FLOW` | `CREATE`, `IMPORT` |
+| `FLOW` | `UPDATE` | `FLOW` | `UPDATE`, `EXECUTE`, `DISABLE`, `ENABLE`, `VALIDATE` |
+| `FLOW` | `DELETE` | `FLOW` | `DELETE` |
+| `EXECUTION` | `READ` | `EXECUTION` | `VIEW`, `LIST`, `ACCESS_LOGS`, `ACCESS_OUTPUTS`, `ACCESS_FILES`, `EXPORT`, `FOLLOW` |
+| `EXECUTION` | `UPDATE` | `EXECUTION` | `UPDATE`, `RESTART`, `KILL`, `REPLAY`, `PAUSE`, `RESUME`, `CHANGE_LABELS`, `UNQUEUE`, `FORCE_RUN` |
+| `EXECUTION` | `DELETE` | `EXECUTION` | `DELETE` |
+| `TRIGGER` | `READ` | `TRIGGER` | `VIEW`, `LIST`, `EXPORT` |
+| `TRIGGER` | `UPDATE` | `TRIGGER` | `UNLOCK`, `RESTART`, `DISABLE`, `ENABLE`, `BACKFILL` |
+| `TRIGGER` | `DELETE` | `TRIGGER` | `DELETE` |
+| `NAMESPACE` | `READ` | `NAMESPACE` | `VIEW`, `LIST`, `EXPORT_PLUGIN_DEFAULTS` |
+| `NAMESPACE` | `UPDATE` | `NAMESPACE` | `UPDATE`, `MANAGE_FILES`, `IMPORT_PLUGIN_DEFAULTS` |
+| `NAMESPACE` | `CREATE` | `NAMESPACE` | `CREATE` |
+| `NAMESPACE` | `DELETE` | `NAMESPACE` | `DELETE` |
+| `APP` | `UPDATE` | `APP` | `UPDATE`, `EXECUTE`, `ACCESS_FILES`, `ACCESS_LOGS` |
+| `APP` | `CREATE` | `APP` | `CREATE` |
+| `APP` | `DELETE` | `APP` | `DELETE` |
+| `USER` | `READ` | `USER` | `VIEW`, `LIST` |
+| `USER` | `UPDATE` | `USER` | `UPDATE`, `MANAGE_GROUP_MEMBERSHIP` |
+| `USER` | `CREATE` | `USER` | `CREATE` |
+| `USER` | `DELETE` | `USER` | `DELETE` |
+| `GROUP` | `READ` | `GROUP` | `VIEW`, `LIST` |
+| `GROUP` | `UPDATE` | `GROUP` | `UPDATE`, `MANAGE_MEMBERS` |
+| `GROUP` | `CREATE` | `GROUP` | `CREATE` |
+| `GROUP` | `DELETE` | `GROUP` | `DELETE` |
+| `AUDITLOG` | `READ` | `AUDITLOG` | `VIEW`, `LIST`, `EXPORT` |
+| `AUDITLOG` | `CREATE` / `UPDATE` / `DELETE` | — | Not migrated (no equivalent actions) |
+| `SETTING` | `READ` | `SYSTEM_SETTINGS` + `TENANT_SETTINGS` | `VIEW` on each |
+| `SETTING` | `UPDATE` | `SYSTEM_SETTINGS` + `TENANT_SETTINGS` | `UPDATE` on each |
+| `SETTING` | `CREATE` / `DELETE` | — | Not migrated |
+| `AI_COPILOT` | `READ` | `COPILOT` | `USE` |
+| `AI_COPILOT` | `CREATE` / `UPDATE` / `DELETE` | — | Not migrated |
+| All other resources | `READ` | Same resource | `VIEW`, `LIST` |
+| All other resources | `CREATE` | Same resource | `CREATE` |
+| All other resources | `UPDATE` | Same resource | `UPDATE` |
+| All other resources | `DELETE` | Same resource | `DELETE` |
+
+## kestractl: updating role definitions
+
+The `kestractl roles create` and `kestractl roles update` commands use `--permission RESOURCE:ACTION[,ACTION]` syntax. The action values are now specific names like `VIEW`, `EXECUTE`, and `ACCESS_LOGS`, not CRUD verbs.
+
+:::alert{type="warning"}
+`READ` is still accepted as a legacy alias and maps to `VIEW` only; it does **not** expand to `VIEW` + `LIST` + `ACCESS_LOGS` + other actions the way the automatic database migration does. Using old CRUD values in new role definitions will silently under-grant permissions.
+:::
+
+Update your `kestractl` scripts to use the new action names:
+
+```bash
+# Before
+kestractl roles create --name operator \
+ --permission FLOW:READ,UPDATE \
+ --permission EXECUTION:READ,UPDATE
+
+# After
+kestractl roles create --name operator \
+ --permission FLOW:VIEW,LIST,EXECUTE,DISABLE,ENABLE \
+ --permission EXECUTION:VIEW,LIST,RESTART,KILL,ACCESS_LOGS,ACCESS_FILES,FOLLOW
+```
+
+If you use `--permissions-file`, update your YAML files in the same way:
+
+```yaml
+# Before
+FLOW:
+ - READ
+ - UPDATE
+EXECUTION:
+ - READ
+ - UPDATE
+
+# After
+FLOW:
+ - VIEW
+ - LIST
+ - EXECUTE
+ - DISABLE
+ - ENABLE
+EXECUTION:
+ - VIEW
+ - LIST
+ - RESTART
+ - KILL
+ - ACCESS_LOGS
+ - ACCESS_FILES
+ - FOLLOW
+```
+
+## Migration steps
+
+1. **Upgrade to Kestra 2.0** — the database migration runs automatically on startup and expands all existing role permissions.
+2. **Review roles that had `TEMPLATE` or `IMPERSONATE` permissions** — these are dropped and must be reconfigured manually.
+3. **Update `kestractl` scripts and permissions files** — replace old CRUD values with the new action names to ensure roles are created with the correct scope.
+4. **Verify access with impersonation** — use the impersonate feature in the Kestra UI to confirm each role behaves as expected after the upgrade.
+
+## Related
+
+- [RBAC overview](../../../07.enterprise/03.auth/rbac/index.md) — resources, actions, and managed roles in 2.0
+- [Permissions reference](../../../07.enterprise/03.auth/rbac/permissions-reference/index.md) — full resource-to-endpoint mapping for the new action model
+- [kestractl roles](../../../kestra-cli/kestractl/index.md#roles) — updating CLI role definitions to use the new action names
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/sdk-authentication/index.md b/src/contents/docs/11.migration-guide/v2.0.0/sdk-authentication/index.md
new file mode 100644
index 00000000000..fe360fdd3ee
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/sdk-authentication/index.md
@@ -0,0 +1,87 @@
+---
+title: SDK Authentication Required for Internal Tasks
+sidebarTitle: SDK Auth Required
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["OSS", "EE"]
+description: Tasks that call the Kestra API internally — such as git sync tasks — now require explicit credentials. In 1.3, these tasks ran without authentication.
+---
+
+In Kestra 2.0, tasks that call the Kestra API internally now use the Kestra SDK and require explicit credentials. In 1.3, these calls were unauthenticated. Affected tasks include `io.kestra.plugin.git.SyncFlows`, `io.kestra.plugin.git.NamespaceSync`, and `io.kestra.plugin.git.SyncNamespaceFiles`.
+
+Without credentials configured, the task fails with a 401 Unauthorized error when it tries to export or import data from the Kestra API:
+
+```
+Caused by: ApiException{code=401, ...}
+Failed to export flows from Kestra for namespace
+```
+
+## Configure credentials
+
+There are three ways to supply credentials, in recommended order.
+
+### 1. Namespace or tenant default (EE)
+
+Configure **Default authentication credentials** in the Kestra UI. Any SDK-based task running in that scope picks up the credentials automatically — no changes to individual flows are required.
+
+- **Tenant**: go to **Tenants → [tenant] → Settings** and scroll to **Default authentication credentials**. Set **Kestra URL** to the webserver address, then enter an API token or a username and password. This applies to all namespaces in the tenant.
+- **Namespace**: go to **Namespaces → [namespace] → Edit** and scroll to **Default authentication credentials**. Set **Kestra URL** and credentials here to override the tenant-level values for this namespace.
+
+The resolution order is: namespace default → tenant default → global config (below).
+
+The namespace and tenant defaults are replaced as whole objects, not merged field by field. If a namespace sets only **Kestra URL** with no credentials, it does not inherit the tenant's credentials — the task will be unauthenticated. Set all required fields (URL and at least one credential) at each level you configure.
+
+### 2. Global configuration fallback
+
+Add credentials to your server configuration. These apply to all SDK-based tasks across the instance that have no namespace or tenant default:
+
+```yaml
+kestra:
+ tasks:
+ sdk:
+ authentication:
+ url: "http://your-webserver:8080" # required in worker deployments; see note below
+ api-token: "${KESTRA_API_TOKEN}" # recommended: use a service account API token
+ # username: my-user # alternative: basic auth
+ # password: "${KESTRA_PASSWORD}"
+```
+
+**OSS:** if `kestra.server.basic-auth` is already configured, Kestra automatically derives the global SDK credentials from it — no additional configuration is needed.
+
+**Worker deployments:** in a setup where workers run separately from the webserver, the SDK call originates from the worker process and will by default target the worker's own host rather than the webserver. Set `url` to the webserver's address so SDK-based tasks route their API calls correctly. You can set this at any level: the global config above, the namespace or tenant default in the UI, or the task-level `auth` block.
+
+### 3. Inline auth on the task
+
+Add an `auth` block directly to the task. This is useful for one-off flows or when you need per-task credential isolation:
+
+```yaml
+tasks:
+ - id: sync
+ type: io.kestra.plugin.git.SyncFlows
+ targetNamespace: company.team
+ gitDirectory: _flows
+ url: https://github.com/your-org/your-repo
+ branch: main
+ auth:
+ url: "http://your-webserver:8080"
+ apiToken: "{{ secret('KESTRA_API_TOKEN') }}"
+ # username and password are also accepted instead of apiToken
+```
+
+## Affected tasks
+
+Any task that calls the Kestra API internally requires credentials. The affected plugin families are:
+
+- `io.kestra.plugin.git.*` — sync tasks such as `SyncFlows`, `SyncNamespaceFiles`, and `NamespaceSync`
+- `io.kestra.plugin.kestra.*` — Kestra SDK tasks such as `CreateCase`. These tasks resolve the webserver URL in this order: the task's `kestraUrl` property → the SDK auth `url` from config or namespace/tenant default → the internal `{{ kestra.url }}` variable. In a split worker/webserver deployment, set `kestraUrl` explicitly or configure `url` at the global/namespace/tenant level. Set `auto: false` on the task to opt out of all default SDK credentials and URL resolution.
+- `io.kestra.plugin.ai.*` — the `KestraFlow` tool
+
+If a task fails with a 401 error after upgrading, adding credentials is the fix.
+
+**Not affected:** Purge tasks (`io.kestra.plugin.core.execution.PurgeExecutions`, etc.) are now SystemTasks that run directly on the Executor and do not go through the SDK.
+
+## What to update
+
+1. Identify which flows use git sync or other tasks that call the Kestra API.
+2. Choose the credential method that fits your setup — namespace/tenant defaults require the fewest per-flow changes.
+3. For inline `auth`, store the token or password as a [secret](../../../07.enterprise/02.governance/05.secrets/index.md) and reference it with `{{ secret('...') }}`.
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/superadmin-renamed-instance-owner/index.md b/src/contents/docs/11.migration-guide/v2.0.0/superadmin-renamed-instance-owner/index.md
new file mode 100644
index 00000000000..d1d6b1d978b
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/superadmin-renamed-instance-owner/index.md
@@ -0,0 +1,70 @@
+---
+title: Super Admin Renamed to Instance Owner
+sidebarTitle: Super Admin → Instance Owner
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["EE", "Cloud"]
+description: The Super Admin privilege is renamed to Instance Owner in 2.0. Deprecated aliases are retained for the CLI and config; HTTP API responses emit instanceOwner instead of superAdmin.
+---
+
+Super Admin is renamed to Instance Owner in 2.0. The privilege is otherwise unchanged.
+
+Deprecated aliases are retained for the CLI, configuration, and HTTP request bodies. The only breaking change is **HTTP API responses**: the field `superAdmin` is renamed to `instanceOwner` with no alias. Update API consumers that read this field.
+
+## What changed
+
+### Terminology
+
+"Super Admin" and "Superadmin" become "Instance Owner" across the UI, docs, and API. The privilege is a boolean on a user or service account — not a role, not a binding, not tenant-scoped.
+
+### CLI (`kestra auths users`)
+
+| Old | New | Alias retained |
+|---|---|---|
+| `--superadmin` flag on `users create` | `--instance-owner` | Yes — `--superadmin` still works |
+| `set-superadmin` subcommand | `set-instance-owner` | Yes — `set-superadmin` still works |
+
+### Configuration
+
+| Old key | New key | Alias retained |
+|---|---|---|
+| `kestra.security.super-admin` | `kestra.security.instance-owner` | Yes — old key still works |
+
+Updated config structure:
+
+```yaml
+kestra:
+ security:
+ instance-owner:
+ username:
+ password:
+ tenant-admin-access:
+ -
+```
+
+### HTTP API
+
+Request bodies still accept `superAdmin` via a JSON alias — no changes required for callers creating or updating users.
+
+Responses now emit `instanceOwner` instead of `superAdmin`:
+
+```diff
+- if (user.superAdmin) { ... }
++ if (user.instanceOwner) { ... }
+```
+
+Deprecated HTTP paths under `/superadmin` still work but will be removed in a future major version.
+
+## What requires action
+
+1. **API consumers reading `superAdmin` from responses** — update to read `instanceOwner`. This is the only change without a deprecated alias.
+2. **Config using `kestra.security.super-admin`** — update to `kestra.security.instance-owner` before the alias is removed.
+3. **CLI scripts using `--superadmin` or `set-superadmin`** — update to `--instance-owner` and `set-instance-owner` before the aliases are removed.
+
+No database migration is required. Existing users and service accounts with the Instance Owner privilege are unaffected.
+
+## Related
+
+- [Instance Owner](../../../07.enterprise/05.instance/00.instance-owner/index.md) — privilege overview, creating and managing Instance Owners
+- [Security and Secrets configuration](../../../configuration/05.security-and-secrets/index.md) — full `kestra.security.instance-owner` reference
+- [kestra server CLI](../../../kestra-cli/kestra-server/index.md) — `--instance-owner` and `set-instance-owner` reference
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/terraform-provider/index.md b/src/contents/docs/11.migration-guide/v2.0.0/terraform-provider/index.md
new file mode 100644
index 00000000000..a5ef0564f99
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/terraform-provider/index.md
@@ -0,0 +1,318 @@
+---
+title: Terraform Provider Changes in Kestra 2.0
+sidebarTitle: Terraform provider
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["EE", "Cloud"]
+description: Breaking changes in the Kestra Terraform provider for 2.0 — role permissions renamed, plugin defaults removed, templates removed, and worker group changes.
+---
+
+The Kestra 2.0 Terraform provider introduces several breaking changes. Terraform state is not affected — resources migrate automatically in the database — but `.tf` files must be updated by hand before you apply against a 2.0 server.
+
+:::alert{type="warning"}
+**You cannot use the 1.3.x provider against a 2.0 server.** Pin to `~> 2.0` when upgrading:
+
+```hcl
+terraform {
+ required_providers {
+ kestra = {
+ source = "kestra-io/kestra"
+ version = "~> 2.0"
+ }
+ }
+}
+```
+:::
+
+## `kestra_role` — permissions renamed to resources and actions
+
+The `permissions` block is renamed to `resources`, and the inner `permissions` list is renamed to `actions`. CRUD verbs (`READ`, `CREATE`, `UPDATE`, `DELETE`) are replaced by per-resource action names.
+
+```hcl
+# Before
+resource "kestra_role" "operator" {
+ name = "Operator"
+
+ permissions {
+ type = "FLOW"
+ permissions = ["READ", "UPDATE"]
+ }
+
+ permissions {
+ type = "EXECUTION"
+ permissions = ["READ", "UPDATE"]
+ }
+
+ permissions {
+ type = "SETTING"
+ permissions = ["READ"]
+ }
+}
+
+# After
+resource "kestra_role" "operator" {
+ name = "Operator"
+
+ resources {
+ type = "FLOW"
+ actions = ["VIEW", "LIST", "EXECUTE", "DISABLE", "ENABLE"]
+ }
+
+ resources {
+ type = "EXECUTION"
+ # READ expands to: VIEW, LIST, ACCESS_LOGS, ACCESS_OUTPUTS, ACCESS_FILES, EXPORT, FOLLOW
+ # UPDATE expands to: UPDATE, RESTART, KILL, REPLAY, PAUSE, RESUME, CHANGE_LABELS, UNQUEUE, FORCE_RUN
+ actions = ["VIEW", "LIST", "ACCESS_LOGS", "ACCESS_OUTPUTS", "ACCESS_FILES", "EXPORT", "FOLLOW", "UPDATE", "RESTART", "KILL", "REPLAY", "PAUSE", "RESUME", "CHANGE_LABELS", "UNQUEUE", "FORCE_RUN"]
+ }
+
+ resources {
+ type = "SYSTEM_SETTINGS"
+ actions = ["VIEW"]
+ }
+
+ resources {
+ type = "TENANT_SETTINGS"
+ actions = ["VIEW"]
+ }
+}
+```
+
+### Resource type renames
+
+| Old type | New type | Notes |
+|---|---|---|
+| `SETTING` | `SYSTEM_SETTINGS` + `TENANT_SETTINGS` | Split into two resources; add a block for each |
+| `AI_COPILOT` | `COPILOT` | Renamed |
+| `TEST` | `TESTSUITE` | Renamed |
+| `APPEXECUTION` | `APP` | Merged; app execution actions are now on `APP` |
+
+### Blocks to remove
+
+| Old block | Action |
+|---|---|
+| `permissions { type = "TEMPLATE" ... }` | Delete — templates are removed in 2.0 |
+| `permissions { type = "IMPERSONATE" ... }` | Delete — impersonate is now an action on `USER`: add `"IMPERSONATE"` to your `USER` actions block instead |
+
+### CRUD → action mapping
+
+Use the [RBAC action model migration guide](../rbac-action-model) for the complete mapping of each CRUD verb to its new action names. The same mapping applies to Terraform `actions` values.
+
+:::alert{type="info"}
+Existing role permissions in the database are migrated automatically. You only need to update `.tf` files — `terraform apply` will show a plan that reflects the renamed attributes.
+:::
+
+## `kestra_namespace` — plugin_defaults and worker_group removed
+
+### `plugin_defaults` removed
+
+The `plugin_defaults` attribute is dropped from `kestra_namespace`. Namespace-level plugin defaults are migrated to Policies during the upgrade. Remove the attribute from your namespace resources:
+
+```hcl
+# Before
+resource "kestra_namespace" "company_team" {
+ namespace_id = "company.team"
+
+ plugin_defaults = <`, `TENANT//`, or `NAMESPACE///`.
+
+See [Policies](../../../07.enterprise/02.governance/policies/index.md) for available rule types and enforcement options.
+
+## Migration steps
+
+1. **Pin the provider** to `~> 2.0` in `required_providers`.
+2. **Update `kestra_role` resources** — rename `permissions` blocks to `resources`, rename the inner `permissions` list to `actions`, replace CRUD verbs with new action names, and update any renamed or removed resource types.
+3. **Remove `plugin_defaults`** from `kestra_namespace` resources. Add `kestra_policy` resources with `Add` rules as the replacement (EE).
+4. **Replace `worker_group`** on `kestra_namespace` with `default_worker_selector` using tags. Set `fallback` explicitly if you relied on the old `WAIT` default (the new default is `FAIL`).
+5. **Add `kestra_worker_queue` and `kestra_worker_group` resources** for any worker routing topology you previously managed through the UI.
+6. **Remove `pluginDefaults`** from flow content strings in `kestra_flow` resources.
+7. **Delete `kestra_template` resources** and data sources.
+8. **Run `terraform plan`** against the upgraded 2.0 server to review changes before applying.
+
+## Related
+
+- [RBAC action model](../rbac-action-model) — full CRUD-to-action mapping table
+- [pluginDefaults removed](../plugin-defaults-removed) — migration options for flow and namespace plugin defaults
+- [Policies](../../../07.enterprise/02.governance/policies/index.md) — the replacement for plugin defaults in EE
+- [Worker Groups](../../../07.enterprise/04.scalability/worker-group/index.md) — tag-based routing, capacity reservation, and JWT auth
+- [Helm gRPC worker-controller](../helm-grpc-worker-controller) — `workerGroup.key` removal and `fallback` default change
diff --git a/src/contents/docs/11.migration-guide/v2.0.0/trigger-conditions-redesign/index.md b/src/contents/docs/11.migration-guide/v2.0.0/trigger-conditions-redesign/index.md
new file mode 100644
index 00000000000..df46f48745a
--- /dev/null
+++ b/src/contents/docs/11.migration-guide/v2.0.0/trigger-conditions-redesign/index.md
@@ -0,0 +1,1028 @@
+---
+title: Trigger Conditions Redesign
+sidebarTitle: Trigger Conditions Redesign
+icon: /src/contents/docs/icons/migration-guide.svg
+release: 2.0.0
+editions: ["OSS", "EE"]
+description: The conditions list on triggers and the preconditions block on Flow triggers are removed in Kestra 2.0. All trigger types use a top-level when Pebble expression. Flow triggers also use dependsOn and window.
+---
+
+Kestra 2.0 replaces the `conditions` and `preconditions` system across all trigger types.
+
+- **All trigger types (Schedule, Webhook, HTTP, Flow, and others)** — the `conditions` list is removed in favor of a top-level `when` Pebble expression.
+- **Flow triggers** — both `conditions` and `preconditions` are removed in favor of `dependsOn` (upstream flow entries) and `window` (time window configuration).
+- **Flow trigger outputs** — scoped by flow ID: `trigger.outputs..`.
+- **Input rendering failures** — now create a `FAILED` execution instead of silently dropping the event.
+
+Both `conditions` and `preconditions` are removed in Kestra 2.0. Flows that still use them will fail to parse after upgrading.
+
+## `conditions` → `when` on all triggers
+
+All trigger types gain a top-level `when` property containing a Pebble expression. When the expression evaluates to `true`, the trigger fires; when `false`, it is skipped. This replaces the `conditions` list, which required a fully qualified Java type for every filtering need and did not compose cleanly across trigger types.
+
+### `when` expression context
+
+The variables available in a `when` expression depend on the trigger type:
+
+| Trigger type | Available variables |
+|---|---|
+| Schedule | `trigger.date`, `trigger.timestamp` |
+| Webhook | `trigger.body`, `trigger.headers` |
+| Flow | `namespace`, `flowId`, `state`, `labels`, `outputs`, `hasRetryAttempt` |
+
+:::alert{type="info"}
+**Schedule date skipping:** When a Schedule trigger has a `when` expression, the scheduler evaluates it against each candidate date. If `when` evaluates to `false`, the scheduler skips that date and advances to the next cron-matching date. This is the same behavior as the previous `conditions` on Schedule triggers; `when` controls which scheduled dates fire, not just whether a single date fires.
+:::
+
+### New Pebble helper functions
+
+These functions are introduced specifically for `when` expressions to replace verbose date formatting patterns:
+
+| Function | Signature | Description |
+|---|---|---|
+| `isPublicHoliday` | `isPublicHoliday(date, countryCode[, subDivision])` | Returns `true` if the date is a public holiday. Backed by Jollyday. Optional third argument for sub-divisions (e.g. `'IDF'`). |
+| `isDayWeekInMonth` | `isDayWeekInMonth(date, dayOfWeek, position)` | Returns `true` if the date is the Nth occurrence of a weekday in its month. `position` accepts `FIRST`, `SECOND`, `THIRD`, `FOURTH`, or `LAST`. |
+| `isWeekend` | `isWeekend(date)` | Returns `true` if the date falls on Saturday or Sunday. |
+| `isLastWorkingDay` | `isLastWorkingDay(date[, workingDays])` | Returns `true` if the date is the last working day of its month. Working days default to Monday–Friday. Optional second argument overrides which days count as working days. |
+| `dayOfWeek` | `dayOfWeek(date)` | Returns the day name as a string (`MONDAY`, `TUESDAY`, …, `SUNDAY`). |
+| `hourOfDay` | `hourOfDay(date)` | Returns the hour as an integer (0–23). |
+| `dayOfMonth` | `dayOfMonth(date)` | Returns the day of the month as an integer (1–31). |
+| `monthOfYear` | `monthOfYear(date)` | Returns the month as an integer (1–12). |
+
+Existing Pebble filters (`startsWith`, `endsWith`, `date`) and operators (`and`, `or`, `not`, `==`, `!=`, `>`, `<`, `>=`, `<=`) cover the remaining use cases.
+
+### Schedule: specific day of week
+
+**Before**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 9 * * *"
+ conditions:
+ - type: io.kestra.plugin.core.condition.DayWeek
+ dayOfWeek: MONDAY
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 9 * * *"
+ when: "{{ dayOfWeek(trigger.date) == 'MONDAY' }}"
+```
+
+### Schedule: weekends only
+
+**Before**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * *"
+ conditions:
+ - type: io.kestra.plugin.core.condition.Weekend
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * *"
+ when: "{{ isWeekend(trigger.date) }}"
+```
+
+### Schedule: weekdays only (exclude weekends)
+
+**Before**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 9 * * *"
+ conditions:
+ - type: io.kestra.plugin.core.condition.Not
+ conditions:
+ - type: io.kestra.plugin.core.condition.Weekend
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 9 * * *"
+ when: "{{ not isWeekend(trigger.date) }}"
+```
+
+### Schedule: exclude Sundays
+
+**Before**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 9 * * *"
+ conditions:
+ - type: io.kestra.plugin.core.condition.Not
+ conditions:
+ - type: io.kestra.plugin.core.condition.DayWeek
+ dayOfWeek: SUNDAY
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 9 * * *"
+ when: "{{ dayOfWeek(trigger.date) != 'SUNDAY' }}"
+```
+
+### Schedule: public holidays
+
+**Before**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * *"
+ conditions:
+ - type: io.kestra.plugin.core.condition.PublicHoliday
+ country: FR
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * *"
+ when: "{{ isPublicHoliday(trigger.date, 'FR') }}"
+```
+
+With a sub-division: `{{ isPublicHoliday(trigger.date, 'FR', 'IDF') }}`.
+
+### Schedule: workdays only (not weekend, not public holiday)
+
+**Before**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * *"
+ conditions:
+ - type: io.kestra.plugin.core.condition.Not
+ conditions:
+ - type: io.kestra.plugin.core.condition.PublicHoliday
+ country: FR
+ - type: io.kestra.plugin.core.condition.Weekend
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * *"
+ when: "{{ not isWeekend(trigger.date) and not isPublicHoliday(trigger.date, 'FR') }}"
+```
+
+### Schedule: first Monday of the month
+
+**Before**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * 1"
+ conditions:
+ - type: io.kestra.plugin.core.condition.DayWeekInMonth
+ dayOfWeek: MONDAY
+ dayInMonth: FIRST
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * 1"
+ when: "{{ isDayWeekInMonth(trigger.date, 'MONDAY', 'FIRST') }}"
+```
+
+### Schedule: date range
+
+**Before**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "*/5 * * * *"
+ conditions:
+ - type: io.kestra.plugin.core.condition.DateTimeBetween
+ after: "2025-12-31T23:59:59Z"
+ before: "2026-06-30T23:59:59Z"
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "*/5 * * * *"
+ when: "{{ trigger.date > '2025-12-31T23:59:59Z' and trigger.date < '2026-06-30T23:59:59Z' }}"
+```
+
+### Schedule: specific hours only
+
+**Before**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 * * * *"
+ conditions:
+ - type: io.kestra.plugin.core.condition.TimeBetween
+ after: "08:00:00"
+ before: "17:00:00"
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 * * * *"
+ when: "{{ hourOfDay(trigger.date) >= 8 and hourOfDay(trigger.date) < 17 }}"
+```
+
+### Schedule: combining multiple conditions
+
+**Before** (first Monday of the month, skip public holidays in France)
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * *"
+ conditions:
+ - type: io.kestra.plugin.core.condition.DayWeekInMonth
+ dayOfWeek: MONDAY
+ dayInMonth: FIRST
+ - type: io.kestra.plugin.core.condition.Not
+ conditions:
+ - type: io.kestra.plugin.core.condition.PublicHoliday
+ country: FR
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: schedule
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 11 * * *"
+ when: "{{ isDayWeekInMonth(trigger.date, 'MONDAY', 'FIRST') and not isPublicHoliday(trigger.date, 'FR') }}"
+```
+
+### Webhook: filter by body
+
+**Before**
+
+```yaml
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: 4wjtkzwVGBM9yKnjm3yv8r
+ conditions:
+ - type: io.kestra.plugin.core.condition.Expression
+ expression: "{{ trigger.body.hello == 'world' }}"
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: 4wjtkzwVGBM9yKnjm3yv8r
+ when: "{{ trigger.body.hello == 'world' }}"
+```
+
+### Webhook: filter by header and body
+
+**Before**
+
+```yaml
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: myKey
+ conditions:
+ - type: io.kestra.plugin.core.condition.Expression
+ expression: "{{ trigger.headers['X-Event-Type'] == 'deploy' }}"
+ - type: io.kestra.plugin.core.condition.Expression
+ expression: "{{ trigger.body.environment == 'production' }}"
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: webhook
+ type: io.kestra.plugin.core.trigger.Webhook
+ key: myKey
+ when: "{{ trigger.headers['X-Event-Type'] == 'deploy' and trigger.body.environment == 'production' }}"
+```
+
+Multiple `Expression` conditions combine into a single `when` expression using `and` / `or`.
+
+### What replaces what
+
+| Old condition type | New `when` expression |
+|---|---|
+| `DayWeek` (e.g. MONDAY) | `{{ dayOfWeek(trigger.date) == 'MONDAY' }}` |
+| `Weekend` | `{{ isWeekend(trigger.date) }}` |
+| `Not` > `Weekend` (weekdays only) | `{{ not isWeekend(trigger.date) }}` |
+| `Not` > `DayWeek` SUNDAY (exclude Sundays) | `{{ dayOfWeek(trigger.date) != 'SUNDAY' }}` |
+| `PublicHoliday` (country: FR) | `{{ isPublicHoliday(trigger.date, 'FR') }}` |
+| `Not` > `PublicHoliday` + `Weekend` (workdays) | `{{ not isWeekend(trigger.date) and not isPublicHoliday(trigger.date, 'FR') }}` |
+| `DayWeekInMonth` (MONDAY, FIRST) | `{{ isDayWeekInMonth(trigger.date, 'MONDAY', 'FIRST') }}` |
+| `DateTimeBetween` (after/before) | `{{ trigger.date > '2025-12-31T23:59:59Z' and trigger.date < '2026-06-30T23:59:59Z' }}` |
+| `TimeBetween` (08:00-17:00) | `{{ hourOfDay(trigger.date) >= 8 and hourOfDay(trigger.date) < 17 }}` |
+| `Expression` (custom Pebble) | Direct `when` expression, no wrapper needed |
+| `Expression` on webhook body/headers | `{{ trigger.body.field == 'value' }}` or `{{ trigger.headers['X-Key'] == 'value' }}` |
+| Multiple `Expression` conditions | Combined with `and` / `or` in a single `when` |
+
+For the full list of Pebble calendar helper functions (`isWeekend`, `isPublicHoliday`, `isDayWeekInMonth`, `isLastWorkingDay`, `hourOfDay`, etc.), see the [date and calendar helpers](../../../expressions/04.functions/06.dates/index.mdx) reference.
+
+## `conditions` and `preconditions` → `dependsOn` on Flow triggers
+
+Both `conditions` (execution-level types such as `ExecutionStatus`, `ExecutionFlow`, `ExecutionNamespace`) and `preconditions` (upstream flow lists with time windows) are replaced by a single `dependsOn` list. Each entry declares one upstream dependency with typed properties.
+
+### `dependsOn` entry properties
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `flowId` | string | — | Exact flow ID to match. Omit to match any flow. |
+| `namespace` | string | — | Exact namespace to match. Use `when` for prefix or pattern matching. |
+| `states` | list | all terminal states and `PAUSED` | Execution states that satisfy this entry. |
+| `labels` | map | — | Labels the upstream execution must carry (all must match). |
+| `when` | string | — | Pebble expression for additional filtering on the upstream execution context. |
+
+Both `flowId` and `namespace` use exact matching: `namespace: company.team` matches only `company.team`, not `company.team.project`. For prefix or pattern matching, use `when` with `startsWith` or `endsWith`.
+
+When no `states` are specified on a `dependsOn` entry, the trigger evaluates against all terminal states (`SUCCESS`, `WARNING`, `FAILED`, `KILLED`, `CANCELLED`, `RETRIED`, `SKIPPED`, `RESUBMITTED`) and `PAUSED`. Specify `states` explicitly to narrow the match.
+
+### Single upstream flow
+
+The `preconditions` block and the `conditions`-based approach both map to a single `dependsOn` entry.
+
+**Before (from `preconditions`)**
+
+```yaml
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ preconditions:
+ id: flows
+ flows:
+ - namespace: company.team
+ flowId: extract
+ states: [SUCCESS]
+```
+
+**Before (from `conditions`)**
+
+```yaml
+triggers:
+ - id: on_completion
+ type: io.kestra.plugin.core.trigger.Flow
+ states: [SUCCESS]
+ conditions:
+ - type: io.kestra.plugin.core.condition.ExecutionFlow
+ namespace: company.team
+ flowId: extract
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: extract
+ namespace: company.team
+ states: [SUCCESS]
+```
+
+### Multiple upstream flows with a deadline
+
+**Before**
+
+```yaml
+triggers:
+ - id: after_staging
+ type: io.kestra.plugin.core.trigger.Flow
+ preconditions:
+ id: staging_deps
+ timeWindow:
+ type: DAILY_TIME_DEADLINE
+ deadline: "09:00:00+01:00"
+ flows:
+ - namespace: company.team
+ flowId: stg_sales
+ states: [SUCCESS]
+ - namespace: company.team
+ flowId: stg_marketing
+ states: [SUCCESS]
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: after_staging
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: stg_sales
+ namespace: company.team
+ - flowId: stg_marketing
+ namespace: company.team
+ window:
+ deadline: "09:00:00"
+```
+
+`states` defaults to all terminal states and `PAUSED` when omitted. `window` moves to the trigger level. See [Window configuration](#window-configuration) for all window types and the `onMiss` property.
+
+### Multiple upstream flows (from `multipleConditions`)
+
+**Before**
+
+```yaml
+triggers:
+ - id: multiple_listen_flow
+ type: io.kestra.plugin.core.trigger.Flow
+ multipleConditions:
+ - id: multiple
+ window: P1D
+ windowAdvance: P0D
+ conditions:
+ flow_a:
+ type: io.kestra.plugin.core.condition.ExecutionFlow
+ namespace: company.team
+ flowId: multiplecondition_flow_a
+ flow_b:
+ type: io.kestra.plugin.core.condition.ExecutionFlow
+ namespace: company.team
+ flowId: multiplecondition_flow_b
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: multiple_listen_flow
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: multiplecondition_flow_a
+ namespace: company.team
+ states: [SUCCESS]
+ - flowId: multiplecondition_flow_b
+ namespace: company.team
+ states: [SUCCESS]
+ window:
+ every: P1D
+```
+
+The arbitrary string keys (`flow_a`, `flow_b`) are dropped; `dependsOn` is always a list. The `windowAdvance` property is removed with no direct equivalent.
+
+### Namespace-wide alerting (prefix matching)
+
+**Before**
+
+```yaml
+triggers:
+ - id: alert_on_failure
+ type: io.kestra.plugin.core.trigger.Flow
+ conditions:
+ - type: io.kestra.plugin.core.condition.ExecutionStatus
+ in:
+ - FAILED
+ - WARNING
+ - type: io.kestra.plugin.core.condition.ExecutionNamespace
+ namespace: company
+ comparison: PREFIX
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: alert_on_failure
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - states: [FAILED, WARNING]
+ when: "{{ namespace | startsWith('company') }}"
+```
+
+`namespace` in `dependsOn` is an exact match. Use `when` with `startsWith` for prefix matching.
+
+### Label-based filtering
+
+**Before**
+
+```yaml
+triggers:
+ - id: after_prod
+ type: io.kestra.plugin.core.trigger.Flow
+ conditions:
+ - type: io.kestra.plugin.core.condition.ExecutionStatus
+ in: [SUCCESS]
+ - type: io.kestra.plugin.core.condition.ExecutionLabels
+ labels:
+ env: production
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: after_prod
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - namespace: company.team
+ labels:
+ env: production
+ states: [SUCCESS]
+```
+
+### Conditional filtering with expressions
+
+**Before**
+
+```yaml
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ preconditions:
+ id: my_filter
+ where:
+ - id: flow1
+ filters:
+ - field: NAMESPACE
+ type: STARTS_WITH
+ value: io.kestra.tests
+ - field: EXPRESSION
+ type: IS_TRUE
+ value: "{{ labels.some == 'label' }}"
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - when: "{{ namespace | startsWith('io.kestra.tests') }}"
+ states: [SUCCESS]
+ labels:
+ some: label
+```
+
+`labels` handles exact key-value matching declaratively. `when` handles everything else.
+
+### Filtering on upstream execution outputs
+
+**Before**
+
+```yaml
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ conditions:
+ - type: io.kestra.plugin.core.condition.ExecutionOutputs
+ expression: "{{ outputs.row_count > 0 }}"
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: extract
+ namespace: company.team
+ when: "{{ outputs.row_count > 0 }}"
+```
+
+### Filtering on retry attempts
+
+**Before**
+
+```yaml
+triggers:
+ - id: after_flaky
+ type: io.kestra.plugin.core.trigger.Flow
+ conditions:
+ - type: io.kestra.plugin.core.condition.HasRetryAttempt
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: after_flaky
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: flaky_pipeline
+ namespace: company.team
+ states: [SUCCESS]
+ when: "{{ hasRetryAttempt == true }}"
+```
+
+### Negation: trigger on any state except SUCCESS
+
+**Before**
+
+```yaml
+triggers:
+ - id: on_non_success
+ type: io.kestra.plugin.core.trigger.Flow
+ conditions:
+ - type: io.kestra.plugin.core.condition.Not
+ conditions:
+ - type: io.kestra.plugin.core.condition.ExecutionStatus
+ in: [SUCCESS]
+```
+
+**After (option 1: explicit states)**
+
+```yaml
+triggers:
+ - id: on_non_success
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: extract
+ namespace: company.team
+ states: [FAILED, WARNING, KILLED, CANCELLED]
+```
+
+**After (option 2: `when` expression)**
+
+```yaml
+triggers:
+ - id: on_non_success
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: extract
+ namespace: company.team
+ when: "{{ state != 'SUCCESS' }}"
+```
+
+### Mixed triggers: success and failure on the same upstream flow
+
+**Before**
+
+```yaml
+triggers:
+ - id: on_completion
+ type: io.kestra.plugin.core.trigger.Flow
+ states: [SUCCESS]
+ conditions:
+ - type: io.kestra.plugin.core.condition.ExecutionFlow
+ namespace: company.team
+ flowId: flow_a
+ - id: on_failure
+ type: io.kestra.plugin.core.trigger.Flow
+ states: [FAILED]
+ preconditions:
+ id: flowsFailure
+ flows:
+ - namespace: company.team
+ flowId: flow_a
+ states: [FAILED]
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: on_completion
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: flow_a
+ namespace: company.team
+ states: [SUCCESS]
+ - id: on_failure
+ type: io.kestra.plugin.core.trigger.Flow
+ dependsOn:
+ - flowId: flow_a
+ namespace: company.team
+ states: [FAILED]
+```
+
+Same `dependsOn` syntax regardless of whether the original used `conditions` or `preconditions`.
+
+### Passing outputs downstream
+
+Flow trigger outputs are now scoped by flow ID. The path format is `trigger.outputs..`.
+
+**Before** (flat map, all upstream outputs merged together)
+
+```yaml
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ inputs:
+ date: "{{ trigger.outputs.date }}"
+ preconditions:
+ id: flows
+ flows:
+ - namespace: company.team
+ flowId: extract
+ states: [SUCCESS]
+```
+
+**After** (scoped by flow ID)
+
+```yaml
+triggers:
+ - id: after_extract
+ type: io.kestra.plugin.core.trigger.Flow
+ inputs:
+ date: "{{ trigger.outputs.extract.date }}"
+ dependsOn:
+ - flowId: extract
+ namespace: company.team
+```
+
+For multi-flow triggers, each upstream flow's outputs are accessed under its own key:
+
+```yaml
+dependsOn:
+ - flowId: stg_sales
+ namespace: company.team
+ - flowId: stg_marketing
+ namespace: company.team
+```
+
+Access as `{{ trigger.outputs.stg_sales.row_count }}` and `{{ trigger.outputs.stg_marketing.row_count }}`.
+
+:::alert{type="warning"}
+**Breaking change for multi-flow triggers.** Update all `trigger.outputs.` references to `trigger.outputs..`. For triggers with a single `dependsOn` entry, the unscoped form `{{ trigger.outputs. }}` still works as a shorthand; no update required.
+:::
+
+#### ForEachItem chain
+
+When using Flow triggers to chain `ForEachItem` child flows, reference the child flow's outputs using its `flowId`:
+
+**Before**
+
+```yaml
+triggers:
+ - id: 01_complete
+ type: io.kestra.plugin.core.trigger.Flow
+ inputs:
+ testFile: "{{ trigger.outputs.myFile }}"
+ preconditions:
+ id: output_01_success
+ flows:
+ - namespace: io.kestra.tests.trigger.foreachitem
+ flowId: flow-trigger-for-each-item-child
+ states: [SUCCESS]
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: 01_complete
+ type: io.kestra.plugin.core.trigger.Flow
+ inputs:
+ testFile: "{{ trigger.outputs.flow-trigger-for-each-item-child.myFile }}"
+ dependsOn:
+ - flowId: flow-trigger-for-each-item-child
+ namespace: io.kestra.tests.trigger.foreachitem
+```
+
+### `mode`: OR and N-of-M logic
+
+The `mode` property controls how `dependsOn` entries are combined when evaluating whether to fire.
+
+| Value | Behavior | Required properties |
+|---|---|---|
+| `ALL` (default) | Fires when all `dependsOn` entries are satisfied | — |
+| `ANY` | Fires as soon as any one entry is satisfied | — |
+| `AT_LEAST` | Fires when at least `minSatisfied` entries are satisfied | `minSatisfied` (integer ≥ 1, ≤ entry count) |
+
+#### OR logic: fire when any upstream completes
+
+Previously, OR logic required N separate Flow triggers. `mode: ANY` consolidates them into one.
+
+**Before** (two separate triggers)
+
+```yaml
+triggers:
+ - id: on_salesforce
+ type: io.kestra.plugin.core.trigger.Flow
+ conditions:
+ - type: io.kestra.plugin.core.condition.ExecutionFlow
+ namespace: company.sources
+ flowId: ingest_salesforce
+ - type: io.kestra.plugin.core.condition.ExecutionStatus
+ in: [SUCCESS]
+ - id: on_hubspot
+ type: io.kestra.plugin.core.trigger.Flow
+ conditions:
+ - type: io.kestra.plugin.core.condition.ExecutionFlow
+ namespace: company.sources
+ flowId: ingest_hubspot
+ - type: io.kestra.plugin.core.condition.ExecutionStatus
+ in: [SUCCESS]
+```
+
+**After**
+
+```yaml
+triggers:
+ - id: react_to_any_source
+ type: io.kestra.plugin.core.trigger.Flow
+ mode: ANY
+ dependsOn:
+ - flowId: ingest_salesforce
+ namespace: company.sources
+ states: [SUCCESS]
+ - flowId: ingest_hubspot
+ namespace: company.sources
+ states: [SUCCESS]
+```
+
+`mode: ANY` fires as soon as either dependency is satisfied. The default `mode: ALL` requires every entry to be satisfied before the trigger fires.
+
+#### OR logic with a time window
+
+```yaml
+triggers:
+ - id: daily_any_source
+ type: io.kestra.plugin.core.trigger.Flow
+ mode: ANY
+ dependsOn:
+ - flowId: ingest_salesforce
+ namespace: company.sources
+ - flowId: ingest_hubspot
+ namespace: company.sources
+ window:
+ deadline: "09:00:00"
+```
+
+Fire before 9 AM when either source completes.
+
+#### N of M: at least 2 out of 3
+
+```yaml
+triggers:
+ - id: partial_success
+ type: io.kestra.plugin.core.trigger.Flow
+ mode: AT_LEAST
+ minSatisfied: 2
+ dependsOn:
+ - flowId: ingest_salesforce
+ namespace: company.sources
+ states: [SUCCESS]
+ - flowId: ingest_hubspot
+ namespace: company.sources
+ states: [SUCCESS]
+ - flowId: ingest_zendesk
+ namespace: company.sources
+ states: [SUCCESS]
+ window:
+ deadline: "09:00:00"
+```
+
+`mode: AT_LEAST` fires when `minSatisfied` entries are satisfied. `minSatisfied` must be ≥ 1 and ≤ the number of `dependsOn` entries.
+
+### What replaces what
+
+| Old property / condition type | New equivalent |
+|---|---|
+| `conditions` list on Flow trigger | `dependsOn` list |
+| `preconditions` block | `dependsOn` list + `window` |
+| `multipleConditions` block | `dependsOn` list + `window.every` |
+| `ExecutionStatus` (`in: [SUCCESS]`) | `states: [SUCCESS]` on the `dependsOn` entry |
+| `ExecutionFlow` (`flowId`, `namespace`) | `flowId` + `namespace` on the `dependsOn` entry |
+| `ExecutionNamespace` (exact) | `namespace` on the `dependsOn` entry |
+| `ExecutionNamespace` (`comparison: PREFIX`) | `when: "{{ namespace \| startsWith('...') }}"` on the entry |
+| `ExecutionLabels` (`labels: {k: v}`) | `labels: {k: v}` on the `dependsOn` entry |
+| `ExecutionOutputs` (`expression`) | `when` with `outputs.` on the entry |
+| `HasRetryAttempt` | `when: "{{ hasRetryAttempt == true }}"` on the entry |
+| `Not` > `ExecutionStatus` | Explicit `states` list or `when: "{{ state != 'SUCCESS' }}"` |
+| Multiple triggers for OR logic | `mode: ANY` with `dependsOn` entries |
+| `preconditions.resetOnSuccess: true` | remove it, this is the only behavior in 2.0 |
+| `timeWindow.type: DAILY_TIME_DEADLINE` | `window.deadline` |
+| `timeWindow.type: DAILY_TIME_WINDOW` | `window.from` + `window.to` |
+| `timeWindow.type: DURATION_WINDOW` | `window.every` |
+| `timeWindow.type: SLIDING_WINDOW` | `window.lookback` |
+
+## Window configuration
+
+The `window` property applies to Flow triggers and controls how Kestra accumulates upstream executions before evaluating `dependsOn` entries. Set exactly one property group per window; combining groups is a validation error.
+
+| Window type | Properties | Behavior |
+|---|---|---|
+| Deadline | `deadline: "09:00:00"` | Upstream flows must complete by a fixed time each day |
+| Daily time range | `from: "06:00:00"` + `to: "12:00:00"` | Only executions within a daily time range count |
+| Fixed interval | `every: P1D` + optional `offset: PT6H` | Recurring window of a fixed size, offset from midnight |
+| Lookback | `lookback: PT1H` | Rolling window looking back from the current evaluation time |
+
+None of the window types changes how often the trigger fires. Once every `dependsOn` entry has been satisfied and an execution has been created, the stored results are reset, so every dependency has to be satisfied again before another execution is created.
+
+### Deadline
+
+```yaml
+window:
+ deadline: "09:00:00"
+```
+
+### Daily time range
+
+```yaml
+window:
+ from: "06:00:00"
+ to: "12:00:00"
+```
+
+### Fixed interval
+
+```yaml
+window:
+ every: P1D
+ offset: PT6H
+```
+
+### Lookback
+
+```yaml
+window:
+ lookback: PT1H
+```
+
+### Replacing `timeWindow` types
+
+| Old `timeWindow.type` | New `window` property |
+|---|---|
+| `DAILY_TIME_DEADLINE` | `deadline: "09:00:00"` |
+| `DAILY_TIME_WINDOW` | `from: "06:00:00"` + `to: "12:00:00"` |
+| `DURATION_WINDOW` | `every: P1D` + optional `offset: PT6H` |
+| `SLIDING_WINDOW` | `lookback: PT1H` |
+
+`preconditions.resetOnSuccess: true` can be removed, since resetting after firing is the only behavior in 2.0. `resetOnSuccess: false` has no equivalent; use `mode: ANY` if you want an execution to be created as soon as any single upstream flow succeeds.
+
+## Behavior changes after upgrading
+
+### Silent failures → FAILED executions
+
+Previously, if an expression on a Flow trigger failed to render (for example, because an upstream output key did not exist), the trigger silently dropped the event and no execution was created. In Kestra 2.0, a `FAILED` execution is created instead, making failures visible in the UI and actionable via downstream alerting.
+
+No migration action is required. Review your Flow trigger `inputs` expressions to ensure they reference valid output keys and avoid unexpected `FAILED` executions after upgrading.
+
+### State store reset and in-flight events
+
+Previously, auto-generated condition keys (`condition_1`, `condition_2`, …) meant that reordering entries could reset accumulated window state. In Kestra 2.0, `dependsOn` entry keys are derived from each entry's `namespace` and `flowId`, making them order-independent.
+
+The trigger-level state store key also changes: the old scheme used `preconditions.id`; the new scheme uses `{flowId}/{triggerId}`. Existing accumulated state from `preconditions` will not be found after upgrading; in-flight multi-flow triggers re-evaluate from scratch. For most deployments this means at most one missed trigger cycle.
+
+Old-format events in the async queue are discarded gracefully (logged as a warning). No user action is required.
+
+## Migration steps
+
+1. **Replace `conditions:` on all triggers** with a `when:` Pebble expression. This applies to Schedule, Webhook, HTTP, and any other trigger type that used `conditions`.
+2. **Replace `conditions:` and `preconditions:` on Flow triggers** with `dependsOn:` entries and (if applicable) `window:`.
+3. **Check `dependsOn` `states` values.** When omitted, `states` defaults to all terminal states and `PAUSED`. Add `states` explicitly on any entry that should match only specific states.
+4. **Update `trigger.outputs` references** in multi-flow triggers from `trigger.outputs.` to `trigger.outputs..`. Single-flow triggers can keep the unscoped form.
+5. **Update `timeWindow` to `window`** using the property mapping table above.
+6. **Validate** by saving updated flows in the Kestra UI or via the API and confirming they parse without errors.
diff --git a/src/contents/docs/13.terraform/resources/flow/index.md b/src/contents/docs/13.terraform/resources/flow/index.md
index 9d701c40599..2bb11069845 100644
--- a/src/contents/docs/13.terraform/resources/flow/index.md
+++ b/src/contents/docs/13.terraform/resources/flow/index.md
@@ -32,10 +32,8 @@ tasks:
message: first {{task.id}}
level: TRACE
-pluginDefaults:
- - type: io.kestra.core.tasks.log.Log
- values:
- message: third {{flow.id}}
+labels:
+ team: data
EOT
}
```
diff --git a/src/contents/docs/14.best-practices/0.flows/index.md b/src/contents/docs/14.best-practices/0.flows/index.md
index f6e81904e7c..47f55a92151 100644
--- a/src/contents/docs/14.best-practices/0.flows/index.md
+++ b/src/contents/docs/14.best-practices/0.flows/index.md
@@ -33,7 +33,7 @@ Depending on the internal queue and repository implementation, there may be a ha
While a flow can contain many tasks, it’s not recommended to include a large number of tasks within a single execution.
-A flow can contain either manually defined tasks or dynamically generated ones. While [ForEach](/plugins/core/flow/io.kestra.plugin.core.flow.foreach) and [ForEachItem](/plugins/core/flow/io.kestra.plugin.core.flow.foreachitem) are powerful for looping over results, they can create hundreds of TaskRuns if used on large datasets. For example, a nested loop of 20 × 20 tasks results in **400 TaskRuns**.
+A flow can contain either manually defined tasks or dynamically generated ones. While the [Loop task](/plugins/core/flow/io.kestra.plugin.core.flow.loop) runs each iteration as an isolated sub-execution, a deeply nested loop can still generate a large number of sub-executions. For example, a nested loop of 20 × 20 tasks results in **400 sub-executions**.
:::alert{type="warning"}
Flows with **over 100 tasks** tend to experience performance degradation and longer execution times.
@@ -48,7 +48,7 @@ While powerful, this feature **should not be used to transfer large amounts of d
For example, the [Query](/plugins/plugin-gcp/google-cloud-bigquery/io.kestra.plugin.gcp.bigquery.query) task in BigQuery has a `fetch` property that retrieves query results as an output attribute. If the query returns a large dataset, the result will be stored in the execution context — meaning it will be serialized and deserialized on each task state change, severely impacting performance.
-This feature is best suited for small datasets, such as querying a few rows to feed into a [Switch](/plugins/core/flow/io.kestra.plugin.core.flow.switch) or [ForEach](/plugins/core/flow/io.kestra.plugin.core.flow.foreach) task.
+This feature is best suited for small datasets, such as querying a few rows to feed into a [Switch](/plugins/core/flow/io.kestra.plugin.core.flow.switch) or [Loop](/plugins/core/flow/io.kestra.plugin.core.flow.loop) task.
:::alert{type="info"}
For large data volumes, use the `stores` property instead. Stored outputs are written to Kestra’s internal storage, and only the file URL is referenced in the execution context.
@@ -74,38 +74,16 @@ This helps prevent stalled executions and ensures resource efficiency.
## Flow trigger on state change
-Kestra can automatically start a flow as soon as another flow completes. This makes it easy to create dependencies between flows, even when they are owned by different teams. For example, a flow can trigger based on the `state` of another flow’s execution. There are multiple ways to configure this behavior, but one approach is recommended as a best practice.
-
-Take the following two triggers polling one specific flow: one using `preconditions.flows.states` to define the required `states` and the other using the `states` property.
-
-**Option 1**
-
-```yaml
-triggers:
- - id: release
- type: io.kestra.plugin.core.trigger.Flow
- preconditions:
- id: flows
- flows:
- - namespace: company.release
- flowId: parent
- states:
- - SUCCESS
-```
-
-or **Option 2**
+Kestra can automatically start a flow as soon as another flow completes. This makes it easy to create dependencies between flows, even when they are owned by different teams. Use `dependsOn` to declare the upstream flow and the required states:
```yaml
triggers:
- id: release
type: io.kestra.plugin.core.trigger.Flow
- states:
- - SUCCESS
- preconditions:
- id: flows
- flows:
- - namespace: company.release
- flowId: parent
+ dependsOn:
+ - namespace: company.release
+ flowId: parent
+ states: [SUCCESS]
```
-While both configurations will work, **Option 1** is the recommended approach. It is more performant and declarative compared to **Option 2**, especially when working with flow triggers dependent on state.
+`states` defaults to all terminal states and `PAUSED` when omitted. Declare it explicitly when you need a narrower set.
diff --git a/src/contents/docs/14.best-practices/1.from-dev-to-prod/index.md b/src/contents/docs/14.best-practices/1.from-dev-to-prod/index.md
index 029f742e612..7ece204f158 100644
--- a/src/contents/docs/14.best-practices/1.from-dev-to-prod/index.md
+++ b/src/contents/docs/14.best-practices/1.from-dev-to-prod/index.md
@@ -42,6 +42,7 @@ For open-source users, it’s recommended to run a **restricted production insta
Kestra supports several deployment strategies:
- [Via the UI](../../09.ui/01.flows/index.md)
+- [Promote](../../07.enterprise/02.governance/promote/index.md) (Enterprise Edition) — move flows between environments directly from the UI with a diff review and confirmation gate, no Git or pipeline required
- [Git synchronization](../../version-control-cicd/04.git/index.md)
- [CI/CD pipelines](../../version-control-cicd/cicd/index.md)
- [Terraform](../../13.terraform/index.mdx)
diff --git a/src/contents/docs/14.best-practices/10.credentials-vs-secrets-vs-kv-store/index.md b/src/contents/docs/14.best-practices/10.credentials-vs-secrets-vs-kv-store/index.md
index 51e043e3db4..262c4e0a70f 100644
--- a/src/contents/docs/14.best-practices/10.credentials-vs-secrets-vs-kv-store/index.md
+++ b/src/contents/docs/14.best-practices/10.credentials-vs-secrets-vs-kv-store/index.md
@@ -36,7 +36,7 @@ Use this rule of thumb:
| A value created by one flow and reused later by another flow | KV Store | It is designed for runtime state shared across flows |
| A value that changes during execution and must be updated programmatically | KV Store | Flows can read and write KV pairs dynamically |
| Sensitive material used by a credential, such as a client secret or private key | Secrets | Credentials should reference secrets rather than embed raw secret values |
-| A non-sensitive setting such as region, endpoint, or bucket name | Task properties, [variables](../../05.workflow-components/04.variables/index.md), or [plugin defaults](../../05.workflow-components/09.plugin-defaults/index.md) | These are configuration values, not authentication objects |
+| A non-sensitive setting such as region, endpoint, or bucket name | Task properties, [variables](../../05.workflow-components/04.variables/index.md), or [Policies](../../07.enterprise/02.governance/policies/index.md) (EE) | These are configuration values, not authentication objects |
## When to use credentials
@@ -135,12 +135,12 @@ This is the preferred pattern when Kestra should mint or refresh tokens for you.
### Pattern 2: Secrets plus non-sensitive configuration
-Use secrets for the confidential part and task properties, [variables](../../05.workflow-components/04.variables/index.md), or [plugin defaults](../../05.workflow-components/09.plugin-defaults/index.md) for the rest.
+Use secrets for the confidential part and task properties or [variables](../../05.workflow-components/04.variables/index.md) for the rest. In Enterprise Edition, use [Policies](../../07.enterprise/02.governance/policies/index.md) to inject repeated non-sensitive settings across all tasks of a type.
Example:
- `password` from `secret('DB_PASSWORD')`
-- `host`, `port`, and `database` from [variables](../../05.workflow-components/04.variables/index.md) or [plugin defaults](../../05.workflow-components/09.plugin-defaults/index.md)
+- `host`, `port`, and `database` from [variables](../../05.workflow-components/04.variables/index.md) or task properties
### Pattern 3: Secret plus KV Store
@@ -153,9 +153,9 @@ Example:
This is common in [polling triggers](../../05.workflow-components/07.triggers/04.polling-trigger/index.md), ingestion, and synchronization flows.
-### Pattern 4: Plugin defaults plus secrets
+### Pattern 4: Policies plus secrets (Enterprise Edition)
-Use [plugin defaults](../../05.workflow-components/09.plugin-defaults/index.md) to centralize repeated connection settings, while referencing secrets for the sensitive fields.
+Use [Policies](../../07.enterprise/02.governance/policies/index.md) to inject repeated connection settings into all tasks of a given type, while referencing secrets for the sensitive fields.
This is often the cleanest approach for large teams because it reduces duplication without putting secret material in the flow body.
@@ -186,7 +186,7 @@ If yes, consider the **KV Store**.
If yes, use **secrets**.
5. Is the value stable non-sensitive configuration reused across many tasks?
-If yes, consider [**plugin defaults**](../../05.workflow-components/09.plugin-defaults/index.md), [variables](../../05.workflow-components/04.variables/index.md), or [namespace-level configuration](../../07.enterprise/02.governance/07.namespace-management/index.md).
+If yes, consider [**Policies**](../../07.enterprise/02.governance/policies/index.md) (Enterprise Edition), [variables](../../05.workflow-components/04.variables/index.md), or task properties.
## Summary
@@ -198,5 +198,5 @@ In most cases, the right answer is not one feature alone, but a combination:
- credentials for token-based authentication
- secrets for sensitive inputs
-- [plugin defaults](../../05.workflow-components/09.plugin-defaults/index.md) or [variables](../../05.workflow-components/04.variables/index.md) for non-sensitive configuration
+- [Policies](../../07.enterprise/02.governance/policies/index.md) (Enterprise Edition) or [variables](../../05.workflow-components/04.variables/index.md) for non-sensitive configuration
- KV Store for changing state
diff --git a/src/contents/docs/14.best-practices/11.foreach-and-foreachitem/index.md b/src/contents/docs/14.best-practices/11.foreach-and-foreachitem/index.md
deleted file mode 100644
index df1bab59bad..00000000000
--- a/src/contents/docs/14.best-practices/11.foreach-and-foreachitem/index.md
+++ /dev/null
@@ -1,284 +0,0 @@
----
-title: "ForEach vs ForEachItem in Kestra: When to Use Each"
-h1: "ForEach vs ForEachItem: Scaling and Output Access"
-sidebarTitle: ForEach vs ForEachItem
-icon: /src/contents/docs/icons/best-practices.svg
-description: Learn when to use ForEach or ForEachItem in Kestra, how they scale differently, and how to access their outputs correctly in downstream tasks.
----
-
-Use `ForEach` and `ForEachItem` for different scaling and orchestration patterns.
-
-## Choose the right loop primitive
-
-Both tasks iterate over multiple items, but they do it in different ways:
-
-- `ForEach` creates child task runs inside the same execution.
-- `ForEachItem` creates one subflow execution per batch of items.
-
-That design difference affects performance, restart behavior, and how you access outputs.
-
-## Decision guide
-
-Use `ForEach` when:
-
-- You already have a small list in memory, such as an input, a small JSON array, or a small fetched result.
-- The work for each item is lightweight.
-- You want to share outputs between sibling tasks inside the loop.
-- You want a simple loop without introducing a subflow.
-
-Use `ForEachItem` when:
-
-- You need to process a large dataset or file.
-- You want to split data into batches and scale processing through subflows.
-- You need better isolation, troubleshooting, and restart behavior for individual batches.
-- The data already lives in Kestra internal storage, or can be written there first.
-
-:::alert{type="warning"}
-`ForEach` can generate many task runs in a single execution. For large fan-out or nested loops, prefer `ForEachItem` or a `Subflow`-based design to avoid oversized execution contexts and slower orchestration.
-:::
-
-:::alert{type="info"}
-`ForEachItem` expects `items` to be a Kestra internal storage URI, for example `{{ outputs.extract.uri }}` or a `FILE` input. If your source data is a regular JSON array, Excel file, Parquet file, or another non line-oriented format, convert it first.
-:::
-
-## `Subflow` vs `ForEachItem`
-
-`Subflow` and `ForEachItem` both create child executions, but they solve different orchestration problems.
-
-Use `Subflow` when:
-
-- You want to trigger one child flow once.
-- You already know the exact inputs to pass to that child flow.
-- You want execution isolation without batching or iteration.
-- You are decomposing a large workflow into smaller reusable modules.
-
-Use `ForEachItem` when:
-
-- You want to start many child flow executions from one dataset or file.
-- You need batching by `rows`, `partitions`, or `bytes`.
-- You want to process file-backed items incrementally at scale.
-- You want Kestra to merge outputs from multiple child executions.
-
-Rule of thumb:
-
-- `Subflow` is one child execution for one unit of work.
-- `ForEachItem` is many child executions for many units of work.
-
-For example, if you need to process one uploaded file in a dedicated child flow, use `Subflow`. If you need to split that file into many batches and process each batch in its own child flow execution, use `ForEachItem`.
-
-## Understand the main difference
-
-`ForEach` iterates over a list of values and exposes:
-
-- `{{ taskrun.value }}` for the current value
-- `{{ taskrun.iteration }}` for the zero-based loop index
-
-`ForEachItem` iterates over batches of file-backed items and exposes:
-
-- `{{ taskrun.items }}` for the current batch file URI
-- `{{ taskrun.iteration }}` for the zero-based batch index
-
-In practice:
-
-- `ForEach` is best when the iteration value itself is the thing you want to work with.
-- `ForEachItem` is best when each iteration should receive a file or batch and hand it off to a subflow.
-
-## Best practices for `ForEach`
-
-- Keep the `values` list small to moderate in size.
-- Use `concurrencyLimit` deliberately rather than leaving fan-out unbounded.
-- If each iteration needs multiple tasks in parallel, put a `Parallel` task inside the loop instead of expecting child tasks to run concurrently by default.
-- If iterating over JSON objects, remember that `taskrun.value` is a JSON string. Use `fromJson(taskrun.value)` to access properties.
-- When referencing outputs from sibling tasks inside the same loop iteration, use `outputs.task_id[taskrun.value]`.
-
-### Example: use sibling outputs correctly inside `ForEach`
-
-```yaml
-id: foreach_outputs
-namespace: company.team
-
-tasks:
- - id: enrich_regions
- type: io.kestra.plugin.core.flow.ForEach
- values: ["north", "south", "west"]
- concurrencyLimit: 2
- tasks:
- - id: metadata
- type: io.kestra.plugin.core.output.OutputValues
- values:
- region: "{{ taskrun.value }}"
- bucket: "landing-{{ taskrun.value }}"
-
- - id: build_message
- type: io.kestra.plugin.core.debug.Return
- format: "Load {{ outputs.metadata[taskrun.value].values.region }} into {{ outputs.metadata[taskrun.value].values.bucket }}"
-
- - id: log_one_result
- type: io.kestra.plugin.core.log.Log
- message: "{{ outputs.build_message['north'].value }}"
-```
-
-Why this pattern works:
-
-- Inside the loop, `outputs.metadata[taskrun.value]` reads the output from the current iteration.
-- Outside the loop, `outputs.build_message['north'].value` reads the output for one specific loop value.
-
-### Example: iterate over JSON objects safely
-
-```yaml
-id: foreach_json
-namespace: company.team
-
-tasks:
- - id: process_users
- type: io.kestra.plugin.core.flow.ForEach
- values:
- - {"id": 101, "email": "a@example.com"}
- - {"id": 102, "email": "b@example.com"}
- tasks:
- - id: log_user
- type: io.kestra.plugin.core.log.Log
- message: "User {{ fromJson(taskrun.value).id }} -> {{ fromJson(taskrun.value).email }}"
-```
-
-## Best practices for `ForEachItem`
-
-- Store the dataset in internal storage first and pass its URI to `items`.
-- If your source file is CSV, JSON, Excel, or another external format, convert it to ION before passing it to `ForEachItem`.
-- Batch by `rows`, `partitions`, or `bytes` based on how the downstream subflow processes data.
-- Design the subflow so it can be rerun independently for one batch.
-- Prefer passing `taskrun.items` to a `FILE` input in the subflow.
-- If the parent flow must depend on child results, keep `wait: true`.
-- If a child failure should fail the parent task, keep `transmitFailed: true`.
-
-### Example: process a file in batches with `ForEachItem`
-
-This pattern is recommended when each batch should run in its own execution.
-
-```yaml
-id: parent_foreachitem
-namespace: company.team
-
-tasks:
- - id: download_orders_csv
- type: io.kestra.plugin.core.http.Download
- uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
-
- - id: orders_to_ion
- type: io.kestra.plugin.serdes.csv.CsvToIon
- from: "{{ outputs.download_orders_csv.uri }}"
-
- - id: process_batches
- type: io.kestra.plugin.core.flow.ForEachItem
- items: "{{ outputs.orders_to_ion.uri }}"
- batch:
- rows: 2
- namespace: company.team
- flowId: process_order_batch
- wait: true
- transmitFailed: true
- inputs:
- orders_file: "{{ taskrun.items }}"
-
- - id: log_merged_outputs_uri
- type: io.kestra.plugin.core.log.Log
- message: "{{ outputs.process_batches_merge.subflowOutputs }}"
-
- - id: preview_merged_outputs
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(outputs.process_batches_merge.subflowOutputs) }}"
-```
-
-And the subflow:
-
-```yaml
-id: process_order_batch
-namespace: company.team
-
-inputs:
- - id: orders_file
- type: FILE
-
-tasks:
- - id: inspect_batch
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(inputs.orders_file) }}"
-
-outputs:
- - id: batch_summary
- type: STRING
- value: "{{ 'Processed batch content: ' ~ read(inputs.orders_file) }}"
-```
-
-Here, `orders_file` is a batch file generated from the ION output of `CsvToIon`. Each subflow execution receives one batch file through `{{ taskrun.items }}`.
-
-## Use `ForEachItem` outputs correctly
-
-`ForEachItem` is best consumed through its internal helper task outputs:
-
-- `{{ outputs.task_id_split.splits }}` contains the file listing generated batch URIs.
-- `{{ outputs.task_id_merge.subflowOutputs }}` contains a file with the merged outputs from the child subflows.
-
-If your `ForEachItem` task id is `process_batches`, those become:
-
-- `{{ outputs.process_batches_split.splits }}`
-- `{{ outputs.process_batches_merge.subflowOutputs }}`
-
-This is different from `ForEach`, where you typically access outputs by loop value, such as `outputs.inner['north'].value`.
-
-### Example: consume merged subflow outputs
-
-If the subflow defines typed flow outputs, `ForEachItem` merges them into a file exposed by the internal merge task. In the example above, each child execution returns a `batch_summary` string, and the merge task gathers those subflow outputs into a single file.
-
-```yaml
-id: parent_read_merged_outputs
-namespace: company.team
-
-tasks:
- - id: download_orders_csv
- type: io.kestra.plugin.core.http.Download
- uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
-
- - id: orders_to_ion
- type: io.kestra.plugin.serdes.csv.CsvToIon
- from: "{{ outputs.download_orders_csv.uri }}"
-
- - id: process_batches
- type: io.kestra.plugin.core.flow.ForEachItem
- items: "{{ outputs.orders_to_ion.uri }}"
- batch:
- rows: 2
- namespace: company.team
- flowId: process_order_batch
- wait: true
- transmitFailed: true
- inputs:
- orders_file: "{{ taskrun.items }}"
-
- - id: log_merged_outputs_uri
- type: io.kestra.plugin.core.log.Log
- message: "{{ outputs.process_batches_merge.subflowOutputs }}"
-
- - id: preview_merged_outputs
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(outputs.process_batches_merge.subflowOutputs) }}"
-```
-
-Use `{{ outputs.process_batches_merge.subflowOutputs }}` when a downstream task needs the collected outputs from all child subflows.
-If you want to inspect the merged file content directly, use `read(outputs.process_batches_merge.subflowOutputs)`.
-
-## Common mistakes to avoid
-
-- Do not use `ForEach` for very large datasets just because the input started as a JSON array.
-- Do not pass a non-storage path or raw inline content to `ForEachItem.items`; it must be a Kestra internal storage URI.
-- Do not assume sibling task outputs in `ForEach` use the plain `outputs.task_id.value` syntax; inside the loop, use `outputs.task_id[taskrun.value]`.
-- Do not expect `ForEach` child tasks to run in parallel unless you either set loop concurrency or add a `Parallel` task inside the loop.
-- Do not forget that `taskrun.iteration` starts at `0` for both `ForEach` and `ForEachItem`.
-
-## Recommended rule of thumb
-
-Use `ForEach` for orchestration over a relatively small list of values.
-
-Use `ForEachItem` for data processing over file-backed items or batches, especially when you need scale, restartability, or subflow isolation.
-
-For API details, see the [ForEach plugin documentation](/plugins/core/flow/io.kestra.plugin.core.flow.foreach), the [ForEachItem plugin documentation](/plugins/core/flow/io.kestra.plugin.core.flow.foreachitem), and the [Outputs documentation](../../05.workflow-components/06.outputs/index.md).
diff --git a/src/contents/docs/14.best-practices/11.loop/index.md b/src/contents/docs/14.best-practices/11.loop/index.md
new file mode 100644
index 00000000000..9d9348425d9
--- /dev/null
+++ b/src/contents/docs/14.best-practices/11.loop/index.md
@@ -0,0 +1,315 @@
+---
+title: "Loop Task Best Practices in Kestra"
+h1: "Best Practices for the Loop Task"
+sidebarTitle: Loop
+icon: /src/contents/docs/icons/best-practices.svg
+description: Best practices for using the Loop task in Kestra — output collection, concurrency, map-reduce patterns, error handling, large-file processing, and subflow isolation.
+---
+
+Use `Loop` for all iteration needs in Kestra.
+
+## Choose the right iteration pattern
+
+`Loop` runs child tasks for each item in a list, map, file, or URI list. Every iteration is an isolated sub-execution.
+
+Use a plain `Loop` when:
+
+- You need to run the same tasks for each item in a list or dataset.
+- Each iteration should process one value, one object, or one file chunk.
+- You want parallel iteration with controlled concurrency.
+- You want per-iteration failure handling without stopping the entire loop.
+
+Use `Loop` + `Subflow` when:
+
+- You need full execution isolation per batch — own retries, own logs, own failure state.
+- Each batch should be independently restartable.
+
+## Access the iteration value
+
+Inside a Loop, use `item.value` for the current value and `item.index` for the zero-based position. These are available in every child task, including those nested inside `If`, `Parallel`, or other flowable tasks — no parent traversal needed.
+
+```yaml
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["north", "south", "west"]
+ tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "region={{ item.value }} index={{ item.index }}"
+```
+
+When iterating over JSON objects, `item.value` is a JSON string. Use `fromJson(item.value).field` to access properties — `item.value.field` does not work.
+
+```yaml
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values:
+ - {"id": 101, "email": "a@example.com"}
+ - {"id": 102, "email": "b@example.com"}
+ tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "User {{ fromJson(item.value).id }} -> {{ fromJson(item.value).email }}"
+```
+
+## Expose outputs explicitly
+
+Task outputs inside a loop are not visible outside it by default. Declare an `outputs:` block on the Loop task to surface values. Choose `fetchType` based on data volume:
+
+- `AUTO` — default, switches automatically based on whether `values` is a URI
+- `FETCH` — collects all iteration results inline (suitable for small iteration counts)
+- `STORE` — writes results to internal storage and exposes a URI (preferred for large iteration counts)
+
+After the loop:
+- `outputs..outputs` is a list of per-iteration results
+- `outputs..outputs[n].outputs.` accesses a specific iteration by index
+- `loopOutputs(outputs..outputs, '')` extracts one field across all iterations as a flat list
+
+### Example: collect outputs and read them downstream
+
+```yaml
+id: loop_outputs
+namespace: company.team
+
+tasks:
+ - id: enrich_regions
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["north", "south", "west"]
+ concurrencyLimit: 2
+ fetchType: AUTO
+ outputs:
+ - id: bucket
+ type: STRING
+ value: "landing-{{ item.value }}"
+ - id: message
+ type: STRING
+ value: "{{ outputs.build_message.value }}"
+ tasks:
+ - id: build_message
+ type: io.kestra.plugin.core.debug.Return
+ format: "Load {{ item.value }} into landing-{{ item.value }}"
+
+ - id: log_first
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ outputs.enrich_regions.outputs[0].outputs.message }}"
+
+ - id: log_all
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ loopOutputs(outputs.enrich_regions.outputs, 'message') }}"
+```
+
+Inside the loop, sibling task outputs are accessed with plain `outputs.task_id.attribute` syntax — each iteration runs in its own isolated context, so there is no ambiguity.
+
+## Use `concurrencyLimit` deliberately
+
+- `1` (default) — sequential execution
+- A positive integer — bounded parallelism; prefer this for heavy workloads
+- `0` — unlimited; all iterations run simultaneously; avoid for large datasets unless you understand the resource implications
+
+## Process large files with Split and Loop
+
+For file-backed datasets, use `Split` to break the file into chunk URIs, then loop over the URI list. Each `item.value` is one chunk URI.
+
+Passing `values: "{{ outputs.split.uris }}"` where `outputs.split.uris` is a **list** is different from passing a single file URI string. A list iterates over elements; a single URI string iterates line-by-line through that file.
+
+```yaml
+tasks:
+ - id: split
+ type: io.kestra.plugin.core.storage.Split
+ from: "{{ inputs.file }}"
+ rows: 100
+
+ - id: per_chunk
+ type: io.kestra.plugin.core.flow.Loop
+ values: "{{ outputs.split.uris }}"
+ concurrencyLimit: 4
+ fetchType: FETCH
+ outputs:
+ - id: result_uri
+ type: STRING
+ value: "{{ outputs.process.value }}"
+ tasks:
+ - id: process
+ type: io.kestra.plugin.core.debug.Return
+ format: "processed chunk {{ item.index }}: {{ item.value }}"
+
+ - id: summary
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ loopOutputs(outputs.per_chunk.outputs, 'result_uri') }}"
+```
+
+## Map-reduce: collect and aggregate file outputs across iterations
+
+Use the map-reduce pattern when each iteration produces a file output that you need to combine and reduce after the loop.
+
+The structure is always the same:
+
+1. **Split** — break the input into chunk URIs
+2. **Loop** — process each chunk in parallel; declare a file URI as a `STRING` output
+3. **Concat** — stitch the per-iteration output files into one
+4. **Aggregate** (or any reduce task) — merge results across chunks
+
+The key is declaring the per-iteration output URI as a `STRING` in the Loop `outputs:` block, then using `loopOutputs()` to extract that list and pass it directly to `Concat`.
+
+```yaml
+id: elt_csv_split_loop_aggregate
+namespace: company.team
+
+tasks:
+ - id: download
+ type: io.kestra.plugin.core.http.Download
+ uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
+
+ - id: to_ion
+ type: io.kestra.plugin.serdes.csv.CsvToIon
+ from: "{{ outputs.download.uri }}"
+ header: true
+
+ - id: split
+ type: io.kestra.plugin.core.storage.Split
+ from: "{{ outputs.to_ion.uri }}"
+ rows: 25
+
+ - id: per_chunk
+ type: io.kestra.plugin.core.flow.Loop
+ values: "{{ outputs.split.uris }}"
+ concurrencyLimit: 4
+ fetchType: FETCH
+ outputs:
+ - id: data
+ type: STRING
+ value: "{{ outputs.aggregate.uri }}"
+ tasks:
+ - id: filter
+ type: io.kestra.plugin.transform.Filter
+ from: "{{ item.value }}"
+ where: todecimal(total) > 10
+
+ - id: aggregate
+ type: io.kestra.plugin.transform.Aggregate
+ from: "{{ outputs.filter.uri }}"
+ outputType: STORE
+ groupBy: [customer_email]
+ aggregates:
+ orders:
+ expr: count()
+ type: INT
+ revenue:
+ expr: sum(todecimal(total))
+ type: DECIMAL
+
+ - id: concat
+ type: io.kestra.plugin.core.storage.Concat
+ files: "{{ loopOutputs(outputs.per_chunk.outputs, 'data') }}"
+ extension: .ion
+
+ - id: reduce
+ type: io.kestra.plugin.transform.Aggregate
+ from: "{{ outputs.concat.uri }}"
+ outputType: STORE
+ groupBy: [customer_email]
+ aggregates:
+ orders:
+ expr: sum(orders)
+ type: INT
+ revenue:
+ expr: sum(revenue)
+ type: DECIMAL
+```
+
+Key points:
+
+- The Loop output declares type `STRING` even though the value is a file URI. `loopOutputs()` extracts strings, and `Concat` accepts a list of URI strings.
+- `fetchType: FETCH` keeps the per-iteration output objects in memory so `loopOutputs()` can read them immediately after the loop. Use `STORE` instead if you have hundreds of iterations.
+- The final `reduce` task re-aggregates because the same key (e.g. a customer) may appear in multiple chunks. The two-pass structure — per-chunk aggregate then cross-chunk reduce — is the canonical map-reduce pattern.
+
+## Handle per-iteration failures
+
+Set `transmitFailed: false` to continue the loop when individual iterations fail. Use `errors:` to run tasks per failed iteration, and `finally:` for a one-time cleanup block after all iterations finish. `errors:` runs on failure regardless of `transmitFailed` — with `transmitFailed: false`, the loop continues to subsequent iterations after handling the error. `finally:` always runs regardless.
+
+```yaml
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["ok", "boom", "ok"]
+ transmitFailed: false
+ tasks:
+ - id: maybe_fail
+ type: io.kestra.plugin.scripts.shell.Commands
+ commands:
+ - |
+ if [ "{{ item.value }}" = "boom" ]; then exit 1; fi
+ echo "ok {{ item.value }}"
+ errors:
+ - id: handle_error
+ type: io.kestra.plugin.core.log.Log
+ message: "Iteration {{ item.index }} ({{ item.value }}) failed"
+ finally:
+ - id: cleanup
+ type: io.kestra.plugin.core.log.Log
+ message: "Loop finished (with or without failures)"
+```
+
+## Use Loop + Subflow for isolated per-batch execution
+
+When each batch needs its own execution — independent retries, logs, and failure ownership — pair `Loop` with `Subflow`. The parent splits the data and fans out; the child flow receives one chunk URI per invocation and returns its result as a flow-level output.
+
+```yaml
+# Parent flow
+tasks:
+ - id: split
+ type: io.kestra.plugin.core.storage.Split
+ from: "{{ inputs.file }}"
+ rows: 100
+
+ - id: per_batch
+ type: io.kestra.plugin.core.flow.Loop
+ values: "{{ outputs.split.uris }}"
+ concurrencyLimit: 4
+ fetchType: FETCH
+ outputs:
+ - id: result_uri
+ type: STRING
+ value: "{{ outputs.run_child.outputs.uri }}"
+ tasks:
+ - id: run_child
+ type: io.kestra.plugin.core.flow.Subflow
+ namespace: company.team
+ flowId: process_batch
+ wait: true
+ transmitFailed: true
+ inputs:
+ batch_uri: "{{ item.value }}"
+
+ - id: concat
+ type: io.kestra.plugin.core.storage.Concat
+ files: "{{ loopOutputs(outputs.per_batch.outputs, 'result_uri') }}"
+ extension: .ion
+```
+
+## Compose Loop with supporting tasks
+
+`Loop` iterates. These tasks handle the rest — batching, transforming, stitching, and reducing. Each does one thing well; combine them around `Loop` to build larger pipelines.
+
+| Task | Role | When to reach for it |
+|---|---|---|
+| `io.kestra.plugin.core.storage.Split` | Batching | Split a single file into chunk URIs by `rows`, `bytes`, `partitions`, or `separator`. Feeds `Loop.values` for map-reduce. |
+| `io.kestra.plugin.core.storage.Concat` | Stitching | Concatenate per-iteration output files into one before a reduce step. |
+| `io.kestra.plugin.transform.Aggregate` | Reduce | Group records by one or more keys with `count()`, `sum()`, `max()`, and more. The reduce side of map-reduce. |
+| `io.kestra.plugin.transform.Filter` | Predicate | Keep only rows where a boolean expression holds. |
+| `io.kestra.plugin.transform.Map` | Project | Per-record rename, drop, or compute fields — SQL `SELECT`-style. |
+| `io.kestra.plugin.transform.Unnest` | Explode | Flatten an array field into one row per element, carrying sibling fields through. |
+| `io.kestra.plugin.core.flow.Subflow` | Isolate | Spawn a separate execution per iteration — own retries, own logs, own failure state. |
+| `io.kestra.plugin.core.flow.Parallel` | Fan-out | Run independent task groups concurrently inside a single iteration. |
+
+## Common mistakes to avoid
+
+- Do not use `taskrun.value` or `taskrun.iteration` — use `item.value` and `item.index`.
+- Do not access `item.value.field` directly on object values — use `fromJson(item.value).field`.
+- Do not expect loop outputs to be visible downstream without declaring an `outputs:` block.
+- Do not use `outputs.task_id[item.value]` inside a loop — sibling outputs are accessed with plain `outputs.task_id.attribute`.
+- Do not set `concurrencyLimit: 0` on very large datasets without considering memory and worker capacity.
+
+For more details, see the [Loop task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.loop) and the [Flowable Tasks reference](../../05.workflow-components/01.tasks/00.flowable-tasks/index.md#loop).
diff --git a/src/contents/docs/14.best-practices/11.purging-data/index.md b/src/contents/docs/14.best-practices/11.purging-data/index.md
index 1c04b0a037d..aec3d3b20c4 100644
--- a/src/contents/docs/14.best-practices/11.purging-data/index.md
+++ b/src/contents/docs/14.best-practices/11.purging-data/index.md
@@ -33,8 +33,8 @@ Use this rule of thumb:
| If you want to remove... | Prefer | Why |
| --- | --- | --- |
-| Old execution records | [`PurgeExecutions`](/plugins/core/execution/io.kestra.plugin.core.execution.purgeexecutions) | It permanently deletes execution metadata and related execution data |
-| Old execution and trigger logs | [`PurgeLogs`](/plugins/core/log/io.kestra.plugin.core.log.purgelogs) | It is designed for bulk log cleanup |
+| Old execution records | [`PurgeExecutions`](/plugins/core/tasks/io.kestra.plugin.core.execution.purgeexecutions) | It permanently deletes execution metadata and related execution data |
+| Old execution logs, trigger logs, or both | [`PurgeLogs`](/plugins/core/log/io.kestra.plugin.core.log.purgelogs) | Use `purgeExecutionLogs` and `purgeNonExecutionLogs` to target each type independently, or leave both `true` (default) to purge all logs |
| Expired runtime state in the KV Store | [`PurgeKV`](/plugins/core/kv/io.kestra.plugin.core.kv.purgekv) or automatic KV expiration purge | It removes stale KV entries without treating them as static configuration |
| Old Namespace file versions | [`PurgeFiles`](/plugins/core/namespace/io.kestra.plugin.core.namespace.purgefiles) | It applies retention rules to Namespace files and their versions |
| Old asset records, usages, or lineage data | [`PurgeAssets`](../../10.administrator-guide/purge/index.md#purge-assets-and-lineage-retention) | It applies retention to asset-related records without touching executions or logs |
@@ -66,6 +66,8 @@ This is usually the right choice when:
Best practice:
- set separate retention periods for executions and logs if your teams use them differently
+- use `purgeExecutionLogs: false` to retain execution logs for failed workflow debugging while still purging trigger logs, or `purgeNonExecutionLogs: false` to do the reverse
+- set `batchSize` on `PurgeLogs` when purging large volumes of logs to limit the number of rows deleted per transaction
- avoid deleting recent data that is still useful for troubleshooting failed workflows
- run purge flows on a schedule instead of waiting for storage pressure
@@ -125,7 +127,7 @@ Best practice:
- define retention based on operational or compliance requirements
- purge only the records you intend to remove, especially if you want to keep lineage or usage data longer than the assets themselves
-[`PurgeAssets`](../../10.administrator-guide/purge/index.md#purge-assets-and-lineage-retention) is an Enterprise Edition feature.
+[`PurgeAssets`](../../10.administrator-guide/purge/index.md#purge-assets-and-lineage-retention) is an Enterprise Edition and Cloud feature.
## Purge tasks vs. UI deletion
diff --git a/src/contents/docs/14.best-practices/2.naming-conventions/index.md b/src/contents/docs/14.best-practices/2.naming-conventions/index.md
index b5c8dd2fde8..7f5b6814bf7 100644
--- a/src/contents/docs/14.best-practices/2.naming-conventions/index.md
+++ b/src/contents/docs/14.best-practices/2.naming-conventions/index.md
@@ -19,7 +19,7 @@ This approach helps with:
## Why use the `company.team` structure
-By defining a **root namespace named after your company**, you can centralize management of [plugin defaults](../../05.workflow-components/09.plugin-defaults/index.md), [variables](../../05.workflow-components/04.variables/index.md), and [secrets](../../06.concepts/04.secret/index.md).
+By defining a **root namespace named after your company**, you can centralize management of [Policies](../../07.enterprise/02.governance/policies/index.md) (Enterprise Edition), [variables](../../05.workflow-components/04.variables/index.md), and [secrets](../../06.concepts/04.secret/index.md).
These configurations can then be inherited by all namespaces under that root.
This structure also simplifies [Git synchronization](../../version-control-cicd/04.git/index.md).
diff --git a/src/contents/docs/14.best-practices/6.git/index.md b/src/contents/docs/14.best-practices/6.git/index.md
index 4b65215a4b7..51e692e6b77 100644
--- a/src/contents/docs/14.best-practices/6.git/index.md
+++ b/src/contents/docs/14.best-practices/6.git/index.md
@@ -3,7 +3,6 @@ title: Version Control with Git in Kestra
h1: "Use Git Effectively with Kestra: SyncFlows, PushFlows & CI/CD"
sidebarTitle: Git Version Control
icon: /src/contents/docs/icons/best-practices.svg
-version: ">= 0.17.0"
description: Best practices for using Git with Kestra for version control, including SyncFlows, PushFlows, and CI/CD integration.
---
diff --git a/src/contents/docs/14.best-practices/7.outputs/index.md b/src/contents/docs/14.best-practices/7.outputs/index.md
index f3315635077..142048f1731 100644
--- a/src/contents/docs/14.best-practices/7.outputs/index.md
+++ b/src/contents/docs/14.best-practices/7.outputs/index.md
@@ -54,7 +54,7 @@ tasks:
- id: load
type: io.kestra.plugin.aws.s3.Upload
- from: "{{ outputs.extractLargeFile.uri }}"
+ from: "{{ outputs.extract.uri }}"
bucket: myBucket
key: largeFiles/orders.csv
diff --git a/src/contents/docs/14.best-practices/8.business-unit-separation/index.md b/src/contents/docs/14.best-practices/8.business-unit-separation/index.md
index 125c330aca9..6ba3e836c2d 100644
--- a/src/contents/docs/14.best-practices/8.business-unit-separation/index.md
+++ b/src/contents/docs/14.best-practices/8.business-unit-separation/index.md
@@ -51,7 +51,7 @@ You can configure dedicated resources for each namespace:
- **Team-based organization**: Separate flows and resources by team within the same tenant, maintaining visibility for users with appropriate permissions.
- **Project-based organization**: Create separate namespaces for projects that need limited isolation while retaining workflow visibility.
- **Dependency management**: Namespaces support cross-team dependencies (e.g., subflows or triggers), simplifying dependency tracking.
-- **RBAC control**: Namespaces allow granular role-based access. A user might have `READ` access in one namespace and full CRUD permissions in another.
+- **RBAC control**: Namespaces allow granular role-based access. A user might have view-only access in one namespace and full management permissions in another.
## Summary of when to use tenants vs. namespaces
diff --git a/src/contents/docs/15.how-to-guides/access-local-files/index.md b/src/contents/docs/15.how-to-guides/access-local-files/index.md
index 0a433998ac4..acae1157d81 100644
--- a/src/contents/docs/15.how-to-guides/access-local-files/index.md
+++ b/src/contents/docs/15.how-to-guides/access-local-files/index.md
@@ -6,7 +6,7 @@ stage: Getting Started
topics:
- Scripting
- Integrations
-description: Access and process files stored on your local machine within Kestra workflows using bind mounts and the Process task runner.
+description: Access files stored on your local machine within Kestra workflows using bind mounts, and batch-upload files to the local filesystem using the local.Uploads task.
---
Access locally stored files on your machine inside Kestra workflows.
@@ -62,3 +62,99 @@ tasks:
commands:
- cat /files/myfile.txt
```
+
+## Batch-uploading files with `local.Uploads`
+
+[`io.kestra.plugin.fs.local.Uploads`](/plugins/plugin-fs/local/io.kestra.plugin.fs.local.uploads) writes multiple Kestra internal storage files to a directory on the local filesystem in a single task. It mirrors the `Uploads` task available on the FTP, FTPS, SFTP, and SMB backends.
+
+### Configure allowed paths
+
+Both [`local.Upload`](/plugins/plugin-fs/local/io.kestra.plugin.fs.local.upload) (single file) and `local.Uploads` (batch) require the destination directory to be listed in the plugin's `allowed-paths` configuration. Add the following to your `kestra.yml`:
+
+```yaml
+kestra:
+ plugins:
+ configurations:
+ - type: io.kestra.plugin.fs.local.Uploads
+ values:
+ allowed-paths:
+ - /data/uploads
+ - type: io.kestra.plugin.fs.local.Upload
+ values:
+ allowed-paths:
+ - /data/uploads
+```
+
+Without this, any write to `/data/uploads` is rejected with a `SecurityException` even if the path is bind-mounted into the container.
+
+### Upload a list of files
+
+Pass a list of Kestra internal storage URIs to `from`. Each file is written to the `to` directory using its original filename.
+
+The flow below runs a data ingestion job that produces run logs and SQL migration scripts, then archives the logs to a local directory:
+
+```yaml
+id: archive_pipeline_logs
+namespace: company.team
+
+tasks:
+ - id: run_pipeline
+ type: io.kestra.plugin.scripts.shell.Commands
+ taskRunner:
+ type: io.kestra.plugin.core.runner.Process
+ outputFiles:
+ - "*.log"
+ - "*.sql"
+ commands:
+ - echo "ingested 1024 rows" > ingest.log
+ - echo "0 errors" > errors.log
+ - echo "ALTER TABLE orders ADD COLUMN status TEXT;" > schema.sql
+ - echo "INSERT INTO orders VALUES (1, 'pending');" > seed.sql
+
+ - id: upload_logs
+ type: io.kestra.plugin.fs.local.Uploads
+ from:
+ - "{{ outputs.run_pipeline.outputFiles['ingest.log'] }}"
+ - "{{ outputs.run_pipeline.outputFiles['errors.log'] }}"
+ to: /data/uploads/logs
+```
+
+### Upload with custom destination filenames
+
+To rename files at the destination, pass a map of `destinationFilename: sourceURI` pairs instead of a list. This is useful for versioning — for example, tagging migration scripts with a version prefix before archiving them.
+
+In the flow above, replace the `upload_logs` task with:
+
+```yaml
+ - id: upload_migrations
+ type: io.kestra.plugin.fs.local.Uploads
+ from:
+ v1_schema.sql: "{{ outputs.run_pipeline.outputFiles['schema.sql'] }}"
+ v1_seed.sql: "{{ outputs.run_pipeline.outputFiles['seed.sql'] }}"
+ to: /data/uploads/migrations
+```
+
+### Filter by regular expression
+
+Use `regExp` to upload only files whose internal storage URI matches a pattern. Files that do not match are skipped.
+
+When a task produces a mixed set of outputs, `regExp` lets you route file types to separate destinations without splitting the upstream task. In the flow above, replace the `upload_logs` task with:
+
+```yaml
+ - id: upload_sql_only
+ type: io.kestra.plugin.fs.local.Uploads
+ from:
+ - "{{ outputs.run_pipeline.outputFiles['ingest.log'] }}"
+ - "{{ outputs.run_pipeline.outputFiles['errors.log'] }}"
+ - "{{ outputs.run_pipeline.outputFiles['schema.sql'] }}"
+ - "{{ outputs.run_pipeline.outputFiles['seed.sql'] }}"
+ regExp: ".*\\.sql$"
+ to: /data/uploads/migrations
+```
+
+### Additional properties
+
+| Property | Default | Description |
+|---|---|---|
+| `maxFiles` | `25` | Upper bound on how many files are written. Excess files are dropped with a warning. |
+| `overwrite` | `true` | When `false`, the task fails if a destination file already exists. |
diff --git a/src/contents/docs/15.how-to-guides/alerting/index.md b/src/contents/docs/15.how-to-guides/alerting/index.md
index 735f2b894e5..741dbb44b1a 100644
--- a/src/contents/docs/15.how-to-guides/alerting/index.md
+++ b/src/contents/docs/15.how-to-guides/alerting/index.md
@@ -79,7 +79,7 @@ errors:
## Flow trigger
-Subflows cut down on duplication, but you still need the `errors` block in every flow. For a fully centralized approach, use a **Flow trigger** that reacts to execution status. Trigger conditions let you target specific states, such as `FAILED` or `WARNING`, and you can define separate triggers per status if needed.
+Subflows cut down on duplication, but you still need the `errors` block in every flow. For a fully centralized approach, use a **Flow trigger** that reacts to execution status. The `when` expression lets you target specific states, such as `FAILED` or `WARNING`, and you can define separate triggers per status if needed.
```yaml
id: failure_alert_slack
@@ -95,11 +95,8 @@ tasks:
triggers:
- id: on_failure
type: io.kestra.plugin.core.trigger.Flow
- conditions:
- - type: io.kestra.plugin.core.condition.ExecutionStatus
- in:
- - FAILED
- - WARNING
+ dependsOn:
+ - states: [FAILED, WARNING]
```
diff --git a/src/contents/docs/15.how-to-guides/ansible-config-drift/index.md b/src/contents/docs/15.how-to-guides/ansible-config-drift/index.md
index 69840efe49f..a2ccf26717e 100644
--- a/src/contents/docs/15.how-to-guides/ansible-config-drift/index.md
+++ b/src/contents/docs/15.how-to-guides/ansible-config-drift/index.md
@@ -48,7 +48,7 @@ server3.example.test ansible_user=admin ansible_ssh_private_key_file=~/.ssh/id_r
## Flow: run Ansible and alert on drift
-This flow runs the playbook with the [Ansible CLI task](/plugins/plugin-ansible/io.kestra.plugin.ansible.cli.ansiblecli), inspects each host result in a [`ForEach`](/plugins/core/flow/io.kestra.plugin.core.flow.foreach), and posts a Slack alert only when a host was changed using the [Slack Incoming Webhook task](/plugins/plugin-slack/slack-notifications/io.kestra.plugin.slack.notifications.slackincomingwebhook). The schedule trigger is disabled by default — enable it to run nightly.
+This flow runs the playbook with the [Ansible CLI task](/plugins/plugin-ansible/io.kestra.plugin.ansible.cli.ansiblecli), inspects each host result in a [`Loop`](/plugins/core/flow/io.kestra.plugin.core.flow.loop), and posts a Slack alert only when a host was changed using the [Slack Incoming Webhook task](/plugins/plugin-slack/slack-notifications/io.kestra.plugin.slack.notifications.slackincomingwebhook). The schedule trigger is disabled by default — enable it to run nightly.
```yaml
id: ansible_config_drift
@@ -71,16 +71,16 @@ tasks:
- ansible-playbook -i inventory.ini myplaybook.yml
- id: loop_hosts
- type: io.kestra.plugin.core.flow.ForEach
+ type: io.kestra.plugin.core.flow.Loop
values: "{{ outputs.set_up_env.vars.outputs }}"
tasks:
- id: check_drift
type: io.kestra.plugin.slack.notifications.SlackIncomingWebhook
- runIf: "{{ taskrun.value | jq('.changed') | first == true }}"
+ runIf: "{{ item.value | jq('.changed') | first == true }}"
url: "{{ secret('SLACK_WEBHOOK') }}"
payload: |
{
- "text": "Configuration updated - {{ taskrun.value | jq('.msg') | first ?? Null }}"
+ "text": "Configuration updated - {{ item.value | jq('.msg') | first ?? Null }}"
}
triggers:
diff --git a/src/contents/docs/15.how-to-guides/ansible-explicit-outputs/index.md b/src/contents/docs/15.how-to-guides/ansible-explicit-outputs/index.md
index 0e983d8e08c..811551e5739 100644
--- a/src/contents/docs/15.how-to-guides/ansible-explicit-outputs/index.md
+++ b/src/contents/docs/15.how-to-guides/ansible-explicit-outputs/index.md
@@ -1,150 +1,232 @@
---
-title: Control Ansible Playbook Outputs to Protect Sensitive Data
-h1: Expose Only What You Need from Ansible Playbooks
+title: Declare Explicit Outputs in Ansible Playbooks with Kestra
+h1: Control Ansible Output Exposure with outputsMode EXPLICIT
icon: /src/contents/docs/icons/ansiblecli.svg
stage: Intermediate
-topics:
- - Integrations
-description: Use outputsMode EXPLICIT and the bundled kestra module to declare exactly which Ansible playbook values become Kestra task outputs, keeping credentials and sensitive data out of execution logs.
+version: ">= 2.0.0"
+description: Use the bundled kestra Ansible module and outputsMode EXPLICIT to declare only the outputs you need, keeping sensitive per-host data out of Kestra's output store.
---
-Control which Ansible playbook values appear as Kestra task outputs using `outputsMode: EXPLICIT` and the bundled `kestra` module.
+By default, the AnsibleCLI task captures the full result payload from every Ansible module on every host, including `module_args` and raw return values. If your playbooks handle credentials, connection strings, or host-specific secrets, that raw output lands in Kestra's internal storage where it is visible to anyone with execution access.
-By default, `AnsibleCLI` captures every per-host result of every playbook task and stores all of them in `{{ outputs..vars.outputs }}`. When a playbook fetches credentials, Active Directory objects, or other sensitive data, those values end up in execution outputs and logs — visible to anyone who can read the execution. Setting `no_log: true` on a task censors the result but breaks output capture entirely, leaving no clean way to pass values downstream.
-
-`outputsMode: EXPLICIT` solves this by letting the playbook declare exactly what becomes a Kestra output, while redacting everything else.
+Setting `outputsMode: EXPLICIT` changes this behavior. Only the values you declare with the bundled `kestra` Ansible module appear in `vars.outputs`. Everything else in `vars.playbooks` is reduced to a status flag (`changed: true/false`), preserving play-level observability without leaking host data.
## How it works
-Add `outputsMode: EXPLICIT` to the `AnsibleCLI` task. Inside the playbook, use the bundled `kestra` module to declare the values downstream tasks should see:
+When `outputsMode: EXPLICIT` is set, the AnsibleCLI task passes `KESTRA_OUTPUTS_MODE=explicit` to the `kestra_logger` callback plugin. The callback then:
-```yaml
-- name: Declare what downstream tasks may see
- kestra:
- outputs:
- records_updated: "{{ records_updated }}"
- work_status: "ok"
-```
+1. Collects outputs only from `kestra` module calls — `kestra: outputs: { key: value }` in your playbook.
+2. Merges all declared outputs into a single flat dict (last write wins across hosts and module calls).
+3. Replaces each host's full result payload in `vars.playbooks` with `{"changed": }`. Failures and unreachable hosts also retain `msg`.
+4. Preserves task status (ok, failed, skipped, unreachable) so you can still inspect play health.
-In `EXPLICIT` mode:
-- Only values declared via `kestra:` appear in `{{ outputs..vars.outputs }}` — as a flat map, not a list
-- Per-host result payloads are redacted to `{"changed": }` in both outputs and live logs
-- Task names, timings, and statuses (`ok` / `failed` / `skipped` / `unreachable`) are preserved
-- Multiple `kestra:` calls merge by key; the last write wins
+The `kestra` module is bundled with the plugin and injected automatically — no extra installation is needed.
-In `ALL` mode (the default), behavior is unchanged — every per-host result is captured as before.
+## Enable EXPLICIT mode
-## Basic example
-
-This playbook fetches a credential object and performs work. Only `records_updated` and `work_status` are exposed to Kestra; the credential object never appears in outputs or logs.
+Add `outputsMode: EXPLICIT` to your `AnsibleCLI` task:
```yaml
-id: ansible_explicit_outputs
-namespace: company.team
-
-variables:
- playbook: |
- ---
- - hosts: localhost
- tasks:
- - name: Fetch credentials needed by the automation
- ansible.builtin.set_fact:
- credential:
- username: svc-automation
- password: "not-for-kestra-outputs"
-
- - name: Do the work
- ansible.builtin.set_fact:
- records_updated: 3
- register: work_result
-
- - name: Declare what downstream tasks may see
- kestra:
- outputs:
- records_updated: "{{ records_updated }}"
- work_status: "{{ 'skipped' if work_result.skipped | default(false) else 'ok' }}"
+id: deploy
+namespace: company.infra
tasks:
- - id: ansible_task
+ - id: run_playbook
type: io.kestra.plugin.ansible.cli.AnsibleCLI
outputsMode: EXPLICIT
- inputFiles:
- playbook.yml: "{{ vars.playbook }}"
containerImage: cytopia/ansible:latest-tools
+ inputFiles:
+ hosts: |
+ [webservers]
+ web-01
+ web-02
+ playbook.yml: |
+ - hosts: webservers
+ tasks:
+ - name: Gather app version
+ shell: cat /opt/app/VERSION
+ register: version_output
+
+ - name: Declare output
+ kestra:
+ outputs:
+ app_version: "{{ version_output.stdout }}"
commands:
- - ansible-playbook -i localhost -c local playbook.yml
+ - ansible-playbook -i hosts playbook.yml
```
-Access the declared values downstream with `{{ outputs.ansible_task.vars.outputs.records_updated }}`.
+:::alert{type="info"}
+Playbooks are passed via `inputFiles`. If your playbook contains Ansible Jinja expressions like `{{ variable }}`, store the playbook content in `variables:` instead to prevent Kestra from evaluating those expressions before Ansible sees them. See [Avoiding expression conflicts](#avoiding-expression-conflicts).
+:::
-## Distinguishing skipped from failed
+## Declare outputs with the kestra module
-A common pattern in Active Directory and CMDB automations is that a missing permission causes Ansible to skip a task rather than fail it. In `ALL` mode this distinction gets buried in the raw result payload. In `EXPLICIT` mode you can surface it cleanly:
+Use `kestra: outputs: { key: value }` anywhere in your playbook tasks. The module name is `kestra` (also resolvable as `ansible.legacy.kestra`).
```yaml
-- name: Attempt AD lookup
- ansible.windows.win_shell: Get-ADUser -Identity {{ ad_user }}
- register: ad_result
- failed_when: false
+- name: Declare database connection info
+ kestra:
+ outputs:
+ db_host: "{{ hostvars[inventory_hostname]['db_host'] }}"
+ schema_version: "{{ schema_version }}"
+```
+
+The module always sets `changed: false`. It is safe to run in check mode.
+
+### Avoid using loop
+
+Do not call the `kestra` module inside an Ansible `loop:`. Ansible aggregates loop results under a `results` key, which the callback cannot parse for output collection. A warning is emitted and declared outputs are silently skipped for that task.
+
+```yaml
+# WRONG — outputs will not be collected
+- name: Declare per-item outputs
+ kestra:
+ outputs:
+ item: "{{ item }}"
+ loop: [a, b, c]
-- name: Expose result status
+# CORRECT — call the module once with all values
+- name: Declare outputs
kestra:
outputs:
- ad_user: "{{ ad_user }}"
- ad_status: "{{ 'skipped' if ad_result.skipped | default(false) else ('failed' if ad_result.rc | default(0) != 0 else 'ok') }}"
+ items: [a, b, c]
```
-Downstream tasks can then branch on `{{ outputs.ansible_task.vars.outputs.ad_status }}` without parsing raw result payloads.
+## Access outputs downstream
-## Multi-host outputs
+Declared outputs are available under `vars.outputs` exactly as in `ALL` mode:
-The `kestra` module runs on whichever host executes the task and the result is always returned to the controller. For outputs that should reflect the run as a whole rather than a single host, use `run_once: true` or `delegate_to: localhost`:
+```yaml
+- id: next_task
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ outputs.run_playbook.vars.outputs.app_version }}"
+```
+
+Where `run_playbook` is the `id` of your `AnsibleCLI` task and `app_version` is the key you declared.
+
+## Output structure: ALL vs EXPLICIT
+
+The shape of `vars.outputs` differs between the two modes.
+
+**ALL mode** (default): a list of per-host result dicts — each entry contains the full Ansible result object for that host.
+
+```json
+[
+ {
+ "host": "web-01",
+ "ok": {
+ "changed": true,
+ "stdout": "1.4.2",
+ "module_args": { ... }
+ }
+ }
+]
+```
+
+**EXPLICIT mode**: a flat dict of the keys you declared — merged across all hosts and `kestra` module calls.
+
+```json
+{
+ "app_version": "1.4.2",
+ "db_host": "pg-prod.internal"
+}
+```
+
+:::alert{type="warning"}
+Switching between `ALL` and `EXPLICIT` changes the type of `vars.outputs` from a list to a dict. Update any downstream expressions that iterate over the list if you migrate an existing flow.
+:::
+
+## Multi-host output merging
+
+When multiple hosts call `kestra: outputs:` with the same key, the last write wins. If hosts run in parallel, which host writes last is non-deterministic. Use one of these patterns for predictable results:
+
+**Run the declaration on one host only:**
```yaml
-- name: Summarize across all hosts
+- name: Declare run-level output
kestra:
outputs:
- hosts_updated: "{{ ansible_play_hosts | length }}"
- completed_at: "{{ ansible_date_time.iso8601 }}"
+ cluster_leader: "{{ inventory_hostname }}"
run_once: true
+ delegate_to: localhost
```
-For per-host outputs, key them by hostname in the playbook so they merge rather than overwrite:
+**Key by hostname for per-host values:**
```yaml
-- name: Expose per-host result
+- name: Declare per-host version
kestra:
outputs:
- "{{ inventory_hostname }}_status": "{{ task_result.rc | default(0) == 0 | ternary('ok', 'failed') }}"
+ "version_{{ inventory_hostname }}": "{{ app_version }}"
+```
+
+## Avoiding expression conflicts
+
+Ansible uses `{{ variable }}` for Jinja2 expressions, and so does Kestra's Pebble templating engine. When a playbook is defined inline in a Kestra flow (for example, in `inputFiles` or as a flow variable), Kestra evaluates its own `{{ }}` expressions before passing the file to Ansible — which breaks Ansible's expressions.
+
+Store playbooks that contain Ansible Jinja expressions in `variables:` to prevent Kestra from evaluating their `{{ }}` expressions:
+
+```yaml
+id: deploy
+namespace: company.infra
+
+variables:
+ playbook: |
+ - hosts: webservers
+ tasks:
+ - name: Gather version
+ shell: cat /opt/app/VERSION
+ register: result
+
+ - name: Declare output
+ kestra:
+ outputs:
+ version: "{{ result.stdout }}"
+
+tasks:
+ - id: run_playbook
+ type: io.kestra.plugin.ansible.cli.AnsibleCLI
+ outputsMode: EXPLICIT
+ containerImage: cytopia/ansible:latest-tools
+ inputFiles:
+ hosts: |
+ [webservers]
+ web-01
+ playbook.yml: "{{ vars.playbook }}"
+ commands:
+ - ansible-playbook -i hosts playbook.yml
```
-## Custom `ansibleConfig`
+The `vars.playbook` reference is resolved by Kestra, returning the raw playbook string. Ansible then evaluates `{{ result.stdout }}` as expected.
+
+## Custom ansibleConfig
-If you supply your own `ansibleConfig`, the generated config is skipped entirely. Include these lines to keep the `kestra` module resolvable and the callback active:
+If you supply a custom `ansibleConfig`, you must manually include the `library` path so Ansible can find the bundled `kestra` module. Without it, `kestra: outputs:` tasks will fail with a module-not-found error.
```yaml
tasks:
- - id: ansible_task
+ - id: run_playbook
type: io.kestra.plugin.ansible.cli.AnsibleCLI
outputsMode: EXPLICIT
ansibleConfig: |
[defaults]
- callback_plugins = ./callback_plugins
+ library = ./library
+ stdout_callback = kestra_logger
callbacks_enabled = kestra_logger
- stdout_callback = ansible.builtin.null
- library = ./library
+ commands:
+ - ansible-playbook -i hosts playbook.yml
```
-`library = ./library` is required for the bundled `kestra` module to resolve. Without it, the `kestra:` task fails with a module-not-found error.
+:::alert{type="warning"}
+The exact `ansibleConfig` lines required alongside `library = ./library` depend on how the plugin configures the callback. Confirm the required fields in a test run before deploying to production. A missing callback line will silently fall back to `ALL` mode behavior.
+:::
-## What EXPLICIT mode does not redact
+## What EXPLICIT mode does not protect
-Redaction only covers what the bundled callback emits. These remain unaffected:
+`outputsMode: EXPLICIT` controls the `vars.outputs` payload. It does not affect:
-- **Ansible's own log file** — if `log_path` is set in `ansibleConfig`, the log file can still contain raw task output. Use `no_log: true` on sensitive tasks alongside `outputsMode: EXPLICIT`.
-- **Verbose stdout** — dropping `stdout_callback = ansible.builtin.null` from `ansibleConfig` re-enables Ansible's default stdout printer, which can print raw results on failures or verbose runs. Keep that line to preserve redaction.
-- **The `debug` module** — `ansible.builtin.debug` always prints to stdout regardless of mode.
+- **Ansible verbose logging** (`-v`, `-vv`, etc.) — verbose output still appears in task logs.
+- **`log_path`** in `ansible.cfg` — if set, Ansible writes its full log to that path.
+- **Kestra task logs** — stdout from shell commands and module output may appear in execution logs.
-:::alert{type="info"}
-For the general `AnsibleCLI` task reference, see the [plugin documentation](/plugins/plugin-ansible/io.kestra.plugin.ansible.cli.ansiblecli). For system audit and reporting examples, see [Use Ansible Playbooks to Check Resources and Automate Updates](../ansible/index.md).
-:::
+If your playbooks handle highly sensitive data, use `no_log: true` on individual tasks to suppress their output from Ansible's own logging, and avoid enabling verbose mode.
diff --git a/src/contents/docs/15.how-to-guides/ansible/index.md b/src/contents/docs/15.how-to-guides/ansible/index.md
index 685123951ae..3f32c36f37e 100644
--- a/src/contents/docs/15.how-to-guides/ansible/index.md
+++ b/src/contents/docs/15.how-to-guides/ansible/index.md
@@ -475,4 +475,4 @@ A trigger allows you to build a historical log of machine health in S3 and Slack
Ansible handles host-level automation — collecting facts, checking software package versions, remediating with the right package manager, and so much more. Kestra now orchestrates the run, stores secrets, uploads the JSON report to S3, and notifies Slack (or your preferred channel) so teams see when upgrades occur. Together they scale this cross-platform playbook from one laptop to a fleet, with repeatable runs and downstream integrations ready to consume the results.
-If your playbooks handle credentials or other sensitive data, see [Control Ansible Playbook Outputs to Protect Sensitive Data](../ansible-explicit-outputs/index.md) for how to use `outputsMode: EXPLICIT` to control exactly which values appear in Kestra task outputs.
+If your playbooks handle sensitive data such as credentials or connection strings, see [Declare Explicit Outputs in Ansible Playbooks](../ansible-explicit-outputs/index.md) to control exactly what appears in `vars.outputs`.
diff --git a/src/contents/docs/15.how-to-guides/azure-workload-id/index.md b/src/contents/docs/15.how-to-guides/azure-workload-id/index.md
index b49e92790b8..445d2906b6b 100644
--- a/src/contents/docs/15.how-to-guides/azure-workload-id/index.md
+++ b/src/contents/docs/15.how-to-guides/azure-workload-id/index.md
@@ -191,42 +191,6 @@ Assuming other properties are populated to your desired values within the `value
helm install -f values.yaml kestra kestra/kestra
```
-## Known Issues
-
-On earlier versions of the Kestra helm chart (<=0.19), it was not possible to define custom labels for individual pods. To use managed workload identity, the following label must be defined on the webserver, scheduler, and worker pods:
-
-```yaml
-azure.workload.identity/use: "true"
-```
-
-This configuration can also be provided in the values.yaml override by specifying the following `podLabels`:
-
-```yaml
-common:
- podLabels:
- azure.workload.identity/use: "true"
-```
-
-Should you be unable to upgrade at this time, here is a workaround:
-- Download the `latest` Kestra helm chart from https://helm.kestra.io
-- Navigate to file `templates/_helpers.tpl`
-- In the section `kestra.selectorsLabels`, add the required label to the list, e.g.:
-
-```yaml
-{{- define "kestra.selectorsLabels" -}}
-app.kubernetes.io/name: {{ include "kestra.name" . }}
-app.kubernetes.io/component: {{ .Component }}
-app.kubernetes.io/instance: {{ .Release.Name }}
-azure.workload.identity/use: "true"
-{{- end -}}
-```
-
-Assuming the above chart is stored locally in `~/helm/kestra`, deploy Kestra using the following command:
-
-```shell
-helm install -f values.yaml kestra ~/helm/kestra
-```
-
## Next steps
Following the steps above, you can use Azure Workload Identity in your Kestra Enterprise Edition instance. If you have any issues replicating this setup, don't hesitate to reach out [via Slack](https://kestra.io/slack) or open a support ticket.
diff --git a/src/contents/docs/15.how-to-guides/clever-cloud/index.md b/src/contents/docs/15.how-to-guides/clever-cloud/index.md
new file mode 100644
index 00000000000..abf3c20eeff
--- /dev/null
+++ b/src/contents/docs/15.how-to-guides/clever-cloud/index.md
@@ -0,0 +1,217 @@
+---
+title: Automate Clever Cloud with Kestra
+h1: Automate Clever Cloud with Kestra
+icon: /src/contents/docs/icons/clevercloud.svg
+stage: Intermediate
+topics:
+ - Integrations
+ - Cloud
+description: Use Kestra flows to deploy applications, provision add-ons, forward logs, and manage organisation members on Clever Cloud.
+---
+
+Orchestrate Clever Cloud infrastructure from Kestra: deploy applications, provision managed databases, route logs to external platforms, and manage organisation members. For the full task and trigger reference, see the [plugin page](https://kestra.io/plugins/plugin-clevercloud).
+
+## Prerequisites
+
+- A [Clever Cloud](https://www.clever-cloud.com/) account
+- A Clever Cloud API token — generate one in the console under **Profile > API tokens**
+- An organisation ID if managing organisation-owned resources (find it in the console URL: `console.clever-cloud.com/organisations/orga_xxx`)
+
+## Authentication
+
+All Clever Cloud tasks share a single `apiToken` property. Store it as a [Kestra secret](../../06.concepts/04.secret/index.md) and reference it with `{{ secret('CC_API_TOKEN') }}`.
+
+Most tasks also accept an optional `organisationId`. When omitted, the plugin targets your personal account via the `/self` API endpoint. Two task groups always require `organisationId`: all `logs` tasks (the APIv4 log endpoints have no `/self` shortcut) and the organisation member tasks (`ListMembers`, `AddMember`, `RemoveMember`).
+
+### Inject credentials with a Policy (EE)
+
+On Enterprise Edition, a [Policy](../../07.enterprise/02.governance/policies/index.md) `Add` rule injects `apiToken` into every Clever Cloud task in a namespace without touching any flow YAML:
+
+```yaml
+id: clever-cloud-auth
+namespace: company.team
+enforcement: ACTIVE
+
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.clevercloud
+ properties:
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+```
+
+## Deploy an application and wait for it
+
+Trigger a redeployment and block the flow until the deployment reaches a terminal state.
+
+```yaml
+id: redeploy_and_wait
+namespace: company.team
+
+inputs:
+ - id: app_id
+ type: STRING
+
+tasks:
+ - id: redeploy
+ type: io.kestra.plugin.clevercloud.applications.Redeploy
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+ applicationId: "{{ inputs.app_id }}"
+
+ - id: get_deployment
+ type: io.kestra.plugin.clevercloud.deployments.List
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+ applicationId: "{{ inputs.app_id }}"
+ limit: 1
+ fetchType: FETCH_ONE
+
+ - id: wait
+ type: io.kestra.plugin.clevercloud.deployments.WaitForState
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+ applicationId: "{{ inputs.app_id }}"
+ deploymentId: "{{ outputs.get_deployment.deployment.uuid }}"
+ targetState: OK
+ failOnUnreached: true
+ timeout: PT20M
+```
+
+Deployment states: `WIP` (in progress), `OK` (success), `FAIL` (error), `CANCELLED`. Set `failOnUnreached: true` to fail the flow if the deployment reaches `FAIL` or `CANCELLED` instead of `OK`. Use `deployments.Trigger` to fire a flow automatically when a deployment reaches a target state.
+
+:::alert{type="warning"}
+`Redeploy` returns no output. The `get_deployment` step fetches the most recent deployment immediately after — if the new deployment has not yet registered in the Clever Cloud API, it may return the previous one. Add a short `Wait` task between `redeploy` and `get_deployment` if you observe this in practice.
+:::
+
+## Provision a database add-on and read its credentials
+
+Provision a managed database and retrieve its connection credentials.
+
+```yaml
+id: provision_postgres
+namespace: company.team
+
+tasks:
+ - id: create_addon
+ type: io.kestra.plugin.clevercloud.addons.Create
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+ organisationId: "{{ secret('CC_ORG_ID') }}"
+ providerId: postgresql-addon
+ plan: dev
+ region: par
+ name: my-app-db
+
+ - id: get_credentials
+ type: io.kestra.plugin.clevercloud.addons.GetEnv
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+ organisationId: "{{ secret('CC_ORG_ID') }}"
+ addonId: "{{ outputs.create_addon.id }}"
+
+ - id: link_to_app
+ type: io.kestra.plugin.clevercloud.addons.LinkToApplication
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+ organisationId: "{{ secret('CC_ORG_ID') }}"
+ applicationId: "{{ secret('CC_APP_ID') }}"
+ addonId: "{{ outputs.create_addon.id }}"
+```
+
+`addons.GetEnv` returns connection credentials in plain text via `{{ outputs.get_credentials.variables }}`. Avoid logging or persisting this output directly — pass values to downstream tasks using Kestra's secret store or output references.
+
+Use `addons.AddonProvisionedTrigger` to fire a flow automatically when a new add-on appears in your account.
+
+## Forward application logs to an external platform
+
+Create a log drain to continuously forward an application's logs to New Relic, Datadog, Elasticsearch, or a syslog endpoint.
+
+```yaml
+id: setup_log_drain
+namespace: company.team
+
+tasks:
+ - id: create_drain
+ type: io.kestra.plugin.clevercloud.logs.CreateDrain
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+ organisationId: "{{ secret('CC_ORG_ID') }}"
+ applicationId: "{{ secret('CC_APP_ID') }}"
+ drainType: NEWRELIC
+ url: https://log-api.newrelic.com/log/v1
+ newRelicApiKey: "{{ secret('CC_NEWRELIC_API_KEY') }}"
+ kind: LOG
+```
+
+Supported `drainType` values:
+
+- `NEWRELIC` — requires `url` and `newRelicApiKey`; use `https://log-api.eu.newrelic.com/log/v1` for EU accounts
+- `DATADOG` — requires `url`
+- `ELASTICSEARCH` — requires `url`
+- `RAW_HTTP`, `SYSLOG_TCP`, `SYSLOG_UDP` — generic syslog or HTTP; use these for OVHcloud (no dedicated drain type)
+
+Use `logs.DeleteDrain` to remove a drain and `logs.ListDrains` to audit active forwarding.
+
+## Trigger a flow when a log line matches a pattern
+
+`LogPatternTrigger` polls application logs at each interval and fires when a line matches a regex. Use it to react to application errors, OOM kills, or deployment signals without polling from a separate flow.
+
+```yaml
+id: alert_on_error
+namespace: company.team
+
+triggers:
+ - id: on_oom
+ type: io.kestra.plugin.clevercloud.logs.LogPatternTrigger
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+ organisationId: "{{ secret('CC_ORG_ID') }}"
+ applicationId: "{{ secret('CC_APP_ID') }}"
+ pattern: "OutOfMemoryError|OOM killed"
+ interval: PT2M
+ limit: 500
+
+tasks:
+ - id: notify
+ type: io.kestra.plugin.core.log.Log
+ message: "OOM detected at {{ trigger.matchedAt }}: {{ trigger.matchedLine }}"
+```
+
+Available trigger outputs:
+
+| Output | Description |
+|--------|-------------|
+| `trigger.matchedLine` | Full text of the matched log line |
+| `trigger.matchedAt` | Timestamp of the matched log line |
+| `trigger.severity` | Log severity level |
+| `trigger.service` | Service that emitted the log |
+
+When multiple lines match in the same poll, only the most recent fires the trigger.
+
+Replace the `notify` task with your preferred alerting method — a Slack message, a PagerDuty call, or an email via `io.kestra.plugin.notifications.slack.SlackIncomingWebhook`.
+
+## Manage organisation members
+
+Add a member and react when the organisation's membership changes.
+
+```yaml
+id: onboard_member
+namespace: company.team
+
+inputs:
+ - id: email
+ type: STRING
+ - id: role
+ type: STRING
+ defaults: DEVELOPER
+
+tasks:
+ - id: add_member
+ type: io.kestra.plugin.clevercloud.organisations.AddMember
+ apiToken: "{{ secret('CC_API_TOKEN') }}"
+ organisationId: "{{ secret('CC_ORG_ID') }}"
+ email: "{{ inputs.email }}"
+ role: "{{ inputs.role }}"
+```
+
+Valid roles: `ADMIN`, `MANAGER`, `DEVELOPER`, `ACCOUNTING`, `READ_ONLY`.
+
+To offboard, call `organisations.RemoveMember` with a `userId` retrieved from `organisations.ListMembers`.
+
+`MemberChangeTrigger` fires when any member is added or removed. It uses the KV store to track membership between polls — the first evaluation always establishes the baseline without firing.
diff --git a/src/contents/docs/09.ui/00.dashboard/add-numerator.png b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/add-numerator.png
similarity index 100%
rename from src/contents/docs/09.ui/00.dashboard/add-numerator.png
rename to src/contents/docs/15.how-to-guides/dashboard-kpi-chart/add-numerator.png
diff --git a/src/contents/docs/09.ui/00.dashboard/chart-options.png b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/chart-options.png
similarity index 100%
rename from src/contents/docs/09.ui/00.dashboard/chart-options.png
rename to src/contents/docs/15.how-to-guides/dashboard-kpi-chart/chart-options.png
diff --git a/src/contents/docs/09.ui/00.dashboard/chart-preview.png b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/chart-preview.png
similarity index 100%
rename from src/contents/docs/09.ui/00.dashboard/chart-preview.png
rename to src/contents/docs/15.how-to-guides/dashboard-kpi-chart/chart-preview.png
diff --git a/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/create-dashboard.png b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/create-dashboard.png
new file mode 100644
index 00000000000..556c8be5e79
Binary files /dev/null and b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/create-dashboard.png differ
diff --git a/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/documentation-view.png b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/documentation-view.png
new file mode 100644
index 00000000000..1f8db619f73
Binary files /dev/null and b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/documentation-view.png differ
diff --git a/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/index.md b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/index.md
new file mode 100644
index 00000000000..7966a5f77f1
--- /dev/null
+++ b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/index.md
@@ -0,0 +1,80 @@
+---
+title: Build a KPI Success Ratio Chart in Kestra Dashboards
+h1: Build a KPI Success Ratio Dashboard Chart
+icon: /src/contents/docs/icons/tutorial.svg
+stage: Getting Started
+topics:
+ - Kestra Concepts
+description: Build a KPI success ratio chart in Kestra using the No Code dashboard editor, then extend it to track failure rates.
+---
+
+Build a KPI chart that displays the percentage of successful executions using Kestra's No Code dashboard editor.
+
+## Prerequisites
+
+- A Kestra instance with at least one completed execution.
+- Access to the **Dashboards** section in the UI.
+
+## Steps
+
+### 1. Open the dashboard editor
+
+From **Dashboards**, select **Default Dashboard** and click **+ Create Dashboard**.
+
+
+
+In the editor, select the **No Code** tab. It appears alongside the YAML editor so you can view both as you work.
+
+
+
+### 2. Set the dashboard properties
+
+Give your dashboard an ID, title, description, and time window. Changes in the No Code form immediately reflect in the YAML editor.
+
+
+
+### 3. Add a KPI chart
+
+Click **+ Add** in the **charts** block. Choose **KPI Chart** as the chart type. Open the **Documentation** tab at any time to view chart-specific guidance without leaving the editor.
+
+
+
+Give the chart an ID and set the data type to **Executions**. Set `field` to `ID` and `agg` to `COUNT` to count all executions.
+
+
+
+### 4. Add a numerator filter
+
+Click **+ Add** under the numerator section. Set `type` to `IN`, add `SUCCESS` as a value, and set `field` to `STATE`. This scopes the numerator to successful executions — the denominator remains all executions.
+
+
+
+### 5. Set display options
+
+Return to the `charts` No Code tab and open **Optional Properties**. Set `displayName`, change `numberType` to `PERCENTAGE`, and set `width` to `3`.
+
+
+
+### 6. Preview and save
+
+Open the **Preview** tab to review the chart. Click **Save** when satisfied.
+
+
+
+## Extend: add a failure ratio chart
+
+To add a failure ratio chart alongside the success ratio, copy the generated YAML for the KPI chart, paste it into the YAML editor as a second chart entry, and replace `SUCCESS` with `FAILED`. The two charts will sit side by side on the dashboard.
+
+## Best practices
+
+**Organize by purpose.** Group related charts into dashboards with a clear goal — for example, separate dashboards for system health, execution performance, and user activity.
+
+**Use consistent naming.** A pattern like `team_metric_type` (e.g., `dataops_executions_latency`) makes dashboards easier to find, version, and export.
+
+**Use YAML for reuse.** When charts share the same structure with small differences in filters or fields, copy-paste the YAML and modify — faster than rebuilding forms.
+
+**Preview before saving.** Catch mismatched fields and aggregation errors early before they make it into a published dashboard.
+
+## Next steps
+
+See the [Dashboards reference](../../09.ui/00.dashboard/index.md) for the full list of chart types, data source fields, and filter options.
diff --git a/src/contents/docs/09.ui/00.dashboard/kpi-chart.png b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/kpi-chart.png
similarity index 100%
rename from src/contents/docs/09.ui/00.dashboard/kpi-chart.png
rename to src/contents/docs/15.how-to-guides/dashboard-kpi-chart/kpi-chart.png
diff --git a/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/no-code-dashboards.png b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/no-code-dashboards.png
new file mode 100644
index 00000000000..e5415754a99
Binary files /dev/null and b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/no-code-dashboards.png differ
diff --git a/src/contents/docs/09.ui/00.dashboard/time-window.png b/src/contents/docs/15.how-to-guides/dashboard-kpi-chart/time-window.png
similarity index 100%
rename from src/contents/docs/09.ui/00.dashboard/time-window.png
rename to src/contents/docs/15.how-to-guides/dashboard-kpi-chart/time-window.png
diff --git a/src/contents/docs/15.how-to-guides/debezium/index.md b/src/contents/docs/15.how-to-guides/debezium/index.md
index 88c690005f3..83bc54dc8f5 100644
--- a/src/contents/docs/15.how-to-guides/debezium/index.md
+++ b/src/contents/docs/15.how-to-guides/debezium/index.md
@@ -8,212 +8,18 @@ topics:
description: Enable Change Data Capture (CDC) in your databases to use Debezium tasks and triggers in Kestra for real-time data ingestion.
---
-To use Debezium tasks and triggers, perform the necessary database setup described below.
+To use Debezium tasks and triggers, enable Change Data Capture (CDC) on your database. Each connector has different prerequisites — follow the links to the Debezium documentation for complete setup steps.
-## Creating a user
+## MySQL
-A Debezium MySQL connector requires a MySQL user account. This MySQL user must have appropriate permissions on all databases for which the Debezium MySQL connector captures changes.
+Before using the Debezium MySQL connector, you must:
+- Create a MySQL user with `SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT` privileges
+- Enable binary logging with `binlog_format = ROW` and `binlog_row_image = FULL`
+- Optionally enable GTIDs for simpler replication management
-**Prerequisites**
+See the [Debezium MySQL connector documentation](https://debezium.io/documentation/reference/3.0/connectors/mysql.html) for complete setup instructions, including hosted options such as Amazon RDS and Aurora.
-- A MySQL server.
-- Basic knowledge of SQL commands.
-
-**Procedure**
-
-1. Create the MySQL user:
-
-```sql
-mysql> CREATE USER 'user'@'localhost' IDENTIFIED BY 'password';
-```
-
-2. Grant the required permissions to the user:
-
-```sql
-mysql> GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'user' IDENTIFIED BY 'password';
-```
-
-For a description of the required permissions, see [Descriptions of user permissions](https://debezium.io/documentation/reference/3.0/connectors/mysql.html#permissions-explained-mysql-connector).
-
-:::alert{type="info"}
-If using a hosted option such as Amazon RDS or Amazon Aurora that does not allow a global read lock, table-level locks are used to create the consistent snapshot. In this case, you need to also grant `LOCK TABLES` permissions to the user that you created. See [snapshots](https://debezium.io/documentation/reference/3.0/connectors/mysql.html#mysql-snapshots) for more details.
-:::
-
-3. Finalize the user’s permissions:
-
-```sql
-mysql> FLUSH PRIVILEGES;
-```
-
-## Enabling the binlog
-
-You must enable binary logging for MySQL replication. The binary logs record transaction updates in a way that enables replicas to propagate those changes.
-
-**Prerequisites**
-
-- A MySQL server.
-- Appropriate MySQL user privileges.
-
-**Procedure**
-
-1. Check whether the `log-bin` option is enabled:
-
-```sql
-// for MySQL 5.x
-mysql> SELECT variable_value as "BINARY LOGGING STATUS (log-bin) ::"
-FROM information_schema.global_variables WHERE variable_name='log_bin';
-// for MySQL 8.x
-mysql> SELECT variable_value as "BINARY LOGGING STATUS (log-bin) ::"
-FROM performance_schema.global_variables WHERE variable_name='log_bin';
-```
-
-2. If the binlog is `OFF`, add the properties in the following table to the configuration file for the MySQL server:
-
-```ini
-server-id = 223344 # Querying variable is called server_id, e.g. SELECT variable_value FROM information_schema.global_variables WHERE variable_name='server_id';
-log_bin = mysql-bin
-binlog_format = ROW
-binlog_row_image = FULL
-binlog_expire_logs_seconds = 864000
-```
-
-3. Confirm your changes by checking the binlog status once more:
-
-```sql
-// for MySQL 5.x
-mysql> SELECT variable_value as "BINARY LOGGING STATUS (log-bin) ::"
-FROM information_schema.global_variables WHERE variable_name='log_bin';
-// for MySQL 8.x
-mysql> SELECT variable_value as "BINARY LOGGING STATUS (log-bin) ::"
-FROM performance_schema.global_variables WHERE variable_name='log_bin';
-```
-
-1. If you run MySQL on Amazon RDS, you must enable automated backups for your database instance for binary logging to occur. If the database instance is not configured to perform automated backups, the binlog is disabled, even if you apply the settings described in the previous steps.
-
-## Enabling GTIDs
-
-Global transaction identifiers (GTIDs) uniquely identify transactions that occur on a server within a cluster. Though not required for a Debezium MySQL connector, using GTIDs simplifies replication and enables you to more easily confirm if primary and replica servers are consistent.
-
-GTIDs are available in MySQL 5.6.5 and later. See the [MySQL documentation](https://dev.mysql.com/doc/refman/8.2/en/replication-options-gtids.html#option_mysqld_gtid-mode) for more details.
-
-**Prerequisites**
-
-- A MySQL server.
-- Basic knowledge of SQL commands.
-- Access to the MySQL configuration file.
-
-**Procedure**
-
-1. Enable `gtid_mode`:
-
-```sql
-mysql> gtid_mode=ON
-```
-
-2. Enable `enforce_gtid_consistency`:
-
-```sql
-mysql> enforce_gtid_consistency=ON
-```
-
-3. Confirm the changes:
-
-```sql
-mysql> show global variables like '%GTID%';
-```
-
-**Result**
-
-+--------------------------+-------+
-| Variable_name | Value |
-+--------------------------+-------+
-| enforce_gtid_consistency | ON |
-| gtid_mode | ON |
-+--------------------------+-------+
-
-## Configuring session timeouts
-
-When an initial consistent snapshot is made for large databases, your established connection could timeout while the tables are being read. You can prevent this behavior by configuring `interactive_timeout` and `wait_timeout` in your MySQL configuration file.
-
-**Prerequisites**
-
-- A MySQL server.
-- Basic knowledge of SQL commands.
-- Access to the MySQL configuration file.
-
-**Procedure**
-
-1. Configure `interactive_timeout`:
-
-```sql
-mysql> interactive_timeout=
-```
-
-2. Configure wait_timeout:
-
-```sql
-mysql> wait_timeout=
-```
-
-## Enabling query log events
-
-You might want to see the original SQL statement for each binlog event. Enabling the `binlog_rows_query_log_events` option in the MySQL configuration file allows you to do this.
-
-This option is available in MySQL 5.6 and later.
-
-**Prerequisites**
-
-- A MySQL server.
-- Basic knowledge of SQL commands.
-- Access to the MySQL configuration file.
-
-**Procedure**
-
-1. Enable `binlog_rows_query_log_events` in MySQL:
-
-```sql
-mysql> binlog_rows_query_log_events=ON
-```
-
-`binlog_rows_query_log_events` is set to a value that enables/disables support for including the original SQL statement in the binlog entry.
-
-- `ON` = enabled
-- `OFF` = disabled
-
-## Validating binlog row value options
-
-Verify the setting of the `binlog_row_value_options` variable in the database. To enable the connector to consume **UPDATE** events, this variable must be set to a value other than `PARTIAL_JSON`.
-
-**Prerequisites**
-
-- A MySQL server.
-- Basic knowledge of SQL commands.
-- Access to the MySQL configuration file.
-
-**Procedure**
-
-1. Check current variable value
-
-```sql
-mysql> show global variables where variable_name = 'binlog_row_value_options';
-```
-
-**Result**
-+--------------------------+-------+
-| Variable_name | Value |
-+--------------------------+-------+
-| binlog_row_value_options | |
-+--------------------------+-------+
-
-If the value of the variable is set to `PARTIAL_JSON`, run the following command to unset it:
-
-```sql
-mysql> set @@global.binlog_row_value_options="" ;
-```
-
-## Running Debezium tasks on MySQL
-
-You are now all set to run the Debezium MySQL based tasks and triggers. Here is an example flow using Debezium MySQL Realtime Trigger:
+### Kestra flow
```yaml
id: debezium_mysql
@@ -234,73 +40,15 @@ triggers:
password: mysql_passwd
```
-Debezium MySQL Realtime Trigger will collect the records from the change data capture as and when they occur. The flow can then process these records as required.
-
-## Debezium with PostgreSQL
-
-In order for Debezium to work with PostgreSQL, you need to enable write ahead logging (WAL) on the PostgreSQL server.
-
-PostgreSQL’s [logical decoding](https://www.postgresql.org/docs/current/static/logicaldecoding-explanation.html) feature was introduced in version 9.4. It is a mechanism that allows the extraction of the changes that were committed to the transaction log and the processing of these changes in a user-friendly manner.
-
-### Local PostgreSQL Installation
-
-Before using the PostgreSQL connector to monitor the changes committed on a PostgreSQL server, decide which logical decoding plug-in you intend to use. If you plan not to use the native pgoutput logical replication stream support, then you must install the logical decoding plug-in into the PostgreSQL server. Afterward, enable a replication slot, and configure a user with sufficient privileges to perform the replication.
-
-If your database is hosted by a service such as [Heroku Postgres](https://www.heroku.com/postgres) you might be unable to install the plug-in. If so, and if you are using PostgreSQL 10+, you can use the pgoutput decoder support to capture changes in your database. If that is not an option, you are unable to use Debezium with your database.
-
-### PostgreSQL in the Cloud
+The `RealtimeTrigger` collects records from change data capture as they occur. The flow processes each record when it arrives.
-#### PostgreSQL on Amazon RDS
+## PostgreSQL
-It is possible to capture changes in a PostgreSQL database that is running in [Amazon RDS](https://aws.amazon.com/rds/). To do this:
+Before using the Debezium PostgreSQL connector, enable Write-Ahead Logging (WAL) on your PostgreSQL server. The `pgoutput` logical decoding plugin is available natively since PostgreSQL 10 and is the recommended choice — no additional installation required.
-- Set the instance parameter `rds.logical_replication` to `1`.
+See the [Debezium PostgreSQL connector documentation](https://debezium.io/documentation/reference/3.0/connectors/postgresql.html) for complete setup instructions, including cloud-hosted options such as Amazon RDS, Azure Database for PostgreSQL, and CrunchyBridge.
-- Verify that the `wal_level` parameter is set to `logical` by running the query `SHOW wal_level` as the database RDS master user. This might not be the case in multi-zone replication setups. You cannot set this option manually. It is automatically changed when the `rds.logical_replication` parameter is set to `1`. If the `wal_level` is not set to `logical` after you make the preceding change, it is probably because the instance has to be restarted after the parameter group change. Restarts occur during your maintenance window, or you can initiate a restart manually.
-
-- Set the Debezium `plugin.name` parameter to `pgoutput`.
-
-- Initiate logical replication from an AWS account that has the `rds_replication` role. The role grants permissions to manage logical slots and to stream data using logical slots. By default, only the master user account on AWS has the `rds_replication` role on Amazon RDS. To enable a user account other than the master account to initiate logical replication, you must grant the account the rds_replication role. For example, `grant rds_replication to `. You must have `superuser` access to grant the `rds_replication` role to a user. To enable accounts other than the master account to create an initial snapshot, you must grant `SELECT` permission to the accounts on the tables to be captured. For more information about security for PostgreSQL logical replication, see the [PostgreSQL documentation](https://www.postgresql.org/docs/current/logical-replication-security.html).
-
-#### PostgreSQL on Azure
-
-It is possible to use Debezium with [Azure Database for PostgreSQL](https://docs.microsoft.com/azure/postgresql/), which has support for the `pgoutput` logical decoding plug-in, which is supported by Debezium.
-
-Set the Azure replication support to `logical`. You can use the [Azure CLI](https://docs.microsoft.com/en-us/azure/postgresql/concepts-logical#using-azure-cli) or the [Azure Portal](https://docs.microsoft.com/en-us/azure/postgresql/concepts-logical#using-azure-portal) to configure this. For example, to use the Azure CLI, here are the `az postgres server` commands that you need to execute:
-
-```bash
-az postgres server configuration set --resource-group mygroup --server-name myserver --name azure.replication_support --value logical
-
-az postgres server restart --resource-group mygroup --name myserver
-```
-
-#### PostgreSQL on CrunchyBridge
-
-It is possible to use Debezium with [CrunchyBridge](https://crunchybridge.com/); logical replication is already turned on. The `pgoutput` plugin is available. You will have to create a replication user and provide correct privileges.
-
-:::alert{type="info"}
-While using the `pgoutput` plug-in, it is recommended that you configure `filtered` as the `publication.autocreate.mode`. If you use `all_tables`, which is the default value for `publication.autocreate.mode`, and the publication is not found, the connector tries to create one by using `CREATE PUBLICATION FOR ALL TABLES;`, but this fails due to lack of permissions.
-:::
-
-### Installing the logical decoding output plug-in
-
-:::alert{type="info"}
-For more detailed instructions about setting up and testing logical decoding plug-ins, see [Logical Decoding Output Plug-in Installation for PostgreSQL](https://debezium.io/documentation/reference/3.0/postgres-plugins.html).
-:::
-
-Starting with PostgreSQL 9.4, the only way to read changes to the write-ahead-log is to install a logical decoding output plug-in. Plug-ins are written in C, compiled, and installed on the machine that runs the PostgreSQL server. Plug-ins use a number of PostgreSQL specific APIs, as described by the [PostgreSQL documentation](https://www.postgresql.org/docs/current/static/logicaldecoding-output-plugin.html).
-
-The PostgreSQL connector works with one of Debezium’s supported logical decoding plug-ins to receive change events from the database in either the [Protobuf format](https://github.com/google/protobuf) or the [pgoutput](https://github.com/postgres/postgres/blob/master/src/backend/replication/pgoutput/pgoutput.c) format. The `pgoutput` plugin comes out-of-the-box with the PostgreSQL database. For more details on using Protobuf via the `decoderbufs` plug-in, see the plug-in [documentation](https://github.com/debezium/postgres-decoderbufs/blob/main/README.md) which discusses its requirements, limitations, and how to compile it.
-
-For simplicity, Debezium also provides a container image based on the upstream PostgreSQL server image, on top of which it compiles and installs the plug-ins. You can [use this image](https://github.com/debezium/container-images/tree/main/postgres/13) as an example of the detailed steps required for the installation.
-
-:::alert{type="warning"}
-The Debezium logical decoding plug-ins have been installed and tested on only Linux machines. For Windows and other operating systems, different installation steps might be required.
-:::
-
-### Running Debezium tasks on PostgreSQL
-
-Once the WAL is enabled, you can run the Debezium PostgreSQL based tasks and triggers. Here is an example flow using Debezium PostgreSQL Realtime Trigger:
+### Kestra flow
```yaml
id: debezium_postgres
@@ -321,154 +69,13 @@ triggers:
password: pg_passwd
```
-Debezium PostgreSQL Realtime Trigger will collect the records from the change data capture as and when they occur. The flow can then process these records as required.
-
-## Using Debezium with Microsoft SQL Server
-
-For Debezium to capture change events from SQL Server tables, a SQL Server administrator with the necessary privileges must first run a query to enable CDC on the database. The administrator must then enable CDC for each table that you want Debezium to capture.
-
-:::alert{type="info"}
-By default, JDBC connections to Microsoft SQL Server are protected by SSL encryption. If SSL is not enabled for a SQL Server database, or if you want to connect to the database without using SSL, you can disable SSL by setting the value of the `database.encrypt` property in connector configuration to `false`.
-:::
-
-After CDC is applied, it captures all of the `INSERT`, `UPDATE`, and `DELETE` operations that are committed to the tables for which CDC is enabled. The Debezium connector can then capture these events and emit them to Kafka topics.
-
-### Enabling CDC on the SQL Server database
-
-Before you can enable CDC for a table, you must enable it for the SQL Server database. A SQL Server administrator enables CDC by running a system stored procedure. System stored procedures can be run by using SQL Server Management Studio, or by using Transact-SQL.
-
-**Prerequisites**
-
-- You are a member of the sysadmin fixed server role for the SQL Server.
-- You are a db_owner of the database.
-- The SQL Server Agent is running.
-
-:::alert{type="info"}
-The SQL Server CDC feature processes changes that occur in user-created tables only. You cannot enable CDC on the SQL Server master database.
-:::
-
-**Procedure**
-
-1. From the **View** menu in SQL Server Management Studio, click **Template Explorer**.
-
-2. In the **Template Browser**, expand **SQL Server Templates**.
-
-3. Expand **Change Data Capture > Configuration** and then click **Enable Database for CDC**.
-
-4. In the template, replace the database name in the `USE` statement with the name of the database that you want to enable for CDC.
-
-5. Run the stored procedure `sys.sp_cdc_enable_db` to enable the database for CDC.
-
-After the database is enabled for CDC, a schema with the name cdc is created, along with a CDC user, metadata tables, and other system objects.
-
-The following example shows how to enable CDC for the database `MyDB`:
-
-```sql
-USE MyDB
-GO
-EXEC sys.sp_cdc_enable_db
-GO
-```
-
-### Enabling CDC on a SQL Server table
-
-A SQL Server administrator must enable change data capture on the source tables that you want to Debezium to capture. The database must already be enabled for CDC. To enable CDC on a table, a SQL Server administrator runs the stored procedure `sys.sp_cdc_enable_table` for the table. The stored procedures can be run by using SQL Server Management Studio, or by using Transact-SQL. SQL Server CDC must be enabled for every table that you want to capture.
-
-**Prerequisites**
-
-- CDC is enabled on the SQL Server database.
-
-- The SQL Server Agent is running.
-
-- You are a member of the `db_owner` fixed database role for the database.
-
-**Procedure**
-
-1. From the **View** menu in SQL Server Management Studio, click **Template Explorer**.
-
-2. In the **Template Browser**, expand **SQL Server Templates**.
-
-3. Expand **Change Data Capture > Configuration**, and then click **Enable Table Specifying Filegroup Option**.
-
-4. In the template, replace the table name in the `USE` statement with the name of the table that you want to capture.
-
-5. Run the stored procedure sys.sp_cdc_enable_table.
-
-The following example shows how to enable CDC for the table `MyTable`:
-
-```sql
-USE MyDB
-GO
-
-EXEC sys.sp_cdc_enable_table
-@source_schema = N'dbo',
-@source_name = N'MyTable',
-@role_name = N'MyRole',
-@filegroup_name = N'MyDB_CT',
-@supports_net_changes = 0
-GO
-```
-
-**source_name**: Specifies the name of the table that you want to capture.
-
-**role_name**: Specifies a role `MyRole` to which you can add users to whom you want to grant `SELECT` permission on the captured columns of the source table. Users in the `sysadmin` or `db_owner` role also have access to the specified change tables. Set the value of `@role_name` to `NULL`, to allow only members in the `sysadmin` or `db_owner` to have full access to captured information.
-
-**filegroup_name**: Specifies the filegroup where SQL Server places the change table for the captured table. The named filegroup must already exist. It is best not to locate change tables in the same filegroup that you use for source tables.
-
-### Verifying that the user has access to the CDC table
-
-A SQL Server administrator can run a system stored procedure to query a database or table to retrieve its CDC configuration information. The stored procedures can be run by using SQL Server Management Studio, or by using Transact-SQL.
-
-**Prerequisites**
-
-- You have `SELECT` permission on all of the captured columns of the capture instance. Members of the `db_owner` database role can view information for all of the defined capture instances.
-
-- You have membership in any gating roles that are defined for the table information that the query includes.
-
-**Procedure**
-
-1. From the **View** menu in SQL Server Management Studio, click **Object Explorer**.
-
-2. From the **Object Explorer**, expand **Databases**, and then expand your database object, for example, `MyDB`.
+## Microsoft SQL Server
-3. Expand **Programmability > Stored Procedures > System Stored Procedures**.
+Before using the Debezium SQL Server connector, a database administrator must enable CDC on the SQL Server database and on each table to capture. The SQL Server Agent must be running.
-4. Run the `sys.sp_cdc_help_change_data_capture` stored procedure to query the table.
+See the [Debezium SQL Server connector documentation](https://debezium.io/documentation/reference/3.0/connectors/sqlserver.html) for complete setup instructions, including SQL Server on Azure and Always On read-only replicas.
-Queries should not return empty results.
-
-The following example runs the stored procedure `sys.sp_cdc_help_change_data_capture` on the database `MyDB`:
-
-```sql
-USE MyDB;
-GO
-EXEC sys.sp_cdc_help_change_data_capture
-GO
-```
-
-The query returns configuration information for each table in the database that is enabled for CDC and that contains change data that the caller is authorized to access. If the result is empty, verify that the user has privileges to access both the capture instance and the CDC tables.
-
-### SQL Server on Azure
-
-The Debezium SQL Server connector can be used with SQL Server on Azure. Refer to [this example](https://learn.microsoft.com/en-us/samples/azure-samples/azure-sql-db-change-stream-debezium/azure-sql%2D%2Dsql-server-change-stream-with-debezium/) for configuring CDC for SQL Server on Azure and using it with Debezium.
-
-### SQL Server Always On
-
-The SQL Server connector can capture changes from an Always On read-only replica.
-
-**Prerequisites**
-
-- Change data capture is configured and enabled on the primary node. SQL Server does not support CDC directly on replicas.
-
-- The configuration option `database.applicationIntent` is set to `ReadOnly`. This is required by SQL Server. When Debezium detects this configuration option, it responds by taking the following actions:
-
- - Sets `snapshot.isolation.mode` to `snapshot`, which is the only one transaction isolation mode supported for read-only replicas.
-
- - Commits the (read-only) transaction in every execution of the streaming query loop, which is necessary to get the latest view of CDC data.
-
-### Running Debezium tasks on Microsoft SQL Server
-
-You are now all set to run the Debezium Microsoft SQL Server based tasks and triggers. Here is an example flow using Debezium Microsoft SQL Server Realtime Trigger:
+### Kestra flow
```yaml
id: debezium_sqlserver
@@ -489,23 +96,13 @@ triggers:
database: deb
```
-Debezium Microsoft SQL Server Realtime Trigger will collect the records from the change data capture as and when they occur. The flow can then process these records as required.
-
-## Debezium with MongoDB
+## MongoDB
-The MongoDB connector uses MongoDB’s change streams to capture the changes, so the connector works only with MongoDB replica sets or with sharded clusters where each shard is a separate replica set. See the MongoDB documentation for setting up a [replica set](https://docs.mongodb.com/manual/replication/) or [sharded cluster](https://docs.mongodb.com/manual/sharding/). Also, be sure to understand how to enable [access control and authentication](https://docs.mongodb.com/manual/tutorial/deploy-replica-set-with-keyfile-access-control/#deploy-repl-set-with-auth) with replica sets.
+Before using the Debezium MongoDB connector, MongoDB must be configured as a replica set or sharded cluster. Create a MongoDB user with the permissions to read the `admin` database (for oplog access) and `listDatabases`, `find`, and `changeStream` cluster-wide privilege actions.
-You must also have a MongoDB user that has the appropriate roles to read the `admin` database where the oplog can be read. Additionally, the user must also be able to read the `config` database in the configuration server of a sharded cluster and must have `listDatabases` privilege action. When change streams are used (the default) the user also must have cluster-wide privilege actions `find` and `changeStream`.
+See the [Debezium MongoDB connector documentation](https://debezium.io/documentation/reference/3.0/connectors/mongodb.html) for complete setup instructions, including MongoDB Atlas (SSL required).
-When you intend to utilize pre-image and populate the `before` field, you need to first enable `changeStreamPreAndPostImages` for a collection using `db.createCollection()`, `create`, or `collMod`.
-
-### MongoDB in the Cloud
-
-You can use the Debezium connector for MongoDB with [MongoDB Atlas](https://www.mongodb.com/atlas/database). Note that MongoDB Atlas only supports secure connections via SSL, i.e. the `[+mongodb.ssl.enabled](https://debezium.io/documentation/reference/3.0/connectors/mongodb.html#mongodb-property-mongodb-ssl-enabled)` connector option must be set to `true`.
-
-### Running Debezium tasks on MongoDB
-
-You are now all set to run the Debezium MongoDB based tasks and triggers. Here is an example flow using Debezium MongoDB Realtime Trigger:
+### Kestra flow
```yaml
id: debezium_mongodb
@@ -522,5 +119,3 @@ triggers:
snapshotMode: INITIAL
connectionString: mongodb://mongo_user:mongo_passwd@mongos0.example.com:27017,mongos1.example.com:27017/
```
-
-Debezium MongoDB Realtime Trigger will collect the records from the change data capture as and when they occur. The flow can then process these records as required.
diff --git a/src/contents/docs/15.how-to-guides/dynamic-flows/index.md b/src/contents/docs/15.how-to-guides/dynamic-flows/index.md
index 6ecf93d3770..f4ebf779f1f 100644
--- a/src/contents/docs/15.how-to-guides/dynamic-flows/index.md
+++ b/src/contents/docs/15.how-to-guides/dynamic-flows/index.md
@@ -240,7 +240,7 @@ tasks:
- id: create_flow
type: io.kestra.plugin.scripts.shell.Commands
inputFiles:
- flow.yaml: "{{ outputs.create_kestra_flow.outputFiles['kestra_flow.yaml'] }}"
+ flow.yaml: "{{ outputs.generate_kestra_flow.outputFiles['kestra_flow.yaml'] }}"
beforeCommands:
- apt-get update
- apt-get -y install curl
diff --git a/src/contents/docs/15.how-to-guides/dynamic-inputs/index.md b/src/contents/docs/15.how-to-guides/dynamic-inputs/index.md
index 6c2c046fdf4..122d323535b 100644
--- a/src/contents/docs/15.how-to-guides/dynamic-inputs/index.md
+++ b/src/contents/docs/15.how-to-guides/dynamic-inputs/index.md
@@ -205,3 +205,151 @@ inputs:
- `subflow()` is only valid in the `expression:` property of a `SELECT` or `MULTISELECT` input. It throws if used in a task or trigger property.
- The subflow must complete within the timeout (default `PT1M`, max `PT5M`). Keep data-fetching subflows fast.
- Recursion is capped at depth 3.
+- Each subflow referenced in a `SELECT` or `MULTISELECT` expression appears in the parent flow's **Dependencies** graph automatically.
+
+## Conditional inputs
+
+Use `dependsOn` and `condition` to show an input only when a previous input matches a value. The following flow shows different inputs depending on which resource type the user selects:
+
+```yaml
+id: request_resources
+namespace: company.team
+
+inputs:
+ - id: resource_type
+ displayName: Resource type
+ type: SELECT
+ values:
+ - Access permissions
+ - SaaS application
+ - Cloud VM
+
+ - id: access_permissions
+ displayName: Access permissions
+ type: SELECT
+ expression: "{{ kv('access_permissions') }}"
+ dependsOn:
+ inputs:
+ - resource_type
+ condition: "{{ inputs.resource_type == 'Access permissions' }}"
+
+ - id: saas_applications
+ displayName: SaaS application
+ type: MULTISELECT
+ expression: "{{ kv('saas_applications') }}"
+ dependsOn:
+ inputs:
+ - resource_type
+ condition: "{{ inputs.resource_type == 'SaaS application' }}"
+
+ - id: cloud_provider
+ displayName: Cloud provider
+ type: SELECT
+ values:
+ - AWS
+ - GCP
+ - Azure
+ dependsOn:
+ inputs:
+ - resource_type
+ condition: "{{ inputs.resource_type == 'Cloud VM' }}"
+
+ - id: cloud_vm
+ displayName: Cloud VM
+ type: SELECT
+ expression: "{{ kv('cloud_vms')[inputs.cloud_provider] }}"
+ dependsOn:
+ inputs:
+ - resource_type
+ - cloud_provider
+ condition: "{{ inputs.resource_type == 'Cloud VM' }}"
+
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Resource type: {{ inputs.resource_type }}"
+```
+
+`dependsOn.inputs` lists the inputs that must be provided first. `dependsOn.condition` is a Pebble expression that controls visibility — the dependent input only appears in the Execute modal when the condition is `true`. An input can depend on multiple parents; all listed inputs must be provided before the condition is evaluated.
+
+Populate the KV store keys before running the flow:
+
+:::collapse{title="Flow to add key-value pairs"}
+```yaml
+id: add_kv_pairs
+namespace: company.team
+
+tasks:
+ - id: access_permissions
+ type: io.kestra.plugin.core.kv.Set
+ key: "{{ task.id }}"
+ kvType: JSON
+ value: |
+ ["Admin", "Developer", "Editor", "Launcher", "Viewer"]
+
+ - id: saas_applications
+ type: io.kestra.plugin.core.kv.Set
+ key: "{{ task.id }}"
+ kvType: JSON
+ value: |
+ ["Slack", "Notion", "HubSpot", "GitHub", "Jira"]
+
+ - id: cloud_vms
+ type: io.kestra.plugin.core.kv.Set
+ key: "{{ task.id }}"
+ kvType: JSON
+ value: |
+ {
+ "AWS": ["t2.micro", "t2.small", "t2.medium", "t2.large"],
+ "GCP": ["f1-micro", "g1-small", "n1-standard-1", "n1-standard-2"],
+ "Azure": ["Standard_B1s", "Standard_B1ms", "Standard_B2s", "Standard_B2ms"]
+ }
+```
+:::
+
+### dependsOn inside FORM inputs
+
+To make one child input inside a FORM depend on another, use the full dotted path in `dependsOn.inputs`:
+
+```yaml
+inputs:
+ - id: cloud
+ type: FORM
+ displayName: Cloud configuration
+ inputs:
+ - id: provider
+ type: SELECT
+ values: [AWS, GCP, Azure]
+
+ - id: region
+ type: SELECT
+ dependsOn:
+ inputs:
+ - cloud.provider
+ condition: "{{ inputs.cloud.provider == 'AWS' }}"
+ values:
+ - us-east-1
+ - eu-west-1
+```
+
+## Label/value pairs for decoupled dropdowns
+
+When your API returns structured data, use a `{label, value}` jq projection so the dropdown shows a human-readable label while `{{ inputs.x }}` resolves to the underlying technical identifier:
+
+```yaml
+id: dynamic_account_selector
+namespace: company.team
+
+inputs:
+ - id: aws_account
+ type: SELECT
+ displayName: AWS Account
+ expression: "{{ http(uri = 'https://api.example.com/accounts') | jq('.accounts[] | {label: .name, value: .id}') }}"
+
+tasks:
+ - id: log_account
+ type: io.kestra.plugin.core.log.Log
+ message: "Selected account ID: {{ inputs.aws_account }}"
+```
+
+The dropdown displays account names; `{{ inputs.aws_account }}` resolves to the account ID. The same pattern works with static `values` lists — see [Label/value pairs in SELECT and MULTISELECT inputs](../../05.workflow-components/05.inputs/index.md#labelvalue-pairs-in-select-and-multiselect-inputs).
diff --git a/src/contents/docs/15.how-to-guides/etl-pipelines/index.md b/src/contents/docs/15.how-to-guides/etl-pipelines/index.md
index e49f51ab525..c3119b20af8 100644
--- a/src/contents/docs/15.how-to-guides/etl-pipelines/index.md
+++ b/src/contents/docs/15.how-to-guides/etl-pipelines/index.md
@@ -72,7 +72,7 @@ tasks:
accessKeyId: "{{ secret('AWS_ACCESS_KEY_ID') }}"
secretKeyId: "{{ secret('AWS_SECRET_KEY_ID') }}"
region: "eu-central-1"
- from: "{{ outputs.get_orders_per_product_csv.uri }}"
+ from: "{{ outputs.ion_to_csv.uri }}"
bucket: "my_bucket"
key: "orders/detailed_orders"
diff --git a/src/contents/docs/15.how-to-guides/generate-and-pseudonymize-data/index.md b/src/contents/docs/15.how-to-guides/generate-and-pseudonymize-data/index.md
new file mode 100644
index 00000000000..59b1dc72fcf
--- /dev/null
+++ b/src/contents/docs/15.how-to-guides/generate-and-pseudonymize-data/index.md
@@ -0,0 +1,192 @@
+---
+title: Generate and Pseudonymize Test Data in Kestra
+h1: Prepare Realistic Data for Dev and Staging Environments
+description: Use the plugin-datagen plugin to generate realistic fake data from scratch or replace PII fields in existing CSV, JSON, or ION files before moving data across environments.
+icon: /src/contents/docs/icons/tutorial.svg
+stage: Intermediate
+topics:
+ - Data Pipelines
+---
+
+Using real production data in dev or staging environments risks exposing customer PII and violating data protection regulations like GDPR and CCPA.
+
+The `plugin-datagen` plugin solves this with two tasks — choose based on what you already have:
+
+| Situation | Task | What it does |
+|---|---|---|
+| You have a production export | `Pseudonymize` | Replaces PII fields in the file with realistic fakes; all other fields pass through unchanged |
+| You have no production data | `Generate` | Creates a synthetic dataset from scratch using [Datafaker](https://www.datafaker.net/documentation/expressions/) expressions |
+
+Both tasks write output to Kestra internal storage.
+
+## Prerequisites
+
+- A running Kestra instance
+- `plugin-datagen` installed — add it from **Administration → Plugins** or via the CLI:
+
+```bash
+kestra plugins install io.kestra.plugin:plugin-datagen:LATEST
+```
+
+---
+
+## Path A: Replace PII in an existing export
+
+Use `Pseudonymize` when you have a real production file and need to replace sensitive fields before it enters a dev or staging environment. The task reads the file, replaces only the fields you specify using Datafaker expressions, and writes the result back to internal storage.
+
+- `from` — URI of the input file in Kestra internal storage (required)
+- `fields` — map of field names to Datafaker expressions; unlisted fields pass through unchanged
+- `contentType` — `CSV`, `JSON`, or `ION`; auto-detected from the file extension when not set
+- `locale` — optional Faker locale list such as `["en", "US"]` or `["fr", "FR"]`
+
+Outputs: `uri` (the pseudonymized file) and `count` (records processed).
+
+### CSV export
+
+`fields` keys match column headers exactly. The header row is preserved verbatim.
+
+```yaml
+id: pseudonymize_customer_export
+namespace: company.team
+
+tasks:
+ - id: pseudonymize
+ type: io.kestra.plugin.datagen.core.Pseudonymize
+ from: "{{ inputs.file }}"
+ contentType: CSV
+ locale: ["en", "US"]
+ fields:
+ first_name: "#{name.firstName}"
+ last_name: "#{name.lastName}"
+ email: "#{internet.emailAddress}"
+ phone: "#{phoneNumber.cellPhone}"
+ national_id: "#{idNumber.ssnValid}"
+
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Pseudonymized {{ outputs.pseudonymize.count }} records → {{ outputs.pseudonymize.uri }}"
+```
+
+### JSON export with nested fields
+
+Use dot-notation to address nested fields (e.g. `user.profile.email`). A path that does not exist in a given record is silently skipped — the record is written as-is.
+
+Replace `export_task` with the ID of your upstream task that produces the JSON file.
+
+```yaml
+id: pseudonymize_json_export
+namespace: company.team
+
+tasks:
+ - id: pseudonymize
+ type: io.kestra.plugin.datagen.core.Pseudonymize
+ from: "{{ outputs.export_task.uri }}"
+ contentType: JSON
+ locale: ["en", "US"]
+ fields:
+ "user.profile.fullName": "#{name.fullName}"
+ "user.profile.email": "#{internet.emailAddress}"
+ "user.address.city": "#{address.city}"
+ "user.address.zipCode": "#{address.zipCode}"
+
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Pseudonymized {{ outputs.pseudonymize.count }} records → {{ outputs.pseudonymize.uri }}"
+```
+
+### Schedule a nightly refresh
+
+Wrap `Pseudonymize` in a scheduled flow to automatically clean a daily production export before it reaches staging.
+
+Set `vars.daily_export_uri` in the flow's **Variables** tab to the Kestra internal storage URI of your daily export file.
+
+```yaml
+id: nightly_pseudonymize_export
+namespace: company.team
+
+triggers:
+ - id: nightly
+ type: io.kestra.plugin.core.trigger.Schedule
+ cron: "0 2 * * *"
+
+tasks:
+ - id: pseudonymize
+ type: io.kestra.plugin.datagen.core.Pseudonymize
+ from: "{{ vars.daily_export_uri }}"
+ contentType: CSV
+ locale: ["en", "US"]
+ fields:
+ customer_name: "#{name.fullName}"
+ customer_email: "#{internet.emailAddress}"
+ customer_phone: "#{phoneNumber.cellPhone}"
+ national_id: "#{idNumber.ssnValid}"
+
+ - id: load_to_staging
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ outputs.pseudonymize.count }} records ready at {{ outputs.pseudonymize.uri }}"
+```
+
+---
+
+## Path B: Generate a synthetic dataset from scratch
+
+Use `Generate` with `JsonObjectGenerator` when you have no production data to start from. Define a template map and use Datafaker expressions (`#{...}`) for any field that should vary per record. Set `store: true` and `batchSize` to write a full batch to internal storage.
+
+Outputs: `uri` (the generated ION file) and `count` (records written).
+
+```yaml
+id: generate_staging_data
+namespace: company.team
+
+tasks:
+ - id: generate
+ type: io.kestra.plugin.datagen.core.Generate
+ store: true
+ batchSize: 1000
+ generator:
+ type: io.kestra.plugin.datagen.generators.JsonObjectGenerator
+ locale: ["en", "US"]
+ value:
+ customer_id: "#{number.numberBetween '10000','99999'}"
+ name: "#{name.fullName}"
+ email: "#{internet.emailAddress}"
+ phone: "#{phoneNumber.cellPhone}"
+ tier: "#{options.option 'free','pro','enterprise'}"
+ address:
+ city: "#{address.city}"
+ zip: "#{address.zipCode}"
+
+ - id: load_to_staging
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ outputs.generate.count }} synthetic records ready at {{ outputs.generate.uri }}"
+```
+
+:::alert{type="info"}
+`Generate` always writes ION format when `store: true`. Pass `{{ outputs.generate.uri }}` to any downstream task that reads from Kestra internal storage.
+:::
+
+---
+
+## Datafaker expressions
+
+Both tasks use [Datafaker](https://www.datafaker.net/documentation/expressions/) expressions in the format `#{category.method}`.
+
+| Expression | Example output |
+|---|---|
+| `#{name.fullName}` | `Jane Doe` |
+| `#{name.firstName}` | `Jane` |
+| `#{internet.emailAddress}` | `jane.doe@example.com` |
+| `#{phoneNumber.cellPhone}` | `(555) 867-5309` |
+| `#{address.city}` | `Springfield` |
+| `#{address.zipCode}` | `90210` |
+| `#{address.fullAddress}` | `123 Main St, Springfield, IL 90210` |
+| `#{idNumber.ssnValid}` | `123-45-6789` |
+| `#{commerce.department}` | `Electronics` |
+| `#{number.numberBetween '1','100'}` | `42` |
+| `#{options.option 'free','pro','enterprise'}` | `pro` |
+
+For the full provider list, see the [Datafaker documentation](https://www.datafaker.net/documentation/expressions/).
+
+If an expression cannot be evaluated, the raw expression string is written to the output field rather than throwing an error.
+
+Both tasks accept a `locale` list to control name, address, and phone formatting — for example, `["fr", "FR"]` for French output or `["de", "DE"]` for German. Omit `locale` to use Faker's default (English).
diff --git a/src/contents/docs/15.how-to-guides/github-repo-backup/index.md b/src/contents/docs/15.how-to-guides/github-repo-backup/index.md
index 0794aa1c715..19d0d6d9d58 100644
--- a/src/contents/docs/15.how-to-guides/github-repo-backup/index.md
+++ b/src/contents/docs/15.how-to-guides/github-repo-backup/index.md
@@ -42,13 +42,13 @@ The `repositories.Search` task exposes a `fetchType` property that controls how
| `STORE` (default) | `uri` — an Ion file written to Kestra internal storage | Large result sets, auditing, or when you need to persist the raw data |
| `NONE` | _(empty)_ | Triggering a side-effect without needing results |
-Use `FETCH` when you want to feed results directly into a ForEach loop with a simple Pebble expression. Use `STORE` when the result set may be large, when you want the raw file persisted in internal storage for inspection or reuse, or when downstream tasks need to read the data multiple times.
+Use `FETCH` when you want to feed results directly into a Loop task with a simple Pebble expression. Use `STORE` when the result set may be large, when you want the raw file persisted in internal storage for inspection or reuse, or when downstream tasks need to read the data multiple times.
---
## Flow Definition — FETCH mode
-`fetchType: FETCH` places results directly in `outputs.search_kestra_repos.rows` as a list of objects. The ForEach `values` expression reads from that list without any file I/O step.
+`fetchType: FETCH` places results directly in `outputs.search_kestra_repos.rows` as a list of objects. The Loop `values` expression reads from that list without any file I/O step.
```yaml
id: github_repo_backup
@@ -64,7 +64,7 @@ tasks:
oauthToken: "{{ secret('GITHUB_TOKEN') }}"
- id: for_each_repo
- type: io.kestra.plugin.core.flow.ForEach
+ type: io.kestra.plugin.core.flow.Loop
description: Iterate over each found repository.
values: "{{ outputs.search_kestra_repos.rows | jq('.[].clone_url') }}"
tasks:
@@ -75,8 +75,8 @@ tasks:
- id: clone_repo
type: io.kestra.plugin.git.Clone
description: Clone the current repository from GitHub.
- url: "{{ taskrun.value }}"
- directory: "{{ taskrun.value | split('/') | last | split('.') | first }}"
+ url: "{{ item.value }}"
+ directory: "{{ item.value | split('/') | last | split('.') | first }}"
- id: zip_repo
type: io.kestra.plugin.scripts.shell.Commands
@@ -120,7 +120,7 @@ tasks:
oauthToken: "{{ secret('GITHUB_TOKEN') }}"
- id: for_each_repo
- type: io.kestra.plugin.core.flow.ForEach
+ type: io.kestra.plugin.core.flow.Loop
description: Iterate over each found repository.
values: "{{ outputs.search_kestra_repos.uri | internalStorage.get() | jq('.[].clone_url') }}"
tasks:
@@ -131,8 +131,8 @@ tasks:
- id: clone_repo
type: io.kestra.plugin.git.Clone
description: Clone the current repository from GitHub.
- url: "{{ taskrun.value }}"
- directory: "{{ taskrun.value | split('/') | last | split('.') | first }}"
+ url: "{{ item.value }}"
+ directory: "{{ item.value | split('/') | last | split('.') | first }}"
- id: zip_repo
type: io.kestra.plugin.scripts.shell.Commands
@@ -162,7 +162,7 @@ tasks:
Both variants share the same structure. The only difference is how the search results move from the `search_kestra_repos` task to the `for_each_repo` loop.
-With `FETCH`, results live in `outputs.search_kestra_repos.rows` as a native list — no file read needed. With `STORE`, results are written to an Ion file and the ForEach expression reads the file via `internalStorage.get()` before applying the `jq` filter.
+With `FETCH`, results live in `outputs.search_kestra_repos.rows` as a native list — no file read needed. With `STORE`, results are written to an Ion file and the Loop `values` expression reads the file via `internalStorage.get()` before applying the `jq` filter.
In both cases:
diff --git a/src/contents/docs/15.how-to-guides/golang/index.md b/src/contents/docs/15.how-to-guides/golang/index.md
deleted file mode 100644
index 9bbe06d0a6c..00000000000
--- a/src/contents/docs/15.how-to-guides/golang/index.md
+++ /dev/null
@@ -1,217 +0,0 @@
----
-title: Run Go Inside Your Flows
-h1: Execute Go Scripts in Kestra Flows Using the Go Plugin
-icon: /src/contents/docs/icons/golang.svg
-stage: Getting Started
-topics:
- - Scripting
-description: Run Go code directly within Kestra flows for high-performance scripting using the Go plugin or inline scripts.
----
-
-Run Go code directly inside your Flows and generate outputs.
-
-
-
-
-
-Go is a powerful programming language often used for cloud-native development, CLI utilities, and more. As Go is compiled, it's often much more performant than Python, making it a great alternative for heavy compute workloads. Combining Go's and Kestra's performance, you can build incredibly fast workflows.
-
-This guide is going to walk you through how to get Go running in a workflow, how to manage input and output files, and how you can pass outputs and metrics back to Kestra to use in later tasks.
-
-## Commands Task
-
-There is an official Go plugin with a `Commands` task and an inline `Script` task. This example executes a Namespace file using `Commands`:
-
-```yaml
-id: golang_commands
-namespace: company.team
-
-tasks:
- - id: go
- type: io.kestra.plugin.scripts.go.Commands
- namespaceFiles:
- enabled: true
- commands:
- - go run main.go
-```
-
-The contents of the `main.go` file contains a simple print statement:
-
-```go
-package main
-import "fmt"
-func main() {
- fmt.Println("hello world")
-}
-```
-
-You'll need to add your Golang code using the built-in Editor or [sync it using Git](../../version-control-cicd/04.git/index.md) so Kestra can see it. You'll also need to set the `enabled` flag for the `namespaceFiles` property to `true` so Kestra can access the file.
-
-You can read more about the Go Commands type in the [Plugin documentation](/plugins/plugin-script-go/io.kestra.plugin.scripts.go.commands).
-
-## Script
-
-You can also add your Golang code inline using the `Script` task.
-
-```yaml
-id: golang_script
-namespace: company.team
-
-tasks:
- - id: go
- type: io.kestra.plugin.scripts.go.Script
- script: |
- package main
- import "fmt"
-
- func main() {
- fmt.Println("hello world")
- }
-```
-
-You can also use expressions directly inside your Go code. In this example, inputs are embedded directly into the code:
-
-```yaml
-id: golang_script_expression
-namespace: company.team
-
-inputs:
- - id: message
- type: STRING
- defaults: "Hello, World!"
-
- - id: number
- type: INT
- defaults: 4
-
-tasks:
- - id: go
- type: io.kestra.plugin.scripts.go.Script
- script: |
- package main
- import "fmt"
-
- func main() {
- fmt.Println("Message: {{ inputs.message }}")
- fmt.Println("Number: {{ inputs.number }}")
- }
-```
-
-You can read more about the Go Script type in the [Plugin documentation](/plugins/plugin-script-go/io.kestra.plugin.scripts.go.script).
-
-
-## Handling Outputs
-
-If you want to get a variable or file from your Golang code, you can use an [output](../../05.workflow-components/06.outputs/index.md).
-
-### Variable Output
-
-You can get the JSON outputs from the Golang script using the `::{}::` pattern. Here is an example:
-
-```yaml
-id: golang_outputs
-namespace: company.team
-
-tasks:
- - id: go
- type: io.kestra.plugin.scripts.go.Script
- script: |
- package main
- import "fmt"
-
- func main() {
- fmt.Println("::{\"outputs\":{\"test\":\"value\",\"int\":2,\"bool\":true,\"float\":3.65}}::")
- }
-```
-
-All the output variables can be viewed in the Outputs tab of the execution.
-
-
-
-You can refer to the outputs in another task as shown in the example below:
-
-```yaml
-id: golang_outputs_usage
-namespace: company.team
-
-tasks:
- - id: go
- type: io.kestra.plugin.scripts.go.Script
- script: |
- package main
- import "fmt"
-
- func main() {
- fmt.Println("::{\"outputs\":{\"test\":\"value\",\"int\":2,\"bool\":true,\"float\":3.65}}::")
- }
-
- - id: return
- type: io.kestra.plugin.core.debug.Return
- format: '{{ outputs.go.vars.test }}'
-```
-
-### File Output
-
-Inside of your Golang code, write a file to the system. You'll need to add the `outputFiles` property to your flow and list the files you're trying to put out. In this case, we want to output `output.txt`. More information on the formats you can use for this property can be found in [Script Output Metrics](../../16.scripts/06.outputs-metrics/index.md).
-
-The example below writes a `output.txt` file containing the "Hello World" text. We can then refer the file using the syntax `{{ outputs.{task_id}.outputFiles[''] }}`, and read the contents of the file using the `read()` function.
-
-```yaml
-id: golang_script
-namespace: company.team
-
-tasks:
- - id: go
- type: io.kestra.plugin.scripts.go.Script
- outputFiles:
- - output.txt
- script: |
- package main
-
- import (
- "os"
- )
-
- func check(e error) {
- if e != nil {
- panic(e)
- }
- }
-
- func main() {
- d1 := []byte("hello go")
- err := os.WriteFile("output.txt", d1, 0644)
- check(err)
- }
-
- - id: log
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(outputs.go.outputFiles['output.txt']) }}"
-```
-
-## Handling Metrics
-
-You can also get [metrics](../../16.scripts/06.outputs-metrics/index.md#outputs-and-metrics-in-script-and-commands-tasks) from your Golang code. Metrics use the same `::{}::` pattern as outputs. This example demonstrates both the counter and timer metrics.
-
-```yaml
-id: golang
-namespace: company.team
-
-tasks:
- - id: go
- type: io.kestra.plugin.scripts.go.Script
- script: |
- package main
- import "fmt"
-
- func main() {
- fmt.Println("There are 20 products in the cart")
- fmt.Println("::{\"outputs\":{\"productCount\":20}}::")
- fmt.Println("::{\"metrics\":[{\"name\":\"productCount\",\"type\":\"counter\",\"value\":20}]}::")
- fmt.Println("::{\"metrics\":[{\"name\":\"purchaseTime\",\"type\":\"timer\",\"value\":32.44}]}::")
- }
-```
-
-Once this has executed, both the metrics can be viewed under **Metrics**.
-
-
diff --git a/src/contents/docs/15.how-to-guides/golang/metrics.png b/src/contents/docs/15.how-to-guides/golang/metrics.png
deleted file mode 100644
index 5d6ba236657..00000000000
Binary files a/src/contents/docs/15.how-to-guides/golang/metrics.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/golang/outputs.png b/src/contents/docs/15.how-to-guides/golang/outputs.png
deleted file mode 100644
index 8d640b4ab38..00000000000
Binary files a/src/contents/docs/15.how-to-guides/golang/outputs.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/google-credentials/index.md b/src/contents/docs/15.how-to-guides/google-credentials/index.md
index 6503adceaa0..74467b9dbde 100644
--- a/src/contents/docs/15.how-to-guides/google-credentials/index.md
+++ b/src/contents/docs/15.how-to-guides/google-credentials/index.md
@@ -119,9 +119,27 @@ With this, we can add this to the `serviceAccount` property like so:
serviceAccount: "{{ secret('GCP_SERVICE_ACCOUNT') }}"
```
-## Set the Service Account with `PluginDefaults`
+## Centralize the service account with a Policy (Enterprise Edition)
+
+If you use multiple tasks that require the same service account, create a [Policy](../../07.enterprise/02.governance/policies/index.md) at the namespace level to inject it automatically:
+
+```yaml
+id: gcp-service-account
+description: "Inject GCP service account into all Google Workspace Drive tasks."
+enforcement: ACTIVE
+rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.googleworkspace.drive
+ values:
+ serviceAccount: "{{ secret('GCP_SERVICE_ACCOUNT') }}"
+```
+
+With this Policy applied, tasks in the namespace need no `serviceAccount` property:
-If you're using multiple tasks that will require the service account secret, you can set up a Plugin Default to apply this property to all tasks of this type. For example:
```yaml
tasks:
- id: upload
@@ -132,16 +150,15 @@ tasks:
name: "My awesome CSV"
contentType: "text/csv"
mimeType: "application/vnd.google-apps.spreadsheet"
-
-pluginDefaults:
- - type: io.kestra.plugin.googleworkspace.drive.Upload
- values:
- serviceAccount: "{{ secret('GCP_SERVICE_ACCOUNT') }}"
```
## Configuring Secrets in the Enterprise Edition
-In Kestra Enterprise Edition, secrets can be managed directly from the UI meaning there's no need to encode them in base64. To learn more about this, see the [secrets page](../../06.concepts/04.secret/index.md#enterprise-edition).
+In Kestra Enterprise Edition, secrets are managed directly from the UI — no base64 encoding required. Navigate to **Namespaces**, open the namespace where your flow runs, and go to the **Secrets** tab. Click **New secret**, set the key to `GCP_SERVICE_ACCOUNT`, and paste the service account JSON directly as the value.
+
+Reference it in your tasks with `{{ secret('GCP_SERVICE_ACCOUNT') }}`, exactly as shown in the examples above.
+
+To learn more about secret backends and enterprise secret managers, see the [secrets page](../../06.concepts/04.secret/index.md#enterprise-edition).
## `GOOGLE_APPLICATION_CREDENTIALS`
diff --git a/src/contents/docs/15.how-to-guides/idempotency/index.md b/src/contents/docs/15.how-to-guides/idempotency/index.md
index 8915560bf47..b17e8e0915d 100644
--- a/src/contents/docs/15.how-to-guides/idempotency/index.md
+++ b/src/contents/docs/15.how-to-guides/idempotency/index.md
@@ -2,7 +2,7 @@
title: Prevent Duplicate Executions with Correlation IDs
icon: /src/contents/docs/icons/tutorial.svg
stage: Intermediate
-editions: ["EE"]
+editions: ["EE", "Cloud"]
topics:
- Kestra Concepts
---
diff --git a/src/contents/docs/15.how-to-guides/inputs-api/index.md b/src/contents/docs/15.how-to-guides/inputs-api/index.md
deleted file mode 100644
index 687dfbc5d41..00000000000
--- a/src/contents/docs/15.how-to-guides/inputs-api/index.md
+++ /dev/null
@@ -1,73 +0,0 @@
----
-title: Pass Inputs via an API Call
-h1: Parameterize Flow Executions by Passing Inputs via API
-icon: /src/contents/docs/icons/api.svg
-stage: Getting Started
-topics:
- - Kestra Workflow Components
-description: Learn how to pass dynamic inputs to Kestra flow executions via API calls to parameterize your workflows at runtime.
----
-
-Passing Inputs via an API Call
-
-Inputs allow you to dynamically pass data to your execution at runtime. For a detailed overview of inputs, see the [Inputs](../../05.workflow-components/05.inputs/index.md) documentation page.
-
-## Example
-
-If you want to trigger a flow and change the value for an input, you can do so by triggering your flow by the API and passing your new input in the form data.
-
-Take the following flow as an example:
-
-```yaml
-id: inputs_demo
-namespace: company.team
-
-inputs:
- - id: user
- type: STRING
- defaults: Rick Astley
-
-tasks:
- - id: hello
- type: io.kestra.plugin.core.log.Log
- message: Hey there, {{ inputs.user }}
-```
-
-This flow has the input `user` which we can modify via an API call. We can do that by triggering this flow and passing our new input using the form data.
-
-```yaml
-id: input_api
-namespace: company.team
-tasks:
- - id: basic_auth_api
- type: io.kestra.plugin.core.http.Request
- uri: http://host.docker.internal:8080/api/v1/main/executions/dev/inputs_demo
- method: POST
- contentType: multipart/form-data
- formData:
- user: John Doe
-```
-
-The above example assumes you are running Kestra locally in Docker. If you are running Kestra in a different environment, replace `http://host.docker.internal:8080` with the correct URL.
-
-If you configured basic authentication for your Kestra instance, you can add the `basicAuthUser` and `basicAuthPassword` options to the `Request` task:
-
-```yaml
-id: api_call
-namespace: company.team
-tasks:
- - id: basic_auth_api
- type: io.kestra.plugin.core.http.Request
- uri: http://host.docker.internal:8080/api/v1/main/executions/dev/inputs_demo
- options:
- basicAuthUser: admin
- basicAuthPassword: admin
- method: POST
- contentType: multipart/form-data
- formData:
- user: John Doe
-```
-
-When you execute the `api_call` flow, this will execute the `input_api` flow with the new `user` input
-
-
diff --git a/src/contents/docs/15.how-to-guides/inputs-api/input_api_log.png b/src/contents/docs/15.how-to-guides/inputs-api/input_api_log.png
deleted file mode 100644
index ab64ba5e3ff..00000000000
Binary files a/src/contents/docs/15.how-to-guides/inputs-api/input_api_log.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/inputs-enum/index.md b/src/contents/docs/15.how-to-guides/inputs-enum/index.md
deleted file mode 100644
index 2c88e53d2a1..00000000000
--- a/src/contents/docs/15.how-to-guides/inputs-enum/index.md
+++ /dev/null
@@ -1,66 +0,0 @@
----
-title: Validate Inputs with the Enum Data Type
-h1: Restrict Input Values Using the Enum Type for Validation
-description: Use ENUM-type inputs in Kestra to restrict flow parameters to a predefined set of values, improving validation and reducing runtime configuration errors.
-icon: /src/contents/docs/icons/tutorial.svg
-stage: Getting Started
-topics:
- - Kestra Workflow Components
----
-
-Input validation with the Enum data type
-
-Inputs allow you to dynamically pass data to your execution at runtime. For a detailed overview of inputs, see the [Inputs](../../05.workflow-components/05.inputs/index.md) documentation page.
-
-## Input validation with Enum data type
-
-The following example shows how to use the `ENUM` input type and the `Switch` task to validate user input and conditionally branch the flow based on the input value.
-
-```yaml
-id: orchestrate_everything
-namespace: company.team
-
-inputs:
- - id: use_case
- description: What do you want to orchestrate?
- type: ENUM
- defaults: Data pipelines
- values:
- - Data pipelines
- - Microservices
- - Business processes
- - Marketing automation
-
-tasks:
- - id: conditional_branching
- type: io.kestra.plugin.core.flow.Switch
- value: "{{ inputs.use_case }}"
- cases:
- Data pipelines:
- - id: data_pipelines
- type: io.kestra.plugin.core.log.Log
- message: Managing important data products
-
- Microservices:
- - id: microservices
- type: io.kestra.plugin.core.log.Log
- message: Orchestrating critical applications
-
- Business processes:
- - id: business_processes
- type: io.kestra.plugin.core.log.Log
- message: Orchestrating critical applications
-
- Marketing automation:
- - id: marketing_automation
- type: io.kestra.plugin.core.log.Log
- message: Orchestrating critical applications
-```
-
-You can add an arbitrary number of cases to the `Switch` task, and each case can contain one or more tasks.
-
-By using the `defaults` attribute, you can specify a default input value that will be prefilled in the dropdown menu in the UI when executing the flow.
-
-:::alert{type="info"}
-It's not possible to launch a workflow execution without selecting a value from the dropdown menu. The requirement for selecting a value guarantees that the flow is only executed with valid input `values` defined by the `ENUM` type.
-:::
diff --git a/src/contents/docs/15.how-to-guides/javascript/index.md b/src/contents/docs/15.how-to-guides/javascript/index.md
index fdd43f8a785..e62168d113f 100644
--- a/src/contents/docs/15.how-to-guides/javascript/index.md
+++ b/src/contents/docs/15.how-to-guides/javascript/index.md
@@ -240,6 +240,58 @@ Kestra.timer('duration', end - start);
Once this has executed, `duration` will be viewable under **Metrics**.
+## Automate JavaScript with triggers
+
+You can also use JavaScript itself as polling logic by using `ScriptTrigger` or `CommandsTrigger`. These trigger types run Node.js code on an interval and start a flow execution only when the `exitCondition` matches.
+
+Use `ScriptTrigger` for inline Node.js code:
+
+```yaml
+id: node_script_trigger
+namespace: company.team
+
+triggers:
+ - id: script_failure
+ type: io.kestra.plugin.scripts.node.ScriptTrigger
+ interval: PT10S
+ exitCondition: "exit 1"
+ edge: true
+ script: |
+ throw new Error("boom");
+
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Triggered with exitCode={{ trigger.exitCode }} (condition={{ trigger.condition }})"
+```
+
+Use `CommandsTrigger` when you want to run Node.js commands instead:
+
+```yaml
+id: node_commands_trigger
+namespace: company.team
+
+triggers:
+ - id: on_fail
+ type: io.kestra.plugin.scripts.node.CommandsTrigger
+ interval: PT5S
+ exitCondition: "exit 1"
+ edge: true
+ commands:
+ - node -e "throw new Error('boom')"
+
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Triggered with exitCode={{ trigger.exitCode }} (condition={{ trigger.condition }})"
+```
+
+These trigger types support:
+
+- `interval` to control how often the script or commands run
+- `exitCondition` to match an exit code such as `exit 1`, or a regex or substring matched against emitted vars and failure logs
+- `edge` to emit only on a transition from not matching to matching
+
## Execute GraalVM Task
Kestra also supports GraalVM integration, allowing you to execute JavaScript code directly on the JVM, with the potential for performance improvements. There are currently two tasks:
@@ -258,7 +310,7 @@ tasks:
uri: http://xkcd.com/info.0.json
- id: graal
- type: io.kestra.plugin.graalvm.python.Eval
+ type: io.kestra.plugin.graalvm.js.Eval
outputs:
- data
script: |
diff --git a/src/contents/docs/15.how-to-guides/julia/index.md b/src/contents/docs/15.how-to-guides/julia/index.md
deleted file mode 100644
index 30c948c21f0..00000000000
--- a/src/contents/docs/15.how-to-guides/julia/index.md
+++ /dev/null
@@ -1,184 +0,0 @@
----
-title: Run Julia Inside Your Flows
-h1: Execute Julia Scripts Directly in Kestra
-description: Execute Julia scripts in Kestra for scientific computing and data analysis. Use Docker to manage Julia dependencies and pass results between tasks.
-icon: /src/contents/docs/icons/julia.svg
-stage: Getting Started
-topics:
- - Scripting
----
-
-Run Julia code directly in your flows and generate outputs.
-
-Julia is renowned for high-performance numerical analysis and computational science. Leverage Kestra to orchestrate your Julia scripts, enhancing their capabilities in large-scale analytics and machine learning applications. From data ingestion to complex numerical simulations, Kestra streamlines your Julia workflows, accelerating development and deployment.
-
-This guide is going to walk you through how to get Julia running in a workflow, how to manage input and output files, and how you can pass outputs and metrics back to Kestra to use in later tasks.
-
-## Executing Julia inside Kestra
-
-Kestra has an official plugin for Julia allowing you to execute Julia code in a flow by either writing your Julia inline or by executing a `.jl` file. You can get outputs and metrics from your Julia code too.
-
-### Scripts
-
-If you want to write a short amount of Julia code to perform a task, you can use the `io.kestra.plugin.scripts.julia.Script` type to write it directly inside your flow. This allows you to keep everything in one place.
-
-```yaml
-id: julia_script
-namespace: company.team
-description: This flow runs the Julia script.
-
-tasks:
- - id: http_download
- type: io.kestra.plugin.core.http.Download
- uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
-
- - id: julia_script_task
- type: io.kestra.plugin.scripts.julia.Script
- script: |
- println("The current execution is {{ execution.id }}")
-
- # Read the file downloaded in `http_download` task
- lines = readlines("{{ outputs.http_download.uri }}")
- println(lines)
-```
-
-You can read more about the Scripts type in the [Plugin documentation](/plugins/plugin-script-julia/io.kestra.plugin.scripts.julia.script)
-
-### Commands
-
-If you would prefer to put your Julia code in a `.jl` file (e.g. your code is much longer or spread across multiple files), you can run the previous example using the `io.kestra.plugin.scripts.julia.Commands` type:
-
-```yaml
-id: julia_commands
-namespace: company.team
-tasks:
- - id: run_julia
- type: io.kestra.plugin.scripts.julia.Commands
- namespaceFiles:
- enabled: true
- commands:
- - julia main.jl
-```
-
-You'll need to add your Julia code using the Editor or [sync it using Git](../../version-control-cicd/04.git/index.md) so Kestra can see it. You'll also need to set the `enabled` flag for the `namespaceFiles` property to `true` so Kestra can access the file.
-
-You can also have the Julia code written inline.
-
-```yaml
-id: julia_commands
-namespace: company.team
-tasks:
- - id: http_download
- type: io.kestra.plugin.core.http.Download
- uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
-
- - id: run_julia
- type: io.kestra.plugin.scripts.julia.Commands
- inputFiles:
- orders.csv: "{{ read(outputs.http_download.uri) }}"
- main.jl: |
- println("The current execution is {{ execution.id }}")
-
- # Read the file downloaded in `http_download` task
- lines = readlines("orders.csv")
- println(lines)
- commands:
- - julia main.jl
-```
-
-You can read more about the Commands type in the [Plugin documentation](/plugins/plugin-script-julia/io.kestra.plugin.scripts.julia.commands).
-
-## Handling Outputs
-
-If you want to get a variable or file from your Julia script, you can use an [output](../../05.workflow-components/06.outputs/index.md).
-
-### Variable Output
-
-You can get the JSON outputs from the Julia commands / script using the `::{}::` pattern. Here is an example:
-
-```yaml
-id: julia_outputs
-namespace: company.team
-description: This flow runs the Julia script, and outputs the variable.
-
-tasks:
- - id: julia_outputs_task
- type: io.kestra.plugin.scripts.julia.Script
- script: |
- println("::{\"outputs\":{\"test\":\"value\",\"int\":2,\"bool\":true,\"float\":3.65}}::")
-```
-
-All the output variables can be viewed in the Outputs tab of the execution.
-
-
-
-You can refer to the outputs in another task as shown in the example below:
-
-```yaml
-id: julia_outputs_usage
-namespace: company.team
-description: This flow runs the Julia script, and outputs the variable.
-
-tasks:
- - id: julia_outputs_task
- type: io.kestra.plugin.scripts.julia.Script
- script: |
- println("::{\"outputs\":{\"test\":\"value\",\"int\":2,\"bool\":true,\"float\":3.65}}::")
-
- - id: return
- type: io.kestra.plugin.core.debug.Return
- format: '{{ outputs.julia_outputs_task.vars.test }}'
-```
-
-_This example works for both `io.kestra.plugin.scripts.julia.Script` and `io.kestra.plugin.scripts.julia.Commands`._
-
-### File Output
-
-Inside of your Julia script, write a file to the system. You'll need to add the `outputFiles` property to your flow and list the files you're trying to put out. In this case, we want to output `output.txt`. More information on the formats you can use for this property can be found in [Script Output Metrics](../../16.scripts/06.outputs-metrics/index.md).
-
-The example below writes a `output.txt` file containing the "Hello World" text. We can then refer the file using the syntax `{{ outputs.{task_id}.outputFiles[''] }}`, and read the contents of the file using the `read()` function.
-
-```yaml
-id: julia_output_file
-namespace: company.team
-description: This flow runs the Julia script to output a file.
-
-tasks:
- - id: julia_outputs_task
- type: io.kestra.plugin.scripts.julia.Script
- outputFiles:
- - output.txt
- script: |
- open("output.txt", "w") do file
- write(file, "Hello World")
- end
-
- - id: log_output
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(outputs.julia_outputs_task.outputFiles['output.txt']) }}"
-```
-
-_This example works for both `io.kestra.plugin.scripts.julia.Script` and `io.kestra.plugin.scripts.julia.Commands`._
-
-## Handling Metrics
-
-You can also get [metrics](../../16.scripts/06.outputs-metrics/index.md#outputs-and-metrics-in-script-and-commands-tasks) from your Julia script. Metrics use the same `::{}::` pattern as outputs. This example demonstrates both the counter and timer metrics.
-
-```yaml
-id: julia_metrics
-namespace: company.team
-description: This flow runs the Julia script, and puts out the metrics.
-
-tasks:
- - id: julia_metrics_task
- type: io.kestra.plugin.scripts.julia.Script
- script: |
- println("There are 20 products in the cart")
- println("::{\"outputs\":{\"productCount\":20}}::")
- println("::{\"metrics\":[{\"name\":\"productCount\",\"type\":\"counter\",\"value\":20}]}::")
- println("::{\"metrics\":[{\"name\":\"purchaseTime\",\"type\":\"timer\",\"value\":32.44}]}::")
-```
-
-Once this has executed, both the metrics can be viewed under **Metrics**.
-
-
diff --git a/src/contents/docs/15.how-to-guides/julia/metrics.png b/src/contents/docs/15.how-to-guides/julia/metrics.png
deleted file mode 100644
index 2a4059e18c9..00000000000
Binary files a/src/contents/docs/15.how-to-guides/julia/metrics.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/julia/outputs.png b/src/contents/docs/15.how-to-guides/julia/outputs.png
deleted file mode 100644
index 32022602977..00000000000
Binary files a/src/contents/docs/15.how-to-guides/julia/outputs.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/long-running-intensive-tasks/index.md b/src/contents/docs/15.how-to-guides/long-running-intensive-tasks/index.md
index e314902512b..3d161197b14 100644
--- a/src/contents/docs/15.how-to-guides/long-running-intensive-tasks/index.md
+++ b/src/contents/docs/15.how-to-guides/long-running-intensive-tasks/index.md
@@ -98,7 +98,7 @@ At the end of the execution, the pod is deleted, and the logs remain available i
-## Embrace Kestra versality with Kubernetes Task Runners
+## Embrace Kestra versatility with Kubernetes Task Runners
While podCreate provides deep control, it takes aways all the benefits of Kestra's rich plugins ecosystem, [dbt plugin](/plugins/plugin-dbt/dbt-cli/io.kestra.plugin.dbt.cli.dbtcli) in this case. Also it can be cumbersome to manage complex Kubernetes pod YAML specification for each task, especially when you have multiple commands to run.
diff --git a/src/contents/docs/15.how-to-guides/loop/index.md b/src/contents/docs/15.how-to-guides/loop/index.md
index 48a15d28366..ec0c9cce1e7 100644
--- a/src/contents/docs/15.how-to-guides/loop/index.md
+++ b/src/contents/docs/15.how-to-guides/loop/index.md
@@ -1,16 +1,14 @@
---
title: Loop Over a List of Values
-h1: Iterate Over Lists with the ForEach Task
+h1: Iterate Over Lists with the Loop Task
icon: /src/contents/docs/icons/tutorial.svg
stage: Intermediate
topics:
- Kestra Workflow Components
-description: Learn how to iterate over lists of values in Kestra workflows using the ForEach task to execute tasks for each item efficiently.
+description: Learn how to iterate over a list of values in Kestra workflows using the Loop task, access iteration context, collect outputs, and run iterations in parallel.
---
-How to iterate over a list of values in your flow.
-
-In this guide, you will learn how to iterate over a list of values using the `ForEach` task. This task enables you to loop through a list of values and execute specific tasks for each value in the list. This approach is useful for scenarios where multiple similar tasks need to be run for different inputs.
+Use the `Loop` task to iterate over a list of values and run tasks for each item. Each iteration runs as an isolated sub-execution with access to the current value via `item.value` and the zero-based index via `item.index`.
## Prerequisites
@@ -19,70 +17,160 @@ Before you begin:
- Deploy [Kestra](../../02.installation/index.mdx) in your preferred development environment.
- Ensure you have a [basic understanding of how to run Kestra flows.](../../03.tutorial/index.mdx)
-## Loop over nested lists of values
+## Basic iteration
+
+The simplest use of `Loop` iterates over a static list and runs child tasks for each item. The example below makes an API call for each author in the list.
+
+```yaml
+id: loop_basic
+namespace: company.team
+
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["pynchon", "dostoyevsky", "hedayat"]
+ tasks:
+ - id: api
+ type: io.kestra.plugin.core.http.Request
+ uri: "https://openlibrary.org/search.json?author={{ item.value }}&sort=new"
+```
+
+Inside each iteration:
+- `{{ item.value }}` — the current value from the list
+- `{{ item.index }}` — the zero-based position (0, 1, 2, …)
+
+After execution, the Gantt view shows a separate task group for each author.
-This example demonstrates how to use `ForEach` to loop over a list of strings and then loop through a nested list for each string.
+When `values` contains objects, each `item.value` is a JSON string. Use `fromJson(item.value).field` to access fields — `item.value.field` does not work.
-You can access the current iteration value using the variable `{{ taskrun.value }}` or `{{ parent.taskrun.value }}` if you are in a nested child task. Additionally, you can access the batch or iteration number with `{{ taskrun.iteration }}`.
+## Nested loops
-To see the flow in action, define the `each_nested` flow as shown below:
+To iterate over multiple dimensions, nest `Loop` tasks. The inner loop accesses the outer loop's value with `{{ item.parent.value }}`. For three or more levels, `{{ item.parents[1].value }}` is the grandparent — `item.parents[0]` is the same as `item.parent`.
```yaml
-id: each_nested
+id: loop_nested
namespace: company.team
tasks:
- - id: 1_each
- type: io.kestra.plugin.core.flow.ForEach
- values: '["s1", "s2", "s3"]'
+ - id: outer
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["bucket1", "bucket2"]
tasks:
- - id: 1-1_return
- type: io.kestra.plugin.core.debug.Return
- format: "{{task.id}} > {{taskrun.value}} > {{taskrun.startDate}}"
- - id: 1-2_each
- type: io.kestra.plugin.core.flow.ForEach
- values: '["a a", "b b"]'
+ - id: inner
+ type: io.kestra.plugin.core.flow.Loop
+ values: [2025, 2026]
tasks:
- - id: 1-2-1_return
- type: io.kestra.plugin.core.debug.Return
- format: "{{task.id}} > {{taskrun.value}} > {{taskrun.startDate}}"
- - id: 1-2-2_return
- type: io.kestra.plugin.core.debug.Return
- format: "{{task.id}} > {{ outputs['1-2-1_return'].s1[taskrun.value].value }} >> get {{ outputs['1-2-1_return']['s1'][taskrun.value].value }} > {{taskrun.startDate}}"
- - id: 1-3_return
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "bucket={{ item.parent.value }} year={{ item.value }}"
+```
+
+## Collect outputs across iterations
+
+By default, outputs produced inside a loop are not visible to tasks that run after it. Declare an `outputs:` block on the Loop task to surface values explicitly. After the loop, `outputs.loop.outputs` is a list of per-iteration results. Use `loopOutputs()` to extract one field across all iterations as a flat list.
+
+```yaml
+id: loop_outputs
+namespace: company.team
+
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["alpha", "beta", "gamma"]
+ fetchType: AUTO
+ outputs:
+ - id: label
+ type: STRING
+ value: "{{ outputs.process.value }}"
+ tasks:
+ - id: process
type: io.kestra.plugin.core.debug.Return
- format: "{{task.id}} > {{ outputs['1-1_return'][taskrun.value].value }} > {{taskrun.startDate}}"
- - id: 2_return
- type: io.kestra.plugin.core.debug.Return
- format: "{{task.id}} > {{outputs['1-2-1_return'].s1['a a'].value}}"
+ format: "processed {{ item.value }}"
+
+ - id: read_outputs
+ type: io.kestra.plugin.core.log.Log
+ message: "All results: {{ loopOutputs(outputs.loop.outputs, 'label') }}"
```
-Save and execute the `each_nested` flow.
+## Run iterations in parallel
+
+Set `concurrencyLimit` to a positive integer to cap how many iterations run at once. Setting it to `0` removes the cap entirely — only do this for small datasets where you understand the resource implications.
+
+```yaml
+id: loop_parallel
+namespace: company.team
+
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
+ concurrencyLimit: 0
+ tasks:
+ - id: parallel
+ type: io.kestra.plugin.core.flow.Parallel
+ tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Processing {{ item.value }}"
+ - id: shell
+ type: io.kestra.plugin.scripts.shell.Commands
+ commands:
+ - "echo done {{ item.value }}"
+```
-The above flow, when executed, iterates over a nested list of values, logging messages at each level of iteration to track the processing of both the outer and inner list items.
+## Fan out with subflows
-Within the flow:
+Use `Loop` with `Subflow` to launch an isolated child execution per iteration. Each subflow gets its own retry policy, logs, and failure state — useful when you want per-item isolation rather than running everything inside a single parent execution.
-- `1_each`: Uses the `ForEach` task to iterate over the list `["s1", "s2", "s3"]`. For each value, it runs the nested tasks defined within.
+The subflow to call per iteration:
- - `1-1_return`: Logs the task ID, the current list value, and the task run start time.
+```yaml
+id: process_item
+namespace: company.team
- - `1-2_each`: Iterates over a second list `["a a", "b b"]` and runs a set of tasks for each value in this nested list.
+inputs:
+ - id: item
+ type: STRING
- - `1-2-1_return`: Logs the task ID, the nested list value, and the start time of the task run.
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Processing: {{ inputs.item }}"
+```
- - `1-2-2_return`: Logs a custom output from `1-2-1_return`, which shows how to access outputs from previous iterations within the nested loop.
+The parent flow that queries a dataset and fans out one subflow per row:
- - `1-3_return`: Logs the output from `1-1_return` after the inner loop is completed and displays the corresponding value processed in the outer loop.
+```yaml
+id: fan_out
+namespace: company.team
-- `2_return`: Fetches the output from the nested loop (`1-2-1_return` for the value `a a`) and logs it.
+tasks:
+ - id: extract
+ type: io.kestra.plugin.jdbc.duckdb.Query
+ sql: |
+ INSTALL httpfs;
+ LOAD httpfs;
+ SELECT * FROM read_csv_auto('https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv', header=True);
+ store: true
+
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: "{{ outputs.extract.uri }}"
+ tasks:
+ - id: process
+ type: io.kestra.plugin.core.flow.Subflow
+ namespace: company.team
+ flowId: process_item
+ wait: true
+ transmitFailed: true
+ inputs:
+ item: "{{ item.value }}"
+```
+Set `wait: true` so the parent tracks each child's outcome. Set `transmitFailed: true` to fail the loop if any subflow fails. Combine with `concurrencyLimit` on the Loop task to cap how many subflows run simultaneously.
## Next steps
-Now that you've seen how to loop over a list of values using `ForEach`, you can apply this technique to any scenario where multiple iterations of similar tasks are needed. You can further extend this flow by:
-- Adding more complex nested loops.
-- Using dynamic input values instead of hardcoded lists.
-- Logging or processing additional data from each iteration.
-
-For more advanced use cases, refer to Kestra’s official [ForEach](/plugins/core/flow/io.kestra.plugin.core.flow.foreach) task documentation and the [Best Practices for ForEach and ForEachItem](../../14.best-practices/11.foreach-and-foreachitem/index.md) guide, which covers how to access sibling task outputs inside and outside the loop, when to use `ForEachItem` instead, and common mistakes to avoid.
+- For the full Loop property reference, see the [Loop task documentation](/plugins/core/flow/io.kestra.plugin.core.flow.loop).
+- For output collection patterns, error handling, and map-reduce examples, see the [Flowable Tasks](../../05.workflow-components/01.tasks/00.flowable-tasks/index.md#loop) reference.
+- For Loop best practices, see the [Loop best practices guide](../../14.best-practices/11.loop/index.md).
diff --git a/src/contents/docs/15.how-to-guides/microservices-unit-tests/index.md b/src/contents/docs/15.how-to-guides/microservices-unit-tests/index.md
index 7f1b091b7e2..2143d7706b1 100644
--- a/src/contents/docs/15.how-to-guides/microservices-unit-tests/index.md
+++ b/src/contents/docs/15.how-to-guides/microservices-unit-tests/index.md
@@ -7,7 +7,7 @@ stage: Intermediate
topics:
- Kestra Workflow Components
- Kestra Concepts
-editions: ["EE"]
+editions: ["EE", "Cloud"]
---
Build an automated guardrail that pings a microservice endpoint, alerts Slack when it fails, and runs only when its unit tests pass.
diff --git a/src/contents/docs/15.how-to-guides/multiplecondition-listener/index.md b/src/contents/docs/15.how-to-guides/multiplecondition-listener/index.md
index 71355361851..c34fe90f7fe 100644
--- a/src/contents/docs/15.how-to-guides/multiplecondition-listener/index.md
+++ b/src/contents/docs/15.how-to-guides/multiplecondition-listener/index.md
@@ -8,35 +8,20 @@ topics:
- Kestra Workflow Components
---
-How to set up a Flow to only trigger when multiple conditions are met.
+How to set up a flow that only triggers when multiple upstream flows have all succeeded.
-In this tutorial, we’ll explore how to set up a flow in Kestra that only triggers when multiple conditions are met. Specifically, we will create a flow that only executes if two other flows, `multiplecondition-flow-a` and `multiplecondition-flow-b`, have executed successfully within the last 24 hours.
+In this guide, we’ll create a flow that only executes if two other flows, `multiplecondition_flow_a` and `multiplecondition_flow_b`, have each completed successfully within the last 24 hours. This pattern uses the `dependsOn` property on the Flow trigger.
-## Why Use Multiple Condition Listeners?
+## When to use this pattern
-The `MultipleCondition` listener allows you to build more complex workflows that depend on the success of several flows. For example, if you have two dependent tasks or processes that need to succeed before triggering another process, this listener ensures that the next workflow is only executed when both conditions are met within a specific time window.
+Use multiple upstream dependencies when a downstream process should only run after several independent upstream flows all succeed. For example, if you have separate ingestion flows for different data sources and want to run a transformation only after all sources have completed, `dependsOn` with a time window is the right tool.
-## Activation Process Overview
+## How it works
-The listener will trigger under the following conditions:
-
-1. Both `multiplecondition-flow-a` and `multiplecondition-flow-b` must have successful executions.
-2. The listener checks if both flows succeeded within the last 24 hours.
-3. If the conditions are met, the flow is activated, and the conditions reset.
-4. Future executions will only re-trigger the flow if both flows succeed again within another 24-hour window.
-
-## How the Process Works
-
-1. Time Window (P1D or 24 hours):
-
- - The `MultipleCondition` listener checks if both flows (`multiplecondition-flow-a` and `multiplecondition-flow-b`) have been executed successfully within the past 24 hours.
-
-2. Resetting Conditions:
-
- - Once the listener triggers, the conditions reset, meaning that even if one of the flows succeeds again, the listener won't trigger until both flows succeed within a new 24-hour period.
-
-3. Flow Dependency:
- - This is particularly useful when you have flows that depend on each other or when the successful execution of multiple workflows is a prerequisite for a downstream task.
+1. Both `multiplecondition_flow_a` and `multiplecondition_flow_b` must complete successfully.
+2. Both must complete within the same 24-hour window (`window.every: P1D`).
+3. Once both conditions are satisfied, the listener flow triggers.
+4. The window resets each day, so both flows must succeed again within the next window to re-trigger the listener.
## First Flow: `multiplecondition_flow_a`
@@ -51,8 +36,8 @@ description: |
tasks:
- id: only
- type: io.kestra.plugin.core.debug.Return
- format: "from parents: {{ execution.id }}"
+ type: io.kestra.plugin.core.log.Log
+ message: "from parents: {{ execution.id }}"
```
This flow is a simple one that returns the execution ID as output. The listener checks whether this flow has executed successfully within the past 24 hours.
@@ -70,8 +55,8 @@ description: |
tasks:
- id: only
- type: io.kestra.plugin.core.debug.Return
- format: "from parents: {{ execution.id }}"
+ type: io.kestra.plugin.core.log.Log
+ message: "from parents: {{ execution.id }}"
```
Just like `multiplecondition_flow_a`, this flow also returns its execution ID. The listener will wait for both this and the first flow to succeed before activating the final flow.
@@ -85,54 +70,34 @@ id: multiplecondition_listener
namespace: company.team
description: |
- This flow will start only if `multiplecondition_flow_a` and `multiplecondition_flow_b` are successful during the last 24h.
+ This flow starts only if `multiplecondition_flow_a` and `multiplecondition_flow_b` both succeed within the same 24-hour window.
tasks:
- id: only_listener
- type: io.kestra.plugin.core.debug.Return
- format: "children"
+ type: io.kestra.plugin.core.log.Log
+ message: "children"
triggers:
- id: multiple_listen_flow
type: io.kestra.plugin.core.trigger.Flow
- conditions:
- - type: io.kestra.plugin.core.condition.ExecutionStatus
- in:
- - SUCCESS
- - id: multiple
- type: io.kestra.plugin.core.condition.MultipleCondition
- window: P1D
- windowAdvance: P0D
- conditions:
- flow_a:
- type: io.kestra.plugin.core.condition.ExecutionFlow
- namespace: company.team
- flowId: multiplecondition_flow_a
- flow_b:
- type: io.kestra.plugin.core.condition.ExecutionFlow
- namespace: company.team
- flowId: multiplecondition_flow_b
+ dependsOn:
+ - flowId: multiplecondition_flow_a
+ namespace: company.team
+ states: [SUCCESS]
+ - flowId: multiplecondition_flow_b
+ namespace: company.team
+ states: [SUCCESS]
+ window:
+ every: P1D
```
-## Explanation of the Flow
+## Explanation of the flow
-1. Tasks Section:
+1. **Tasks** — `only_listener` outputs a static value when the trigger fires. Replace this with whatever downstream logic you need.
+2. **`dependsOn`** — declares two upstream flow dependencies. Both entries must be satisfied before the trigger fires. `states: [SUCCESS]` means only successful executions count.
- - The task `only_listener` outputs a static value (`children`) when the trigger conditions are met. This part can be customized to perform more complex tasks after the conditions are satisfied.
-
-2. Triggers Section:
-
-
- - The `multiple_listen_flow` trigger listens for both `multiplecondition_flow_a` and `multiplecondition_flow_b`.
- - Execution Status Condition: Ensures that only successful executions (status `SUCCESS`) are considered.
- - MultipleCondition: This condition checks that both `flow_a` and `flow_b` have successfully completed within the last 24 hours (`P1D`).
-
-3. Window:
-
-
- - The `window: P1D` ensures that the listener checks for executions within the past 24 hours.
- - The `windowAdvance: P0D` parameter ensures that the time window starts immediately, without any delay.
+3. **`window.every: P1D`** — defines a 24-hour evaluation window. Kestra accumulates upstream executions within this window and fires the trigger once all `dependsOn` entries are satisfied within the same window period.
## Expected Output
@@ -150,6 +115,4 @@ When both multiplecondition_flow_a and multiplecondition_flow_b succeed within 2
## Conclusion
-In this tutorial, we’ve demonstrated how to set up a `MultipleCondition` listener that checks for the success of multiple flows within a specified time window. This is a powerful feature for managing complex workflows that depend on the successful execution of multiple tasks.
-
-By using this listener, you can ensure that downstream processes are only triggered when all necessary upstream conditions are met.
+This guide demonstrated how to use `dependsOn` with a time window to trigger a flow only when multiple upstream flows all succeed within the same period. Use this pattern whenever a downstream process must wait on several independent upstream flows before running.
diff --git a/src/contents/docs/15.how-to-guides/neon/index.md b/src/contents/docs/15.how-to-guides/neon/index.md
index cebda134fe5..e2edcb13857 100644
--- a/src/contents/docs/15.how-to-guides/neon/index.md
+++ b/src/contents/docs/15.how-to-guides/neon/index.md
@@ -41,34 +41,22 @@ Next, click on the '+' icon to add a table, name it, and create it. You can leav
With the setup in Neon done, we can go Kestra to set up our connection. While there's no official Neon plugin, we can connect using the [PostgreSQL plugin](/plugins/plugin-jdbc-postgres), which supports a number of tasks such as `Query`, `CopyIn`, and `CopyOut`.
-To connect, we can copy the URL provided from before. To prevent exposing the password in our flow, take the password saved earlier and store it as a [secret](../../06.concepts/04.secret/index.md). Then, in the URL, switch out the password for the secret expression: `{{ secret('NEON_PASSWORD') }}`.
-
-By using [Plugin Defaults](../../05.workflow-components/09.plugin-defaults/index.md), we can configure our connection to Neon once for all tasks in our flow rather than individually for each task.
-
-Once configured, our connection in Kestra will look like the example below:
+To connect, copy the URL provided from before. Store the password as a [secret](../../06.concepts/04.secret/index.md) and reference it in the URL with `{{ secret('NEON_PASSWORD') }}`. Add the `url` property directly to each PostgreSQL task:
```yaml
-pluginDefaults:
- - forced: true
- type: io.kestra.plugin.jdbc.postgresql
- values:
- url: "jdbc:postgresql://ep-gentle-tree-a25pyhxb-pooler.eu-central-1.aws.neon.tech/neondb?user=neondb_owner&password={{ secret('NEON_PASSWORD') }}&sslmode=require"
-
+url: "jdbc:postgresql://ep-gentle-tree-a25pyhxb-pooler.eu-central-1.aws.neon.tech/neondb?user=neondb_owner&password={{ secret('NEON_PASSWORD') }}&sslmode=require"
```
:::alert{type="info"}
-You can also use the `username` and `password` properties rather than combining it all into the `url` property:
+You can also split the connection string into separate `url`, `username`, and `password` properties:
```yaml
-pluginDefaults:
- - forced: true
- type: io.kestra.plugin.jdbc.postgresql
- values:
- url: "jdbc:postgresql://ep-gentle-tree-a25pyhxb-pooler.eu-central-1.aws.neon.tech/neondb"
- username: "neondb_owner"
- password: "{{ secret('NEON_PASSWORD') }}"
+url: "jdbc:postgresql://ep-gentle-tree-a25pyhxb-pooler.eu-central-1.aws.neon.tech/neondb"
+username: "neondb_owner"
+password: "{{ secret('NEON_PASSWORD') }}"
```
+In Enterprise Edition, you can centralize connection properties across flows using a [Policy](../../07.enterprise/02.governance/policies/index.md) with an `Add` rule targeting `io.kestra.plugin.jdbc.postgresql`.
:::
## Copying a CSV File into Neon in a Flow
@@ -82,6 +70,7 @@ namespace: company.team
tasks:
- id: create_columns
type: io.kestra.plugin.jdbc.postgresql.Queries
+ url: "jdbc:postgresql://ep-gentle-tree-a25pyhxb-pooler.eu-central-1.aws.neon.tech/neondb?user=neondb_owner&password={{ secret('NEON_PASSWORD') }}&sslmode=require"
sql: |
ALTER TABLE kestra_example
ADD COLUMN order_id int,
@@ -91,15 +80,9 @@ tasks:
ADD COLUMN price double precision,
ADD COLUMN quantity int,
ADD COLUMN total double precision;
-
-pluginDefaults:
- - forced: true
- type: io.kestra.plugin.jdbc.postgresql
- values:
- url: "jdbc:postgresql://ep-gentle-tree-a25pyhxb-pooler.eu-central-1.aws.neon.tech/neondb?user=neondb_owner&password={{ secret('NEON_PASSWORD') }}&sslmode=require"
```
-Once your columns are configured, you can use the [CopyIn](/plugins/plugin-jdbc-postgres/io.kestra.plugin.jdbc.postgresql.copyin) task combined with the [HTTP Download](/plugins/core/http/io.kestra.plugin.core.http.download) task to download the CSV file and copy it directly into the table. As we set up the database connection with our [Plugin Defaults](#connecting-neon-to-kestra), the CopyIn task will connect directly and copy the CSV file into the database.
+Once your columns are configured, use the [CopyIn](/plugins/plugin-jdbc-postgres/io.kestra.plugin.jdbc.postgresql.copyin) task combined with the [HTTP Download](/plugins/core/http/io.kestra.plugin.core.http.download) task to download the CSV file and copy it directly into the table.
```yaml
id: neon_db_copyin
@@ -112,17 +95,12 @@ tasks:
- id: copy_in
type: io.kestra.plugin.jdbc.postgresql.CopyIn
+ url: "jdbc:postgresql://ep-gentle-tree-a25pyhxb-pooler.eu-central-1.aws.neon.tech/neondb?user=neondb_owner&password={{ secret('NEON_PASSWORD') }}&sslmode=require"
table: "kestra_example"
from: "{{ outputs.download.uri }}"
header: true
columns: [order_id,customer_name,customer_email,product_id,price,quantity,total]
delimiter: ","
-
-pluginDefaults:
- - forced: true
- type: io.kestra.plugin.jdbc.postgresql
- values:
- url: "jdbc:postgresql://ep-gentle-tree-a25pyhxb-pooler.eu-central-1.aws.neon.tech/neondb?user=neondb_owner&password={{ secret('NEON_PASSWORD') }}&sslmode=require"
```
Once this flow completes, we can view the contents of our database in Neon:
diff --git a/src/contents/docs/15.how-to-guides/parallel-vs-sequential/index.md b/src/contents/docs/15.how-to-guides/parallel-vs-sequential/index.md
index 5867517f707..c78e24b4d69 100644
--- a/src/contents/docs/15.how-to-guides/parallel-vs-sequential/index.md
+++ b/src/contents/docs/15.how-to-guides/parallel-vs-sequential/index.md
@@ -72,8 +72,8 @@ tasks:
- 'echo "running {{task.id}}"'
- 'sleep 1'
- id: last
- type: io.kestra.plugin.core.debug.Return
- format: "{{task.id}} > {{taskrun.startDate}}"
+ type: io.kestra.plugin.core.log.Log
+ message: "{{task.id}} > {{taskrun.startDate}}"
```
@@ -141,6 +141,6 @@ tasks:
- 'echo "running {{task.id}}"'
- 'sleep 1'
- id: last
- type: io.kestra.plugin.core.debug.Return
- format: "{{task.id}} > {{taskrun.startDate}}"
+ type: io.kestra.plugin.core.log.Log
+ message: "{{task.id}} > {{taskrun.startDate}}"
```
diff --git a/src/contents/docs/15.how-to-guides/pause-resume/index.md b/src/contents/docs/15.how-to-guides/pause-resume/index.md
index 8c294073b38..feb025e2b46 100644
--- a/src/contents/docs/15.how-to-guides/pause-resume/index.md
+++ b/src/contents/docs/15.how-to-guides/pause-resume/index.md
@@ -32,9 +32,9 @@ tasks:
The `Pause` task will pause the execution and the `Log` task will run only once the workflow has been resumed.
-## Pausing and resuming a workflow from the UI
+## Pause and resume a workflow from the UI
-You can either use the Pause task or manually Pause from the Execution overview page. Once the execution is paused, you can inspect the current logs and outputs. Then, you can resume it from the UI by clicking on the `Resume` button in the `Overview` tab:
+You can either use the Pause task or manually pause from the Execution overview page. Once the execution is paused, inspect the logs and outputs, then resume using the **Resume** button in the **Overview** tab:
diff --git a/src/contents/docs/15.how-to-guides/perl/index.md b/src/contents/docs/15.how-to-guides/perl/index.md
deleted file mode 100644
index ddcc2af76b8..00000000000
--- a/src/contents/docs/15.how-to-guides/perl/index.md
+++ /dev/null
@@ -1,196 +0,0 @@
----
-title: Run Perl Inside Your Flows
-h1: Execute Perl Scripts in Kestra with Docker
-icon: /src/contents/docs/icons/perl.svg
-stage: Getting Started
-topics:
- - Scripting
-description: Execute Perl scripts inside Kestra workflows. Run automation and text-processing tasks with Perl, using Docker containers for clean dependency isolation.
----
-
-Run Perl code directly in your flows and generate outputs.
-
-There isn't an official Perl plugin but we can use the `Shell` `Commands` task to execute arbitrary commands inside a Docker container. We can also specify a container image that contains the necessary libraries to run the specific programming language.
-
-In this example, we're using the Docker Task Runner with the `perl:latest` image so that Perl can be executed.
-
-```yaml
-id: perl_commands
-namespace: company.team
-
-tasks:
- - id: perl
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: perl:latest
- namespaceFiles:
- enabled: true
- commands:
- - chmod +x main.pl
- - perl main.pl
-```
-
-The contents of the `main.pl` file contains a simple print statement:
-
-```perl
-#!/usr/bin/perl
-
-print "Hello World";
-```
-
-You'll need to add your Perl code using the Editor or [sync it using Git](../../version-control-cicd/04.git/index.md) so Kestra can see it. You'll also need to set the `enabled` flag for the `namespaceFiles` property to `true` so Kestra can access the file.
-
-You can also have the Perl code written inline using the `inputFiles` property.
-
-```yaml
-id: perl_commands
-namespace: company.team
-
-tasks:
- - id: perl
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: perl:latest
- inputFiles:
- main.pl: |
- #!/usr/bin/perl
- print "Hello World";
- commands:
- - chmod +x main.pl
- - perl main.pl
-```
-
-You can read more about the Shell Commands type in the [Plugin documentation](/plugins/plugin-script-shell/io.kestra.plugin.scripts.shell.commands).
-
-## Handling Outputs
-
-If you want to get a variable or file from your Perl code, you can use an [output](../../05.workflow-components/06.outputs/index.md).
-
-### Variable Output
-
-You can get the JSON outputs from the Perl script using the `::{}::` pattern. Here is an example:
-
-```yaml
-id: perl_outputs
-namespace: company.team
-
-tasks:
- - id: perl
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: perl:latest
- inputFiles:
- main.pl: |
- #!/usr/bin/perl
- print '::{"outputs":{"test":"value","int":2,"bool":true,"float":3.65}}::';
- commands:
- - chmod +x main.pl
- - perl main.pl
-```
-
-All the output variables can be viewed in the Outputs tab of the execution.
-
-
-
-You can refer to the outputs in another task as shown in the example below:
-
-```yaml
-id: perl_outputs
-namespace: company.team
-
-tasks:
- - id: perl
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: perl:latest
- inputFiles:
- main.pl: |
- #!/usr/bin/perl
- print '::{"outputs":{"test":"value","int":2,"bool":true,"float":3.65}}::';
- commands:
- - chmod +x main.pl
- - perl main.pl
-
- - id: return
- type: io.kestra.plugin.core.debug.Return
- format: '{{ outputs.perl.vars.test }}'
-```
-
-### File Output
-
-Inside of your Perl code, write a file to the system. You'll need to add the `outputFiles` property to your flow and list the files you're trying to put out. In this case, we want to output `output.txt`. More information on the formats you can use for this property can be found in [Script Output Metrics](../../16.scripts/06.outputs-metrics/index.md).
-
-The example below writes a `output.txt` file containing the "Hello World" text. We can then refer the file using the syntax `{{ outputs.{task_id}.outputFiles[''] }}`, and read the contents of the file using the `read()` function.
-
-```yaml
-id: perl_script
-namespace: company.team
-
-tasks:
- - id: perl
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: perl:latest
- inputFiles:
- main.pl: |
- #!/usr/bin/perl
- use strict;
- use warnings;
-
- # Open the file for writing
- open(my $fh, '>', 'output.txt') or die "Cannot open file: $!";
-
- # Write to the file
- print $fh "Hello World";
-
- # Close the file
- close($fh);
-
- print "Successfully wrote to the file.\n";
- outputFiles:
- - output.txt
- commands:
- - chmod +x main.pl
- - perl main.pl
-
- - id: log
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(outputs.perl.outputFiles['output.txt']) }}"
-```
-
-## Handling Metrics
-
-You can also get [metrics](../../16.scripts/06.outputs-metrics/index.md#outputs-and-metrics-in-script-and-commands-tasks) from your Perl code. Metrics use the same `::{}::` pattern as outputs. This example demonstrates both the counter and timer metrics.
-
-```yaml
-id: perl_metrics
-namespace: company.team
-
-tasks:
- - id: perl
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: perl:latest
- inputFiles:
- main.pl: |
- #!/usr/bin/perl
-
- print "There are 20 products in the cart\n";
- print "::{\"outputs\":{\"productCount\":20}}::\n";
- print "::{\"metrics\":[{\"name\":\"productCount\",\"type\":\"counter\",\"value\":20}]}::\n";
- print "::{\"metrics\":[{\"name\":\"purchaseTime\",\"type\":\"timer\",\"value\":32.44}]}::\n";
-
- commands:
- - chmod +x main.pl
- - perl main.pl
-```
-
-Once this has executed, both the metrics can be viewed under **Metrics**.
-
-
diff --git a/src/contents/docs/15.how-to-guides/perl/metrics.png b/src/contents/docs/15.how-to-guides/perl/metrics.png
deleted file mode 100644
index d675f539357..00000000000
Binary files a/src/contents/docs/15.how-to-guides/perl/metrics.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/perl/outputs.png b/src/contents/docs/15.how-to-guides/perl/outputs.png
deleted file mode 100644
index f9653bbe123..00000000000
Binary files a/src/contents/docs/15.how-to-guides/perl/outputs.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/populate-demo-data/index.md b/src/contents/docs/15.how-to-guides/populate-demo-data/index.md
index adf912afabe..928c88e951c 100644
--- a/src/contents/docs/15.how-to-guides/populate-demo-data/index.md
+++ b/src/contents/docs/15.how-to-guides/populate-demo-data/index.md
@@ -169,7 +169,7 @@ triggers:
-- 4. data_pipeline_assets (acme.company.data)
INSERT INTO flows (key, value, source_code) VALUES (
'acme.company.data_data_pipeline_assets_1',
- '{"id":"data_pipeline_assets","namespace":"acme.company.data","tenantId":"main","revision":1,"deleted":false,"tasks":[{"id":"create_staging_layer_asset","type":"io.kestra.plugin.jdbc.duckdb.Query","sql":"CREATE TABLE IF NOT EXISTS trips AS select VendorID, passenger_count, trip_distance from sample_data.nyc.taxi limit 10;"},{"id":"for_each","type":"io.kestra.plugin.core.flow.ForEach","values":["passenger_count","trip_distance"],"tasks":[{"id":"create_mart_layer_asset","type":"io.kestra.plugin.jdbc.duckdb.Query","sql":"SELECT AVG({{taskrun.value}}) AS avg_{{taskrun.value}} FROM trips;"}]}]}'::jsonb,
+ '{"id":"data_pipeline_assets","namespace":"acme.company.data","tenantId":"main","revision":1,"deleted":false,"tasks":[{"id":"create_staging_layer_asset","type":"io.kestra.plugin.jdbc.duckdb.Query","sql":"CREATE TABLE IF NOT EXISTS trips AS select VendorID, passenger_count, trip_distance from sample_data.nyc.taxi limit 10;"},{"id":"for_each","type":"io.kestra.plugin.core.flow.Loop","values":["passenger_count","trip_distance"],"tasks":[{"id":"create_mart_layer_asset","type":"io.kestra.plugin.jdbc.duckdb.Query","sql":"SELECT AVG({{item.value}}) AS avg_{{item.value}} FROM trips;"}]}]}'::jsonb,
'id: data_pipeline_assets
namespace: acme.company.data
@@ -181,14 +181,14 @@ tasks:
select VendorID, passenger_count, trip_distance from sample_data.nyc.taxi limit 10;
- id: for_each
- type: io.kestra.plugin.core.flow.ForEach
+ type: io.kestra.plugin.core.flow.Loop
values:
- passenger_count
- trip_distance
tasks:
- id: create_mart_layer_asset
type: io.kestra.plugin.jdbc.duckdb.Query
- sql: SELECT AVG({{taskrun.value}}) AS avg_{{taskrun.value}} FROM trips;'
+ sql: SELECT AVG({{item.value}}) AS avg_{{item.value}} FROM trips;'
) ON CONFLICT (key) DO NOTHING;
-- 5. system_health_check (acme.operations)
diff --git a/src/contents/docs/15.how-to-guides/python-uv/index.md b/src/contents/docs/15.how-to-guides/python-uv/index.md
index ff23d152bd0..5b67c106f78 100644
--- a/src/contents/docs/15.how-to-guides/python-uv/index.md
+++ b/src/contents/docs/15.how-to-guides/python-uv/index.md
@@ -8,15 +8,19 @@ topics:
description: Use uv in Kestra to manage Python dependencies and virtual environments for faster and more reliable script execution.
---
-Manage your Python Dependencies in Kestra using `uv`.
+Manage Python dependencies in Kestra using `uv`.
+
+:::alert{type="info"}
+For most use cases, the native `dependencies` property on Python `Script` and `Commands` tasks is the simplest way to install packages — Kestra handles installation automatically without needing `beforeCommands`. Use `uv` when you need faster resolution, virtual environments with the Process runner, or a custom Docker image with `uv` pre-installed.
+:::
-`uv` is a new Python package and project manager designed to be extremely fast. Written in rust, it aims to fix some of the pitfalls of pip while also combining multiple python dependency management tools like `virtualenv`, `poetry`, and more into one unified tool.
+`uv` is a fast Python package and project manager written in Rust. It combines tools like `virtualenv`, `poetry`, and `pip` into one unified interface and is pre-installed in Kestra's default Python image `kestrapy`.
-`uv` can be used in Kestra to install dependencies as well as manage virtual environments in combination with the [Process Task Runner](../../task-runners/04.types/01.process-task-runner/index.md).
+`uv` is useful in Kestra for managing virtual environments with the [Process Task Runner](../../task-runners/04.types/01.process-task-runner/index.md) or when you need explicit control over dependency resolution speed.
## Install Dependencies
diff --git a/src/contents/docs/15.how-to-guides/python/index.md b/src/contents/docs/15.how-to-guides/python/index.md
index ea2a7b71035..35219ee574a 100644
--- a/src/contents/docs/15.how-to-guides/python/index.md
+++ b/src/contents/docs/15.how-to-guides/python/index.md
@@ -337,12 +337,13 @@ flow.execute('example', 'python_scripts', {'greeting': 'hello from Python'})
Read more about it on the [execution page](../../05.workflow-components/03.execution/index.md).
-## Automate Python with Triggers
+## Automate Python with triggers
You can combine your Python code with a trigger to automatically execute your code. There's a few key ways you can automate it:
- Run on a schedule
- Run when a webhook is called
- Run when a file is available in a data lake or storage bucket
+- Run Python code on a polling interval and emit only when a condition matches
@@ -436,6 +437,61 @@ triggers:
maxKeys: 1
```
+### Run Python code as a polling trigger
+
+You can also use Python itself as polling logic by using `ScriptTrigger` or `CommandsTrigger`. These trigger types run Python code on an interval and start a flow execution only when the `exitCondition` matches.
+
+Use `ScriptTrigger` for inline Python code:
+
+```yaml
+id: python_script_trigger
+namespace: company.team
+
+triggers:
+ - id: script_failure
+ type: io.kestra.plugin.scripts.python.ScriptTrigger
+ interval: PT10S
+ exitCondition: "exit 1"
+ edge: true
+ script: |
+ raise Exception("boom")
+
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Triggered with exitCode={{ trigger.exitCode }} (condition={{ trigger.condition }})"
+```
+
+Use `CommandsTrigger` when you want to run Python commands instead:
+
+```yaml
+id: python_commands_trigger
+namespace: company.team
+
+triggers:
+ - id: on_fail
+ type: io.kestra.plugin.scripts.python.CommandsTrigger
+ interval: PT10S
+ exitCondition: "exit 1"
+ edge: true
+ containerImage: python:3.13-slim
+ commands:
+ - python3 -c "raise Exception('boom')"
+
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Triggered with exitCode={{ trigger.exitCode }} (condition={{ trigger.condition }})"
+```
+
+These triggers support:
+
+- `interval` to control how often the Python code runs
+- `exitCondition` to match an exit code such as `exit 1`, or a regex or substring matched against emitted vars and failure logs
+- `edge` to emit only when the condition changes from not matching to matching
+
+Use these trigger types when you want Python itself to decide whether a polling condition has been met, rather than relying on a separate external-system trigger.
+
## Execute GraalVM Task
diff --git a/src/contents/docs/15.how-to-guides/r/index.md b/src/contents/docs/15.how-to-guides/r/index.md
deleted file mode 100644
index 7a5c05fe216..00000000000
--- a/src/contents/docs/15.how-to-guides/r/index.md
+++ /dev/null
@@ -1,186 +0,0 @@
----
-title: Run R Inside Your Flows
-h1: Execute R Scripts in Kestra Workflows
-description: Run R scripts in Kestra for statistical computing and data analysis. Use Docker to manage package dependencies and capture outputs for downstream tasks.
-icon: /src/contents/docs/icons/r.svg
-stage: Getting Started
-topics:
- - Scripting
----
-
-Run R code directly in your flows and generate outputs.
-
-R is essential for statistical analysis, visualization, and data manipulation. With Kestra, you can effortlessly automate data ingestion, conduct complex statistical analysis, and handle real-time data processing. Kestra's robust orchestration capabilities ensure that your R scripts run smoothly and efficiently, streamlining your data-driven projects.
-
-This guide is going to walk you through how to get R running in a workflow, how to manage input and output files, and how you can pass outputs and metrics back to Kestra to use in later tasks.
-
-Kestra has an official plugin for R allowing you to execute R code in a flow by either writing your R code inline or by executing an `.R` file. You can get outputs and metrics from your R code too.
-
-## Scripts
-
-If you want to write a short amount of R code to perform a task, you can use the `io.kestra.plugin.scripts.r.Script` type to write it directly in your flow. This allows you to keep everything in one place.
-
-```yaml
-id: r_script
-namespace: company.team
-description: This flow runs the R script.
-
-tasks:
- - id: http_download
- type: io.kestra.plugin.core.http.Download
- uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
-
- - id: r_script_task
- type: io.kestra.plugin.scripts.r.Script
- script: |
- print("The current execution is {{ execution.id }}")
-
- # Read the file downloaded in `http_download` task
- data <- read.csv("{{ outputs.http_download.uri }}", header=TRUE)
- print(data)
-```
-
-You can read more about the Scripts type in the [Plugin documentation](/plugins/plugin-script-r/io.kestra.plugin.scripts.r.script)
-
-## Commands
-
-If you would prefer to put your R code in an `.R` file (e.g. your code is much longer or spread across multiple files), you can run the previous example using the `io.kestra.plugin.scripts.r.Commands` type:
-
-```yaml
-id: r_commands
-namespace: company.team
-tasks:
- - id: run_r
- type: io.kestra.plugin.scripts.r.Commands
- namespaceFiles:
- enabled: true
- commands:
- - Rscript main.R
-```
-
-The contents of the `main.R` file can be:
-
-```r
-print("Hello World")
-```
-
-You'll need to add your R code using the Editor or [sync it using Git](../../version-control-cicd/04.git/index.md) so Kestra can see it. You'll also need to set the `enabled` flag for the `namespaceFiles` property to `true` so Kestra can access the file.
-
-You can also have the R code written inline.
-
-```yaml
-id: r_commands
-namespace: company.team
-tasks:
- - id: http_download
- type: io.kestra.plugin.core.http.Download
- uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
-
- - id: run_r
- type: io.kestra.plugin.scripts.r.Commands
- inputFiles:
- orders.csv: "{{ read(outputs.http_download.uri) }}"
- main.R: |
- print("The current execution is {{ execution.id }}")
-
- # Read the file
- data <- read.csv("orders.csv", header=TRUE)
- print(data)
- commands:
- - Rscript main.R
-```
-
-You can read more about the Commands type in the [Plugin documentation](/plugins/plugin-script-r/io.kestra.plugin.scripts.r.commands).
-
-## Handling Outputs
-
-If you want to get a variable or file from your R script, you can use an [output](../../05.workflow-components/06.outputs/index.md).
-
-### Variable Output
-
-You can get the JSON outputs from the R commands / script using the `::{}::` pattern. Here is an example:
-
-```yaml
-id: r_outputs
-namespace: company.team
-description: This flow runs the R script, and outputs the variable.
-
-tasks:
- - id: r_outputs_task
- type: io.kestra.plugin.scripts.r.Script
- script: |
- cat('::{"outputs":{"test":"value","int":2,"bool":true,"float":3.65}}::')
-```
-
-All the output variables can be viewed in the Outputs tab of the execution.
-
-
-
-You can refer to the outputs in another task as shown in the example below:
-
-```yaml
-id: r_outputs
-namespace: company.team
-description: This flow runs the R script, and outputs the variable.
-
-tasks:
- - id: r_outputs_task
- type: io.kestra.plugin.scripts.r.Script
- script: |
- cat('::{"outputs":{"test":"value","int":2,"bool":true,"float":3.65}}::')
-
- - id: return
- type: io.kestra.plugin.core.debug.Return
- format: '{{ outputs.r_outputs_task.vars.test }}'
-```
-
-_This example works for both `io.kestra.plugin.scripts.r.Script` and `io.kestra.plugin.scripts.r.Commands`._
-
-### File Output
-
-Inside of your R script, write a file to the system. You'll need to add the `outputFiles` property to your flow and list the files you're trying to put out. In this case, we want to output `output.txt`. More information on the formats you can use for this property can be found in [Script Output Metrics](../../16.scripts/06.outputs-metrics/index.md).
-
-The example below writes a `output.txt` file containing the "Hello World" text. We can then refer the file using the syntax `{{ outputs.{task_id}.outputFiles[''] }}`, and read the contents of the file using the `read()` function.
-
-```yaml
-id: r_output_file
-namespace: company.team
-description: This flow runs the R script to output a file.
-
-tasks:
- - id: r_outputs_task
- type: io.kestra.plugin.scripts.r.Script
- outputFiles:
- - output.txt
- script: |
- writeLines("Hello World", "output.txt")
-
- - id: log_output
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(outputs.r_outputs_task.outputFiles['output.txt']) }}"
-```
-
-_This example works for both `io.kestra.plugin.scripts.r.Script` and `io.kestra.plugin.scripts.r.Commands`._
-
-## Handling Metrics
-
-You can also get [metrics](../../16.scripts/06.outputs-metrics/index.md#outputs-and-metrics-in-script-and-commands-tasks) from your R script. Metrics use the same `::{}::` pattern as outputs. This example demonstrates both the counter and timer metrics.
-
-```yaml
-id: r_metrics
-namespace: company.team
-description: This flow runs the R script, and puts out the metrics.
-
-tasks:
- - id: r_metrics_task
- type: io.kestra.plugin.scripts.r.Script
- script: |
- print('There are 20 products in the cart')
- cat('::{"outputs":{"productCount":20}}::\n')
- cat('::{"metrics":[{"name":"productCount","type":"counter","value":20}]}::\n')
- cat('::{"metrics":[{"name":"purchaseTime","type":"timer","value":32.44}]}::\n')
-```
-
-Once this has executed, both the metrics can be viewed under **Metrics**.
-
-
diff --git a/src/contents/docs/15.how-to-guides/r/metrics.png b/src/contents/docs/15.how-to-guides/r/metrics.png
deleted file mode 100644
index 297f5905fe8..00000000000
Binary files a/src/contents/docs/15.how-to-guides/r/metrics.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/r/outputs.png b/src/contents/docs/15.how-to-guides/r/outputs.png
deleted file mode 100644
index db9043b936b..00000000000
Binary files a/src/contents/docs/15.how-to-guides/r/outputs.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/rollback-and-revision-history/index.md b/src/contents/docs/15.how-to-guides/rollback-and-revision-history/index.md
deleted file mode 100644
index cf33b6be159..00000000000
--- a/src/contents/docs/15.how-to-guides/rollback-and-revision-history/index.md
+++ /dev/null
@@ -1,28 +0,0 @@
----
-title: Revision History and Rollback in Kestra
-h1: Track Changes and Roll Back to Previous Flow Versions
-icon: /src/contents/docs/icons/tutorial.svg
-stage: Getting Started
-topics:
- - Kestra Concepts
- - Version Control
-description: Use Kestra's revision history to track changes, compare flow versions, and easily rollback to previous configurations.
----
-
-Use revision history to rollback to an older version of a flow.
-
-
-
-
-
-Kestra stores revision history which allows you to roll back to any older version of the flow. Navigate to the "Revisions" tab on the flow's page to view older versions. By default, the page opens up a comparison of the current version of the flow against the previous version.
-
-
-
-You can compare any two versions by choosing the appropriate revision number from the drop-down on both sides, allowing you to see the changes made between the two selected versions.
-
-
-
-There is a `Restore` button allowing you to roll back to the selected version. The `Restore` button is disabled for the current live version as there is nothing to restore.
-
-
diff --git a/src/contents/docs/15.how-to-guides/rollback-and-revision-history/restore_option.png b/src/contents/docs/15.how-to-guides/rollback-and-revision-history/restore_option.png
deleted file mode 100644
index 7a20a96382a..00000000000
Binary files a/src/contents/docs/15.how-to-guides/rollback-and-revision-history/restore_option.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/rollback-and-revision-history/revision_comparison.png b/src/contents/docs/15.how-to-guides/rollback-and-revision-history/revision_comparison.png
deleted file mode 100644
index 2665d6447e6..00000000000
Binary files a/src/contents/docs/15.how-to-guides/rollback-and-revision-history/revision_comparison.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/rollback-and-revision-history/revision_dropdown.png b/src/contents/docs/15.how-to-guides/rollback-and-revision-history/revision_dropdown.png
deleted file mode 100644
index 913ea22eb95..00000000000
Binary files a/src/contents/docs/15.how-to-guides/rollback-and-revision-history/revision_dropdown.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/ruby/index.md b/src/contents/docs/15.how-to-guides/ruby/index.md
deleted file mode 100644
index 3531187b51a..00000000000
--- a/src/contents/docs/15.how-to-guides/ruby/index.md
+++ /dev/null
@@ -1,216 +0,0 @@
----
-title: Run Ruby Inside Your Flows
-h1: Execute Ruby Scripts in Kestra Workflows
-description: Execute Ruby scripts in Kestra. Automate tasks with Ruby code, install gems at runtime, and pass outputs to downstream tasks for flexible scripting.
-icon: /src/contents/docs/icons/ruby.svg
-stage: Getting Started
-topics:
- - Scripting
----
-
-Run Ruby code directly in your flows and generate outputs.
-
-Ruby is well known for web development but has many other powerful use cases too, such as automation, web scraping, data processing and command-line tools. With Kestra, you can effortlessly automate data ingestion, as well as manage complex automations. Kestra's robust orchestration capabilities ensure that your Ruby scripts run smoothly and efficiently, streamlining your data-driven projects.
-
-This guide is going to walk you through how to get Ruby running in a workflow, how to manage input and output files, and how you can pass outputs and metrics back to Kestra to use in later tasks.
-
-Kestra has an official plugin for Ruby allowing you to execute Ruby code in a flow by either writing your Ruby code inline or by executing an `.rb` file. You can get outputs and metrics from your Ruby code too.
-
-
-## Scripts
-
-If you want to write a short amount of Ruby code to perform a task, you can use the `io.kestra.plugin.scripts.ruby.Script` type to write it directly in your flow. This allows you to keep everything in one place.
-
-```yaml
-id: ruby_output_file
-namespace: company.team
-description: This flow runs the Ruby script to output a file.
-
-tasks:
- - id: ruby_outputs_task
- type: io.kestra.plugin.scripts.ruby.Script
- outputFiles:
- - output.txt
- script: |
- File.open("output.txt", "w") do |file|
- file.write("Hello World")
- end
-
- - id: log_output
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(outputs.ruby_outputs_task.outputFiles['output.txt']) }}"
-```
-
-You can read more about the Scripts type in the [Plugin documentation](/plugins/plugin-script-ruby/io.kestra.plugin.scripts.ruby.script)
-
-## Commands
-
-If you would prefer to put your Ruby code in a `.rb` file (e.g. your code is much longer or spread across multiple files), you can run the previous example using the `io.kestra.plugin.scripts.ruby.Commands` type:
-
-```yaml
-id: ruby_commands
-namespace: company.team
-tasks:
- - id: run_ruby
- type: io.kestra.plugin.scripts.ruby.Commands
- namespaceFiles:
- enabled: true
- commands:
- - ruby main.rb
-```
-
-The contents of the `main.rb` file can be:
-
-```ruby
-puts "Hello World"
-```
-
-You'll need to add your Ruby code using the Editor or [sync it using Git](../../version-control-cicd/04.git/index.md) so Kestra can see it. You'll also need to set the `enabled` flag for the `namespaceFiles` property to `true` so Kestra can access the file.
-
-You can also have the Ruby code written inline.
-
-```yaml
-id: ruby_commands
-namespace: company.team
-tasks:
- - id: http_download
- type: io.kestra.plugin.core.http.Download
- uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
-
- - id: run_ruby
- type: io.kestra.plugin.scripts.ruby.Commands
- inputFiles:
- orders.csv: "{{ read(outputs.http_download.uri) }}"
- main.rb: |
- puts "The current execution is {{ execution.id }}"
-
- # Read the file downloaded in `http_download` task
- lines = File.readlines("orders.csv")
- puts lines
- commands:
- - ruby main.rb
-```
-
-You can read more about the Commands type in the [Plugin documentation](/plugins/plugin-script-ruby/io.kestra.plugin.scripts.ruby.commands).
-
-## Handling Outputs
-
-If you want to get a variable or file from your Ruby script, you can use an [output](../../05.workflow-components/06.outputs/index.md).
-
-### Variable Output
-
-You can get the JSON outputs from the Ruby commands / script using the `::{}::` pattern. Here is an example:
-
-```yaml
-id: ruby_outputs
-namespace: company.team
-description: This flow runs the Ruby script, and outputs the variable.
-
-tasks:
- - id: ruby_outputs_task
- type: io.kestra.plugin.scripts.ruby.Script
- script: |
- puts '::{"outputs":{"test":"value","int":2,"bool":true,"float":3.65}}::'
-```
-
-All the output variables can be viewed in the Outputs tab of the execution.
-
-
-
-You can refer to the outputs in another task as shown in the example below:
-
-```yaml
-id: ruby_outputs
-namespace: company.team
-description: This flow runs the Ruby script, and outputs the variable.
-
-tasks:
- - id: ruby_outputs_task
- type: io.kestra.plugin.scripts.ruby.Script
- script: |
- puts '::{"outputs":{"test":"value","int":2,"bool":true,"float":3.65}}::'
-
- - id: return
- type: io.kestra.plugin.core.debug.Return
- format: '{{ outputs.ruby_outputs_task.vars.test }}'
-```
-
-_This example works for both `io.kestra.plugin.scripts.ruby.Script` and `io.kestra.plugin.scripts.ruby.Commands`._
-
-### File Output
-
-Inside of your Ruby script, write a file to the system. You'll need to add the `outputFiles` property to your flow and list the files you're trying to put out. In this case, we want to output `output.txt`. More information on the formats you can use for this property can be found in [Script Output Metrics](../../16.scripts/06.outputs-metrics/index.md).
-
-The example below writes a `output.txt` file containing the "Hello World" text. We can then refer the file using the syntax `{{ outputs.{task_id}.outputFiles[''] }}`, and read the contents of the file using the `read()` function.
-
-```yaml
-id: ruby_output_file
-namespace: company.team
-description: This flow runs the Ruby script to output a file.
-
-tasks:
- - id: ruby_outputs_task
- type: io.kestra.plugin.scripts.ruby.Script
- outputFiles:
- - output.txt
- script: |
- File.open("output.txt", "w") do |file|
- file.write("Hello World")
- end
-
- - id: log_output
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(outputs.ruby_outputs_task.outputFiles['output.txt']) }}"
-```
-
-_This example works for both `io.kestra.plugin.scripts.ruby.Script` and `io.kestra.plugin.scripts.ruby.Commands`._
-
-## Handling Metrics
-
-You can also get [metrics](../../16.scripts/06.outputs-metrics/index.md#outputs-and-metrics-in-script-and-commands-tasks) from your Ruby script. Metrics use the same `::{}::` pattern as outputs. This example demonstrates both the counter and timer metrics.
-
-```yaml
-id: ruby_metrics
-namespace: company.team
-description: This flow runs the Ruby script, and puts out the metrics.
-
-tasks:
- - id: ruby_metrics_task
- type: io.kestra.plugin.scripts.ruby.Script
- script: |
- puts 'There are 20 products in the cart'
- puts '::{"outputs":{"productCount":20}}::'
- puts '::{"metrics":[{"name":"productCount","type":"counter","value":20}]}::'
- puts '::{"metrics":[{"name":"purchaseTime","type":"timer","value":32.44}]}::'
-```
-
-Once this has executed, both the metrics can be viewed under **Metrics**.
-
-
-
-## Execute GraalVM Task
-
-Kestra also supports GraalVM integration, allowing you to execute Ruby code directly on the JVM, with the potential for performance improvements. There are currently two tasks:
-- [Eval](/plugins/plugin-graalvm/ruby-graalvm/io.kestra.plugin.graalvm.ruby.eval)
-- [FileTransform](/plugins/plugin-graalvm/ruby-graalvm/io.kestra.plugin.graalvm.ruby.filetransform)
-
-In this example, the `Eval` is used to manipulate data from a previous task. GraalVM makes it easy to generate outputs from variables in Python using the `outputs` property. This is useful if you want to manipulate data and pass the new format to another task.
-
-```yaml
-id: parse_json_data
-namespace: company.team
-
-tasks:
- - id: download
- type: io.kestra.plugin.core.http.Download
- uri: http://xkcd.com/info.0.json
-
- - id: graal
- type: io.kestra.plugin.graalvm.ruby.Eval
- outputs:
- - data
- script: |
- data = {{ read(outputs.download.uri) }}
- data["next_month"] = '{{ read(outputs.download.uri) | jq(".month") | first }}'.to_i + 1
- return {data: data}
-```
diff --git a/src/contents/docs/15.how-to-guides/ruby/metrics.png b/src/contents/docs/15.how-to-guides/ruby/metrics.png
deleted file mode 100644
index b0aac9f1c4c..00000000000
Binary files a/src/contents/docs/15.how-to-guides/ruby/metrics.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/ruby/outputs.png b/src/contents/docs/15.how-to-guides/ruby/outputs.png
deleted file mode 100644
index 32165bbdf29..00000000000
Binary files a/src/contents/docs/15.how-to-guides/ruby/outputs.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/rust/index.md b/src/contents/docs/15.how-to-guides/rust/index.md
deleted file mode 100644
index 9839b2f7946..00000000000
--- a/src/contents/docs/15.how-to-guides/rust/index.md
+++ /dev/null
@@ -1,139 +0,0 @@
----
-title: Run Rust Inside Your Flows
-h1: Execute High-Performance Rust Code in Kestra with Docker
-icon: /src/contents/docs/icons/rust.svg
-stage: Getting Started
-topics:
- - Scripting
-description: Execute Rust code directly within your Kestra flows using Docker to leverage Rust's performance for your data processing tasks.
----
-
-Run Rust code directly in your flows and generate outputs.
-
-Rust has jumped in popularity over the past few years, mainly due to its performance and reliability in production settings. Compared to Python, Rust is a great choice for performance-critical workloads so might be a good choice to use in your flows.
-
-This guide is going to walk you through how to get Rust running in a workflow, how to manage input and output files, and how you can pass outputs and metrics back to Kestra to use in later tasks.
-
-There isn't an official Rust plugin but we can use the `Shell` `Commands` task to execute arbitrary commands in a Docker container. We can also specify a container image that contains the necessary libraries to run the specific programming language.
-
-In this example, we're using the Docker Task Runner with the `rust:latest` image so that Rust code can be executed.
-
-```yaml
-id: rust_commands
-namespace: company.team
-tasks:
- - id: rust
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: rust:latest
- namespaceFiles:
- enabled: true
- commands:
- - rustc main.rs && ./main
-```
-
-The contents of the `main.rs` file contains a simple print statement:
-
-```rust
-fn main() {
- println!("Hello World");
-}
-```
-
-You'll need to add your Rust code using the built-in Editor or [using our Git plugin](../../version-control-cicd/04.git/index.md) so Kestra can see it. You'll also need to set the `enabled` flag for the `namespaceFiles` property to `true` so Kestra can access the file.
-
-You can also add your Rust code inline using the `inputFiles` property.
-
-```yaml
-id: rust_commands
-namespace: company.team
-tasks:
- - id: rust
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: rust:latest
- inputFiles:
- main.rs: |
- fn main() {
- println!("Hello World!");
- }
- commands:
- - rustc main.rs && ./main
-```
-
-You can read more about the Shell Commands type in the [Plugin documentation](/plugins/plugin-script-shell/io.kestra.plugin.scripts.shell.commands).
-
-## Handling Outputs
-
-Your Rust code can generate file-based [outputs](../../05.workflow-components/06.outputs/index.md).
-
-In your Rust code, write a file to the local directory. Then, use the `outputFiles` property to point Kestra to the path of those [output files](../../16.scripts/06.outputs-metrics/index.md).
-
-In this example, `output.txt` file containing the text "Hello World" is written to the local directory. To read that output file in another downstream task, you can use the syntax `{{ outputs.{task_id}.outputFiles[''] }}`, and if you need a file's content as a string rather than a file path, you can wrap that expression in a `read()` function e.g. `{{ read(outputs.mytask.outputFiles['outputs.txt']) }}`.
-
-```yaml
-id: rust_script
-namespace: company.team
-
-tasks:
- - id: rust
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: rust:latest
- inputFiles:
- main.rs: |
- use std::fs::File;
- use std::io::Write; // For the `write_all` method
-
- fn main() -> std::io::Result<()> {
- // Create or open the file `output.txt` in write mode
- let mut file = File::create("output.txt")?;
-
- // Write the string "Hello, World" to the file
- file.write_all(b"Hello World")?;
-
- // Confirm successful write operation
- println!("Successfully wrote to the file.");
-
- Ok(())
- }
- outputFiles:
- - output.txt
- commands:
- - rustc main.rs && ./main
-
- - id: read_file
- type: io.kestra.plugin.core.log.Log
- message: "{{ read(outputs.rust.outputFiles['output.txt']) }}"
-```
-
-## Orchestrate with Rust
-
-Rust is a great choice for performance-critical workloads. If you're working with huge datasets, Rust could be a good choice for ETL. Below is an example of how you can setup Rust in Kestra to perform an ETL process.
-
-The example flow uses a Rust image created using the following [sample ETL project](https://github.com/kestra-io/examples/tree/main/examples/rust). The image contains the CLI command `etl` to allow us to start the process.
-
-```yaml
-id: rust_in_container
-namespace: company.team
-
-tasks:
- - id: rust
- type: io.kestra.plugin.scripts.shell.Commands
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
- containerImage: ghcr.io/kestra-io/rust:latest
- outputFiles:
- - "*.csv"
- commands:
- - etl
-```
-
-Once the container finishes execution, you'll be able to download all CSV files generated by the Rust container from the Outputs tab. Kestra makes it easy to both process heavy compute workloads while providing an intuitive interface to access the results.
-
-:::alert{type="info"}
-The `ghcr.io/kestra-io/rust:latest` image is public, so you can directly use the example shown above.
-:::
diff --git a/src/contents/docs/15.how-to-guides/secops-with-kestra/index.md b/src/contents/docs/15.how-to-guides/secops-with-kestra/index.md
index ae2225f2af3..ea20c73962a 100644
--- a/src/contents/docs/15.how-to-guides/secops-with-kestra/index.md
+++ b/src/contents/docs/15.how-to-guides/secops-with-kestra/index.md
@@ -223,13 +223,11 @@ triggers:
- id: postVMCreation
type: io.kestra.plugin.core.trigger.Flow
inputs:
- ipAddress: "{{ trigger.outputs.externalIPAddress }}"
- preconditions:
- id: vmCreationSuccess
- flows:
- - namespace: company.ops.it
- flowId: createVMRevamped
- states: [ SUCCESS, WARNING ]
+ ipAddress: "{{ trigger.outputs.createVMRevamped.externalIPAddress }}"
+ dependsOn:
+ - namespace: company.ops.it
+ flowId: createVMRevamped
+ states: [SUCCESS, WARNING]
```
## Step 7: Review the Topology
diff --git a/src/contents/docs/15.how-to-guides/selected-plugin-installation/index.md b/src/contents/docs/15.how-to-guides/selected-plugin-installation/index.md
index 68bb3461fda..d8e41e1ff7b 100644
--- a/src/contents/docs/15.how-to-guides/selected-plugin-installation/index.md
+++ b/src/contents/docs/15.how-to-guides/selected-plugin-installation/index.md
@@ -5,14 +5,14 @@ icon: /src/contents/docs/icons/tutorial.svg
stage: Getting Started
topics:
- Kestra Concepts
-description: Learn how to install specific Kestra plugins in the open-source version for a lightweight build and faster startup using the -no-plugins Docker image.
+description: Learn how to install specific Kestra plugins in the open-source version for a lightweight build and faster startup using the -slim Docker image.
---
Install a selection of Kestra plugins in the open-source version.
Pick and choose Kestra plugins to create lightweight builds and achieve a faster startup. This guide explains how to:
-- Install specific plugins when using the `-no-plugins` Docker image
+- Install specific plugins when using the `-slim` Docker image (formerly `-no-plugins`)
- Understand plugin versioning across Open Source and [Enterprise](../../07.enterprise/01.overview/01.enterprise-edition/index.md)
- Automate plugin installation using Docker Compose
- Link to plugin documentation and versioning support
@@ -23,7 +23,7 @@ To download plugins for a standalone worker or local development environment, us
## Plugin basics in Kestra Open Source
-Kestra plugins are distributed as individual JAR files and loaded at runtime. Plugins are not embedded by default in `-no-plugins` Docker images. You can:
+Kestra plugins are distributed as individual JAR files and loaded at runtime. Plugins are not embedded by default in `-slim` Docker images (formerly published as `-no-plugins`). You can:
- Download specific [plugin JARs](https://repo.maven.apache.org/maven2/io/kestra/plugin/) manually or via `kestra plugins install`.
- Mount them into `/app/plugins/` in your [Docker Compose](../../02.installation/03.docker-compose/index.md) setup.
@@ -42,14 +42,14 @@ You can run this inside a container (interactively or as part of Dockerfile) to
## Automate plugin selection with Docker Compose
-If you're using the `kestra/kestra:*-no-plugins` image and want to add only selected plugins:
+If you're using the `kestra/kestra:*-slim` image and want to add only selected plugins:
### Option 1: Use `kestra plugins install` inside the container
```yaml
services:
kestra:
- image: kestra/kestra:latest-no-plugins
+ image: kestra/kestra:latest-slim
entrypoint: /bin/sh -c "
kestra plugins install io.kestra.plugin:plugin-dbt:LATEST && \
kestra plugins install io.kestra.plugin:plugin-scripts:LATEST && \
@@ -104,7 +104,7 @@ Learn more about versioned plugins in Enterprise:
| Use Case | Recommendation |
| -------------------------- | ---------------------------------------------------- |
-| Minimal runtime image | Use `kestra/kestra:*-no-plugins` with mounted JARs |
+| Minimal runtime image | Use `kestra/kestra:*-slim` with mounted JARs |
| Dynamic plugin setup | Use `kestra plugins install` in entrypoint |
| Controlled plugin versions | Use Enterprise with versioned plugins |
| Custom plugin development | Build and copy plugins into `/app/plugins/` manually |
diff --git a/src/contents/docs/15.how-to-guides/shell/index.md b/src/contents/docs/15.how-to-guides/shell/index.md
index 4c1c70a12cd..3e65390d219 100644
--- a/src/contents/docs/15.how-to-guides/shell/index.md
+++ b/src/contents/docs/15.how-to-guides/shell/index.md
@@ -117,9 +117,9 @@ tasks:
commands:
- echo '::{"outputs":{"test":"value","int":2,"bool":true,"float":3.65}}::'
- - id: return
- type: io.kestra.plugin.core.debug.Return
- format: '{{ outputs.shell_outputs_task.vars.test }}'
+ - id: log_vars
+ type: io.kestra.plugin.core.log.Log
+ message: '{{ outputs.shell_outputs_task.vars.test }}'
```
_This example works for both `io.kestra.plugin.scripts.shell.Script` and `io.kestra.plugin.scripts.shell.Commands`._
@@ -172,3 +172,57 @@ tasks:
Once this has executed, both the metrics can be viewed under **Metrics**.
+
+## Automate Shell with triggers
+
+You can also use shell code as polling logic by using `ScriptTrigger` or `CommandsTrigger`. These trigger types run shell code on an interval and start a flow execution only when the `exitCondition` matches.
+
+Use `ScriptTrigger` for inline shell code:
+
+```yaml
+id: script_trigger
+namespace: company.team
+
+triggers:
+ - id: script_failure
+ type: io.kestra.plugin.scripts.shell.ScriptTrigger
+ interval: PT10S
+ exitCondition: "exit 1"
+ edge: true
+ containerImage: ubuntu
+ script: |
+ cat /path/that/does/not/exist
+
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Triggered with exitCode={{ trigger.exitCode }} (condition={{ trigger.condition }})"
+```
+
+Use `CommandsTrigger` when you want to run shell commands instead:
+
+```yaml
+id: commands_trigger
+namespace: company.team
+
+triggers:
+ - id: commands_failure
+ type: io.kestra.plugin.scripts.shell.CommandsTrigger
+ interval: PT10S
+ exitCondition: "exit 1"
+ edge: true
+ containerImage: ubuntu
+ commands:
+ - cat /path/that/does/not/exist
+
+tasks:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Triggered with exitCode={{ trigger.exitCode }} (condition={{ trigger.condition }})"
+```
+
+These trigger types support:
+
+- `interval` to control how often the script or commands run
+- `exitCondition` to match an exit code such as `exit 1`, or a regex or substring matched against emitted vars and failure logs
+- `edge` to emit only on a transition from not matching to matching
diff --git a/src/contents/docs/15.how-to-guides/shipyard-migration/documentation_view.png b/src/contents/docs/15.how-to-guides/shipyard-migration/documentation_view.png
deleted file mode 100644
index 050d48da631..00000000000
Binary files a/src/contents/docs/15.how-to-guides/shipyard-migration/documentation_view.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/shipyard-migration/index.md b/src/contents/docs/15.how-to-guides/shipyard-migration/index.md
deleted file mode 100644
index fd9b5eb7164..00000000000
--- a/src/contents/docs/15.how-to-guides/shipyard-migration/index.md
+++ /dev/null
@@ -1,59 +0,0 @@
----
-title: Migrate from Shipyard to Kestra
-h1: Map Shipyard Concepts to Kestra Flows and Tasks
-icon: /src/contents/docs/icons/tutorial.svg
-stage: Getting Started
-topics:
- - Best Practices
-description: A comprehensive guide for migrating workflows from Shipyard to Kestra, mapping concepts like Fleets and Vessels to Flows and Tasks.
----
-
-Migrate from Shipyard to Kestra.
-
-This is a guide for users who are considering migration of their workflows from [Shipyard](https://www.shipyardapp.com/) to [Kestra](https://kestra.io/).
-
-[Kestra](https://kestra.io/) is a language-agnostic orchestration platform allowing users to build workflows as code and from the UI. Similarly to Shipyard, Kestra uses YAML for workflow logic, and its extensive plugin ecosystem makes migration straightforward.
-
-## Technical Glossary
-
-| Shipyard Concept | Equivalent Concept in Kestra | Description |
-|------------------|-----------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| [Fleet](https://www.shipyardapp.com/docs/reference/fleets/fleets-overview/) | [Flow](../../05.workflow-components/01.flow/index.md) | a container for tasks, their inputs, outputs, handling of errors and overall orchestration logic |
-| [Vessel](https://www.shipyardapp.com/docs/reference/vessels/) | [Task](../../05.workflow-components/01.tasks/index.mdx) | a discrete action within a flow, capable of taking inputs and variables from the flow, and producing outputs for downstream consumption by end users and other tasks |
-| [Project](https://www.shipyardapp.com/docs/reference/projects/) | [Namespace](../../05.workflow-components/02.namespace/index.md) | a logical grouping of flows, used to organize workflows and manage access to secrets, plugin defaults and variables |
-| [Triggers](https://www.shipyardapp.com/docs/reference/triggers/triggers-overview/) | [Triggers](../../05.workflow-components/07.triggers/index.mdx) | a mechanism that automates the execution of a flow; triggers can be scheduled or event-based |
-| [Blueprints](https://www.shipyardapp.com/docs/blueprint-library/) | [Blueprints](/blueprints) | a collection of premade templates ready to be used in your workflows; Blueprints work very similarly between both platforms — the main difference is that Shipyard's blueprints are like plugins in kestra since they are used to run a task (vessel). Kestra's blueprints are more comprehensive, they often contain multiple tasks composed together to accomplish some use case end-to-end. |
-| [Inputs](https://www.shipyardapp.com/docs/reference/inputs/) | [Inputs](../../05.workflow-components/05.inputs/index.md) | a list of dynamic values passed to the flow at runtime; the main difference between both is that Shipyard's inputs are provided to the task (i.e. vessel), while Kestra's inputs are defined at a flow (i.e. fleet) level |
-| UI | [UI](../../09.ui/index.mdx) | Shipyard's UI allows building workflows via drag-and-drop and autogenerates a YAML configuration; in Kestra, users typically write the YAML configuration first and then they can optionally modify the workflow or add new tasks from low-code UI forms. |
-
-## Getting Started with Kestra
-
-To get started, follow the [Quickstart Guide](../../01.quickstart/index.md) to install Kestra and start building your first workflows.
-
-## How to Migrate
-
-Every fleet in Shipyard generates a YAML configuration. You can retrieve it from the UI as shown below, or get it from the version control system like Git in case you maintained one for Shipyard.
-
-
-
-For every vessel in the fleet, try to find a matching [Kestra Plugin](/plugins). For example, the equivalent of **Amazon S3 - Delete Files** vessel in Shipyard will be [io.kestra.plugin.aws.s3.Delete](/plugins/plugin-aws/aws-s3/io.kestra.plugin.aws.s3.delete) and [io.kestra.plugin.aws.s3.DeleteList](/plugins/plugin-aws/aws-s3/io.kestra.plugin.aws.s3.deletelist).
-
-In the same fashion as you would configure a vessel, you can configure a task in Kestra. Use the built-it task documentation in the Kestra UI to help you configure all task properties (the **Source and documentation** view). Find plugins directly within the built-in UI editor using the auto-complete feature. Each task documentation comes with an example and a detailed description of each task property.
-
-
-
-There is no concept of **connections** in Kestra. By default, all tasks are executed sequentially. To adjust the execution logic e.g. to run some tasks in parallel, wrap your tasks in [flowable tasks](../../05.workflow-components/01.tasks/00.flowable-tasks/index.md). As always, the combination of the [core documentation](../../index.mdx), [Plugin documentation](/plugins) and [Blueprints](/blueprints) will help you figure out how to do that.
-
-Once you have the fleet equivalent (i.e. a flow) ready in Kestra, you can use the **Source and topology view** to validate whether your Kestra flow matches the connections in your Shipyard fleet.
-
-
-
-You can now Save and Execute your flow. Then, check the Logs, Gantt and Outputs tab of your Execution to validate that your workflow behaves as expected.
-
-## Need Help?
-
-Check out our extensive [plugin catalog](/plugins) for descriptions and examples of each task and trigger.
-
-Use our [blueprints](/blueprints) for guidance on creating various flows.
-
-For assistance, join our free [Slack community](/slack) and ask your questions in the `#help` channel. We respond to every message!
diff --git a/src/contents/docs/15.how-to-guides/shipyard-migration/shipyard_yaml_configuration.png b/src/contents/docs/15.how-to-guides/shipyard-migration/shipyard_yaml_configuration.png
deleted file mode 100644
index 114f8ca8e38..00000000000
Binary files a/src/contents/docs/15.how-to-guides/shipyard-migration/shipyard_yaml_configuration.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/shipyard-migration/topology_view.png b/src/contents/docs/15.how-to-guides/shipyard-migration/topology_view.png
deleted file mode 100644
index 6523ed77cdf..00000000000
Binary files a/src/contents/docs/15.how-to-guides/shipyard-migration/topology_view.png and /dev/null differ
diff --git a/src/contents/docs/15.how-to-guides/slack-webhook/index.md b/src/contents/docs/15.how-to-guides/slack-webhook/index.md
index cedcc42afc8..cfaff5c4fcc 100644
--- a/src/contents/docs/15.how-to-guides/slack-webhook/index.md
+++ b/src/contents/docs/15.how-to-guides/slack-webhook/index.md
@@ -109,11 +109,11 @@ tasks:
import logging
from fastapi import FastAPI, Request, BackgroundTasks
from fastapi.responses import JSONResponse
- from modal import Image, Stub, asgi_app
+ from modal import Image, App, asgi_app
import requests
web_app = FastAPI()
- stub = Stub("slack_app")
+ app = App("slack_app")
image = Image.debian_slim().pip_install("requests")
@@ -121,7 +121,7 @@ tasks:
logger = logging.getLogger(__name__)
def process_event(event):
- # TODO adjust the URL below to your Kestra Webhook URL
+ # Replace with your Kestra Webhook URL
url = "http://your_kestra_host:8080/api/v1/main/executions/webhook/prod/slack_events/superStrongSecretKey42"
headers = {"Content-Type": "application/json"}
response = requests.post(url, headers=headers, json=event)
@@ -144,7 +144,7 @@ tasks:
logger.info("Responding immediately to Slack")
return JSONResponse(content={"status": "ok"})
- @stub.function(image=image)
+ @app.function(image=image)
@asgi_app()
def fastapi_app():
return web_app
diff --git a/src/contents/docs/15.how-to-guides/splunk-alert-trigger/index.md b/src/contents/docs/15.how-to-guides/splunk-alert-trigger/index.md
index 2201ab413b9..26b392cb56d 100644
--- a/src/contents/docs/15.how-to-guides/splunk-alert-trigger/index.md
+++ b/src/contents/docs/15.how-to-guides/splunk-alert-trigger/index.md
@@ -5,7 +5,7 @@ icon: /src/contents/docs/icons/splunk.svg
stage: Intermediate
topics:
- Integrations
-editions: ["EE"]
+editions: ["EE", "Cloud"]
description: Install the Kestra Add-on for Splunk and configure the Trigger Kestra Flow alert action to start Kestra flows from Splunk saved-search alerts using a Bearer API token.
---
diff --git a/src/contents/docs/15.how-to-guides/supabase-db/index.md b/src/contents/docs/15.how-to-guides/supabase-db/index.md
index a5d21e3e02f..5dce7ba1439 100644
--- a/src/contents/docs/15.how-to-guides/supabase-db/index.md
+++ b/src/contents/docs/15.how-to-guides/supabase-db/index.md
@@ -8,17 +8,13 @@ topics:
description: Learn how to connect your Supabase Database to Kestra workflows using the PostgreSQL plugin to query, copy, and manage your data.
---
-Connect your Supabase Database to your workflows using the PostgreSQL plugin.
+Use the native [Supabase plugin](/plugins/plugin-supabase) for new integrations. This guide documents the PostgreSQL plugin approach for direct database access.
-:::alert{type="info"}
-There is a dedicated [Supabase plugin](/plugins/plugin-supabase) to replace these steps.
-:::
-
-Supabase is an open-source Backend-as-a-service (BaaS) platform that helps developers build applications faster and more efficiently. They provide a number of services, including hosted PostgreSQL databases, which can be used within Flows in Kestra.
+Supabase is an open-source Backend-as-a-service (BaaS) platform that provides hosted PostgreSQL databases you can query directly from Kestra flows.
Before you begin, ensure you have a [Supabase account](https://supabase.com/) set up and a [Kestra installation](../../02.installation/index.mdx) running.
@@ -44,34 +40,22 @@ Inside of Supabase, select the **Connect** button at the top to get information
-To connect, we can copy the URL provided for the Transaction pooler and replace `[YOUR-PASSWORD]` with the password set earlier. To prevent exposing the password in our flow, store it as a [secret](../../06.concepts/04.secret/index.md).
-
-By using [Plugin Defaults](../../05.workflow-components/09.plugin-defaults/index.md), we can configure our connection to Supabase once for all tasks in our flow rather than individually for each task.
-
-Once configured, our connection in Kestra will look like the example below:
+Copy the URL provided for the Transaction pooler and replace `[YOUR-PASSWORD]` with the password set earlier. Store the password as a [secret](../../06.concepts/04.secret/index.md) and add the `url` property directly to each PostgreSQL task:
```yaml
-pluginDefaults:
- - forced: true
- type: io.kestra.plugin.jdbc.postgresql
- values:
- url: "jdbc:postgresql://aws-0-eu-west-2.pooler.supabase.com:6543/postgres?user=postgres.nqxaafovehwkjapsqqlk&password={{ secret('SUPABASE_PASSWORD') }}"
-
+url: "jdbc:postgresql://aws-0-eu-west-2.pooler.supabase.com:6543/postgres?user=postgres.nqxaafovehwkjapsqqlk&password={{ secret('SUPABASE_PASSWORD') }}"
```
:::alert{type="info"}
-You can also use the `username` and `password` properties rather than combining it all into the `url` property:
+You can also split the connection string into separate properties:
```yaml
-pluginDefaults:
- - forced: true
- type: io.kestra.plugin.jdbc.postgresql
- values:
- url: "jdbc:postgresql://aws-0-eu-west-2.pooler.supabase.com:6543/postgres"
- username: "postgres.nqxaafovehwkjapsqqlk"
- password: "{{ secret('SUPABASE_PASSWORD') }}"
+url: "jdbc:postgresql://aws-0-eu-west-2.pooler.supabase.com:6543/postgres"
+username: "postgres.nqxaafovehwkjapsqqlk"
+password: "{{ secret('SUPABASE_PASSWORD') }}"
```
+In Enterprise Edition, you can centralize connection properties across flows using a [Policy](../../07.enterprise/02.governance/policies/index.md) with an `Add` rule targeting `io.kestra.plugin.jdbc.postgresql`.
:::
## Copying a CSV File into Supabase DB in a Flow
@@ -85,6 +69,7 @@ namespace: company.team
tasks:
- id: create_columns
type: io.kestra.plugin.jdbc.postgresql.Queries
+ url: "jdbc:postgresql://aws-0-eu-west-2.pooler.supabase.com:6543/postgres?user=postgres.nqxaafovehwkjapsqqlk&password={{ secret('SUPABASE_PASSWORD') }}"
sql: |
ALTER TABLE kestra_example
ADD COLUMN order_id int,
@@ -94,15 +79,9 @@ tasks:
ADD COLUMN price double precision,
ADD COLUMN quantity int,
ADD COLUMN total double precision;
-
-pluginDefaults:
- - forced: true
- type: io.kestra.plugin.jdbc.postgresql
- values:
- url: "jdbc:postgresql://aws-0-eu-west-2.pooler.supabase.com:6543/postgres?user=postgres.nqxaafovehwkjapsqqlk&password={{ secret('SUPABASE_PASSWORD') }}"
```
-Once your columns are configured, you can use the [CopyIn](/plugins/plugin-jdbc-postgres/io.kestra.plugin.jdbc.postgresql.copyin) task combined with the [HTTP Download](/plugins/core/http/io.kestra.plugin.core.http.download) task to download the CSV file and copy it directly into our database. As we set up the database connection with our [Plugin Defaults](#connecting-supabase-to-kestra), the CopyIn task will connect directly and copy the CSV file into the database.
+Once your columns are configured, use the [CopyIn](/plugins/plugin-jdbc-postgres/io.kestra.plugin.jdbc.postgresql.copyin) task combined with the [HTTP Download](/plugins/core/http/io.kestra.plugin.core.http.download) task to download the CSV file and copy it directly into the database.
```yaml
id: supabase_db_copyin
@@ -115,17 +94,12 @@ tasks:
- id: copy_in
type: io.kestra.plugin.jdbc.postgresql.CopyIn
+ url: "jdbc:postgresql://aws-0-eu-west-2.pooler.supabase.com:6543/postgres?user=postgres.nqxaafovehwkjapsqqlk&password={{ secret('SUPABASE_PASSWORD') }}"
table: "kestra_example"
from: "{{ outputs.download.uri }}"
header: true
columns: [order_id,customer_name,customer_email,product_id,price,quantity,total]
delimiter: ","
-
-pluginDefaults:
- - forced: true
- type: io.kestra.plugin.jdbc.postgresql
- values:
- url: "jdbc:postgresql://aws-0-eu-west-2.pooler.supabase.com:6543/postgres?user=postgres.nqxaafovehwkjapsqqlk&password={{ secret('SUPABASE_PASSWORD') }}"
```
Once this flow completes, we can view the contents of our database in Supabase:
diff --git a/src/contents/docs/15.how-to-guides/synchronous-executions-api/index.md b/src/contents/docs/15.how-to-guides/synchronous-executions-api/index.md
index 4135aabce03..29cf438508c 100644
--- a/src/contents/docs/15.how-to-guides/synchronous-executions-api/index.md
+++ b/src/contents/docs/15.how-to-guides/synchronous-executions-api/index.md
@@ -57,7 +57,6 @@ By default, the Executions API is asynchronous. It will invoke the execution of
"startDate": "2024-07-12T05:07:28.447110427Z"
},
"originalId": "1KWLxLeaXEXNDaXWP7YSKA",
- "deleted": false,
"metadata": {
"attemptNumber": 1,
"originalCreatedDate": "2024-07-12T05:07:28.447113302Z"
@@ -70,7 +69,7 @@ By default, the Executions API is asynchronous. It will invoke the execution of
To wait for an execution to finish and return the flow outputs in the response, call the Executions API with the `wait=true` query parameter. This would make the API call synchronous, and you will receive all outputs in the response that are explicitly exposed in the flow. You can invoke the Executions API in a synchronous fashion as follows:
```bash
-curl -X POST 'http://localhost:8080e/api/v1/main/xecutions/company.team/myflow?wait=true'
+curl -X POST 'http://localhost:8080/api/v1/main/executions/company.team/myflow?wait=true'
```
Here is the output of this API invocation:
@@ -84,9 +83,6 @@ Here is the output of this API invocation:
"taskRunList": [
{
"id": "4536yghIDGwqeRWZEE7AEE",
- "executionId": "24znmto07B2ZGrI9IQoSSH",
- "namespace": "company.team",
- "flowId": "myflow",
"taskId": "mytask",
"attempts": [
{
@@ -112,9 +108,6 @@ Here is the output of this API invocation:
}
}
],
- "outputs": {
- "value": "hello from kestra"
- },
"state": {
"current": "SUCCESS",
"histories": [
@@ -161,7 +154,6 @@ Here is the output of this API invocation:
"startDate": "2024-07-12T05:13:41.789Z"
},
"originalId": "24znmto07B2ZGrI9IQoSSH",
- "deleted": false,
"metadata": {
"attemptNumber": 1,
"originalCreatedDate": "2024-07-12T05:13:41.789Z"
@@ -169,7 +161,7 @@ Here is the output of this API invocation:
}
```
-As expected, the API response returned the outputs from the flow. It also contains all execution states.
+The API response includes the flow-level `outputs` (the values explicitly declared in the flow's `outputs:` block) and the execution state. Task run outputs are not included in the execution response — use `GET /api/v1/{tenant}/outputs/{executionId}/{taskRunId}` to retrieve per-task outputs.
## Authentication
diff --git a/src/contents/docs/15.how-to-guides/syncnamespacefiles/index.md b/src/contents/docs/15.how-to-guides/syncnamespacefiles/index.md
index 36cb97f6cd6..a02a2a0f598 100644
--- a/src/contents/docs/15.how-to-guides/syncnamespacefiles/index.md
+++ b/src/contents/docs/15.how-to-guides/syncnamespacefiles/index.md
@@ -113,7 +113,7 @@ tasks:
type: io.kestra.plugin.git.SyncNamespaceFiles
username: git_username
password: "{{ secret('GITHUB_ACCESS_TOKEN') }}"
- url: https://github.com/git_username/scipts
+ url: https://github.com/git_username/scripts
branch: main
namespace: git
gitDirectory: _files
diff --git a/src/contents/docs/15.how-to-guides/terraform-templating/index.md b/src/contents/docs/15.how-to-guides/terraform-templating/index.md
index 711f88133de..321f1a169b0 100644
--- a/src/contents/docs/15.how-to-guides/terraform-templating/index.md
+++ b/src/contents/docs/15.how-to-guides/terraform-templating/index.md
@@ -262,7 +262,7 @@ tasks:
type: io.kestra.plugin.jdbc.postgresql.Query
url: jdbc:postgresql://MY_HOST/MY_DATABASE
username: MY_USER
- password: "{{ secrets.get('my-postgres-password') }}"
+ password: "{{ secret('my-postgres-password') }}"
sql: "{{ inputs.sqlQuery }}"
fetchType: FETCH
@@ -294,7 +294,7 @@ Executing the subflow will prompt you to enter the SQL query you want to execute
```yaml
- id: query_last_job
- type: io.kestra.core.tasks.flows.Subflow
+ type: io.kestra.plugin.core.flow.Subflow
namespace: company.team
flowId: query_my_postgres_database
inputs:
@@ -303,9 +303,8 @@ Executing the subflow will prompt you to enter the SQL query you want to execute
transmitFailed: true
- id: use_result
- type: io.kestra.core.tasks.debugs.Return
- # Use the query result from the subflow
- format: "{{ outputs.query_last_job.outputs.query_result }}"
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ outputs.query_last_job.outputs.query_result }}"
```
1. Connection details are stored in the subflow, and only the SQL query is exposed to the user.
diff --git a/src/contents/docs/15.how-to-guides/vault-read-only-secrets/index.md b/src/contents/docs/15.how-to-guides/vault-read-only-secrets/index.md
index b8841a99b30..70632248edc 100644
--- a/src/contents/docs/15.how-to-guides/vault-read-only-secrets/index.md
+++ b/src/contents/docs/15.how-to-guides/vault-read-only-secrets/index.md
@@ -105,17 +105,12 @@ tasks:
- id: copy_in
type: io.kestra.plugin.jdbc.postgresql.CopyIn
+ url: "jdbc:postgresql://ep-ancient-flower-a2e73um1-pooler.eu-central-1.aws.neon.tech/neondb?user=neondb_owner&password={{ secret('my-app', subkey='NEON_PASSWORD') }}"
table: "kestra_example_secret"
from: "{{ outputs.download.uri }}"
header: true
columns: [order_id,customer_name,customer_email,product_id,price,quantity,total]
delimiter: ","
-
-pluginDefaults:
- - forced: true
- type: io.kestra.plugin.jdbc.postgresql
- values:
- url: jdbc:postgresql://ep-ancient-flower-a2e73um1-pooler.eu-central-1.aws.neon.tech/neondb?user=neondb_owner&password={{ secret('my-app', subkey='NEON_PASSWORD') }}
```
:::
diff --git a/src/contents/docs/15.how-to-guides/webhooks/index.md b/src/contents/docs/15.how-to-guides/webhooks/index.md
index ee92727d97c..848c3ef745d 100644
--- a/src/contents/docs/15.how-to-guides/webhooks/index.md
+++ b/src/contents/docs/15.how-to-guides/webhooks/index.md
@@ -32,7 +32,7 @@ description: |
tasks:
- id: out
type: io.kestra.plugin.core.debug.Return
- format: "{{ trigger | json }}"
+ format: "{{ trigger | toJson }}"
triggers:
@@ -85,7 +85,7 @@ description: |
tasks:
- id: out
type: io.kestra.plugin.core.debug.Return
- format: "{{ trigger | json }}"
+ format: "{{ trigger | toJson }}"
triggers:
- id: webhook_trigger
@@ -109,5 +109,5 @@ where:
With this information, you can test your flow by running the following command in the terminal to trigger the flow:
```bash
-curl http://my.kestra.clod/api/v1/my_tenant/executions/webhook/company.team/webhook_eE_example/1KERKzRQZSMtLdMdNI7Nkr
+curl http://localhost:8080/api/v1/my_tenant/executions/webhook/company.team/webhook_ee_example/1KERKzRQZSMtLdMdNI7Nkr
```
diff --git a/src/contents/docs/16.scripts/00.languages/index.md b/src/contents/docs/16.scripts/00.languages/index.md
index 0260c90f29f..2303436f723 100644
--- a/src/contents/docs/16.scripts/00.languages/index.md
+++ b/src/contents/docs/16.scripts/00.languages/index.md
@@ -41,6 +41,8 @@ Each of these plugins provides two task types:
- `Script` for short inline code in your flow definition.
- `Commands` for code stored in files or split across multiple commands.
+Some plugins also provide `ScriptTrigger` and `CommandsTrigger` variants, which run code on a polling interval and start a flow execution only when an exit condition matches. See [Polling trigger](../../05.workflow-components/07.triggers/04.polling-trigger/index.md) for the concept, and the language-specific guides below for working examples (Python, Shell, Ruby, Go, and JavaScript).
+
Here is a minimal example that uses the Python `Script` task:
```yaml
@@ -76,15 +78,10 @@ tasks:
Use these guides for complete examples, outputs, metrics, and dependency management:
- [Run Python inside your flows](../../15.how-to-guides/python/index.md)
-- [Run R inside your flows](../../15.how-to-guides/r/index.md)
- [Run JavaScript inside your flows](../../15.how-to-guides/javascript/index.md)
- [Run Shell scripts inside your flows](../../15.how-to-guides/shell/index.md)
- [Run PowerShell inside your flows](../../15.how-to-guides/powershell/index.md)
- [Run C# inside your flows](../../15.how-to-guides/dotnet/index.md)
-- [Run Julia inside your flows](../../15.how-to-guides/julia/index.md)
-- [Run Go inside your flows](../../15.how-to-guides/golang/index.md)
-- [Run Perl inside your flows](../../15.how-to-guides/perl/index.md)
-- [Run Rust inside your flows](../../15.how-to-guides/rust/index.md)
## Run other languages with the Shell plugin
@@ -127,7 +124,7 @@ fn main() {
}
```
-See the full [Rust guide](../../15.how-to-guides/rust/index.md) for outputs and file handling.
+For outputs and file handling from compiled languages, see [Shell outputs and metrics](../06.outputs-metrics/index.md#shell).
### Java example
diff --git a/src/contents/docs/16.scripts/03.task-runners/index.md b/src/contents/docs/16.scripts/03.task-runners/index.md
index 102a0a24344..b6a7f2a2245 100644
--- a/src/contents/docs/16.scripts/03.task-runners/index.md
+++ b/src/contents/docs/16.scripts/03.task-runners/index.md
@@ -3,9 +3,6 @@ title: "Task Runners in Scripts: Control Execution Environment"
h1: Manage Docker and Process Runners for Script Execution
sidebarTitle: Task Runners
icon: /src/contents/docs/icons/dev.svg
-deprecated:
- since: "0.18.0"
- migrationGuide: /docs/migration-guide/0.18.0/runners
description: Manage execution environments for your scripts using Kestra's Task Runners, including Docker and Process runners.
---
diff --git a/src/contents/docs/16.scripts/04.custom-docker-image/index.md b/src/contents/docs/16.scripts/04.custom-docker-image/index.md
index 657d5e4bbc1..bd13e581fb0 100644
--- a/src/contents/docs/16.scripts/04.custom-docker-image/index.md
+++ b/src/contents/docs/16.scripts/04.custom-docker-image/index.md
@@ -34,7 +34,7 @@ services:
image: kestra-python:latest
```
-Once you start Kestra containers using `docker compose up -d`, you can create a flow that directly runs Python tasks with your custom dependencies using the `PROCESS` runner:
+Once you start Kestra containers using `docker compose up -d`, you can create a flow that directly runs Python tasks with your custom dependencies using the Process task runner:
```yaml
id: python_process
@@ -42,7 +42,8 @@ namespace: company.team
tasks:
- id: custom_dependencies
type: io.kestra.plugin.scripts.python.Script
- runner: PROCESS
+ taskRunner:
+ type: io.kestra.plugin.core.runner.Process
script: |
import pandas as pd
import requests
diff --git a/src/contents/docs/16.scripts/06.outputs-metrics/index.md b/src/contents/docs/16.scripts/06.outputs-metrics/index.md
index abd512b8729..f41f24a8101 100644
--- a/src/contents/docs/16.scripts/06.outputs-metrics/index.md
+++ b/src/contents/docs/16.scripts/06.outputs-metrics/index.md
@@ -173,7 +173,7 @@ tasks:
type: io.kestra.plugin.scripts.shell.Script
containerImage: ubuntu
script: |
- echo '{"outputs":{"test":"value","int":2,"bool":true,"float":3.65}}'
+ echo '::{"outputs":{"test":"value","int":2,"bool":true,"float":3.65}}::'
echo '::{"metrics":[{"name":"count","type":"counter","value":1,"tags":{"tag1":"i","tag2":"win"}}]}::'
echo '::{"metrics":[{"name":"time","type":"timer","value":2.12,"tags":{"tag1":"i","tag2":"destroy"}}]}::'
```
diff --git a/src/contents/docs/16.scripts/07.input-output-files/index.md b/src/contents/docs/16.scripts/07.input-output-files/index.md
index 279df88f333..5c81776c95e 100644
--- a/src/contents/docs/16.scripts/07.input-output-files/index.md
+++ b/src/contents/docs/16.scripts/07.input-output-files/index.md
@@ -114,10 +114,6 @@ To generate files in your script and make them available for download and use in
### Generating outputs from a script task using `outputFiles`
-:::alert{type="info"}
-From 0.17.0, `outputDir` has been deprecated. Use the `outputFiles` property instead.
-:::
-
The `outputFiles` property allows to specify a list of files to be persisted in Kestra's internal storage. Here is an example:
```yaml
@@ -147,3 +143,31 @@ tasks:
Note how the `outputFiles` property is used to specify the list of files to be persisted in Kestra's internal storage. The `outputFiles` property supports [glob patterns](https://en.wikipedia.org/wiki/Glob_(programming)).
The subsequent task can access the output file by leveraging the syntax `{{outputs.yourTaskId.outputFiles['yourFileName.fileExtension']}}`.
+
+### Referencing output file paths inside the script
+
+For local runners (Process and Docker), writing files by plain name works because Kestra sets the process working directory automatically. For remote task runners (Kubernetes, AWS Batch, Azure Batch, etc.), the working directory is an execution-specific absolute path. Rather than constructing it manually with `{{ workingDir }}/filename`, you can use `{{ outputFiles["filename"] }}` to get the resolved absolute path by name:
+
+```yaml
+id: output_file_remote
+namespace: company.team
+
+tasks:
+ - id: shell
+ type: io.kestra.plugin.scripts.shell.Commands
+ taskRunner:
+ type: io.kestra.plugin.ee.kubernetes.runner.Kubernetes
+ config:
+ masterUrl: https://my-cluster:6443
+ caCertData: "{{ secret('K8S_CA_CERT_DATA') }}"
+ outputFiles:
+ - out.txt
+ commands:
+ - echo "Hello from Kubernetes" > {{ outputFiles["out.txt"] }}
+```
+
+The `{{ outputFiles["filename"] }}` expression resolves to the absolute path of the named file in the task's working directory — the same value as `{{ workingDir }}/filename`. The same form is used in JDBC tasks (e.g., `COPY ... TO '{{ outputFiles["out.csv"] }}'`), so the pattern is consistent across task types.
+
+:::alert{type="info"}
+Only named files are available as Pebble expressions. Glob patterns such as `*.csv` are collected post-run but cannot be referenced as `{{ outputFiles["*.csv"] }}` inside the script. Declare named files for any output you need to reference by path, and use globs only for bulk collection.
+:::
diff --git a/src/contents/docs/16.scripts/09.execution-context/index.md b/src/contents/docs/16.scripts/09.execution-context/index.md
new file mode 100644
index 00000000000..65a7fb8c86f
--- /dev/null
+++ b/src/contents/docs/16.scripts/09.execution-context/index.md
@@ -0,0 +1,134 @@
+---
+title: "Execution Context in Scripts: Read Metadata as Data"
+h1: Read the Execution Context from Your Scripts
+sidebarTitle: Execution Context
+icon: /src/contents/docs/icons/dev.svg
+description: Read inputs, outputs, labels and variables inside a script from a JSON file instead of interpolating Pebble expressions into your code.
+---
+
+Read the execution metadata from inside your script.
+
+## Why read the context as data
+
+A script task can reach the execution metadata through Pebble expressions:
+
+```python
+csv_path = "{{ outputs.extract.uri }}"
+```
+
+That works, but the expression is not valid Python, so a script kept as a [Namespace File](/docs/concepts/namespace-files) and edited in an IDE fails linting. It also mixes orchestration into business logic, and it breaks outright when a value contains a quote or a newline, because the value is pasted into your source before the interpreter sees it.
+
+Setting `executionContext: true` writes the same metadata to a `.kestra-execution-context.json` file in the task working directory, so your script reads it as data and stays plain, valid code:
+
+```python
+from kestra import context
+
+csv_path = context.outputs.extract.uri
+```
+
+## Enable the execution context
+
+Add `executionContext: true` to any script task:
+
+```yaml
+id: execution_context
+namespace: company.team
+
+inputs:
+ - id: dataset
+ type: STRING
+ defaults: products
+
+labels:
+ env: prod
+
+variables:
+ threshold: 42
+
+tasks:
+ - id: extract
+ type: io.kestra.plugin.core.http.Download
+ uri: https://huggingface.co/datasets/kestra/datasets/raw/main/csv/orders.csv
+
+ - id: transform
+ type: io.kestra.plugin.scripts.python.Script
+ executionContext: true
+ beforeCommands:
+ - pip install kestra pandas
+ inputFiles:
+ orders.csv: "{{ outputs.extract.uri }}"
+ script: |
+ import pandas as pd
+ from kestra import context
+
+ df = pd.read_csv("orders.csv")
+
+ print(f"dataset: {context.inputs.dataset}")
+ print(f"environment: {context.labels.env}")
+ print(f"threshold: {context.vars.threshold}")
+ print(f"internal storage URI: {context.outputs.extract.uri}")
+ print(f"rows: {len(df)}")
+```
+
+Note that the context gives you the *metadata* of a previous task, including its `kestra:///` internal storage URI. To read the file itself, map it into the working directory with [`inputFiles`](/docs/scripts/input-output-files) as above — a `kestra:///` URI is not a path your script can open directly.
+
+The property is available on every script task, so `Python`, `Node.js`, `Shell`, `Powershell`, `R`, `Julia`, `Ruby`, `Go`, `Groovy`, `PHP`, `Perl`, `Lua`, `Deno`, `Bun`, `.NET` and `JBang` all accept it, with every task runner.
+
+## What the context contains
+
+The file holds the same variables your flow can render with Pebble, as a single JSON object:
+
+| Key | Contents |
+| --- | --- |
+| `inputs` | The execution inputs |
+| `outputs` | Outputs of the tasks that already ran |
+| `labels` | Execution labels |
+| `vars` | Flow variables |
+| `trigger` | Trigger metadata, on a triggered execution |
+| `flow`, `execution`, `task`, `taskrun` | Identifiers, state and dates |
+
+Two variables are left out on purpose: `envs` and `globals`. Both already reach your script as environment variables, so read them with `os.environ` instead.
+
+## Read the context from Python
+
+The [`kestra` Python package](https://pypi.org/project/kestra/) exposes the file as a `context` object. Attributes traverse to any depth, including objects nested in lists:
+
+```python
+from kestra import context
+
+my_labels = context.labels
+start_date = context.trigger.startDate
+csv_path = context.outputs.extract.uri
+first_file = context.inputs.files[0].name
+```
+
+`context` is a read-only mapping, so `len()`, `in`, iteration, `keys()`, `values()`, `items()` and `dict(context)` all behave as expected. Use the bracket form for keys that are not valid Python identifiers, or that collide with one of those method names:
+
+```python
+context["items"] # a key literally named "items"
+context.get("optional_key", "fallback")
+context.to_dict() # the raw dict, ready for json.dumps
+```
+
+Reading a key that is not there raises an `AttributeError` listing what is available, and the context is loaded lazily, so `import kestra` still works in a script that runs outside of Kestra. Reaching for `context` there raises a `FileNotFoundError`.
+
+## Read the context from another language
+
+The context is a plain JSON file, so any language can read it without a Kestra client:
+
+```javascript
+const fs = require('fs');
+
+const context = JSON.parse(fs.readFileSync('.kestra-execution-context.json', 'utf8'));
+
+console.log(context.outputs.extract.uri);
+```
+
+## Secrets and lifecycle
+
+The file contains the values your task could already render, and that includes inputs of type `SECRET` in plaintext. Two guardrails follow from that:
+
+- The file name is reserved, and never collected by your patterns. An `outputFiles: ["*"]` glob, a `WorkingDirectory` `cache.patterns: ["**"]`, or an `UploadFiles` task will not pick it up, so it cannot reach internal storage.
+- The file is removed once the task's commands are done, so a later task in the same [Working Directory](/docs/scripts/working-directory) never inherits another task's context.
+
+`executionContext` is disabled by default for the same reason — writing the metadata to disk stays an explicit choice.
diff --git a/src/contents/docs/16.scripts/index.mdx b/src/contents/docs/16.scripts/index.mdx
index a39b4778f67..7efd1286628 100644
--- a/src/contents/docs/16.scripts/index.mdx
+++ b/src/contents/docs/16.scripts/index.mdx
@@ -32,4 +32,4 @@ If you use the [Enterprise Edition](/docs/enterprise), you can also run your scr
The following pages dive into details of each task runner, supported programming languages, and how to manage dependencies.
-
\ No newline at end of file
+
diff --git a/src/contents/docs/ai-tools/ai-copilot/ai-copilot.png b/src/contents/docs/ai-tools/01.ai-copilot/ai-copilot.png
similarity index 100%
rename from src/contents/docs/ai-tools/ai-copilot/ai-copilot.png
rename to src/contents/docs/ai-tools/01.ai-copilot/ai-copilot.png
diff --git a/src/contents/docs/ai-tools/01.ai-copilot/ask-mode-diagnose.png b/src/contents/docs/ai-tools/01.ai-copilot/ask-mode-diagnose.png
new file mode 100644
index 00000000000..ae78ad8c7fc
Binary files /dev/null and b/src/contents/docs/ai-tools/01.ai-copilot/ask-mode-diagnose.png differ
diff --git a/src/contents/docs/ai-tools/ai-copilot/copilot-suggestion.png b/src/contents/docs/ai-tools/01.ai-copilot/copilot-suggestion.png
similarity index 100%
rename from src/contents/docs/ai-tools/ai-copilot/copilot-suggestion.png
rename to src/contents/docs/ai-tools/01.ai-copilot/copilot-suggestion.png
diff --git a/src/contents/docs/ai-tools/ai-copilot/copilot-trigger.png b/src/contents/docs/ai-tools/01.ai-copilot/copilot-trigger.png
similarity index 100%
rename from src/contents/docs/ai-tools/ai-copilot/copilot-trigger.png
rename to src/contents/docs/ai-tools/01.ai-copilot/copilot-trigger.png
diff --git a/src/contents/docs/ai-tools/01.ai-copilot/edit-step-1-build.png b/src/contents/docs/ai-tools/01.ai-copilot/edit-step-1-build.png
new file mode 100644
index 00000000000..48e560903b2
Binary files /dev/null and b/src/contents/docs/ai-tools/01.ai-copilot/edit-step-1-build.png differ
diff --git a/src/contents/docs/ai-tools/01.ai-copilot/edit-step-2-errors.png b/src/contents/docs/ai-tools/01.ai-copilot/edit-step-2-errors.png
new file mode 100644
index 00000000000..c61f57399a6
Binary files /dev/null and b/src/contents/docs/ai-tools/01.ai-copilot/edit-step-2-errors.png differ
diff --git a/src/contents/docs/ai-tools/01.ai-copilot/edit-step-3-inputs.png b/src/contents/docs/ai-tools/01.ai-copilot/edit-step-3-inputs.png
new file mode 100644
index 00000000000..7220e6238ed
Binary files /dev/null and b/src/contents/docs/ai-tools/01.ai-copilot/edit-step-3-inputs.png differ
diff --git a/src/contents/docs/ai-tools/ai-copilot/fix-with-ai-gantt.png b/src/contents/docs/ai-tools/01.ai-copilot/fix-with-ai-gantt.png
similarity index 100%
rename from src/contents/docs/ai-tools/ai-copilot/fix-with-ai-gantt.png
rename to src/contents/docs/ai-tools/01.ai-copilot/fix-with-ai-gantt.png
diff --git a/src/contents/docs/ai-tools/01.ai-copilot/index.md b/src/contents/docs/ai-tools/01.ai-copilot/index.md
new file mode 100644
index 00000000000..3270746978d
--- /dev/null
+++ b/src/contents/docs/ai-tools/01.ai-copilot/index.md
@@ -0,0 +1,429 @@
+---
+title: AI Copilot in Kestra – Generate and Edit Flows
+h1: Generate and Refine Flows with Natural Language Prompts
+description: Use Kestra AI Copilot to generate and edit flows with natural language prompts. Get AI-assisted suggestions for tasks, triggers, and configurations.
+sidebarTitle: AI Copilot
+icon: /src/contents/docs/icons/ai.svg
+version: ">= 2.0.0"
+---
+
+Build and modify flows, ask questions about Kestra, and get AI-driven plans — all from a persistent chat sidebar.
+
+The AI Copilot opens as a right-side panel from the **AI** button in the top toolbar. Click **New chat +** to start a conversation, or use **Recents** to return to a previous one. Conversations persist for the browser session. You can type prompts or click the microphone button to dictate with speech-to-text.
+
+## Modes
+
+The Copilot has three modes, selectable from the dropdown at the bottom left of the chat panel:
+
+| Mode | What it does |
+|---|---|
+| **Ask** | Answers questions about Kestra using docs-grounded responses. Ask about features, configuration, plugin options, or get help diagnosing a failed execution. |
+| **Edit** | Generates and iteratively edits flow YAML. Describe what you want to build; the Copilot drafts the change and asks for confirmation before applying it. |
+| **Plan** | Proposes a step-by-step plan for a complex task. Each step requires individual approval before the Copilot executes it. Rejecting any step cancels the plan. |
+
+Switch modes at any point in a conversation — the Copilot carries the conversation history across mode switches.
+
+| If you want to… | Use |
+|---|---|
+| Build, modify, or refactor a flow | Edit |
+| Diagnose a failed execution | Ask |
+| Ask about Kestra features, plugins, or configuration | Ask |
+| Complete a multi-step task with approval at each step | Plan |
+
+## Context
+
+The Copilot automatically attaches the resource you are viewing as context when you open the panel. Attached resources appear as dismissible pills above the input. You can remove any pill to narrow the Copilot’s focus, and the transcript records each add and remove so you always know what the agent is looking at.
+
+Resources that can be attached as context:
+
+- Flow
+- Namespace
+- Execution
+- Dashboard
+- App
+- Test suite
+- Blueprint
+- Plugin
+
+Copilot also reads Namespace metadata — Policies, Variables, Secrets, and Key-Value pairs — so prompts like "Create a task that integrates with MongoDB" can reuse your configured credentials and variables without extra hints.
+
+## Confirmation
+
+In **Edit** and **Plan** modes, actions that modify resources (creating or updating a flow, restarting an execution) require explicit confirmation before the Copilot executes them. A confirmation prompt appears in the chat with an optional field to steer the next step. Approving executes the action; rejecting resumes the conversation in Edit mode or cancels the current plan in Plan mode.
+
+## Edit mode
+
+Edit mode generates and iteratively refines declarative flow YAML. Describe what you want to build; the Copilot searches available plugins, validates the generated YAML, and proposes the change for your approval. Once accepted, you can keep iterating — adding triggers, adjusting tasks, or refactoring a section — without the Copilot touching unrelated parts of the flow.
+
+Edit mode is available anywhere you build in Kestra — Flows, Apps, Unit tests, and Dashboards.
+
+## Usage limits
+
+When no custom provider is configured, Kestra uses a built-in AI service with a daily generation limit per instance. The UI shows how many generations you have left, and the limit resets daily at midnight UTC.
+
+To remove the limit, configure your own LLM provider in the `kestra.ai.providers` block. See [Configuration](#configuration) below.
+
+## Configuration
+
+To add Copilot to your flow editor, add the following to your [Enterprise and Advanced configuration](../../configuration/06.enterprise-and-advanced/index.md). The `providers` array lets you register multiple LLMs and pick a default (`is-default: true`):
+
+```yaml
+kestra:
+ ai:
+ enabled: true # set to false to disable AI Copilot entirely
+ providers:
+ - id: gemini
+ display-name: Gemini - Private
+ type: gemini
+ configuration:
+ model-name: gemini-2.5-flash
+ api-key: YOUR_GEMINI_API_KEY
+ - id: gpt
+ display-name: OpenAI
+ type: openai
+ is-default: true
+ configuration:
+ model-name: gpt-4o
+ api-key: YOUR_OPENAI_API_KEY
+```
+
+:::alert{type="info"}
+Legacy single-provider configs (`kestra.ai.type` + provider block) still work, but the `providers` array lets you register multiple providers and choose a default (`is-default: true`).
+:::
+
+### Disabling AI Copilot
+
+To fully disable the AI Copilot — including the built-in fallback to the `api.kestra.io` service — set `kestra.ai.enabled` to `false`:
+
+```yaml
+kestra:
+ ai:
+ enabled: false
+```
+
+When disabled, the Copilot UI will not appear and all AI endpoints will be deactivated. The property defaults to `true`.
+
+### Multiple providers
+
+When multiple providers are configured, users can switch models from a dropdown in the Copilot UI instead of relying only on the default.
+
+For a full reference of every configuration property — including generation parameters, extended reasoning, mTLS, custom headers, and per-provider availability — see the [AI Copilot configuration reference](../../configuration/06.enterprise-and-advanced/index.md#ai-copilot).
+- `timeout`: Specifies the maximum duration to wait for an AI model API request to complete before timing out. ISO 8601 duration format (Java Duration): `PT30S` = 30 seconds. You can set it per provider to enforce strict SLAs.
+
+:::alert{type="info"}
+Enterprise Edition includes an [RBAC permission](../../07.enterprise/03.auth/rbac/index.md) that lets administrators allow or disallow Copilot usage per role at tenant or namespace scope.
+:::
+
+:::alert{type="info"}
+The open-source version supports only Google Gemini models. Enterprise Edition users can configure any LLM provider, including Amazon Bedrock, Anthropic, Azure OpenAI, DeepSeek, Google Gemini, Google Vertex AI, Mistral, OpenAI, OpenRouter, and all open-source models supported by Ollama. See [Enterprise Edition Copilot configurations](#enterprise-edition-copilot-configurations) below. If you use a different provider, [reach out to us](https://kestra.io/demo) and we'll add it.
+:::
+
+## Build flows with Edit mode
+
+Open the Copilot sidebar, select **Edit** mode, and describe what you want to build. The Copilot searches for the right plugins, generates validated YAML, and proposes the change for your approval. The flow is marked **Valid** before the proposal is shown — you will not be asked to apply broken YAML.
+
+**Step 1: Build the initial flow**
+
+```txt
+Create a flow that downloads a CSV from S3 and loads it into Postgres
+```
+
+
+
+The Copilot searches for the S3 and Postgres plugins, authors the flow with secrets referenced via `{{ secret('...') }}`, and presents the proposal. Select **Apply** to write it to the editor, or **Open in editor** to review the diff before accepting.
+
+**Step 2: Add error handling**
+
+```txt
+Add error handling that sends a Slack alert if any task fails
+```
+
+
+
+The Copilot updates only the `errors` block — the existing `download_from_s3` and `load_to_postgres` tasks are untouched. The Copilot explains what it changed before presenting the proposal.
+
+**Step 3: Parameterize hardcoded values**
+
+```txt
+Parameterize the S3 bucket name and Postgres table as flow inputs
+```
+
+
+
+The Copilot reads the current flow (note the `read-flow` step in the sidebar), adds an `inputs` block with `s3_bucket`, `s3_key`, and `postgres_table`, and rewires the hardcoded values to `{{ inputs.* }}` references throughout the flow. The flow and namespace context pills are attached automatically while working inside the editor.
+
+Each accepted change is saved as a revision. You can track the full edit history from the **Revisions** tab, or use [Git sync](../../version-control-cicd/04.git/index.md) to push revisions to your repository.
+
+## Ask mode
+
+Use Ask mode to ask natural-language questions about Kestra without generating any code. Ask mode grounds its answers in the Kestra documentation and can analyze execution failures by reading the execution logs directly.
+
+**Diagnosing a failed execution**
+
+When you open the Copilot from a failed execution view, the execution is automatically attached as context. Ask "Why did this execution fail?" and the Copilot reads the execution metadata and logs, then gives a structured answer: which task failed, the root-cause error, and what to fix.
+
+
+
+In the example above, the Copilot ran `read-execution` and `read-execution-logs`, identified that the `download_from_s3` task failed due to `SecretNotFoundException`, and listed exactly which secrets — `AWS_ACCESS_KEY_ID`, `AWS_SECRET_KEY_ID`, `POSTGRES_USERNAME`, `POSTGRES_PASSWORD`, `POSTGRES_HOST`, and `SLACK_WEBHOOK` — need to be configured before running the flow again.
+
+Other example questions:
+- "What is the difference between a Worker Group and a Task Runner?"
+- "How do I configure namespace-level Policies?"
+- "What secrets and variables are available in this namespace?" (with a namespace attached as context)
+
+Ask mode is also a useful starting point before switching to Edit or Plan — use it to understand your options, then switch modes to act on the answer.
+
+## Plan mode
+
+Use Plan mode when a task involves multiple ordered steps that you want to approve individually before the Copilot executes them. Plan mode presents the full plan upfront as a numbered list, then waits for your confirmation before starting. You can approve and execute the plan, or reply to revise it before anything runs.
+
+
+
+In the example above, the prompt "Build an ELT pipeline: extract from Salesforce, transform with dbt on DuckDB, load into Snowflake, and send a Slack summary on completion or failure" produced a four-step plan. The `company.team` namespace pill is attached, so the Copilot can reference available plugins and credentials in that namespace.
+
+Rejecting a step cancels the remaining steps. If you want to adjust the plan before it runs, use **Reply to revise** to send feedback and get a revised plan.
+
+Use Plan mode for tasks like:
+- Building a multi-stage pipeline where you want to review the structure before any YAML is generated
+- Migrating flows from one pattern to another (for example, from `ForEach` to the Loop task) across multiple steps
+- Setting up namespaces, variables, and RBAC in sequence for a new team
+
+## Fix with AI
+
+From the Logs and Gantt views, click the three-dot menu on any failed task and select **Fix with AI**. The flow editor opens with the Copilot pre-loaded with the error context in Edit mode, ready to propose a fix.
+
+## Starter prompts
+
+:::collapse{title="Edit mode prompts"}
+```markdown
+- Create a flow that runs a dbt build command on DuckDB
+- Create a flow cloning https://github.com/kestra-io/dbt-example Git repository from a main branch, then add a dbt CLI task using DuckDB backend that will run dbt build command for that cloned repository using my_dbt_project profile and dev target. The dbt project is located in the root directory so no dbt project needs to be configured.
+- Create a flow that sends a POST request to https://dummyjson.com/products/add
+- Send a POST request to https://dummyjson.com/products/add
+- Write a Python script that sends a POST request to https://dummyjson.com/products/add
+- Write a Node.js script that sends a POST request to https://dummyjson.com/products/add
+- Create a flow with a Python script that fetches weather data for New York City
+- Make a REST API call to https://kestra.io/api/mock and allow failure
+- Create a flow that logs "Hello from AI" to the console
+- Create a flow that returns Hello as output
+- Create a flow that outputs Hello as value
+- Run a flow every 10 minutes
+- Run a flow every day at 9 AM
+- Run a shell command echo 'Hello Docker' in a Docker container
+- Run a command python main.py in a Docker container
+- Run a script main.py stored as namespace file
+- Build a Docker image from an inline Dockerfile and push it to a GitHub Container Registry
+- Build a Docker image from an inline Dockerfile and push it to a DockerHub Container Registry
+- Create a flow that adds a string KV pair called MYKEY with value myvalue to namespace company
+- Fetch value for KV pair called MYKEY from namespace company
+- Create a flow that downloads a file mydata.csv from S3 bucket named mybucket
+- Create a flow that downloads all files from the folder kestra/plugins/ from S3 bucket mybucket in us-east-1
+- Send a Slack notification that approval is needed and Pause the flow for manual approval
+- Send a Slack alert whenever any execution from namespace company fails
+- Fetch value for string kv pair called mykey from Redis
+- Fetch value for mykey from Redis
+- Set value for mykey in Redis to myvalue
+- Sync all flows and scripts for selected namespaces from Git to Kestra
+- Create a flow that clones a Git repository and runs a Python script
+- Export a Postgres table called mytable to a CSV file
+- Query a Postgres table called mytable
+- Find documents in a MongoDB collection called mycollection
+- Load documents into a MongoDB mycollection using a file from input mydata
+- Trigger an Airbyte connection sync and retry it up to 3 times
+- Run an Airflow DAG called mydag
+- Orchestrate an Ansible playbook stored in Namespace Files
+- Run a DuckDB query that reads a CSV file
+- Fetch AWS ECR authorization token to push Docker images to Amazon ECR
+- Run a flow whenever 5 records are available in Kafka topic mytopic
+- Submit a run for a Databricks job
+```
+:::
+
+:::collapse{title="Ask mode prompts"}
+```markdown
+- Why did this execution fail? (attach the execution as context)
+- What secrets and variables are available in this namespace? (attach the namespace as context)
+- What is the difference between a Worker Group and a Task Runner?
+- What plugins are available for working with Kafka?
+- How do I configure RBAC so developers can run flows but not edit them?
+- What is the best way to handle retries for a flaky HTTP API?
+- How do I pass outputs from one task to the next?
+- What does the errors block do and when should I use it?
+- How do I schedule a flow to run only on weekdays?
+- What is the difference between Namespace Variables and the KV Store?
+```
+:::
+
+:::collapse{title="Plan mode prompts"}
+```markdown
+- Build an ELT pipeline: extract from Salesforce, transform with dbt on DuckDB, load into Snowflake, and send a Slack summary on completion or failure
+- Migrate all ForEach tasks in this flow to use the Loop task
+- Add retry logic, error notifications, and a timeout to every task in this flow
+- Set up namespaces for dev, staging, and prod with RBAC roles for the engineering team
+- Create a flow that ingests data from five different S3 paths in parallel, merges the results, and loads them into BigQuery
+```
+:::
+
+## Enterprise Edition Copilot configurations
+
+Enterprise Edition supports Amazon Bedrock, Anthropic, Azure OpenAI, DeepSeek, Google Gemini, Google Vertex AI, Mistral, OpenAI, OpenRouter, and all open-source models via Ollama. Add one or more provider blocks inside `kestra.ai.providers` and set `is-default: true` on the one Copilot should use by default.
+
+:::alert{type="warning"}
+Some Ollama model tags resolve to thinking models behind the scenes (for example, `qwen3:30b-a3b` points to a thinking variant). Thinking models that cannot have thinking disabled will produce incorrect YAML. Check that your chosen model supports a non-thinking mode, or use `thinking-enabled: false` where the provider supports it.
+:::
+
+### Amazon Bedrock
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: bedrock
+ display-name: Amazon Bedrock
+ type: bedrock
+ configuration:
+ model-name: amazon.nova-lite-v1:0
+ access-key-id: BEDROCK_ACCESS_KEY_ID
+ secret-access-key: BEDROCK_SECRET_ACCESS_KEY
+```
+
+### Anthropic
+
+Anthropic does not accept an `api-key` configuration field. Set the `ANTHROPIC_API_KEY` environment variable on the Kestra server instead.
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: anthropic
+ display-name: Anthropic
+ type: anthropic
+ configuration:
+ model-name: claude-opus-4-5
+```
+
+### Azure OpenAI
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: azure-openai
+ display-name: Azure OpenAI
+ type: azure-openai
+ configuration:
+ model-name: gpt-4o-2024-11-20
+ api-key: AZURE_OPENAI_API_KEY
+ tenant-id: AZURE_TENANT_ID
+ client-id: AZURE_CLIENT_ID
+ client-secret: AZURE_CLIENT_SECRET
+ endpoint: "https://your-resource.openai.azure.com/"
+```
+
+### Deepseek
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: deepseek
+ display-name: DeepSeek
+ type: deepseek
+ configuration:
+ model-name: deepseek-chat
+ api-key: DEEPSEEK_API_KEY
+ base-url: "https://api.deepseek.com/v1"
+```
+
+### Google Gemini
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: gemini
+ display-name: Google Gemini
+ type: gemini
+ configuration:
+ model-name: gemini-2.5-flash
+ api-key: YOUR_GEMINI_API_KEY
+```
+
+### Google Vertex AI
+
+Authenticates via Application Default Credentials; no `api-key` field is needed. Ensure the Kestra runtime has ADC configured (e.g. `GOOGLE_APPLICATION_CREDENTIALS` env var or Workload Identity).
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: vertex
+ display-name: Google Vertex AI
+ type: googlevertexai
+ configuration:
+ model-name: gemini-2.5-flash
+ project: GOOGLE_PROJECT_ID
+ location: us-central1
+```
+
+### Mistral
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: mistral
+ display-name: Mistral
+ type: mistralai
+ configuration:
+ model-name: mistral:7b
+ api-key: MISTRALAI_API_KEY
+ base-url: "https://api.mistral.ai/v1"
+```
+
+### Ollama
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: ollama
+ display-name: Ollama
+ type: ollama
+ configuration:
+ model-name: llama3
+ base-url: http://localhost:11434
+```
+
+:::alert{type="info"}
+If Ollama is running locally on your host machine while Kestra is running inside a container, connection errors may occur when using `localhost`. In this case, use the Docker internal network URL instead. For example, set the base URL to `http://host.docker.internal:11434`.
+:::
+
+### OpenAI
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: openai
+ display-name: OpenAI
+ type: openai
+ configuration:
+ model-name: gpt-5-nano
+ api-key: OPENAI_API_KEY
+ base-url: https://api.openai.com/v1
+```
+
+### OpenRouter
+
+```yaml
+kestra:
+ ai:
+ providers:
+ - id: openrouter
+ display-name: OpenRouter
+ type: open-router
+ configuration:
+ api-key: OPENROUTER_API_KEY
+ model-name: "anthropic/claude-sonnet-4"
+```
diff --git a/src/contents/docs/ai-tools/01.ai-copilot/plan-mode.png b/src/contents/docs/ai-tools/01.ai-copilot/plan-mode.png
new file mode 100644
index 00000000000..723aeb84730
Binary files /dev/null and b/src/contents/docs/ai-tools/01.ai-copilot/plan-mode.png differ
diff --git a/src/contents/docs/ai-tools/kestra-mcp-resources/index.md b/src/contents/docs/ai-tools/02.kestra-mcp-resources/index.md
similarity index 92%
rename from src/contents/docs/ai-tools/kestra-mcp-resources/index.md
rename to src/contents/docs/ai-tools/02.kestra-mcp-resources/index.md
index cbc4b99564e..28c2ef96a69 100644
--- a/src/contents/docs/ai-tools/kestra-mcp-resources/index.md
+++ b/src/contents/docs/ai-tools/02.kestra-mcp-resources/index.md
@@ -130,8 +130,8 @@ For prose questions — such as _"What does the `whenMissingInSource` property d
The Kestra MCP server complements the other AI tools in this section:
-- **[AI Copilot](../ai-copilot/index.md)**: generates and refines flows from natural language inside the Kestra UI.
-- **[AI Agents](../ai-agents/index.md)**: autonomous task execution inside Kestra flows.
-- **[Agent Skills](../agent-skills/index.md)**: structured knowledge files that teach coding agents how to build Kestra flows and operate environments.
+- **[AI Copilot](../01.ai-copilot/index.md)**: generates and refines flows from natural language inside the Kestra UI.
+- **[AI Agents](../05.ai-agents/index.md)**: autonomous task execution inside Kestra flows.
+- **[Agent Skills](../04.agent-skills/index.md)**: structured knowledge files that teach coding agents how to build Kestra flows and operate environments.
If you primarily work in an AI coding agent like Claude Code or Cursor, the MCP server gives you current Kestra context while building flows.
diff --git a/src/contents/docs/ai-tools/03.mcp-server/index.md b/src/contents/docs/ai-tools/03.mcp-server/index.md
new file mode 100644
index 00000000000..7a508941822
--- /dev/null
+++ b/src/contents/docs/ai-tools/03.mcp-server/index.md
@@ -0,0 +1,127 @@
+---
+title: MCP Server in Kestra – Expose Flows as AI Tools
+h1: Configure Kestra MCP Servers and Connect AI Agents
+description: Configure Kestra MCP servers to expose flows as tools for AI agents. Learn how to create servers, set authentication, and connect Claude Desktop, Claude Code, and Cursor.
+sidebarTitle: Kestra Flow MCP Servers
+icon: /src/contents/docs/icons/ai.svg
+version: "2.0.0"
+editions: ["OSS", "EE"]
+---
+
+A Kestra MCP server exposes flows as named tools over HTTP for AI agents to discover and call.
+
+A Kestra MCP server is a tenant-scoped entity that uses the [Model Context Protocol](https://modelcontextprotocol.io). Any flow with an [`McpToolTrigger`](../../05.workflow-components/07.triggers/06.mcp-tool-trigger/index.md) is automatically registered as a named tool on its target server. AI agents discover the tool list at connection time, so adding or removing triggers takes effect without restarting clients.
+
+## Two directions: Kestra as server vs. Kestra as client
+
+Kestra supports MCP in both directions:
+
+| Direction | How | When to use |
+|---|---|---|
+| **Kestra as MCP server** | `McpToolTrigger` + MCP server entity | AI agents (Claude, Cursor) call your flows as tools |
+| **Kestra as MCP client** | MCP client tasks (`SseMcpClient`, `StreamableHttpMcpClient`, `StdioMcpClient`, `DockerMcpClient`) | Your flows call external MCP servers as part of an AI Agent task |
+
+This page covers Kestra as an MCP server. For using external MCP servers from within flows, see [AI Agents](../05.ai-agents/index.md).
+
+## Default server
+
+A `default` MCP server is automatically provisioned for every tenant on startup. You can use it immediately with no setup needed. The `McpToolTrigger`'s `mcpServer` property defaults to `"default"`, so a minimal trigger requires no explicit server reference.
+
+## Managing MCP servers
+
+Navigate to **Tenant → MCP Servers** in the left sidebar to view, create, edit, and manage MCP servers.
+
+Each server has the following fields:
+
+| Field | Description |
+|---|---|
+| `name` | Display name for the server. |
+| `description` | Optional description shown in the UI. |
+| `systemPrompt` | Instructions prepended to every AI agent session connected to this server. Use this to guide agent behavior, for example to restrict which tools to call or define the agent's persona. |
+| `serverType` | `PRIVATE` (default) or `PUBLIC`. A private server requires authentication; a public server accepts unauthenticated connections. |
+| `authType` | `BASIC` (username/password, available in OSS and EE), `API_TOKEN` (EE and Cloud only), or `OAUTH` (EE and Cloud only). |
+
+### Authentication types
+
+| Auth type | Available in | Notes |
+|---|---|---|
+| `BASIC` | OSS, EE | Username and password required on connect. |
+| `API_TOKEN` | EE and Cloud | API token required on connect. Rejected on OSS. |
+| `OAUTH` | EE and Cloud | OAuth 2.0 flow. Required for browser-based MCP clients such as Claude web. Configure the OAuth provider name via `oauthProvider`. |
+
+Keep servers private unless you have a specific reason to expose them publicly. A public server allows any MCP client to call any flow registered on it without authentication.
+
+### Configuring OAuth authentication
+
+`OAUTH` auth requires an OIDC provider configured in your Kestra instance (the same configuration used for SSO). See [SSO configuration](../../07.enterprise/03.auth/sso/index.md) for how to set up an OIDC provider under `micronaut.security.oauth2.clients`.
+
+Once a provider is configured, set `authType` to `OAUTH` on the MCP server. The `oauthProvider` field specifies which provider to use by its configured name:
+
+```yaml
+# kestra.yml
+micronaut:
+ security:
+ oauth2:
+ clients:
+ my-oidc-provider:
+ client-id:
+ client-secret:
+ openid:
+ issuer: https://accounts.example.com
+```
+
+When a client connects to an `OAUTH`-protected MCP server and presents no token, the server responds with a `WWW-Authenticate` header pointing to the OAuth Protected Resource Metadata endpoint (RFC 9728). MCP-compliant clients such as Claude web discover the OIDC provider from this automatically and initiate the authorization code + PKCE flow; no manual client configuration is needed beyond the MCP server URL.
+
+:::alert{type="info"}
+The authenticating user must already have a Kestra account. The OAuth token's `email` or `preferred_username` claim is matched against existing Kestra users. If no matching user is found, the connection is rejected.
+:::
+
+If multiple OIDC providers are configured, set `oauthProvider` to the name of the specific provider (e.g. `my-oidc-provider` from the example above). If `oauthProvider` is omitted, the provider is matched automatically by the JWT's issuer claim.
+
+## Connecting an AI agent client
+
+Open a server in the UI and click the **Connect** tab. It shows the server URL and ready-to-paste configuration snippets for each supported client:
+
+- **Claude Desktop** — JSON block to add to `claude_desktop_config.json`. Claude Desktop does not natively support HTTP MCP servers, so the snippet uses `npx mcp-remote` as a bridge over SSE.
+- **Claude Code** — `claude mcp add` command using `--transport http`. Run it in your terminal, then start Claude Code from the same terminal session so the auth header is in scope.
+- **Cursor** — server URL to paste into Cursor Settings → MCP → Add new MCP server
+- **Codex** — connection configuration
+
+For Basic Auth, the snippet references `${KESTRA_BASIC_AUTH}` as a placeholder. Replace it with the actual base64-encoded `username:password` value inline; the variable reference does not expand at connection time:
+
+```bash
+claude mcp add \
+ --transport http \
+ --header "Authorization: Basic $(echo -n 'username:password' | base64)"
+```
+
+## Viewing registered tools
+
+The **Tool Flows** tab on each server lists all flows that have an `McpToolTrigger` pointing at that server. Use this to audit which flows are exposed and to navigate directly to a flow's trigger configuration.
+
+## RBAC (Enterprise)
+
+In the Enterprise Edition, `MCP_SERVER` is a first-class RBAC resource. See [RBAC](../../07.enterprise/03.auth/rbac/index.md#mcp-server-permissions) for the default role assignments.
+
+Access to a private server is also flow-scoped: a user can connect to a private MCP server only if they have `FLOW: EXECUTE` permission on at least one namespace that has a flow with an `McpToolTrigger` pointing at that server.
+
+## MCP server cache configuration
+
+By default, each webserver node caches MCP server configuration in memory and hot-reloads it when a server is created, updated, or deleted. Two optional properties control the cache behavior:
+
+| Property | Default | Description |
+|---|---|---|
+| `kestra.mcp.server-cache-config.maximum-size` | `500` | Maximum number of MCP server entries to cache. |
+| `kestra.mcp.server-cache-config.expire-after-access` | `PT5M` | How long a cache entry remains valid after last access. |
+
+Example configuration:
+
+```yaml
+kestra:
+ mcp:
+ server-cache-config:
+ maximum-size: 200
+ expire-after-access: PT10M
+```
+
+Adjust these only if you have a large number of MCP servers or tight memory constraints. The defaults are sufficient for most deployments.
diff --git a/src/contents/docs/ai-tools/agent-skills/index.md b/src/contents/docs/ai-tools/04.agent-skills/index.md
similarity index 83%
rename from src/contents/docs/ai-tools/agent-skills/index.md
rename to src/contents/docs/ai-tools/04.agent-skills/index.md
index 94ca695125a..e576dc5bfa9 100644
--- a/src/contents/docs/ai-tools/agent-skills/index.md
+++ b/src/contents/docs/ai-tools/04.agent-skills/index.md
@@ -13,9 +13,9 @@ Give AI coding agents structured knowledge to generate Kestra flows and operate
Agent Skills are structured knowledge files (`SKILL.md`) that teach external AI coding agents how to work with Kestra. They provide the context, commands, and guardrails an agent needs to generate valid flow YAML or operate a Kestra environment via the CLI.
-Unlike [AI Copilot](../ai-copilot/index.md), which works inside the Kestra UI, Agent Skills bring Kestra expertise to the tools you already use in your editor or terminal — Claude Code, Cursor, Windsurf, OpenAI Codex, and others.
+Unlike [AI Copilot](../01.ai-copilot/index.md), which works inside the Kestra UI, Agent Skills bring Kestra expertise to the tools you already use in your editor or terminal: Claude Code, Cursor, Windsurf, OpenAI Codex, and others.
-Unlike [AI Agents](../ai-agents/index.md), which are autonomous tasks running inside Kestra flows, Agent Skills equip your external coding agent with Kestra-specific knowledge so it can help you build and operate flows from your development environment.
+Unlike [AI Agents](../05.ai-agents/index.md), which are autonomous tasks running inside Kestra flows, Agent Skills equip your external coding agent with Kestra-specific knowledge so it can help you build and operate flows from your development environment.
Agent Skills follow an emerging standard for giving AI tools domain-specific knowledge. Learn more at [agentskills.io](https://agentskills.io/home), the community hub for agent skills across tools and domains.
@@ -25,7 +25,7 @@ Kestra provides multiple skills in the [kestra-io/agent-skills](https://github.c
### kestra-flow
-Generate, modify, or debug Kestra Flow YAML grounded in the live flow schema — the same approach used by Kestra's AI Copilot.
+Generate, modify, or debug Kestra Flow YAML grounded in the live flow schema (the same approach used by Kestra's AI Copilot).
**Use when:**
- Generating a new flow from a description
@@ -46,7 +46,7 @@ Use kestra-flow to write a flow that polls a REST API every 30 minutes and store
### kestra-flow-hardening
-Audit existing flows and add production-hardening controls — the consulting counterpart to `kestra-flow`. It is **audit-first**: it produces a severity-ranked findings report (risk, caveat, and proposed fix per finding), then applies only the edits you confirm.
+Audit existing flows and add production-hardening controls, the consulting counterpart to `kestra-flow`. It is **audit-first**: it produces a severity-ranked findings report (risk, caveat, and proposed fix per finding), then applies only the edits you confirm.
**Use when:**
- Hardening one or more flows for production
@@ -119,7 +119,7 @@ Use migrate-airflow-kestra to migrate dags/ingest_pipeline.py from Airflow to Ke
## Setup
-The easiest way to install Kestra agent skills is with [skills.sh](https://skills.sh) — it auto-detects your AI coding agent and places the skill files in the right location:
+The easiest way to install Kestra agent skills is with [skills.sh](https://skills.sh); it auto-detects your AI coding agent and places the skill files in the right location:
```bash
npx skills add kestra-io/agent-skills
@@ -139,7 +139,7 @@ curl -sL https://raw.githubusercontent.com/kestra-io/agent-skills/main/skills/ke
-o .claude/skills/kestra-ops/SKILL.md
```
-Repeat for any other skill you need (e.g. `kestra-flow`). Adjust the target directory for your agent — `.cursor/rules/` for Cursor, `.agents/skills/` for OpenAI Codex, etc. Some skills ship supporting files alongside `SKILL.md` (for example, `kestra-flow-hardening` includes `references/hardening-patterns.md`) — copy the whole skill directory so those files are available. Using `skills.sh` handles this automatically.
+Repeat for any other skill you need (e.g. `kestra-flow`). Adjust the target directory for your agent: `.cursor/rules/` for Cursor, `.agents/skills/` for OpenAI Codex, etc. Some skills ship supporting files alongside `SKILL.md` (for example, `kestra-flow-hardening` includes `references/hardening-patterns.md`); copy the whole skill directory so those files are available. Using `skills.sh` handles this automatically.
## Example Workflows
@@ -164,7 +164,7 @@ Use kestra-flow-hardening to audit ./flows/extract.yaml and add retries,
timeouts, and failure alerting where they're missing.
```
-The agent will produce a severity-ranked findings report — for example, flagging missing retries on external HTTP calls (Medium), a silent fallback masking a geocoding failure (data-correctness risk), and no failure alerting on a scheduled flow (High) — then apply only the fixes you confirm, validating each edit against the live schema.
+The agent will produce a severity-ranked findings report (for example, flagging missing retries on external HTTP calls (Medium), a silent fallback masking a geocoding failure (data-correctness risk), and no failure alerting on a scheduled flow (High)), then apply only the fixes you confirm, validating each edit against the live schema.
### Validate and deploy with kestra-ops
diff --git a/src/contents/docs/ai-tools/ai-agents/ai-agent-brevity.png b/src/contents/docs/ai-tools/05.ai-agents/ai-agent-brevity.png
similarity index 100%
rename from src/contents/docs/ai-tools/ai-agents/ai-agent-brevity.png
rename to src/contents/docs/ai-tools/05.ai-agents/ai-agent-brevity.png
diff --git a/src/contents/docs/ai-tools/ai-agents/ai-agent-inputs.png b/src/contents/docs/ai-tools/05.ai-agents/ai-agent-inputs.png
similarity index 100%
rename from src/contents/docs/ai-tools/ai-agents/ai-agent-inputs.png
rename to src/contents/docs/ai-tools/05.ai-agents/ai-agent-inputs.png
diff --git a/src/contents/docs/ai-tools/ai-agents/ai-agent-summary.png b/src/contents/docs/ai-tools/05.ai-agents/ai-agent-summary.png
similarity index 100%
rename from src/contents/docs/ai-tools/ai-agents/ai-agent-summary.png
rename to src/contents/docs/ai-tools/05.ai-agents/ai-agent-summary.png
diff --git a/src/contents/docs/ai-tools/05.ai-agents/index.md b/src/contents/docs/ai-tools/05.ai-agents/index.md
new file mode 100644
index 00000000000..d372710eb62
--- /dev/null
+++ b/src/contents/docs/ai-tools/05.ai-agents/index.md
@@ -0,0 +1,261 @@
+---
+title: AI Agents in Kestra – Autonomous Orchestration
+h1: Build LLM-Powered AI Agents That Think, Remember & Use Tools
+description: Build autonomous AI agents in Kestra for LLM-powered orchestration. Create agents that think, remember, and use tools like web search for complex workflows.
+sidebarTitle: AI Agents
+icon: /src/contents/docs/icons/ai.svg
+version: "1.0.0"
+---
+
+Launch autonomous processes with an LLM, memory, and tools.
+
+## Build autonomous AI agents in Kestra
+
+Add autonomous AI-driven tasks to flows that can think, remember, and dynamically orchestrate tools and tasks.
+
+
+
+
+
+An AI Agent is an autonomous system that uses a Large Language Model (LLM). Each run combines a **system message** and a **prompt**. The system message defines the agent's role and behavior, while the prompt carries the actual user input for that execution. Together, they guide the agent's response.
+
+With AI Agents, workflows are no longer limited to a predefined sequence of tasks. An AI Agent task launches an autonomous process with the help of an LLM, memory, and tools such as web search, task execution, and flow calling, and can dynamically decide which actions to take and in what order. Unlike traditional flows, an AI Agent can loop tasks until a condition is met, adapt to new information, and orchestrate complex multi-step objectives on its own. This enables agentic orchestration patterns in Kestra, where agents can operate independently or collaborate in multi-agent systems, all while remaining fully observable and manageable in code.
+
+To start using this feature, you can add an [**AI Agent**](/plugins/plugin-ai/agent) task to your flow. The AI Agent will then use the tools you provide to achieve its goal, leveraging capabilities such as web search, task execution, and flow calling. Thanks to memory, your AI Agent can remember information across executions to provide context for future tasks and subsequent prompts.
+
+## AI Agent flow example
+
+
+
+The following flow summarizes arbitrary text with controllable length and language. Each component of the flow is broken down below.
+
+```yaml
+id: simple_summarizer_agent
+namespace: company.ai
+inputs:
+ - id: summary_length
+ displayName: Summary Length
+ type: SELECT
+ defaults: medium
+ values:
+ - short
+ - medium
+ - long
+ - id: language
+ displayName: Language ISO code
+ type: SELECT
+ defaults: en
+ values:
+ - en
+ - fr
+ - de
+ - es
+ - it
+ - ru
+ - ja
+ - id: text
+ type: STRING
+ displayName: Text to summarize
+ defaults: |
+ Kestra is an open-source orchestration platform that:
+ - Allows you to define workflows declaratively in YAML
+ - Allows non-developers to automate tasks with a no-code interface
+ - Keeps everything versioned and governed, so it stays secure and auditable
+ - Extends easily for custom use cases through plugins and custom scripts.
+ Kestra follows a "start simple and grow as needed" philosophy. You can schedule a basic workflow in a few minutes, then later add Python scripts, Docker containers, or complicated branching logic if the situation calls for it.
+tasks:
+ - id: multilingual_agent
+ type: io.kestra.plugin.ai.agent.AIAgent
+ provider:
+ type: io.kestra.plugin.ai.provider.GoogleGemini
+ modelName: gemini-3.5-flash-lite
+ apiKey: "{{ secret('GEMINI_API_KEY') }}"
+ configuration:
+ logRequests: true
+ logResponses: true
+ responseFormat:
+ type: TEXT
+ systemMessage: |
+ You are a precise technical assistant.
+ Produce a {{ inputs.summary_length }} summary in {{ inputs.language }}.
+ Keep it factual, remove fluff, and avoid marketing language.
+ If the input is empty or non-text, return a one-sentence explanation.
+ Output format:
+ - 1-2 sentences for 'short'
+ - 2-5 sentences for 'medium'
+ - Up to 5 paragraphs for 'long'
+ prompt: |
+ Summarize the following content: {{ inputs.text }}
+
+ - id: english_brevity
+ type: io.kestra.plugin.ai.agent.AIAgent
+ provider:
+ type: io.kestra.plugin.ai.provider.GoogleGemini
+ modelName: gemini-3.5-flash-lite
+ apiKey: "{{ secret('GEMINI_API_KEY') }}"
+ configuration:
+ logRequests: true
+ logResponses: true
+ responseFormat:
+ type: TEXT
+ prompt: Generate exactly 1 sentence English summary of "{{ outputs.multilingual_agent.textOutput }}"
+```
+
+### Inputs
+
+The flow uses three inputs (`summary_length`, `language`, and `text`) to control the summary length, language, and source text.
+
+All inputs have a default value. Any of them can be referenced in downstream tasks with [expressions](../../expressions/index.mdx). When executing the flow, any input can be selected or modified from its default.
+
+
+
+The example selects `short` for the summary length and German (`de`) for the summary language.
+
+### Tasks
+
+The flow has two tasks using the [AI Agent plugin](/plugins/plugin-ai/agent): `multilingual_agent` and `english_brevity`. The first task, `multilingual_agent`, uses the `systemMessage` property to set the agent's role and behavior. The system message references the input selections for summary length and language, and defines what to output for each length option.
+
+The `prompt` property instructs the agent to summarize the input text. For a short summary, `multilingual_agent` produces a 1–2 sentence German summary of Kestra.
+
+
+
+The `english_brevity` task only needs a `prompt` because the `systemMessage` is inherited from plugin defaults. Whether the original output is in a different language or needs shortening, `english_brevity` produces a one-sentence English summary.
+
+
+
+These outputs can then be passed on as notifications or system messages to external tools or subflows within Kestra. Other useful outputs include `tokenUsage` to compare different providers for the same tasks. At runtime, Kestra also emits counter metrics — `ai.agent.tool.calls`, `ai.provider.calls`, and `ai.embedding.store.calls` — tagged by class name, which you can scrape with Prometheus or export via OpenTelemetry to monitor AI task usage. For more examples and details about properties, outputs, and definitions, refer to the AI [Agent plugin documentation](/plugins/plugin-ai/agent).
+
+### Centralizing provider configuration
+
+Each task using the AI Agent requires the `provider` property. To avoid repeating it on every task, use a [Policy](../../07.enterprise/02.governance/policies/index.md) with an `Add` rule to inject the `provider` block into all `AIAgent` tasks across a namespace — this is an Enterprise Edition and Cloud feature. For your provider API key, store it as a [Secret](../../06.concepts/04.secret/index.md) and reference it with `{{ secret('...') }}`.
+
+## Agent tools
+
+The AI Agent can be extended with **tools** — capabilities the LLM can choose to invoke at runtime to complete its task. Tools are listed under the `tools` property of an `AIAgent` task.
+
+### Skills
+
+The [**Skill**](/plugins/plugin-ai/tool/skill) tool lets you attach structured instructions to an agent that it can activate on demand. Rather than including all instructions in the system message, skills let you define discrete, reusable knowledge blocks — each with a name, a description the LLM uses to decide when to activate it, and the actual instruction content.
+
+This is useful when an agent has multiple possible modes of operation, such as translating text, reviewing code, or formatting data, where you want the LLM to select and apply the right instructions based on context rather than always receiving all instructions at once.
+
+Each skill requires:
+- `name` — a unique identifier for the skill
+- `description` — explains to the LLM when to activate the skill
+- `content` or `contentUri` — the instruction content, either inline or loaded from Kestra internal storage
+
+#### Inline skill content
+
+The simplest way to define a skill is with inline `content`:
+
+```yaml
+id: agent_with_skills
+namespace: company.ai
+
+tasks:
+ - id: agent
+ type: io.kestra.plugin.ai.agent.AIAgent
+ prompt: Translate the following text to French - "Hello, how are you today?"
+ provider:
+ type: io.kestra.plugin.ai.provider.GoogleGemini
+ modelName: gemini-3.5-flash-lite
+ apiKey: "{{ secret('GEMINI_API_KEY') }}"
+ tools:
+ - type: io.kestra.plugin.ai.tool.Skill
+ skills:
+ - name: translation_expert
+ description: Expert translator for multiple languages
+ content: |
+ You are an expert translator. When translating text:
+ 1. Preserve the original meaning and tone
+ 2. Use natural phrasing in the target language
+ 3. Keep proper nouns unchanged
+```
+
+#### Loading skill content from storage
+
+For longer or reusable instructions, store the skill content as a file in Kestra internal storage and reference it with `contentUri`. This is especially useful when skill content is generated or updated by an earlier task in the same flow:
+
+```yaml
+id: agent_with_skill_from_storage
+namespace: company.ai
+
+tasks:
+ - id: write_instructions
+ type: io.kestra.plugin.core.storage.Write
+ content: |
+ You are a senior code reviewer. When reviewing code:
+ 1. Check for security vulnerabilities
+ 2. Ensure proper error handling
+ 3. Verify naming conventions are followed
+ 4. Flag any code duplication
+
+ - id: agent
+ type: io.kestra.plugin.ai.agent.AIAgent
+ prompt: "Review this Python function - 'def add(a, b): return a + b'"
+ provider:
+ type: io.kestra.plugin.ai.provider.GoogleGemini
+ modelName: gemini-3.5-flash-lite
+ apiKey: "{{ secret('GEMINI_API_KEY') }}"
+ tools:
+ - type: io.kestra.plugin.ai.tool.Skill
+ skills:
+ - name: code_review_expert
+ description: Expert code reviewer with strict guidelines
+ contentUri: "{{ outputs.write_instructions.uri }}"
+```
+
+A single `Skill` tool can define multiple skills. Each skill must have a unique name. `content` and `contentUri` are mutually exclusive — exactly one must be set per skill. For more details on all available properties, refer to the [Skill plugin documentation](/plugins/plugin-ai/tool/skill).
+
+### Kestra-native tools
+
+- [**KestraFlow**](/plugins/plugin-ai/tool/kestraflow) — triggers a Kestra flow as a tool, either with a predefined namespace and flow ID or dynamically based on the agent's prompt.
+- [**KestraTask**](/plugins/plugin-ai/tool/kestratask) — exposes one or more Kestra runnable tasks as tools, letting the agent supply values for properties left unset.
+
+### Web search
+
+- [**TavilyWebSearch**](/plugins/plugin-ai/tool/tavilywebsearch) — gives the agent access to live web results via the Tavily search API.
+- [**GoogleCustomWebSearch**](/plugins/plugin-ai/tool/googlecustomwebsearch) — gives the agent access to live web results via a Google Custom Search Engine.
+
+### Code execution
+
+- [**CodeExecution**](/plugins/plugin-ai/tool/codeexecution) — lets the agent write and run JavaScript snippets in a Judge0 sandbox (via RapidAPI).
+
+### Nested agents
+
+- [**AIAgent**](/plugins/plugin-ai/tool/aiagent) — wraps another AI agent as a callable tool so a parent agent can delegate sub-tasks to a specialized child agent.
+- [**A2AClient**](/plugins/plugin-ai/tool/a2aclient) — forwards prompts to a remote AI agent over the Agent-to-Agent (A2A) protocol and returns its response.
+
+### MCP clients
+
+Kestra supports MCP in two directions. These clients cover the **Kestra-as-client** direction: your flow calls tools on an *external* MCP server. For the opposite direction — exposing your flows *as* MCP tools for external AI agents to call — see [MCP Server](../03.mcp-server/index.md) and the [McpToolTrigger](../../05.workflow-components/07.triggers/06.mcp-tool-trigger/index.md).
+
+Connect the agent to any [Model Context Protocol (MCP)](https://modelcontextprotocol.io) server to expose its tools:
+
+- [**DockerMcpClient**](/plugins/plugin-ai/tool/dockermcpclient) — runs an MCP server inside a Docker container.
+- [**SseMcpClient**](/plugins/plugin-ai/tool/ssemcpclient) — connects to a remote MCP server over Server-Sent Events (SSE).
+- [**StdioMcpClient**](/plugins/plugin-ai/tool/stdiomcpclient) — spawns a local MCP server process and communicates over stdio.
+- [**StreamableHttpMcpClient**](/plugins/plugin-ai/tool/streamablehttpmcpclient) — connects to an MCP server over HTTP streaming.
+
+The [Kestra Python MCP server](https://github.com/kestra-io/mcp-server-python) is an example of an external MCP server you can connect to from a Kestra AI Agent task using one of the clients above.
+
+## Execution details
+
+When you open an execution in the topology view, the details panel for `AIAgent`, `ChatCompletion`, and `rag.ChatCompletion` tasks shows the LLM configuration and post-execution context for each call.
+
+**Pre-execution:**
+- Model name and provider
+- System prompt (collapsible)
+- Tools available to the agent
+- RAG retriever and embedding store configuration (when applicable)
+
+**Post-execution:**
+
+| Signal | Description |
+|---|---|
+| LLM response | The final text or JSON output rendered inline |
+| Tool call timeline | Each tool invocation in order: name, arguments, and result |
+| Token usage | Input tokens, output tokens, total, and an estimated cost by provider and model |
+| Reasoning chain | Intermediate responses and extended thinking steps when present |
+| RAG sources | Retrieved chunks ranked by similarity score, showing which context grounded the answer |
+| Finish reason | Why the model stopped: natural stop, max tokens reached, or a guardrail trigger |
diff --git a/src/contents/docs/ai-tools/ai-workflows/index.md b/src/contents/docs/ai-tools/06.ai-workflows/index.md
similarity index 97%
rename from src/contents/docs/ai-tools/ai-workflows/index.md
rename to src/contents/docs/ai-tools/06.ai-workflows/index.md
index 8189722ed6a..dc77ebb7bda 100644
--- a/src/contents/docs/ai-tools/ai-workflows/index.md
+++ b/src/contents/docs/ai-tools/06.ai-workflows/index.md
@@ -137,7 +137,7 @@ tasks:
}
- id: create_tasks
- type: io.kestra.plugin.core.flow.ForEach
+ type: io.kestra.plugin.core.flow.Loop
values: "{{ outputs.create_task_fields.response | jq('.tasks') | first }}"
tasks:
- id: create_task
@@ -149,9 +149,9 @@ tasks:
Authorization: "Bearer {{ secret('TODOIST_API_TOKEN') }}"
body: |
{
- "content": "{{ taskrun.value | jq('.title') | first }}",
- "description": "{{ taskrun.value | jq('.description') | first }}",
- "due_datetime": "{{ taskrun.value | jq('.due_date') | first }}"
+ "content": "{{ item.value | jq('.title') | first }}",
+ "description": "{{ item.value | jq('.description') | first }}",
+ "due_datetime": "{{ item.value | jq('.due_date') | first }}"
}
```
:::
diff --git a/src/contents/docs/ai-tools/ai-rag-workflows/index.md b/src/contents/docs/ai-tools/07.ai-rag-workflows/index.md
similarity index 95%
rename from src/contents/docs/ai-tools/ai-rag-workflows/index.md
rename to src/contents/docs/ai-tools/07.ai-rag-workflows/index.md
index bb2c2e1a144..0f83f51553c 100644
--- a/src/contents/docs/ai-tools/ai-rag-workflows/index.md
+++ b/src/contents/docs/ai-tools/07.ai-rag-workflows/index.md
@@ -41,6 +41,8 @@ tasks:
type: io.kestra.plugin.ai.completion.ChatCompletion
provider:
type: io.kestra.plugin.ai.provider.GoogleGemini
+ apiKey: "{{ secret('GEMINI_API_KEY') }}"
+ modelName: gemini-3.5-flash-lite
messages:
- type: USER
content: Which features were released in Kestra 0.24?
@@ -49,19 +51,16 @@ tasks:
type: io.kestra.plugin.ai.rag.ChatCompletion
chatProvider:
type: io.kestra.plugin.ai.provider.GoogleGemini
+ apiKey: "{{ secret('GEMINI_API_KEY') }}"
+ modelName: gemini-3.5-flash-lite
embeddingProvider:
type: io.kestra.plugin.ai.provider.GoogleGemini
+ apiKey: "{{ secret('GEMINI_API_KEY') }}"
modelName: gemini-embedding-exp-03-07
embeddings:
type: io.kestra.plugin.ai.embeddings.KestraKVStore
systemMessage: You are a helpful assistant that can answer questions about Kestra.
prompt: Which features were released in Kestra 0.24?
-
-pluginDefaults:
- - type: io.kestra.plugin.ai.provider.GoogleGemini
- values:
- apiKey: "{{ secret('GEMINI_API_KEY') }}"
- modelName: gemini-3.5-flash-lite
```
### How it works
diff --git a/src/contents/docs/ai-tools/ai-agents/index.md b/src/contents/docs/ai-tools/ai-agents/index.md
deleted file mode 100644
index bd289347ed4..00000000000
--- a/src/contents/docs/ai-tools/ai-agents/index.md
+++ /dev/null
@@ -1,125 +0,0 @@
----
-title: AI Agents in Kestra – Autonomous Orchestration
-h1: Build LLM-Powered AI Agents That Think, Remember & Use Tools
-description: Build autonomous AI agents in Kestra for LLM-powered orchestration. Create agents that think, remember, and use tools like web search for complex workflows.
-sidebarTitle: AI Agents
-icon: /src/contents/docs/icons/ai.svg
-version: "1.0.0"
----
-
-Launch autonomous processes with an LLM, memory, and tools.
-
-## Build autonomous AI agents in Kestra
-
-Add autonomous AI-driven tasks to flows that can think, remember, and dynamically orchestrate tools and tasks.
-
-
-
-
-
-An AI Agent is an autonomous system that uses a Large Language Model (LLM). Each run combines a **system message** and a **prompt**. The system message defines the agent's role and behavior, while the prompt carries the actual user input for that execution. Together, they guide the agent's response.
-
-With AI Agents, workflows are no longer limited to a predefined sequence of tasks. An AI Agent task launches an autonomous process with the help of an LLM, memory, and tools such as web search, task execution, and flow calling, and can dynamically decide which actions to take and in what order. Unlike traditional flows, an AI Agent can loop tasks until a condition is met, adapt to new information, and orchestrate complex multi-step objectives on its own. This enables agentic orchestration patterns in Kestra, where agents can operate independently or collaborate in multi-agent systems, all while remaining fully observable and manageable in code.
-
-To start using this feature, you can add an [**AI Agent**](/plugins/plugin-ai/agent) task to your flow. The AI Agent will then use the tools you provide to achieve its goal, leveraging capabilities such as web search, task execution, and flow calling. Thanks to memory, your AI Agent can remember information across executions to provide context for future tasks and subsequent prompts.
-
-## AI Agent flow example
-
-
-
-To demonstrate, below is a flow that summarizes arbitrary text with controllable length and language. Each component of the flow is broken down.
-
-```yaml
-id: simple_summarizer_agent
-namespace: company.ai
-inputs:
- - id: summary_length
- displayName: Summary Length
- type: SELECT
- defaults: medium
- values:
- - short
- - medium
- - long
- - id: language
- displayName: Language ISO code
- type: SELECT
- defaults: en
- values:
- - en
- - fr
- - de
- - es
- - it
- - ru
- - ja
- - id: text
- type: STRING
- displayName: Text to summarize
- defaults: |
- Kestra is an open-source orchestration platform that:
- - Allows you to define workflows declaratively in YAML
- - Allows non-developers to automate tasks with a no-code interface
- - Keeps everything versioned and governed, so it stays secure and auditable
- - Extends easily for custom use cases through plugins and custom scripts.
- Kestra follows a "start simple and grow as needed" philosophy. You can schedule a basic workflow in a few minutes, then later add Python scripts, Docker containers, or complicated branching logic if the situation calls for it.
-tasks:
- - id: multilingual_agent
- type: io.kestra.plugin.ai.agent.AIAgent
- systemMessage: |
- You are a precise technical assistant.
- Produce a {{ inputs.summary_length }} summary in {{ inputs.language }}.
- Keep it factual, remove fluff, and avoid marketing language.
- If the input is empty or non-text, return a one-sentence explanation.
- Output format:
- - 1-2 sentences for 'short'
- - 2-5 sentences for 'medium'
- - Up to 5 paragraphs for 'long'
- prompt: |
- Summarize the following content: {{ inputs.text }}
-
- - id: english_brevity
- type: io.kestra.plugin.ai.agent.AIAgent
- prompt: Generate exactly 1 sentence English summary of "{{ outputs.multilingual_agent.textOutput }}"
-
-pluginDefaults:
- - type: io.kestra.plugin.ai.agent.AIAgent
- values:
- provider:
- type: io.kestra.plugin.ai.provider.GoogleGemini
- modelName: gemini-3.5-flash-lite
- apiKey: "{{ secret('GEMINI_API_KEY') }}"
- configuration:
- logRequests: true
- logResponses: true
- responseFormat:
- type: TEXT
-```
-
-### Inputs
-
-The goal of the AI Agent is to summarize text. The flow uses three inputs -- `summary_length`, `language`, and `text` -- to control the length, language, and source text for the summary.
-
-All inputs have a default value, and more or less can be used and referenced in downstream agentic tasks depending on the use case with [expressions](../../expressions/index.mdx). When executing the flow, all the inputs can be selected or modified from the defaults.
-
-
-
-Continuing below for reference, we select `short` for the summary length and German (`de`) for the summary language.
-
-### Tasks
-
-In the flow, there are two tasks using the [AI Agent plugin](/plugins/plugin-ai/agent): `multilingual_agent` and `english_brevity`. The first task, `multilingual_agent`, includes the `systemMessage` property which dictates the system message to the LLM provider. The system message references the input selections for the desired summary length and in what language to generate the summary in. It also defines what should be outputted when the input is short, medium, or long.
-
-Now that the AI Agent is familiar with its role, the `prompt` property tells it what to do, which is to summarize the inputted text. Taking a look at the output for a short summary, the `multilingual_agent` task does provide a 1–2 sentence summary of Kestra in German.
-
-
-
-Following `multilingual_agent` is the `english_brevity` task, which only needs a `prompt` because the `systemMessage` moves downstream in the flow. Whether a shorter English translation is needed, or the original outputted summary is in a different language, the `english_brevity` task provides a different output to match the need. In the execution context, the output is abbreviated and limited to exactly one sentence per the prompt.
-
-
-
-These outputs can then be passed on as notifications or system messages to external tools or subflows within Kestra. Other useful outputs include `tokenUsage` to compare different providers for the same tasks. For more examples and details about properties, outputs, and definitions, refer to the AI [Agent plugin documentation](/plugins/plugin-ai/agent).
-
-### Plugin defaults
-
-Each task using the AI Agent requires the `provider` property. To avoid repetition and simplify the flow building experience, first consider using [Kestra's AI Copilot](../ai-copilot/index.md), next consider using [Plugin Defaults](../../05.workflow-components/09.plugin-defaults/index.md) to ensure consistency and remove repetition. Additionally, for your provider API key, secure it either through the [Key-Value Store](../../06.concepts/05.kv-store/index.md) or as a [Secret](../../06.concepts/04.secret/index.md) if using [Kestra Enterprise Edition](../../07.enterprise/01.overview/01.enterprise-edition/index.md).
diff --git a/src/contents/docs/ai-tools/ai-copilot/index.md b/src/contents/docs/ai-tools/ai-copilot/index.md
deleted file mode 100644
index 6701d12cd65..00000000000
--- a/src/contents/docs/ai-tools/ai-copilot/index.md
+++ /dev/null
@@ -1,347 +0,0 @@
----
-title: AI Copilot in Kestra – Generate and Edit Flows
-h1: Generate and Refine Flows with Natural Language Prompts
-description: Use Kestra AI Copilot to generate and edit flows with natural language prompts. Get AI-assisted suggestions for tasks, triggers, and configurations.
-sidebarTitle: AI Copilot
-icon: /src/contents/docs/icons/ai.svg
-version: "1.0.0"
----
-
-Build and modify flows directly from natural language prompts.
-
-## Create and edit flows with AI Copilot
-
-The AI Copilot can generate and iteratively edit declarative flow code with AI-assisted suggestions.
-
-
-
-
-
-The AI Copilot is designed to help build and modify flows directly from natural language prompts. Describe what you are trying to build, and Copilot will generate the YAML flow code for you to accept or adjust. Once your initial flow is created, you can iteratively refine it with Copilot’s help, adding new tasks or adjusting triggers without touching unrelated parts of the flow. Everything stays as code and in Kestra's usual declarative syntax.
-
-Copilot is available anywhere you build in Kestra — Flows, Apps, Unit tests, and Dashboards — so you can keep iterating with the same AI assistant across the product surface.
-
-You can type prompts or click the microphone button in the Copilot panel to dictate them with speech-to-text directly from the UI.
-
-Copilot grounds its suggestions in your Namespace metadata. It automatically reads Plugin Defaults, Variables, Secrets, and Key-Value pairs configured in the current Namespace, so prompts like "Create a task that integrates with MongoDB" can reuse your existing `pluginDefaults`, stored credentials, or variables without extra hints.
-
-## Usage limits
-
-When no custom provider is configured, Kestra uses a built-in AI service with a daily generation limit per instance. The UI shows how many generations you have left, and the limit resets daily at midnight UTC.
-
-To remove the limit, configure your own LLM provider in the `kestra.ai.providers` block. See [Configuration](#configuration) below.
-
-## Configuration
-
-To add Copilot to your flow editor, add the following to your [Enterprise and Advanced configuration](../../configuration/06.enterprise-and-advanced/index.md). The `providers` array lets you register multiple LLMs and pick a default (`isDefault: true`):
-
-```yaml
-kestra:
- ai:
- enabled: true # set to false to disable AI Copilot entirely
- providers:
- - id: gemini
- display-name: Gemini - Private
- type: gemini
- configuration:
- model-name: gemini-3.1-flash-lite
- api-key: YOUR_GEMINI_API_KEY
- - id: gpt
- display-name: Open AI
- type: openai
- isDefault: true
- configuration:
- model-name: gpt-4
- api-key: YOUR_OPENAI_API_KEY
-```
-
-:::alert{type="info"}
-Legacy single-provider configs (`kestra.ai.type` + provider block) still work, but the `providers` array lets you register multiple providers and choose a default (`isDefault: true`).
-:::
-
-### Disabling AI Copilot
-
-To fully disable the AI Copilot — including the built-in fallback to the `api.kestra.io` service — set `kestra.ai.enabled` to `false`:
-
-```yaml
-kestra:
- ai:
- enabled: false
-```
-
-When disabled, the Copilot UI will not appear and all AI endpoints will be deactivated. The property defaults to `true`.
-
-### Multiple providers
-
-When multiple providers are configured, users can switch models from a dropdown in the Copilot UI instead of relying only on the default.
-
-Replace `api-key` with your provider credentials. Copilot appears in the top right corner of the flow editor. Optionally, you can add the following properties inside each provider `configuration` block (availability varies by provider):
-
-- `temperature`: Controls randomness in responses — lower values make outputs more focused and deterministic, while higher values increase creativity and variability.
-- `topP` (nucleus sampling): Ranges from 0.0–1.0; lower values (0.1–0.3) produce safer, more focused responses for technical tasks, while higher values (0.7–0.9) encourage more creative and varied outputs.
-- `topK`: Typically ranges from 1–200+ depending on the API; lower values restrict choices to a few predictable tokens, while higher values allow more options and greater variety in responses.
-- `maxOutputTokens`: Sets the maximum number of tokens the model can generate, capping the response length.
-- `logRequests`: Creates logs in Kestra for LLM requests.
-- `logResponses`: Creates logs in Kestra for LLM responses.
-- `baseURL`: Specifies the endpoint address where the LLM API is hosted.
-- `clientPem`: (Required for mTLS) PEM bundle with client cert + private key (e.g., `cat client.crt.pem client.key.pem > client-bundle.pem`). Used for mutual TLS.
-- `caPem`: CA PEM file to add a custom CA without `trustAll`. Usually not needed since hosts already trust the CA.
-- `customHeaders`: Specify custom HTTP headers for authentication and routing through internal AI gateways. Custom headers should be passed as a map inside the property.
-- `timeout`: Specifies the maximum duration to wait for an AI model API request to complete before timing out. ISO 8601 duration format (Java Duration): `PT30S` = 30 seconds. You can set it per provider to enforce strict SLAs.
-
-:::alert{type="info"}
-Enterprise Edition includes an [RBAC permission](../../07.enterprise/03.auth/rbac/index.md) that lets administrators allow or disallow Copilot usage per role at tenant or namespace scope.
-:::
-
-
-
-:::alert{type="info"}
-The open-source version supports only Google Gemini models. Enterprise Edition users can configure any LLM provider, including Amazon Bedrock, Anthropic, Azure OpenAI, DeepSeek, Google Gemini, Google Vertex AI, Mistral, and all open-source models supported by Ollama. Navigate down to the Enterprise configurations section for your provider. If you use a different provider, please [reach out to us](https://kestra.io/demo) and we'll add it.
-:::
-
-## Build flows with Copilot
-
-
-
-In the above demo, we want to create a flow that uses a [Python script](/plugins/plugin-script-python/io.kestra.plugin.scripts.python.script) to fetch New York City weather data. To get started, open the Copilot and write a prompt. For example:
-
-```txt
-Create a flow with a Python script that fetches weather data for New York City
-```
-
-Once prompted, the Copilot generates YAML directly in the flow editor that can be accepted or refused in the bottom right corner.
-
-
-
-If accepted, the flow is created and can be saved for execution, iterated on manually, or continually iterated upon by the Copilot. For example, you want a trigger added to the flow to run it on a schedule. Reopen the Copilot and prompt it with the desired trigger setup such as:
-
-```txt
-Add a trigger to run the flow every day at 9 AM
-```
-
-The Copilot again makes a suggestion to add to the flow, but only in the targeted section, in this case a `triggers` block. This is also the case if you want the Copilot only to consider a specific task, input, plugin default, and so on.
-
-
-
-You can continuously collaborate with Copilot until the flow is exactly as you imagined. If accepted, suggestions are always declaratively written and manageable as code. You can keep track of the revision history using the built-in Revisions tab or with the help of Git Sync.
-
-## Fix with AI
-
-With Copilot configured, there is also the added benefit of consulting Copilot to resolve execution errors from the Logs and Gantt views. For failed tasks, you can open the task and click the three dots to "**Fix with AI**". This option reopens the flow editor with the Copilot automatically prompted with the error context to help resolve any issues with the task.
-
-
-
-## Starter prompts
-
-To get started with Copilot, here are some example prompts to test, iterate on, and use as a starting point for collaboratively building flows with AI in Kestra:
-
-:::collapse{title="Example prompts to get started"}
-```markdown
-- Create a flow that runs a dbt build command on DuckDB
-- Create a flow cloning https://github.com/kestra-io/dbt-example Git repository from a main branch, then add a dbt CLI task using DuckDB backend that will run dbt build command for that cloned repository using my_dbt_project profile and dev target. The dbt project is located in the root directory so no dbt project needs to be configured.
-- Create a flow that sends a POST request to https://dummyjson.com/products/add
-- Send a POST request to https://dummyjson.com/products/add
-- Write a Python script that sends a POST request to https://dummyjson.com/products/add
-- Write a Node.js script that sends a POST request to https://dummyjson.com/products/add
-- Create a flow with a Python script that fetches weather data for New York City
-- Make a REST API call to https://kestra.io/api/mock and allow failure
-- Create a flow that logs "Hello from AI" to the console
-- Create a flow that returns Hello as output
-- Create a flow that outputs Hello as value
-- Run a flow every 10 minutes
-- Run a flow every day at 9 AM
-- Run a shell command echo 'Hello Docker' in a Docker container
-- Run a command python main.py in a Docker container
-- Run a script main.py stored as namespace file
-- Build a Docker image from an inline Dockerfile and push it to a GitHub Container Registry
-- Build a Docker image from an inline Dockerfile and push it to a DockerHub Container Registry
-- Create a flow that adds a string KV pair called MYKEY with value myvalue to namespace company
-- Fetch value for KV pair called MYKEY from namespace company
-- Create a flow that downloads a file mydata.csv from S3 bucket named mybucket
-- Create a flow that downloads all files from the folder kestra/plugins/ from S3 bucket mybucket in us-east-1
-- Send a Slack notification that approval is needed and Pause the flow for manual approval
-- Send a Slack alert whenever any execution from namespace company fails
-- Fetch value for string kv pair called mykey from Redis
-- Fetch value for mykey from Redis
-- Set value for mykey in Redis to myvalue
-- Sync all flows and scripts for selected namespaces from Git to Kestra
-- Create a flow that clones a Git repository and runs a Python script
-- Export a Postgres table called mytable to a CSV file
-- Query a Postgres table called mytable
-- Find documents in a MongoDB collection called mycollection
-- Load documents into a MongoDB mycollection using a file from input mydata
-- Trigger an Airbyte connection sync and retry it up to 3 times
-- Run an Airflow DAG called mydag
-- Orchestrate an Ansible playbook stored in Namespace Files
-- Run a DuckDB query that reads a CSV file
-- Fetch AWS ECR authorization token to push Docker images to Amazon ECR
-- Run a flow whenever 5 records are available in Kafka topic mytopic
-- Submit a run for a Databricks job
-```
-:::
-
-## Enterprise Edition Copilot configurations
-
-Enterprise Edition users can configure any LLM provider, including Amazon Bedrock, Anthropic, Azure OpenAI, DeepSeek, Google Gemini, Google Vertex AI, Mistral, OpenAI, OpenRouter, and all open-source models supported by Ollama. Add one or more of the snippets below as entries inside `kestra.ai.providers` (set `isDefault: true` on the default provider). Each configuration has slight differences, so adjust it for your provider.
-
-Only non-thinking modes are supported. If the used LLM is a pure thinking model (one that possesses thinking ability and cannot be disabled), the generated Flow will be incorrect and contain thinking elements.
-
-### Amazon Bedrock
-
-```yaml
-kestra:
- ai:
- providers:
- - id: bedrock
- display-name: Amazon Bedrock
- type: bedrock
- configuration:
- model-name: amazon.nova-lite-v1:0
- access-key-id: BEDROCK_ACCESS_KEY_ID
- secret-access-key: BEDROCK_SECRET_ACCESS_KEY
-```
-
-### Anthropic
-
-```yaml
-kestra:
- ai:
- providers:
- - id: anthropic
- display-name: Anthropic
- type: anthropic
- configuration:
- model-name: claude-opus-4-1-20250805
- api-key: CLAUDE_API_KEY
-```
-
-### Azure OpenAI
-
-```yaml
-kestra:
- ai:
- providers:
- - id: azure-openai
- display-name: Azure OpenAI
- type: azure-openai
- configuration:
- model-name: gpt-4o-2024-11-20
- api-key: AZURE_OPENAI_API_KEY
- tenant-id: AZURE_TENANT_ID
- client-id: AZURE_CLIENT_ID
- client-secret: AZURE_CLIENT_SECRET
- endpoint: "https://your-resource.openai.azure.com/"
-```
-
-### Deepseek
-
-```yaml
-kestra:
- ai:
- providers:
- - id: deepseek
- display-name: DeepSeek
- type: deepseek
- configuration:
- model-name: deepseek-chat
- api-key: DEEPSEEK_API_KEY
- base-url: "https://api.deepseek.com/v1"
-```
-
-### Google Gemini
-
-```yaml
-kestra:
- ai:
- providers:
- - id: gemini
- display-name: Google Gemini
- type: gemini
- configuration:
- model-name: gemini-3.1-flash-lite
- api-key: YOUR_GEMINI_API_KEY
-```
-
-### Google Vertex AI
-
-```yaml
-kestra:
- ai:
- providers:
- - id: vertex
- display-name: Google Vertex AI
- type: googlevertexai
- configuration:
- model-name: gemini-3.1-flash-lite
- project: GOOGLE_PROJECT_ID
- location: GOOGLE_CLOUD_REGION
- endpoint: VERTEX-AI-ENDPOINT
-```
-
-### Mistral
-
-```yaml
-kestra:
- ai:
- providers:
- - id: mistral
- display-name: Mistral
- type: mistralai
- configuration:
- model-name: mistral:7b
- api-key: MISTRALAI_API_KEY
- base-url: "https://api.mistral.ai/v1"
-```
-
-### Ollama
-
-```yaml
-kestra:
- ai:
- providers:
- - id: ollama
- display-name: Ollama
- type: ollama
- configuration:
- model-name: llama3
- base-url: http://localhost:11434
-```
-
-:::alert{type="info"}
-If Ollama is running locally on your host machine while Kestra is running inside a container, connection errors may occur when using `localhost`. In this case, use the Docker internal network URL instead — for example, set the base URL to `http://host.docker.internal:11434`.
-:::
-
-:::alert{type="info"}
-Some Ollama model names can be confusing. For example, at the time of writing, the model `qwen3:30b-a3b` is pointing to SHA `ad815644918f`, which is the `qwen3:30b-a3b-thinking-2507-q4_K_M` model behind the scenes. This is a thinking model that doesn't support disabling it.
-Please double-check that the chosen model has a non-thinking version or that a toggle is available.
-:::
-
-### OpenAI
-
-```yaml
-kestra:
- ai:
- providers:
- - id: openai
- display-name: OpenAI
- type: openai
- configuration:
- model-name: gpt-5-nano
- api-key: OPENAI_API_KEY
- base-url: https://api.openai.com/v1
-```
-
-### OpenRouter
-
-```yaml
-kestra:
- ai:
- providers:
- - id: openrouter
- display-name: OpenRouter
- type: openrouter
- configuration:
- api-key: OPENROUTER_API_KEY
- base-url: "https://openrouter.ai/api/v1"
- model-name: "anthropic/claude-sonnet-4"
-```
diff --git a/src/contents/docs/ai-tools/index.mdx b/src/contents/docs/ai-tools/index.mdx
index 61ede9925e8..a8e730c26a7 100644
--- a/src/contents/docs/ai-tools/index.mdx
+++ b/src/contents/docs/ai-tools/index.mdx
@@ -1,49 +1,92 @@
---
-title: "AI Tools in Kestra: Copilot, Agents, MCP Server & More"
-h1: Accelerate Orchestration with Kestra AI Tools
+title: "AI in Kestra: Copilot, Coding Agents, MCP & AI Agents"
+h1: Choose How You Use AI with Kestra
sidebarTitle: AI Tools
icon: /src/contents/docs/icons/ai.svg
-description: Learn how Kestra's AI Copilot, AI Agents, Agent Skills, and MCP server can accelerate your workflow creation and enable autonomous orchestration.
+description: Compare every way to use AI with Kestra — AI Copilot in the UI, your own coding agent via MCP and Agent Skills, and AI Agents running inside your flows.
---
import ChildCard from "~/components/docs/ChildCard.astro"
-Create, refine, and orchestrate workflows using natural language or autonomous decision-making.
+Compare every way to use AI with Kestra and pick the one that fits how you work.
-## Learn how Kestra AI tools accelerate orchestration
+AI shows up in Kestra in two unrelated ways, and it helps to keep them apart:
-Kestra provides built-in AI features and external integrations that work at every layer of your workflow development:
+- **AI that helps you build and operate Kestra** — writing flow YAML, fixing a failed execution, deploying to a namespace. The AI is a way to drive Kestra faster; it never runs as part of your workflow.
+- **AI that runs inside your flows** — a task calls a model, an agent decides what to do next, a flow answers questions over your documents. Here the AI is part of what you orchestrate.
-- **[Kestra MCP resources](./kestra-mcp-resources/index.md)**: connect any MCP-compatible AI tool (Claude Code, Cursor, etc.) to live Kestra plugin docs, blueprints, and documentation search — the fastest way to get accurate Kestra context in your AI coding agent.
-- **[AI Copilot](./ai-copilot/index.md)**: generate and refine flows from natural language inside the Kestra UI.
-- **[AI Agents](./ai-agents/index.md)**: autonomous orchestration where tasks are chosen dynamically at runtime rather than following a fixed sequence.
-- **[Agent Skills](./agent-skills/index.md)**: structured knowledge files that give AI coding agents the expertise to generate valid Kestra flows and operate environments via `kestractl`.
+None of this is required. Kestra is an orchestrator, and every flow you can generate with AI you can also write by hand in YAML or build in the No-Code editor.
-## Kestra MCP resources
+## Build and operate Kestra with AI
-The [Kestra MCP resources](./kestra-mcp-resources/index.md) endpoint connects any MCP-compatible AI tool — Claude Code, Cursor, and others — to live Kestra plugin documentation, blueprints, and product docs. Instead of relying on training data, your AI agent queries it at runtime for current, accurate information about task properties, configuration options, and usage patterns.
+Start from where you already work:
-## AI Copilot
+| Where you work | What to use | What it gives you |
+| --- | --- | --- |
+| The Kestra UI | [AI Copilot](./01.ai-copilot/index.md) | Generate and refine flows from a prompt in the editor, and "Fix with AI" on a failed execution |
+| Your terminal or editor (Claude Code, Cursor, Codex, Windsurf) | [MCP server](./02.kestra-mcp-resources/index.md) + [Agent Skills](./04.agent-skills/index.md) | Your existing coding agent writes, validates, and deploys flows alongside the rest of your codebase |
+| Scripts and CI | [`kestractl`](../kestra-cli/kestractl/index.md) | Deterministic flow, execution, and namespace operations with no AI in the loop |
-AI Copilot allows users to generate and refine flow definitions from natural language prompts. Instead of manually writing YAML, you can describe the desired behavior (for example, _”Make a REST API call to https://kestra.io/api/mock and allow failure”_) and Copilot will generate the corresponding flow code. The generated YAML can then be reviewed, accepted, or modified. Copilot can also update existing flows incrementally, such as adding tasks or adjusting triggers, without affecting unrelated parts of the flow.
+### AI Copilot in the Kestra UI
-## AI Agents
+[AI Copilot](./01.ai-copilot/index.md) is built into the Kestra editor. Describe what you want — for example, _"make a REST API call to https://kestra.io/api/mock and allow failure"_ — and Copilot generates the flow YAML for you to review, accept, or edit. It also modifies existing flows incrementally, touching only the part you asked about, and powers **Fix with AI** on failed tasks.
-AI Agents provide autonomous orchestration capabilities. An AI Agent task uses a large language model (LLM), optional memory, and configured tools such as web search, task execution, or flow calling. The agent can dynamically decide which actions to take, loop until conditions are satisfied, and adapt based on new information. Unlike static flows that follow a fixed sequence, agents operate adaptively while remaining observable and fully defined as code.
+Copilot is the fastest path when you are already in the UI and want the change applied to the flow in front of you.
-## Agent Skills
+### Your own coding agent
-Agent Skills are structured knowledge files that teach external AI coding agents — such as Claude Code, Cursor, and Windsurf — how to generate Kestra flows and operate Kestra environments using `kestractl`. Unlike AI Copilot (which works inside the Kestra UI) or AI Agents (which run inside flows), Agent Skills bring Kestra expertise directly to your editor or terminal.
+If your flows live in Git next to your application code, you may prefer to stay in Claude Code, Cursor, Codex, or Windsurf. Three separate pieces make that work, and each covers a different gap:
-## Summary
+- **[MCP server](./02.kestra-mcp-resources/index.md) — context.** A hosted [Model Context Protocol](https://modelcontextprotocol.io/) endpoint that gives your agent live access to task schemas for all 1,400+ plugins, the blueprints library, and documentation search. Without it, an agent invents plugin properties from stale training data.
+- **[Agent Skills](./04.agent-skills/index.md) — know-how.** `SKILL.md` files that teach your agent how to author valid flows (`kestra-flow`), harden them for production (`kestra-flow-hardening`), operate an environment (`kestra-ops`), and port Airflow DAGs (`migrate-airflow-kestra`). Skills carry the guardrails: no invented task types, no hardcoded secrets, validate before deploy.
+- **[`kestractl`](../kestra-cli/kestractl/index.md) — hands.** The Kestra CLI the agent actually calls to validate, deploy, and run flows against a real instance. The `kestra-ops` skill is a wrapper around these commands, so an agent without `kestractl` can write flows but cannot deploy them.
-Together, these approaches offer complementary ways to work with AI:
+#### Set it up
-- **Kestra MCP resources**: gives external AI coding agents live access to Kestra plugin docs, blueprints, and documentation — no training data required.
-- **AI Copilot**: speeds up flow creation and modification by translating natural language instructions into YAML.
-- **AI Agents**: enable adaptive orchestration patterns where task sequences are not predetermined but are chosen dynamically at runtime.
-- **Agent Skills**: give external AI coding agents structured knowledge to generate valid Kestra flows and operate environments from your development tools.
+Connect the MCP server. For Claude Code:
-AI Copilot and AI Agents are built into Kestra, while Kestra MCP resources and Agent Skills extend Kestra expertise to the external tools you already use.
+```bash
+claude mcp add kestra --transport http https://api.kestra.io/v1/mcp
+```
-
\ No newline at end of file
+Any MCP-compatible tool can connect to `https://api.kestra.io/v1/mcp` over HTTP — see [MCP server](./02.kestra-mcp-resources/index.md) for Cursor and other configurations.
+
+Install the skills. [skills.sh](https://skills.sh) detects your agent and writes the files to the right location:
+
+```bash
+npx skills add kestra-io/agent-skills
+```
+
+Install `kestractl` and point it at your instance:
+
+```bash
+curl -fsSL https://raw.githubusercontent.com/kestra-io/kestractl/main/install-scripts/install.sh | bash
+kestractl config add default http://localhost:8080 main --username YOUR_USERNAME --password YOUR_PASSWORD --default
+```
+
+See [`kestractl`](../kestra-cli/kestractl/index.md) for Enterprise token auth and the full command reference.
+
+#### What it looks like end to end
+
+With all three in place, a single prompt covers authoring and deployment:
+
+```plaintext
+Use kestra-flow to write a flow in namespace company.data that fetches
+https://api.example.com/metrics every 30 minutes and stores the response in
+the KV store under the key last_metrics. Validate it with kestractl, then
+deploy it to the dev instance.
+```
+
+The agent queries the MCP server for the correct plugin schema, follows the skill's guardrails (no hardcoded credentials, valid YAML structure), runs `kestractl flow validate`, and deploys with `kestractl flow create`.
+
+### Expose flows as AI tools
+
+[MCP Server](./03.mcp-server/index.md) lets you go the other direction: instead of an agent driving Kestra, Kestra flows become callable tools for any MCP-compatible AI agent. Add an `McpToolTrigger` to any flow and it is automatically registered as a named tool. AI agents such as Claude Desktop, Claude Code, and Cursor can then discover and invoke your flows directly, with inputs and outputs mapped to a JSON schema.
+
+## Run AI inside your flows
+
+- **[AI Agents](./05.ai-agents/index.md)**: autonomous orchestration where an LLM decides which actions to take at runtime rather than following a fixed sequence.
+- **[AI Workflows](./06.ai-workflows/index.md)**: pre-built flow templates for common AI patterns such as summarization, classification, and structured extraction.
+- **[RAG Workflows](./07.ai-rag-workflows/index.md)**: retrieval-augmented generation pipelines that connect your documents to a language model.
+
+
diff --git a/src/contents/docs/api-reference/kestra-sdk/go-sdk/index.md b/src/contents/docs/api-reference/kestra-sdk/go-sdk/index.md
index c2f37d768fa..e8fbcba2d13 100644
--- a/src/contents/docs/api-reference/kestra-sdk/go-sdk/index.md
+++ b/src/contents/docs/api-reference/kestra-sdk/go-sdk/index.md
@@ -256,24 +256,85 @@ func deleteKVKey(ctx context.Context, apiClient *openapiclient.APIClient) {
## Read execution logs
-Fetch logs for a completed execution.
+Fetch all log entries for a completed execution. Log operations use `KestraClient` from the same package — not the generated `APIClient`.
```go
-func listLogs(ctx context.Context, apiClient *openapiclient.APIClient) {
- tenant := "main"
- result, _, err := apiClient.LogsAPI.
- ListLogsFromExecution(ctx, "your-execution-id", tenant).
- Execute()
+import (
+ "context"
+ "fmt"
+ kestra "github.com/kestra-io/client-sdk/go-sdk/kestra_api_client"
+)
+
+func listLogs() {
+ ctx := context.Background()
+ client := kestra.NewClient("http://localhost:8080",
+ kestra.WithBasicAuth("root@root.com", "Root!1234"))
+
+ logs, err := client.Logs().ListLogsFromExecution(ctx, "your-execution-id", "main", nil, nil, nil, nil)
if err != nil {
fmt.Printf("Error fetching logs: %v\n", err)
return
}
- for _, log := range result.GetResults() {
+ for _, log := range logs {
fmt.Printf("[%s] %s\n", log.GetLevel(), log.GetMessage())
}
}
```
+Pass a minimum log level to filter results — for example `kestra.PtrString("INFO")` as the fourth argument. Remaining arguments (`taskRunId`, `taskId`, `attempt`) narrow by task; pass `nil` to skip each filter.
+
+---
+
+## Stream execution logs (SSE)
+
+Stream logs from a running execution in real time. `FollowLogsFromExecution` opens an SSE connection and returns a `<-chan *LogEntry`. Entries arrive as the execution produces them. The channel closes when the execution ends or the context is cancelled.
+
+```go
+func followLogs() {
+ ctx, cancel := context.WithCancel(context.Background())
+ defer cancel()
+
+ client := kestra.NewClient("http://localhost:8080",
+ kestra.WithBasicAuth("root@root.com", "Root!1234"))
+
+ ch, err := client.Logs().FollowLogsFromExecution(ctx, "your-execution-id", "main", nil)
+ if err != nil {
+ fmt.Printf("Error starting log stream: %v\n", err)
+ return
+ }
+
+ for entry := range ch {
+ if entry.GetExecutionId() == "" {
+ continue // skip the synthetic "start" entry the server sends on stream open
+ }
+ fmt.Printf("[%s] %s\n", entry.GetLevel(), entry.GetMessage())
+ }
+}
+```
+
+To stop streaming early, cancel the context. The server-side SSE connection closes and the channel drains within milliseconds:
+
+```go
+ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+defer cancel()
+
+ch, err := client.Logs().FollowLogsFromExecution(ctx, "your-execution-id", "main", nil)
+if err != nil {
+ fmt.Printf("Error starting log stream: %v\n", err)
+ return
+}
+for entry := range ch {
+ if entry.GetExecutionId() == "" {
+ continue
+ }
+ fmt.Printf("[%s] %s\n", entry.GetLevel(), entry.GetMessage())
+}
+```
+
+:::alert{type="info"}
+The fourth argument is an optional minimum log level filter (`*string`). Pass `kestra.PtrString("INFO")` to receive only INFO and above, or `nil` to receive all levels.
+:::
+
---
## Manage triggers
diff --git a/src/contents/docs/api-reference/kestra-sdk/java-sdk/index.md b/src/contents/docs/api-reference/kestra-sdk/java-sdk/index.md
index 0a25f7b26f2..dc454cf92e3 100644
--- a/src/contents/docs/api-reference/kestra-sdk/java-sdk/index.md
+++ b/src/contents/docs/api-reference/kestra-sdk/java-sdk/index.md
@@ -63,6 +63,26 @@ Construct the client once (singleton or DI) and reuse it. Use either `.basicAuth
---
+## Configure timeouts
+
+By default, both connect and read timeouts are infinite. Configure them on the builder before calling long-running endpoints such as `runTestSuite`:
+
+```java
+import io.kestra.sdk.KestraClient;
+import java.time.Duration;
+
+KestraClient client = KestraClient.builder()
+ .url("https://kestra.example.com")
+ .tokenAuth(System.getenv("KESTRA_TOKEN"))
+ .connectTimeout(Duration.ofSeconds(10)) // time to establish the connection
+ .readTimeout(Duration.ofMinutes(30)) // time to wait for a response
+ .build();
+```
+
+Pass `Duration.ZERO` or omit the call to keep the default (infinite). Both methods accept any `java.time.Duration`.
+
+---
+
## Create a flow
Send the flow definition as a YAML string. This matches what you would define in the UI.
@@ -324,27 +344,26 @@ public class LogsExamples {
### Stream logs live
-```java
-import io.kestra.sdk.model.Level;
+`followLogsFromExecution` returns a reactive `Flux`. Each `FollowLogEvent` carries the same fields as `LogEntry` (plus `tenantId`). The server sends an initial keepalive frame with all fields `null` — filter it out before processing.
+```java
public class LogsExamples {
public static void followLogs() {
String executionId = "your-execution-id";
String tenant = "main";
- var event = KestraClients.INSTANCE.logs()
- .followLogsFromExecution(executionId, tenant, null);
-
- if (event != null && event.getData() != null) {
- System.out.printf("[%s] %s%n",
- event.getData().getLevel(), event.getData().getMessage());
- }
+ KestraClients.INSTANCE.logs()
+ .followLogsFromExecution(executionId, tenant, null) // null = no filters
+ .filter(event -> event.getExecutionId() != null) // skip keepalive frames
+ .doOnNext(event -> System.out.printf("[%s] %s%n",
+ event.getLevel(), event.getMessage()))
+ .blockLast(); // blocks until the stream ends
}
}
```
:::alert{type="info"}
-Use `listLogsFromExecution` after an execution finishes. Use `followLogsFromExecution` to fetch the latest log event from a running execution.
+Use `listLogsFromExecution` after an execution finishes. Use `followLogsFromExecution` to stream logs in real time from a running execution.
:::
---
diff --git a/src/contents/docs/api-reference/kestra-sdk/javascript-sdk/index.md b/src/contents/docs/api-reference/kestra-sdk/javascript-sdk/index.md
index 8f1877e3207..1bcb717b17b 100644
--- a/src/contents/docs/api-reference/kestra-sdk/javascript-sdk/index.md
+++ b/src/contents/docs/api-reference/kestra-sdk/javascript-sdk/index.md
@@ -167,6 +167,47 @@ The server emits an initial keepalive event with no `state` — skip it before p
---
+## Read execution logs
+
+### List logs
+
+Fetch all log entries for a completed execution:
+
+```javascript
+import * as Logs from "@kestra-io/kestra-sdk/logs";
+
+async function listLogs() {
+ const logs = await Logs.listLogsFromExecution({
+ executionId: "your-execution-id",
+ });
+ logs?.forEach(entry => console.log(`[${entry.level}] ${entry.message}`));
+}
+```
+
+To filter by minimum log level, pass a `filters` array:
+
+```javascript
+const logs = await Logs.listLogsFromExecution({
+ executionId: "your-execution-id",
+ filters: [{ field: "LEVEL", operation: "GREATER_THAN_OR_EQUAL_TO", value: "INFO" }],
+});
+```
+
+### Search logs
+
+Search across all executions with pagination:
+
+```javascript
+import * as Logs from "@kestra-io/kestra-sdk/logs";
+
+async function searchLogs() {
+ const result = await Logs.searchLogs({ page: 1, size: 50 });
+ result?.results?.forEach(entry => console.log(`[${entry.level}] ${entry.message}`));
+}
+```
+
+---
+
## KV Store
The KV Store lets you read and write key-value pairs scoped to a namespace.
diff --git a/src/contents/docs/api-reference/kestra-sdk/python-sdk/index.md b/src/contents/docs/api-reference/kestra-sdk/python-sdk/index.md
index cc11c81bddb..292cf497141 100644
--- a/src/contents/docs/api-reference/kestra-sdk/python-sdk/index.md
+++ b/src/contents/docs/api-reference/kestra-sdk/python-sdk/index.md
@@ -50,6 +50,36 @@ Use environment variables rather than hardcoding credentials. You can also use t
---
+## Configure timeouts
+
+By default, requests wait indefinitely. Pass a `timeout` argument to `KestraClient` to limit how long requests wait before raising `requests.Timeout`.
+
+Using a `Configuration` object:
+
+```python
+from kestrapy import Configuration, KestraClient
+
+configuration = Configuration(host="http://localhost:8080", username="root@root.com", password="Root!1234")
+
+kestra_client = KestraClient(configuration, timeout=30.0) # float: connect + read combined
+kestra_client = KestraClient(configuration, timeout=(10.0, 300.0)) # tuple: (connect, read)
+kestra_client = KestraClient(configuration, timeout=None) # None: no timeout (default)
+```
+
+Using keyword arguments directly:
+
+```python
+from kestrapy import KestraClient
+
+kestra_client = KestraClient(host="http://localhost:8080", token="your-api-token", timeout=30.0)
+kestra_client = KestraClient(host="http://localhost:8080", token="your-api-token", timeout=(10.0, 300.0))
+kestra_client = KestraClient(host="http://localhost:8080", token="your-api-token", timeout=None)
+```
+
+The `timeout` value is forwarded directly to [`requests`](https://docs.python-requests.org/en/latest/user/advanced/#timeouts), so any form that `requests` accepts is valid.
+
+---
+
## Create a flow
Pass the flow definition as a YAML string to [`create_flow`](https://github.com/kestra-io/client-sdk/blob/main/python-sdk/docs/FlowsApi.md#create_flow).
@@ -192,6 +222,46 @@ The first SSE payload is an empty keepalive — skip it before processing subseq
---
+## Read execution logs
+
+### List logs
+
+Fetch all log entries for a completed execution:
+
+```python
+def list_logs():
+ tenant = "main"
+ logs = kestra_client.logs.list_logs_from_execution(
+ execution_id="your-execution-id",
+ tenant=tenant,
+ min_level="INFO", # optional; filters to INFO and above
+ )
+ for entry in logs:
+ print(f"[{entry.level}] {entry.message}")
+```
+
+### Stream logs live
+
+`follow_logs_from_execution` yields `LogEntry` items as the execution produces them. The server sends an initial keepalive frame with all fields `None` — skip entries where `execution_id` is `None`.
+
+```python
+def follow_logs():
+ tenant = "main"
+ for entry in kestra_client.logs.follow_logs_from_execution(
+ execution_id="your-execution-id",
+ tenant=tenant,
+ ):
+ if entry.execution_id is None:
+ continue # keepalive frame
+ print(f"[{entry.level}] {entry.message}")
+```
+
+:::alert{type="info"}
+The `min_level` parameter on `follow_logs_from_execution` is not applied by the Kestra 2.0 server — pass no filter and handle level filtering in the consumer loop if needed.
+:::
+
+---
+
## KV Store
The KV Store lets you read and write key-value pairs scoped to a namespace.
diff --git a/src/contents/docs/configuration/01.configuration-basics/index.md b/src/contents/docs/configuration/01.configuration-basics/index.md
index 989ff2e9743..c06378d9815 100644
--- a/src/contents/docs/configuration/01.configuration-basics/index.md
+++ b/src/contents/docs/configuration/01.configuration-basics/index.md
@@ -139,24 +139,31 @@ The same applies to `kestra.storage.gcs.project-id`, `kestra.storage.gcs.service
## SDK default authentication
-SDK-based plugins can use default authentication if configured. Kestra resolves credentials in this order:
+SDK-based plugins resolve authentication in this order:
-1. namespace-level default service account
-2. tenant-level default service account
-3. global SDK defaults
+1. Namespace-level default service account
+2. Tenant-level default service account
+3. Global SDK defaults (`kestra.tasks.sdk.authentication`)
-Example:
+**In OSS,** when `kestra.server.basic-auth` is configured, Kestra automatically derives the global SDK credentials from it — no additional configuration is needed. SDK-based tasks using `DEFAULT` or `AUTO` authentication work without further setup.
+
+If you need to use different credentials from those in `kestra.server.basic-auth`, or to authenticate with an API token, override the global default explicitly:
```yaml
-tasks:
- sdk:
- authentication:
- username: ${kestra.server.basic-auth.username}
- password: ${kestra.server.basic-auth.password}
- # token: ${KESTRA_API_TOKEN}
+kestra:
+ tasks:
+ sdk:
+ authentication:
+ username: my-user # overrides basic-auth username
+ password: my-password # overrides basic-auth password
+ # api-token: ${KESTRA_API_TOKEN} # use an API token instead
```
-If no namespace, tenant, or global default is configured, SDK-based tasks that use `DEFAULT` or `AUTO` authentication fail because no API credentials are available.
+:::alert{type="warning"}
+If only one of `username` or `password` is set — either explicitly or resolved from `basic-auth` — Kestra throws an error when a task first attempts to use the SDK. Both must be provided together or neither.
+:::
+
+If no credential is available at any level, SDK-based tasks using `DEFAULT` or `AUTO` authentication fail. This applies to OSS instances without basic auth configured, and to EE/Cloud instances without a namespace or tenant-level service account.
## What belongs on the other configuration pages
diff --git a/src/contents/docs/configuration/02.runtime-and-storage/index.md b/src/contents/docs/configuration/02.runtime-and-storage/index.md
index ae965ae4364..17da59f5ccb 100644
--- a/src/contents/docs/configuration/02.runtime-and-storage/index.md
+++ b/src/contents/docs/configuration/02.runtime-and-storage/index.md
@@ -16,6 +16,10 @@ Every Kestra deployment must define:
- queue type
- internal storage type
+Optionally, in Kestra 2.0 and later, you can also configure a separate log store:
+
+- log data store (defaults to the repository backend if not set)
+
The common production path is PostgreSQL for queue and repository, plus an object store or durable internal storage backend.
Queues and repositories must stay compatible:
@@ -24,9 +28,22 @@ Queues and repositories must stay compatible:
- JDBC queue with H2, MySQL, or PostgreSQL repository
- Kafka queue with Elasticsearch repository in Enterprise Edition
+## Allocated CPU cores
+
+Kestra sizes several internal thread pools based on the number of CPU cores available to the process. By default, it uses the number of CPU cores reported by the runtime environment.
+
+If you want Kestra to size those pools using a different value, set `kestra.allocated-cpu-cores`:
+
+```yaml
+kestra:
+ allocated-cpu-cores: 2
+```
+
+This is useful when you want to limit how aggressively Kestra allocates worker, scheduler, and queue-related threads without changing container limits or host-level CPU settings.
+
## Database and datasources
-Start here if you are choosing the persistence layer for a new Kestra instance or moving from a local setup to a durable environment. In most teams, this is the first configuration page they revisit after initial installation.
+Start here if you are choosing the queue and repository backend for a new Kestra instance or moving from a local setup to a durable environment. In most teams, this is the first configuration page they revisit after initial installation.
Use `kestra.queue.type` and `kestra.repository.type` to select your backend:
@@ -111,6 +128,22 @@ Use H2 for local development. For production, prefer PostgreSQL, or MySQL if Pos
For PostgreSQL performance issues, consider `random_page_cost=1.1` and `kestra.queue.postgres.disable-seq-scan=true` if queue polling is choosing poor query plans.
:::
+## Log data store
+
+By default, execution logs are stored in the same database as flows and executions. In Kestra 2.0+, you can route logs to a separate store by setting `kestra.logs.type`. If this key is not set, logs continue to use the repository backend — no migration needed.
+
+```yaml
+kestra:
+ logs:
+ type: postgres # or h2, mysql, elasticsearch
+ postgres:
+ url: jdbc:postgresql://logs-db:5432/kestra_logs
+ username: kestra
+ password: k3str4
+```
+
+The External Log Data Store is an Enterprise Edition feature. Both JDBC (H2, Postgres, MySQL) and Elasticsearch backends require EE. For full configuration examples and the capability reference, see the [External Log Data Store](../../10.administrator-guide/log-data-store/index.md) guide.
+
## Connection pooling and JDBC queue tuning
Most users can keep the defaults here until they see either connection pressure or queue latency. This section matters most for larger deployments, split-component topologies, or databases that are already under load.
@@ -206,7 +239,23 @@ If you are not troubleshooting queue throughput or database pressure, you can us
`kestra.storage.type` controls where Kestra stores internal files such as task outputs, namespace files, and execution artifacts. Choose the backend based on durability and whether all Kestra components can reach the same storage.
-**Supported backends:** [Local](#local) · [AWS S3](#aws-s3) · [Google Cloud Storage](#google-cloud-storage) · [Azure Blob Storage](#azure-blob-storage) · [MinIO / S3-compatible](#minio--s3-compatible) · [SeaweedFS](#seaweedfs) · [Cloudflare R2](#cloudflare-r2) · [Huawei OBS](#huawei-obs)
+Common options include:
+
+- `local` for local testing
+- `s3`
+- `s3files`
+- `gcs`
+- `azure`
+- `minio`
+- other object-storage-compatible backends
+
+The default local storage is fine for local testing but not for every production topology. The important distinction is whether every Kestra component can see the same files.
+
+### Local storage deployment guidance
+
+Local storage works well for standalone deployments with a persistent volume. In distributed deployments, it only works safely when all components share the same filesystem through a `ReadWriteMany` volume or an equivalent shared storage layer.
+
+If that shared filesystem does not exist, move to object storage instead of trying to share host paths between services.
### Storage isolation
@@ -405,6 +454,28 @@ Assign the `roles/storage.objectAdmin` predefined role on the bucket (not the pr
---
+### S3 Files
+
+Use `s3files` when Kestra runs on a host where an [S3 Files](https://aws.amazon.com/blogs/aws/launching-s3-files-making-s3-buckets-accessible-as-file-systems/) NFS filesystem is already mounted locally. This backend reads and writes directly through the local filesystem — no S3 SDK or AWS credentials are required.
+
+Mount the NFS filesystem on every host that runs a Kestra component before configuring this backend. All components must share the same mount path.
+
+```yaml
+kestra:
+ storage:
+ type: s3files
+ s3files:
+ mount-path: "/mnt/s3files"
+```
+
+`mount-path` must point to a directory that exists and is readable and writable by the Kestra process. Kestra will not create the directory on startup.
+
+Object metadata is stored in `.meta` sidecar files alongside each object on the filesystem. Custom S3 object metadata is not exposed through this backend.
+
+If you prefer to keep the S3 SDK path (for example, because not every host has the NFS mount), use the standard `s3` backend with `s3-files-compatible: true` instead.
+
+---
+
### Azure Blob Storage
:::alert{type="info"}
@@ -497,9 +568,15 @@ kestra:
minio:
endpoint: my.minio.domain.com
port: 9000
- bucket: kestra-storage
- access-key: YOUR_ACCESS_KEY
- secret-key: YOUR_SECRET_KEY
+ secure: false
+ access-key: ${AWS_ACCESS_KEY_ID}
+ secret-key: ${AWS_SECRET_ACCESS_KEY}
+ region: "default"
+ bucket: my-bucket
+ part-size: 5MB
+ # httpConnectTimeout: PT10S # optional; omit to use OkHttp default (10 s)
+ # httpReadTimeout: PT10S # optional; omit to use OkHttp default (10 s)
+ # httpWriteTimeout: PT10S # optional; omit to use OkHttp default (10 s)
```
#### Configuration reference
@@ -552,6 +629,8 @@ For Ceph RGW, SeaweedFS, or other S3-compatible backends, apply the equivalent b
---
+If large bucket operations such as `deleteByPrefix()` produce `SocketException: Socket closed` errors on heavily loaded or large buckets, increase `httpReadTimeout` or set it to `PT0S` to disable the timeout entirely.
+
### SeaweedFS
```yaml
@@ -725,6 +804,8 @@ kestra:
Use `html-head` sparingly for environment banners, extra CSS, or internal scripts that must load with the app shell.
+### Local file access
+
To allow universal file access from host-mounted paths, both mount the directory and add it to the allowlist:
```yaml
@@ -737,6 +818,22 @@ kestra:
Without the allowlist, file-access URIs pointing at local host paths will be rejected even if the path is mounted into the container.
+The `io.kestra.plugin.fs.local.Upload` and `io.kestra.plugin.fs.local.Uploads` tasks enforce their own `allowed-paths` check, independent of `kestra.local-files.allowed-paths`. Configure permitted directories under `plugins.configurations`:
+
+```yaml
+kestra:
+ plugins:
+ configurations:
+ - type: io.kestra.plugin.fs.local.Uploads
+ values:
+ allowed-paths:
+ - /data/uploads
+ - type: io.kestra.plugin.fs.local.Upload
+ values:
+ allowed-paths:
+ - /data/uploads
+```
+
## When to use this page
- Need logs, telemetry, metrics, endpoints, CORS, or SSL: [Observability and Networking](../03.observability-and-networking/index.md)
diff --git a/src/contents/docs/configuration/03.observability-and-networking/index.md b/src/contents/docs/configuration/03.observability-and-networking/index.md
index 5686d68fa5f..1ba7a409b6d 100644
--- a/src/contents/docs/configuration/03.observability-and-networking/index.md
+++ b/src/contents/docs/configuration/03.observability-and-networking/index.md
@@ -160,6 +160,20 @@ For example, with `country` and `environment` configured, an execution that has
kestra_executions_total{flow_id="my-flow",namespace_id="default",state="SUCCESS",label_country="Germany",label_environment="__none__"} 1
```
+To collect metrics from other service instances and re-expose them on the webserver's monitoring endpoint, use `sharedServiceInstanceMetrics`. Each key is a service type (`EXECUTOR`, `INDEXER`, `SCHEDULER`, `WEBSERVER`, `WORKER`) and each value is a list of fully-qualified metric names:
+
+```yaml
+kestra:
+ metrics:
+ sharedServiceInstanceMetrics:
+ WORKER:
+ - kestra.worker.job.pending
+ - kestra.worker.job.thread
+ - kestra.worker.job.running
+```
+
+See [Service Instance Metrics](../../10.administrator-guide/service-instance-metrics/index.md) for details.
+
For traces, metrics, and logs exported through OpenTelemetry, use the dedicated [OpenTelemetry guide](../../10.administrator-guide/open-telemetry/index.md).
## Network and HTTP settings
@@ -258,9 +272,11 @@ These settings are lighter-weight than the Micronaut server settings above. Use
The webserver-related configuration also includes:
+- disabling the UI to run the webserver API-only
- Google Analytics ID
- additional HTML tags
- mail server settings
+- security response headers
Examples:
@@ -279,6 +295,91 @@ kestra:
Mail server settings are useful when you need platform emails for invitations and notifications.
+### Security response headers
+
+Kestra adds browser security response headers to every HTTP response, including error responses (401, 403) and static file responses.
+
+Three headers are enabled by default:
+
+| Header | Default value |
+|---|---|
+| `X-Frame-Options` | `SAMEORIGIN` |
+| `X-Content-Type-Options` | `nosniff` |
+| `Referrer-Policy` | `strict-origin-when-cross-origin` |
+
+Two additional headers are disabled by default:
+
+| Header | When emitted |
+|---|---|
+| `Content-Security-Policy` | When `content-security-policy` is set to a non-blank value |
+| `Strict-Transport-Security` | When `strict-transport-security` is set and the request arrives at Kestra over HTTPS directly |
+
+Each header is added only if not already present in the response.
+
+Configuration reference:
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `kestra.webserver.security-headers.enabled` | boolean | `true` | Master toggle. When `false`, the filter is not created and no headers are added. |
+| `kestra.webserver.security-headers.frame-options` | string | `SAMEORIGIN` | Value for `X-Frame-Options`. Set to `""` to disable this header only. |
+| `kestra.webserver.security-headers.content-type-options` | string | `nosniff` | Value for `X-Content-Type-Options`. Set to `""` to disable this header only. |
+| `kestra.webserver.security-headers.referrer-policy` | string | `strict-origin-when-cross-origin` | Value for `Referrer-Policy`. Set to `""` to disable this header only. |
+| `kestra.webserver.security-headers.content-security-policy` | string | (absent) | Value for `Content-Security-Policy`. Not set unless configured. |
+| `kestra.webserver.security-headers.content-security-policy-report-only` | boolean | `false` | When `true`, the CSP value is sent under `Content-Security-Policy-Report-Only` instead of `Content-Security-Policy`. Use this to test a policy before enforcing it. |
+| `kestra.webserver.security-headers.strict-transport-security` | string | (absent) | Value for `Strict-Transport-Security`. Not set unless configured, and only emitted when Kestra is serving the request over HTTPS directly (see note below). |
+
+Enable CSP in report-only mode — report-only sends the policy without blocking any resources, so you can test that your CSP does not break any Kestra UI functionality before switching to enforcement:
+
+```yaml
+kestra:
+ webserver:
+ security-headers:
+ content-security-policy: "default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com; font-src 'self' https://fonts.gstatic.com; img-src 'self' data:; connect-src 'self' wss:; frame-ancestors 'none'"
+ content-security-policy-report-only: true
+```
+
+Remove `content-security-policy-report-only` or set it to `false` to switch from reporting to enforcement.
+
+Disable an individual header by setting its value to an empty string:
+
+```yaml
+kestra:
+ webserver:
+ security-headers:
+ frame-options: ""
+```
+
+Disable all security headers:
+
+```yaml
+kestra:
+ webserver:
+ security-headers:
+ enabled: false
+```
+
+:::alert{type="warning"}
+**HSTS and reverse proxies**: `Strict-Transport-Security` is only emitted when Kestra itself receives the request over HTTPS (i.e., TLS is terminated on the Kestra server). In the common deployment pattern where a reverse proxy (nginx, Traefik, AWS ALB, Cloudflare) terminates TLS and forwards plain HTTP to Kestra, the request arrives at Kestra as plain HTTP and HSTS is never sent — even if `strict-transport-security` is configured. In this case, configure HSTS at the reverse proxy instead.
+:::
+
+### Disabling the UI (API-only mode)
+
+The bundled web UI is enabled by default. Set `kestra.webserver.ui.enabled` to `false` to run the webserver as an API-only service — useful when Kestra is driven entirely through the REST API or fronted by your own application, and you don't want the UI exposed.
+
+```yaml
+kestra:
+ webserver:
+ ui:
+ enabled: false
+```
+
+When the UI is disabled:
+
+- requests to `/ui/**` return `404`, and `/` no longer redirects to the UI;
+- the REST API (`/api/v1/**`) and health endpoints keep working as usual.
+
+The setting can also be provided through the `KESTRA_WEBSERVER_UI_ENABLED` environment variable.
+
## Typical use cases
Use this section when you need to:
@@ -288,3 +389,4 @@ Use this section when you need to:
- adjust access log format for GCP or ECS
- configure Prometheus-style metrics ingestion
- change management endpoint behavior
+- configure security response headers (CSP, HSTS, framing)
diff --git a/src/contents/docs/configuration/04.plugins-and-execution/index.md b/src/contents/docs/configuration/04.plugins-and-execution/index.md
index f885204f1dc..6832c3d8f3b 100644
--- a/src/contents/docs/configuration/04.plugins-and-execution/index.md
+++ b/src/contents/docs/configuration/04.plugins-and-execution/index.md
@@ -60,38 +60,63 @@ kestra:
Most teams only need custom repositories if they publish private plugins or mirror public artifacts through an internal registry.
-## Global plugin defaults and shared behavior
+## Static policies (global defaults and enforcement)
-Use plugin defaults when many flows should inherit the same behavior. This is usually preferable to repeating the same task settings across dozens of flow definitions.
+In Kestra 2.0, global plugin defaults are replaced by static policies declared under `kestra.policies` in server configuration. Static policies form the outermost scope, apply across all tenants, and cannot be overridden through the API.
-Apply global defaults that flows can still override:
+:::alert{type="warning"}
+`kestra.plugins.defaults` is removed in Kestra 2.0. See the [pluginDefaults Removed migration guide](../../11.migration-guide/v2.0.0/plugin-defaults-removed/index.md) to convert existing defaults to policies.
+:::
+
+Apply an installation-wide default that flows can still override:
```yaml
kestra:
- plugins:
- defaults:
- - type: io.kestra.plugin.core.log.Log
- values:
- level: ERROR
+ policies:
+ - id: global-log-level
+ description: "Default log level for all Log tasks."
+ rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ where:
+ - field: type
+ operator: EQUAL_TO
+ value: io.kestra.plugin.core.log.Log
+ values:
+ level: ERROR
```
-Use forced defaults when teams must not override the value:
+Enforce a value that tasks cannot override (`override: true`):
```yaml
kestra:
- plugins:
- defaults:
- - type: io.kestra.plugin.scripts.shell.Commands
- forced: true
- values:
- containerImage: ubuntu:latest
- taskRunner:
- type: io.kestra.plugin.scripts.runner.docker.Docker
+ policies:
+ - id: enforce-docker-isolation
+ description: "Force Docker task runner for all shell script tasks."
+ rules:
+ - type: io.kestra.plugin.ee.rules.Add
+ on: PLUGIN
+ override: true
+ where:
+ - field: type
+ operator: STARTS_WITH
+ value: io.kestra.plugin.scripts.shell
+ values:
+ containerImage: ubuntu:latest
+ taskRunner:
+ type: io.kestra.plugin.scripts.runner.docker.Docker
```
-:::alert{type="warning"}
-Plugin defaults are evaluated by the Executor and propagated to other components, so every server should use the same `kestra.plugins.defaults`.
-:::
+Static policies are evaluated by the Executor and propagated to all components, so every server should have the same `kestra.policies` configuration. A malformed static policy prevents server startup (fail-closed) — validate in a staging environment first.
+
+Precedence works as follows:
+
+- Static policies (`kestra.policies`) form the outermost scope
+- Tenant-level and namespace-level Policies apply inside that
+- Task properties fill any remaining unset values
+- `override: true` on a policy rule always wins over the author's value
+
+Use `override: false` (the default) for convenience defaults and `override: true` when the platform must enforce a value such as a specific task runner.
Enable or preconfigure plugin features globally:
@@ -164,6 +189,7 @@ This part of the configuration also includes:
- retries
- temporary task storage
+- HTTP task URL filtering
- tutorial flows
- system flows
- local flow synchronization
@@ -218,6 +244,28 @@ volumes:
- /home/kestra:/home/kestra
```
+### HTTP task URL filtering
+
+Use `kestra.tasks.http` to restrict which URLs HTTP plugin tasks can call. Configure an allow-list, a deny-list, or both:
+
+```yaml
+kestra:
+ tasks:
+ http:
+ allowed-list:
+ - https://api.example.com
+ denied-list:
+ - http://169.254.169.254
+ - http://localhost
+```
+
+| Key | Default | Description |
+|---|---|---|
+| `kestra.tasks.http.allowed-list` | `[]` | When non-empty, a request URI must start with at least one entry or the task fails. |
+| `kestra.tasks.http.denied-list` | `[]` | A request URI that starts with any entry causes the task to fail. Evaluated after the allowed-list. |
+
+For security guidance and matching behavior, see [HTTP task URL filtering](../../10.administrator-guide/security-hardening/index.md#http-task-url-filtering).
+
Reserve `system` for background workflows, or rename it if your organization already uses that namespace for something else:
```yaml
@@ -289,7 +337,7 @@ Increase `max-timeout` only if your data-fetching subflows genuinely need longer
## Related docs
-- Flow-level plugin defaults: [Plugin Defaults](../../05.workflow-components/09.plugin-defaults/index.md)
+- Installation-wide configuration rules: [Policies](../../07.enterprise/02.governance/policies/index.md) (Enterprise Edition)
- Universal file access: [File Access](../../06.concepts/file-access/index.md)
- Storage backends, JVM, and global variables: [Runtime and Storage](../02.runtime-and-storage/index.md)
- Execution data isolation and enterprise-only runtime features: [Enterprise and Advanced](../06.enterprise-and-advanced/index.md)
diff --git a/src/contents/docs/configuration/05.security-and-secrets/index.md b/src/contents/docs/configuration/05.security-and-secrets/index.md
index a0f7d6281de..830b2a7ed2d 100644
--- a/src/contents/docs/configuration/05.security-and-secrets/index.md
+++ b/src/contents/docs/configuration/05.security-and-secrets/index.md
@@ -211,7 +211,7 @@ This section is about hardening the running platform rather than managing secret
This group includes:
-- super-admin behavior
+- instance owner behavior
- default roles
- invitation expiration
- password rules
@@ -238,20 +238,24 @@ endpoints:
password: your-password
```
-### Super-admin
+### Instance Owner
-The super-admin account has the highest level of platform access and should be reserved for break-glass administration:
+The instance owner account has the highest level of platform access and should be reserved for break-glass administration:
```yaml
kestra:
security:
- super-admin:
+ instance-owner:
username: your_username
- password: ${KESTRA_SUPERADMIN_PASSWORD}
+ password: ${KESTRA_INSTANCE_OWNER_PASSWORD}
tenant-admin-access:
-
```
+:::alert{type="info"}
+`kestra.security.super-admin` is a deprecated alias for `kestra.security.instance-owner` and still works.
+:::
+
:::alert{type="warning"}
Never store clear-text passwords in config. Use environment variables or your platform secret mechanism.
:::
@@ -267,7 +271,7 @@ kestra:
name: default
description: "Default role"
permissions:
- FLOW: ["CREATE", "READ", "UPDATE", "DELETE"]
+ FLOW: ["VIEW", "LIST", "CREATE", "UPDATE", "DELETE", "EXECUTE"]
```
In multi-tenant environments, scope that role to one tenant:
@@ -279,7 +283,7 @@ kestra:
name: default
description: "Default role"
permissions:
- FLOW: ["CREATE", "READ", "UPDATE", "DELETE"]
+ FLOW: ["VIEW", "LIST", "CREATE", "UPDATE", "DELETE", "EXECUTE"]
tenant-id: staging
```
@@ -298,15 +302,22 @@ kestra:
expire-after: P30D
```
-For username/password auth, enforce password complexity explicitly:
+For username/password auth, configure password complexity explicitly:
```yaml
kestra:
security:
basic-auth:
- password-regexp: ""
+ password-min-length: 8
+ password-require-special: true
+ password-min-digits: 1
+ password-min-lower-case: 1
+ password-min-upper-case: 1
+ password-allowed-special-characters: "!@#$%^&*"
```
+These rules apply anywhere Kestra asks a user to set or reset a password, including the initial setup flow, invitation acceptance, and user management screens.
+
### Delete configuration files after startup
If the runtime reads secrets from configuration files, delete them after startup so tasks cannot read them later from disk:
@@ -378,6 +389,24 @@ kestra:
Keep the external process manager timeout longer than Kestra's own termination grace period. Otherwise Kubernetes, Docker, or systemd can kill the process before graceful shutdown finishes.
:::
+## Regex timeout
+
+Kestra protects worker threads from ReDoS (catastrophic backtracking) by enforcing a timeout on all regex operations. This applies to [Pebble expression filters](../../expressions/index.mdx) (`regexMatch`, `regexReplace`, `regexExtract`, `replace` with `regexp=true`) and to `validator` patterns on `STRING` and `SECRET` inputs. When a pattern exceeds the limit, the task fails immediately with a timeout error rather than hanging indefinitely.
+
+The default timeout is **10 seconds**. To change it, set `kestra.regex.timeout` in your configuration:
+
+```yaml
+kestra:
+ regex:
+ timeout: 30s
+```
+
+Accepts ISO 8601 duration strings (e.g., `PT30S`, `PT1M`) or Micronaut shorthand (e.g., `5s`, `1m`).
+
+:::alert{type="info"}
+The timeout is set once at startup and cannot be changed at runtime without restarting the server.
+:::
+
## Related docs
- Secrets manager concepts: [External Secrets Manager](../../07.enterprise/02.governance/secrets-manager/index.md)
diff --git a/src/contents/docs/configuration/06.enterprise-and-advanced/index.md b/src/contents/docs/configuration/06.enterprise-and-advanced/index.md
index 2ad2255356f..5793c1590c8 100644
--- a/src/contents/docs/configuration/06.enterprise-and-advanced/index.md
+++ b/src/contents/docs/configuration/06.enterprise-and-advanced/index.md
@@ -17,6 +17,7 @@ This area includes:
- Enterprise license configuration
- Enterprise Java security
+- gRPC TLS/mTLS for worker ↔ controller communication
- UI sidebar customization
- historical multi-tenancy and default tenant settings
- custom links in the UI
@@ -86,7 +87,211 @@ kestra:
- "Blueprints/Flow Blueprints"
```
-The old multi-tenancy and default-tenant configuration was removed in `0.23.0`; keep it only in mind for migration work.
+The old multi-tenancy and default-tenant configuration is no longer supported.
+
+## Worker topology bootstrap (kestra.ee.setup)
+
+`kestra.ee.setup` lets you declare worker queues, worker groups, subscriptions, and registration tokens in configuration so the full topology is provisioned at startup without any runtime API calls.
+
+```yaml
+kestra:
+ ee:
+ setup:
+ enabled: true
+
+ worker-queues:
+ - id: gpu
+ tags: [gpu, linux]
+ allowed-tenants: [acme] # optional; empty = unrestricted
+ - id: etl
+ tags: [etl]
+
+ worker-groups:
+ - id: gpu-workers
+ name: GPU workers
+ registration-tokens:
+ - name: bootstrap
+ token-file: /var/run/secrets/kestra/gpu-workers-token
+ subscriptions:
+ - worker-queue-id: gpu
+ reserved-percent: 70
+ - worker-queue-id: etl
+```
+
+Only the webserver and standalone server roles apply this configuration at startup. Worker processes never apply it.
+
+Each declared entity is created only if it does not already exist in the database. Existing entities are skipped as a whole — no subscriptions are changed and no tokens are added or revoked. The database remains the source of truth once an entity exists.
+
+**Token secret handling.** Each registration token entry requires exactly one of:
+
+| Field | Description |
+|---|---|
+| `token-file` | Path to a mounted secret file containing the token. Preferred in Kubernetes environments. |
+| `token` | Environment variable placeholder resolved at startup, e.g. `"${MY_TOKEN}"`. |
+
+Plaintext tokens in committed configuration files are not recommended.
+
+**Configuration reference**
+
+| Property | Required | Description |
+|---|---|---|
+| `kestra.ee.setup.enabled` | No | Set to `true` to activate declarative bootstrap. Defaults to `false`. |
+| `worker-queues[].id` | Yes | RFC 1123 label. `default` and `system` are reserved and cannot be used. |
+| `worker-queues[].tags` | Yes | Non-empty list of routing tags. Must be unique across queues. |
+| `worker-queues[].allowed-tenants` | No | Tenant ids permitted to route through this queue. Empty = unrestricted. |
+| `worker-groups[].id` | Yes | RFC 1123 label. Use `default` to seed the default group. |
+| `worker-groups[].name` | No | Display name. Defaults to the id when omitted. |
+| `worker-groups[].registration-tokens[].name` | No | Token display name. Defaults to `bootstrap`. |
+| `worker-groups[].registration-tokens[].token-file` | One of | Path to a file containing a pre-generated registration token. Surrounding whitespace is trimmed. |
+| `worker-groups[].registration-tokens[].token` | One of | Environment variable placeholder, e.g. `"${MY_TOKEN}"`. Surrounding whitespace is trimmed. |
+| `worker-groups[].subscriptions[].worker-queue-id` | Yes | Id of a queue declared under `worker-queues` or already in the database. |
+| `worker-groups[].subscriptions[].reserved-percent` | No | Per-worker capacity floor, 1–100. Sum across subscriptions must not exceed 100. |
+| `worker-groups[].subscriptions[].mode` | No | `STRICT` or `ELASTIC`. See [Capacity reservation](../../07.enterprise/04.scalability/worker-group/index.md#capacity-reservation). Defaults to `STRICT`. |
+
+See [Declarative configuration](../../07.enterprise/04.scalability/worker-group/index.md#declarative-configuration) for full semantics, validation behavior, and a Kubernetes deployment example.
+
+## gRPC TLS/mTLS (EE only)
+
+Use this section when running Kestra in a distributed topology where the Worker Controller and Workers communicate over gRPC and you need to encrypt that channel. By default, gRPC traffic is plaintext. Enabling TLS here encrypts the controller ↔ worker channel; enabling mTLS additionally requires workers to present a certificate the controller trusts.
+
+This feature is active on any component with server type `CONTROLLER`, `WORKER`, or `STANDALONE`.
+
+### One-way TLS
+
+The controller presents a certificate; workers verify it against a truststore. Configure the controller (server side) with a keystore and the workers (client side) with a matching truststore:
+
+**Controller:**
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ key-store:
+ path: /etc/kestra/tls/controller-keystore.p12
+ type: PKCS12
+ password: ""
+```
+
+**Worker:**
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ trust-store:
+ path: /etc/kestra/tls/ca-truststore.p12
+ type: PKCS12
+ password: ""
+```
+
+If no truststore is provided on the worker side, the JVM default trust store is used. This is appropriate when the controller certificate is signed by a well-known CA.
+
+### Mutual TLS (mTLS)
+
+Set `client-auth: REQUIRE` on the controller to enforce that workers present a certificate. Both sides need a keystore and a truststore:
+
+**Controller:**
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ client-auth: REQUIRE
+ key-store:
+ path: /etc/kestra/tls/controller-keystore.p12
+ type: PKCS12
+ password: ""
+ trust-store:
+ path: /etc/kestra/tls/ca-truststore.p12
+ type: PKCS12
+ password: ""
+```
+
+**Worker:**
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ key-store:
+ path: /etc/kestra/tls/worker-keystore.p12
+ type: PKCS12
+ password: ""
+ trust-store:
+ path: /etc/kestra/tls/ca-truststore.p12
+ type: PKCS12
+ password: ""
+```
+
+`client-auth` also accepts `OPTIONAL`, which requests a client certificate but does not require one.
+
+### Authority override for static discovery
+
+When using static discovery, the gRPC channel authority is the synthetic value `controllers` rather than a real hostname. If the controller certificate's Subject Alternative Names (SANs) do not include `controllers`, TLS verification will fail. Set `authority-override` on the worker to a hostname that is present in the certificate's SANs:
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ authority-override: kestra-controller
+ trust-store:
+ path: /etc/kestra/tls/ca-truststore.p12
+ type: PKCS12
+ password: ""
+```
+
+This is not needed with DNS-based discovery, where the authority is derived from the actual hostname.
+
+### JKS keystores
+
+PKCS12 is the recommended format. For JKS keystores, set `type: JKS`. JKS also supports a separate key password (used when the private key entry password differs from the store password):
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ key-store:
+ path: /etc/kestra/tls/keystore.jks
+ type: JKS
+ password: ""
+ key-password: ""
+```
+
+### Development: skip certificate verification
+
+:::alert{type="warning"}
+`insecure-trust-all-certificates: true` disables CA verification entirely. Use only in local development or CI environments where certificates are self-signed and not managed. Never enable this in production.
+:::
+
+```yaml
+kestra:
+ grpc:
+ tls:
+ enabled: true
+ insecure-trust-all-certificates: true
+```
+
+### Configuration reference
+
+| Property | Default | Description |
+| --- | --- | --- |
+| `kestra.grpc.tls.enabled` | `false` | Enable TLS for gRPC communication |
+| `kestra.grpc.tls.key-store.path` | — | Path to keystore file |
+| `kestra.grpc.tls.key-store.type` | `PKCS12` | Keystore format (`PKCS12` or `JKS`) |
+| `kestra.grpc.tls.key-store.password` | — | Keystore password |
+| `kestra.grpc.tls.key-store.key-password` | — | Private key entry password (JKS only) |
+| `kestra.grpc.tls.trust-store.path` | — | Path to truststore file |
+| `kestra.grpc.tls.trust-store.type` | `PKCS12` | Truststore format |
+| `kestra.grpc.tls.trust-store.password` | — | Truststore password |
+| `kestra.grpc.tls.client-auth` | `NONE` | Client auth mode: `NONE`, `OPTIONAL`, or `REQUIRE` |
+| `kestra.grpc.tls.insecure-trust-all-certificates` | `false` | Skip CA verification (development only) |
+| `kestra.grpc.tls.authority-override` | — | Override TLS authority for static discovery |
## Elasticsearch, Kafka, and indexing
@@ -268,6 +473,25 @@ kestra:
If indexing falls behind, tune indexer batch settings before changing flow definitions. Those settings control how aggressively Kafka-backed events are flushed into Elasticsearch.
+## MCP server cache
+
+Each webserver node caches MCP server configuration in memory and hot-reloads it when a server is created, updated, or deleted. Two properties control this cache:
+
+| Property | Default | Description |
+|---|---|---|
+| `kestra.mcp.server-cache-config.maximum-size` | `500` | Maximum number of MCP server entries held in the cache. |
+| `kestra.mcp.server-cache-config.expire-after-access` | `PT5M` | Duration after which a cache entry expires if not accessed. |
+
+```yaml
+kestra:
+ mcp:
+ server-cache-config:
+ maximum-size: 200
+ expire-after-access: PT10M
+```
+
+Tune these only if you have a large number of MCP servers or tight memory constraints. The defaults are sufficient for most deployments.
+
## AI and isolated environments
These are the most optional settings on the page. They matter only if you are enabling Copilot integrations or operating Kestra in restricted network environments.
@@ -279,66 +503,274 @@ This page also includes:
### AI Copilot
-Set `kestra.ai.enabled` to `false` to fully disable the AI Copilot, including the built-in fallback to `api.kestra.io`. Defaults to `true`.
+AI Copilot configuration lives under `kestra.ai` and controls which LLM providers are active, how each provider is authenticated and tuned, and how the agent runtime behaves.
+
+#### Enabling and disabling
+
+`kestra.ai.enabled` (default: `true`) controls whether AI Copilot is active. Set it to `false` to disable the feature entirely, including the built-in fallback to `api.kestra.io`.
-Enterprise Edition supports multiple providers in one configuration, which is useful when teams need both a default internal model and a fallback external model:
+#### Providers
+
+Enterprise Edition lets you configure multiple providers in a single deployment. Each entry in `kestra.ai.providers` is an independent provider the UI can offer to users.
```yaml
kestra:
ai:
- enabled: true # set to false to disable AI Copilot entirely
+ enabled: true
providers:
- - id: gemini
- display-name: Gemini - Private
+ - id: internal-gemini
+ display-name: Gemini (internal)
type: gemini
configuration:
- model-name: gemini-3.5-flash-lite
api-key: YOUR_GEMINI_API_KEY
- - id: gpt
- display-name: OpenAI
+ model-name: gemini-2.5-flash
+ - id: openai-gpt
+ display-name: OpenAI GPT
type: openai
- isDefault: true
+ is-default: true
configuration:
- model-name: gpt-4
api-key: YOUR_OPENAI_API_KEY
+ model-name: gpt-4o
```
-Optional provider settings include `temperature`, `top-p`, `top-k`, `max-output-tokens`, `log-requests`, `log-responses`, and `base-url`.
+**Provider wrapper fields**
-### Air-gapped mode
+| Field | Required | Description |
+|---|---|---|
+| `id` | ✅ | Unique identifier for this provider entry. |
+| `display-name` | ✅ | Label shown to users in the Copilot UI. |
+| `type` | ✅ | Provider type. One of: `openai`, `azure-openai`, `gemini`, `googlevertexai`, `anthropic`, `bedrock`, `deepseek`, `mistralai`, `ollama`, `open-router`. |
+| `is-default` | ❌ | When `true`, this provider is selected automatically when no explicit choice is made. |
+| `configuration` | ❌ | Provider-specific settings. See the property reference and provider sections below. |
+| `system-prompt` | ❌ | (EE only) Override the built-in system prompt per Copilot mode. A non-blank value for a mode fully replaces the built-in prompt for that mode. |
+| `system-prompt.ask` | ❌ | Custom system prompt for Ask mode. |
+| `system-prompt.plan` | ❌ | Custom system prompt for Plan mode. |
+| `system-prompt.edit` | ❌ | Custom system prompt for Edit mode. |
-Use air-gapped mode when the UI and blueprint experience must avoid external dependencies:
+#### Configuration property reference
-```yaml
-kestra:
- ee:
- airgapped: true
-```
+These properties appear inside the `configuration:` block of a provider entry. Not every property is available on every provider — the per-provider sections below show which apply and which are required.
-When enabled, the UI hides or adapts features that normally depend on external services, such as hosted fonts, external blueprint sources, or embedded internet content.
+**Authentication**
+
+| Property | Description |
+|---|---|
+| `api-key` | API key for the provider. Most providers require this; Anthropic and Google Vertex AI do not (see their sections). |
+| `access-key-id` | AWS access key ID. Amazon Bedrock only. |
+| `secret-access-key` | AWS secret access key. Amazon Bedrock only. |
+| `client-pem` | PEM-encoded client certificate used for mutual TLS (mTLS) when the provider endpoint requires client authentication. |
+| `ca-pem` | PEM-encoded CA certificate to add additional TLS trust beyond the system trust store. Not required for standard provider endpoints. |
+
+**Model selection**
+
+| Property | Description |
+|---|---|
+| `model-name` | The model identifier to use. The accepted values are provider-specific (e.g. `gpt-4o`, `gemini-2.5-flash`, `claude-opus-4-5`). Each provider section lists its default. |
+
+**Generation parameters**
+
+| Property | Description |
+|---|---|
+| `temperature` | Controls randomness in sampling. Lower values (e.g. `0.2`) produce more deterministic output; higher values (e.g. `1.0`) produce more varied responses. Most providers default to `0.7`; OpenAI and OpenRouter default to `1`. |
+| `top-p` | Nucleus sampling: only tokens whose cumulative probability reaches `top-p` are considered. An alternative to `temperature` — set one or the other, not both. |
+| `top-k` | Limits sampling to the top K most probable tokens at each step. Supported by Gemini, Google Vertex AI, Anthropic, Amazon Bedrock, and Ollama. |
+| `max-output-tokens` | Maximum number of tokens the model may produce in a single response. Defaults to `8000` on providers that support it. Mistral AI does not expose this setting. |
+
+**Extended reasoning**
+
+| Property | Description |
+|---|---|
+| `thinking-enabled` | When `true`, enables the provider's extended reasoning or thinking mode. Supported by OpenAI, Azure OpenAI, Gemini, Anthropic, and Amazon Bedrock. Anthropic requires `temperature: 1` and disables `top-p` and `top-k` when thinking is on — these constraints are applied automatically. |
+| `thinking-effort` | Provider-neutral reasoning effort: `LOW`, `MEDIUM`, or `HIGH`. Supported by OpenAI, Azure OpenAI, and Gemini. Each provider maps this to its own vocabulary (e.g. OpenAI `reasoning_effort`, Gemini `thinkingLevel`). |
+| `thinking-budget-tokens` | Token budget for the reasoning process. Supported by Gemini, Anthropic, and Amazon Bedrock. Anthropic requires a minimum of `1024`; when `thinking-enabled` is `true` and no budget is set, `1024` is used automatically. |
+
+**Network and connectivity**
+
+| Property | Description |
+|---|---|
+| `base-url` | Override the default API endpoint. Use this for self-hosted deployments, proxies, or OpenAI-compatible local servers. Available on OpenAI, Gemini, DeepSeek, Mistral AI, Ollama, and OpenRouter. For Ollama, `base-url` is required (there is no cloud endpoint). |
+| `custom-headers` | A flat map of extra HTTP headers sent with every request to the provider. Useful for passing organization IDs, routing headers, or authentication tokens that the provider requires alongside the API key. |
+| `timeout` | Maximum duration for a single HTTP request to the provider (e.g. `PT30S`, `PT2M`). Does not apply to Google Vertex AI. |
+
+**Logging**
+
+| Property | Description |
+|---|---|
+| `log-requests` | When `true`, logs the full request body sent to the provider. Useful for debugging prompt construction. Avoid in production — request bodies may contain sensitive data. |
+| `log-responses` | When `true`, logs the full response body received from the provider. Same caveats as `log-requests`. |
+| `log-requests-and-responses` | Azure OpenAI equivalent of `log-requests` + `log-responses` combined in a single toggle. |
+
+#### Provider types
+
+Each provider section lists only its required fields and properties unique to that provider. All other properties from the reference above are available unless noted.
+
+##### OpenAI (`type: openai`)
+
+| Property | Required | Default |
+|---|---|---|
+| `api-key` | ✅ | — |
+| `model-name` | ❌ | `gpt-5-nano` |
+| `temperature` | ❌ | `1` |
+| `max-output-tokens` | ❌ | `8000` |
+
+Supports `thinking-enabled` and `thinking-effort`. Supports `base-url` for OpenAI-compatible self-hosted endpoints.
+
+Does not support `top-k`.
+
+##### Azure OpenAI (`type: azure-openai`)
+
+Supports two authentication methods: API key or Azure Active Directory (AAD).
+
+| Property | Required | Default |
+|---|---|---|
+| `endpoint` | ✅ | — |
+| `model-name` | ✅ | — |
+| `api-key` | ❌ (use this or AAD) | — |
+| `tenant-id` | ❌ (AAD auth) | — |
+| `client-id` | ❌ (AAD auth) | — |
+| `client-secret` | ❌ (AAD auth) | — |
+| `service-version` | ❌ | — |
+| `temperature` | ❌ | `1` |
+| `max-output-tokens` | ❌ | `8000` |
+
+Supports `thinking-enabled` and `thinking-effort`.
+
+Uses `log-requests-and-responses` instead of separate `log-requests` / `log-responses` toggles. Does not support `top-k`, `base-url`, `client-pem`, or `ca-pem`.
-### Execution data in internal storage
+##### Gemini (`type: gemini`)
-If EE outputs and inputs must be isolated per tenant or namespace, store execution data in internal storage:
+| Property | Required | Default |
+|---|---|---|
+| `api-key` | ✅ | — |
+| `model-name` | ❌ | `gemini-2.5-flash` |
+| `temperature` | ❌ | `0.7` |
+| `max-output-tokens` | ❌ | `8000` |
+
+Supports `thinking-enabled`, `thinking-effort`, and `thinking-budget-tokens`. Supports `base-url` and `top-k`.
+
+##### Google Vertex AI (`type: googlevertexai`)
+
+Authenticates via [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials). No `api-key` field — ensure the Kestra runtime environment has ADC configured (e.g. a service account key via `GOOGLE_APPLICATION_CREDENTIALS`, Workload Identity, or `gcloud auth application-default login`).
+
+| Property | Required | Default |
+|---|---|---|
+| `project` | ✅ | — |
+| `location` | ✅ | — |
+| `model-name` | ✅ | — |
+| `temperature` | ❌ | `0.7` |
+
+Supports `top-k`. Does not support `thinking-enabled`, `thinking-effort`, `thinking-budget-tokens`, `base-url`, `client-pem`, `ca-pem`, `max-output-tokens`, or `timeout`.
+
+##### Anthropic (`type: anthropic`)
+
+No `api-key` configuration field. Set the `ANTHROPIC_API_KEY` environment variable on the Kestra server instead.
+
+| Property | Required | Default |
+|---|---|---|
+| `model-name` | ✅ | — |
+| `temperature` | ❌ | `0.7` |
+| `max-output-tokens` | ❌ | `8000` |
+
+Supports `thinking-enabled` and `thinking-budget-tokens` (minimum `1024`; defaults to `1024` when thinking is enabled without an explicit budget). When `thinking-enabled` is `true`, Anthropic requires `temperature: 1` and ignores `top-p` and `top-k` — these constraints are applied automatically regardless of what you configure. Supports `top-k`.
+
+Does not support `thinking-effort`.
+
+##### Amazon Bedrock (`type: bedrock`)
+
+| Property | Required | Default |
+|---|---|---|
+| `access-key-id` | ✅ | — |
+| `secret-access-key` | ✅ | — |
+| `model-name` | ✅ | — |
+| `temperature` | ❌ | `0.7` |
+| `max-output-tokens` | ❌ | `8000` |
+
+Supports `thinking-enabled` and `thinking-budget-tokens`. Supports `top-k`.
+
+Does not support `thinking-effort`, `base-url`, `client-pem`, or `ca-pem`.
+
+##### DeepSeek (`type: deepseek`)
+
+| Property | Required | Default |
+|---|---|---|
+| `api-key` | ✅ | — |
+| `model-name` | ❌ | `deepseek-chat` |
+| `temperature` | ❌ | `0.7` |
+| `max-output-tokens` | ❌ | `8000` |
+
+Supports `base-url` for self-hosted or compatible endpoints. Does not support `top-k`, `thinking-enabled`, or `thinking-effort`.
+
+##### Mistral AI (`type: mistralai`)
+
+| Property | Required | Default |
+|---|---|---|
+| `api-key` | ✅ | — |
+| `model-name` | ✅ | — |
+| `temperature` | ❌ | `0.7` |
+
+Supports `base-url`. Does not support `max-output-tokens`, `top-k`, `thinking-enabled`, or `thinking-effort`.
+
+##### Ollama (`type: ollama`)
+
+Ollama runs locally — there is no cloud API key. `base-url` points to your Ollama server and is required.
+
+| Property | Required | Default |
+|---|---|---|
+| `base-url` | ✅ | — |
+| `model-name` | ✅ | — |
+| `temperature` | ❌ | `0.7` |
+| `max-output-tokens` | ❌ | `8000` |
+
+Supports `top-k`. Does not support `api-key`, `thinking-enabled`, or `thinking-effort`.
+
+##### OpenRouter (`type: open-router`)
+
+| Property | Required | Default |
+|---|---|---|
+| `api-key` | ✅ | — |
+| `model-name` | ❌ | `gpt-5-nano` |
+| `temperature` | ❌ | `1` |
+| `max-output-tokens` | ❌ | `8000` |
+
+Supports `base-url`. Does not support `top-k`, `thinking-enabled`, or `thinking-effort`.
+
+#### Agent runtime settings
+
+`kestra.ai.agent` controls the Copilot agent runtime. The defaults suit most deployments — tune only when hitting provider rate limits, memory pressure, or needing to adjust conversation scope.
+
+| Property | Default | Description |
+|---|---|---|
+| `model-call-timeout` | `PT5M` | Maximum duration of a single streaming model call. If a provider call hangs beyond this threshold, the turn is failed rather than leaving a thread pinned indefinitely. |
+| `docs-mcp-url` | `https://api.kestra.io/v1/mcp` | Kestra docs MCP endpoint used for context grounding in Ask mode. Override this in air-gapped deployments that run a local docs MCP server. |
+| `max-sequential-tools-invocations` | `25` | Maximum number of sequential tool-calling round-trips within a single turn. Bounds runaway reasoning loops — each round-trip is a paid model call. |
+| `max-turns-per-thread` | `50` | Maximum number of user turns in a single conversation thread before new turns are refused. |
+| `max-concurrent-turns` | `32` | Per-node ceiling on simultaneously running agent turns. New turns receive a 429 response when the ceiling is reached rather than queuing. Bounds concurrent provider load — agent turns run on virtual threads so the thread count itself is not a concern. |
+| `max-context-turns` | `10` | How many of the most recent turns are replayed into the model context per turn. Older turns remain stored for history but are windowed out of the prompt. Windowing operates on whole turns so tool-call and result pairs are never split. |
+| `in-memory-conversation-ttl` | `PT1H` | In-memory store only: how long a conversation is retained after its last activity before eviction. Ignored when a durable backend is configured. |
+| `max-in-memory-conversations` | `50` | In-memory store only: hard cap on retained conversations. The least-recently-active conversation is evicted when the cap is exceeded. Ignored when a durable backend is configured. |
```yaml
kestra:
- ee:
- execution-data:
- internal-storage:
- enabled: true
+ ai:
+ agent:
+ model-call-timeout: PT5M
+ max-sequential-tools-invocations: 25
+ max-concurrent-turns: 32
+ max-context-turns: 10
```
-To enforce that behavior everywhere:
+### Air-gapped mode
+
+Use air-gapped mode when the UI and blueprint experience must avoid external dependencies:
```yaml
kestra:
ee:
- execution-data:
- internal-storage:
- force-globally: true
+ airgapped: true
```
+When enabled, the UI hides or adapts features that normally depend on external services, such as hosted fonts, external blueprint sources, or embedded internet content.
+
### Mail service
Invitation and password-reset emails rely on the EE mail service:
diff --git a/src/contents/docs/docs-style-guide.md b/src/contents/docs/docs-style-guide.md
new file mode 100644
index 00000000000..6d183426490
--- /dev/null
+++ b/src/contents/docs/docs-style-guide.md
@@ -0,0 +1,201 @@
+---
+title: Kestra Docs Style Guide
+h1: Kestra Docs Style Guide
+description: Writing and editorial standards for Kestra documentation contributors.
+hideSidebar: true
+---
+
+Writing and editorial standards for the Kestra docs. Use it when writing new pages, reviewing pull requests, or editing existing content.
+
+## Core principles
+
+- **Be clear before being clever.** Optimize for task completion, not elegance.
+- **Explain the why, not just the how.** Readers who understand the reason make better decisions when the instructions don't perfectly match their situation.
+- **Write for a global technical audience.** Avoid idioms, culturally specific references, and assumed familiarity.
+- **Prefer consistency over novelty.** Match existing Kestra terminology and page patterns before introducing new phrasing.
+- **Keep the page focused.** One dominant page type per page. Mixed-type pages are harder to write and harder to use.
+
+## Page types
+
+Every docs page should have a single dominant type. Mixed-type pages are harder to write and harder to use.
+
+| Type | Purpose | Examples |
+|---|---|---|
+| **Landing page** | Orient readers and route them to sub-pages | Section index pages |
+| **Concept** | Explain what something is and why it matters | Architecture, Multi-tenancy |
+| **How-to guide** | Drive one specific task to completion | How to set up webhooks |
+| **Reference** | Help readers look up facts quickly | API reference, CLI commands |
+| **Migration guide** | Explain what changed between versions and how to adapt | pluginDefaults Removed |
+
+If you find yourself writing a concept explanation inside a how-to, extract it. If you find a how-to inside a reference page, move it.
+
+## Page openings
+
+The first sentence is the most important on the page. It appears in ChildCards, search results, and link previews.
+
+**Requirements:**
+- Explains what the page covers in plain language
+- Stands alone without context from the title or URL
+- Stays under 30 words
+- Uses no filler: avoid "In this page," "This document will," "Welcome to," or "This guide covers"
+
+**Before:**
+> In this guide, we will walk through how you can configure webhooks to trigger Kestra flows.
+
+**After:**
+> Configure a Webhook trigger to start a flow in response to an HTTP POST request.
+
+## Voice and tone
+
+- **Second person** — address the reader as "you." Never use "we," "our," or first-person plural; Kestra docs address the reader, not the writing team.
+- **Active voice** — prefer "Kestra executes the task" over "the task is executed by Kestra."
+- **Present tense** — describe product behavior as it is now, not as it will be or was.
+- **Direct** — cut filler phrases. "Note that," "It's worth mentioning," "Simply," and "Just" add no meaning.
+- **Calm and technical** — avoid enthusiasm markers ("powerful," "amazing," "easy"), anthropomorphism ("Kestra knows," "the task understands"), and marketing language.
+
+## Headings
+
+- Use **sentence case** for all body headings — capitalise only the first word and proper nouns.
+- Use **Title Case** for `title` and `sidebarTitle` in frontmatter.
+- Do not restate the page title as the first H2. The first heading should introduce the first distinct section.
+- Keep heading depth shallow. Prefer H2 and H3; use H4 only when genuinely necessary.
+- Make headings descriptive. "Configuration" is vague; "Configure internal storage" is specific.
+
+**Before:**
+> ## Introduction to webhooks
+> ### What is a webhook?
+> #### Example of webhook usage
+
+**After:**
+> ## How webhooks work
+> ### Configure a webhook trigger
+
+## Capitalization
+
+Kestra has specific capitalization conventions. The key principle: capitalize a term when referring to the named product entity; use lowercase for the generic concept.
+
+| Term | Generic (lowercase) | Product entity (capitalized) |
+|---|---|---|
+| flow | "orchestrate your flows" | "open the Flow editor" |
+| task | "add a task to the flow" | — (always generic) |
+| namespace | "organize flows by namespace" | — (always generic) |
+| execution | "view past executions" | "open the Executions tab" |
+| trigger | "add a trigger" | "a Webhook trigger," "a Flow trigger" |
+
+**Always match visible UI labels exactly**, including capitalization and punctuation. If the UI says "Worker Queues," write "Worker Queues," not "worker queues."
+
+Kestra feature names that are always capitalized:
+- Policies, Worker Queues, Apps, Blueprints, AI Copilot
+- Internal Storage (when referring to the storage layer)
+- RBAC (always acronym)
+
+## Procedures
+
+A procedure drives one task to completion. Structure it as:
+
+1. **Goal sentence** — one line describing what the reader will accomplish.
+2. **Prerequisites** — what the reader needs before starting. Use a dedicated prerequisites section, not buried alerts or body text.
+3. **Numbered steps** — each step is one action, written as an imperative verb ("Click **Create**," "Set the `region` property").
+4. **Verification** — how the reader confirms it worked.
+5. **Next steps** — where to go from here, if relevant.
+
+Avoid collapsing multiple actions into one step. "Click Create, fill in the form, and save" is three steps.
+
+## Code and examples
+
+- Introduce every code block. Tell the reader what they are about to see before they see it.
+- Specify the language on every fenced code block: ` ```yaml `, ` ```bash `, ` ```python `.
+- Prefer complete, runnable examples in how-to guides. If an example is intentionally incomplete, say so explicitly.
+- Explain the result or the key decision after the block when it isn't obvious.
+- Use `company.team` as the namespace in all flow examples.
+
+**Before:**
+> Here is an example:
+> ```yaml
+> kestra:
+> storage:
+> type: s3
+> ```
+
+**After:**
+> Configure internal storage to use AWS S3:
+> ```yaml
+> kestra:
+> storage:
+> type: s3
+> s3:
+> bucket: "kestra-internal-storage"
+> region: "us-east-1"
+> ```
+> Replace `bucket` and `region` with your own values. See [internal storage configuration](./configuration/02.runtime-and-storage/index.md) for all available options.
+
+## Alerts
+
+Use alerts sparingly. Reserve them for content that would cause failure or confusion if missed.
+
+| Alert type | When to use |
+|---|---|
+| `warning` | Destructive, irreversible, or data-loss actions |
+| `info` | Prerequisites, edition requirements, or non-obvious constraints that affect correctness |
+| `success` | Rarely needed; only for outcomes that are genuinely non-obvious |
+
+**Do not use alerts for:**
+- General information that works equally well as a sentence in body text
+- Tips or suggestions that are optional
+- Two alerts in a row — consolidate or convert one to prose
+
+## Links and media
+
+**Links:**
+- Use descriptive link text. "See [Policies](./policies)" not "click [here](./policies)."
+- Link to the source of truth rather than duplicating content across pages.
+- Use relative links between docs pages.
+
+**Screenshots:**
+- Include a screenshot only when it materially helps — a complex UI interaction, a before/after comparison, or a result that is hard to describe in words.
+- Write descriptive alt text that describes what is shown, not just what the image is called.
+- Keep screenshots current. An outdated screenshot is worse than no screenshot.
+
+**Videos:**
+- Embed videos after the page's opening sentence, before the main content, using the `video-container` div class.
+- Do not use videos as a substitute for written content — readers on slow connections or screen readers cannot access them.
+
+## What to remove
+
+Cut the following on sight:
+
+- **Promotional language** — "powerful," "flexible," "seamless," "robust," "best-in-class"
+- **Vague claims** — "easy to use," "quick and simple," "at any scale" without qualification
+- **Throat-clearing** — "In this guide we will," "Before we get started," "It is important to note that"
+- **Filler adverbs** — "simply," "just," "easily," "quickly"
+- **Decorative screenshots** — images that show the UI but add no information the text doesn't already convey
+- **Unexplained code blocks** — a block with no introduction or follow-up explanation
+- **Redundant alerts** — information already in the body text repeated in an alert box
+
+## Version references
+
+Docs describe the current supported behavior by default. Avoid phrases like "as of version X.Y," "starting in version X.Y," or "currently" in normal docs pages — these go stale and create confusion about what applies to the reader's version.
+
+**Exceptions:**
+- Migration guides — always version-scoped by design
+- A `version` frontmatter property on a page, which renders a version badge automatically
+- An inline `:::badge` component when a single section of a page applies only to certain versions
+
+Outside these cases, if version scoping is necessary, prefer structured metadata over prose mentions.
+
+## Review checklist
+
+Before submitting or approving a docs change, verify:
+
+1. The page type is clear and consistent throughout.
+2. The first sentence stands alone as a summary.
+3. All body headings use sentence case.
+4. Capitalization matches Kestra feature names and visible UI labels.
+5. Prose is second person, active voice, and present tense.
+6. No filler phrases, hype, or anthropomorphism.
+7. Terminology is consistent within the page and matches the rest of the docs.
+8. Every code block is introduced and (where non-obvious) followed by an explanation.
+9. Links use descriptive text and point to the source of truth.
+10. Screenshots are current and earn their place.
+11. Alerts are used only for genuinely warning- or constraint-worthy content.
+12. The reader can verify success or find the next relevant path.
diff --git a/src/contents/docs/expressions/01.context/index.mdx b/src/contents/docs/expressions/01.context/index.mdx
index c6dce8c5103..0f4548ce589 100644
--- a/src/contents/docs/expressions/01.context/index.mdx
+++ b/src/contents/docs/expressions/01.context/index.mdx
@@ -25,6 +25,7 @@ The execution context usually includes:
- `namespace` in Enterprise Edition when namespace variables are configured
- `envs` for environment variables
- `globals` for global configuration values
+- `item` inside a [Loop](../../05.workflow-components/01.tasks/00.flowable-tasks/index.md#loop) task iteration
:::alert{type="info"}
To inspect the full runtime context, use `{{ fetchContext() }}` in the Debug Expression console.
@@ -54,11 +55,10 @@ The Debug Expression console is available in the Kestra UI under **Executions
| `{{ taskrun.startDate }}` | Start date of the current task run |
| `{{ taskrun.attemptsCount }}` | Retry and restart attempt count |
| `{{ taskrun.parentId }}` | Parent task run identifier for nested tasks |
-| `{{ taskrun.value }}` | Current loop or flowable value |
-| `{{ parent.taskrun.value }}` | Value of the nearest parent task run |
| `{{ parent.outputs }}` | Outputs of the nearest parent task run |
| `{{ parents }}` | List of parent task runs |
| `{{ labels }}` | Execution labels accessible by key |
+| `{{ trace.parent }}` | W3C `traceparent` header for the current execution; only populated when [OpenTelemetry tracing is enabled](../../10.administrator-guide/open-telemetry/index.md#traces) |
Example:
@@ -93,6 +93,36 @@ When the execution is started by a `Flow` trigger:
| `{{ trigger.flowId }}` | ID of the triggering flow |
| `{{ trigger.flowRevision }}` | Revision of the triggering flow |
+## Loop iteration context
+
+Inside a [Loop](../../05.workflow-components/01.tasks/00.flowable-tasks/index.md#loop) task, each iteration runs as an isolated sub-execution. The `item` variable is available to all tasks within that sub-execution.
+
+| Expression | Description |
+|---|---|
+| `{{ item.index }}` | Zero-based index of the current iteration |
+| `{{ item.value }}` | Value of the current iteration |
+| `{{ item.key }}` | Map key of the current iteration; only set when `values` is a map |
+| `{{ item.parent.index }}` | Index of the nearest enclosing loop (nested loops only) |
+| `{{ item.parent.value }}` | Value of the nearest enclosing loop (nested loops only) |
+| `{{ item.parents[n].value }}` | Value of the nth ancestor loop, counting from innermost (`[0]` = immediate parent) |
+
+Because `item` is bound to the loop execution rather than individual task runs, flowable tasks nested inside a `Loop` (such as `If` or `Parallel`) can access `item` directly without any `parent.` prefix.
+
+```yaml
+tasks:
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
+ values: ["value 1", "value 2", "value 3"]
+ tasks:
+ - id: check
+ type: io.kestra.plugin.core.flow.If
+ condition: '{{ item.value == "value 2" }}'
+ then:
+ - id: log
+ type: io.kestra.plugin.core.log.Log
+ message: "Matched at index {{ item.index }}: {{ item.value }}"
+```
+
## Environment and global variables
Kestra provides access to environment variables prefixed with `ENV_` by default, unless configured otherwise in the [runtime and storage configuration](/docs/configuration/runtime-and-storage).
diff --git a/src/contents/docs/expressions/03.filters/01.json/index.mdx b/src/contents/docs/expressions/03.filters/01.json/index.mdx
index cac644dab60..455cc78a6bf 100644
--- a/src/contents/docs/expressions/03.filters/01.json/index.mdx
+++ b/src/contents/docs/expressions/03.filters/01.json/index.mdx
@@ -41,10 +41,10 @@ Examples:
{{ [1, 2, 3] | jq('.[0]') | first }}
```
-Example flow using `jq` inside a `ForEach`:
+Example flow using `jq` inside a `Loop`:
```yaml
-id: jq_with_foreach
+id: jq_with_loop
namespace: company.team
tasks:
@@ -56,15 +56,15 @@ tasks:
{"name": "bravo", "value": 2}
]
- - id: foreach
- type: io.kestra.plugin.core.flow.ForEach
+ - id: loop
+ type: io.kestra.plugin.core.flow.Loop
values: "{{ fromJson(outputs.generate.value) }}"
tasks:
- id: log_filtered
type: io.kestra.plugin.core.log.Log
message: |
- Name: {{ fromJson(taskrun.value).name }}
- Doubled value: {{ fromJson(taskrun.value) | jq('.value * 2') | first }}
+ Name: {{ fromJson(item.value).name }}
+ Doubled value: {{ fromJson(item.value) | jq('.value * 2') | first }}
```
The practical rule with `jq` is that it is great for extracting or transforming a small part of a larger payload, but it is usually overkill when plain dot access already gets you the value you need.
diff --git a/src/contents/docs/expressions/03.filters/03.strings/index.mdx b/src/contents/docs/expressions/03.filters/03.strings/index.mdx
index c9112ce1e0c..fa4e67279a1 100644
--- a/src/contents/docs/expressions/03.filters/03.strings/index.mdx
+++ b/src/contents/docs/expressions/03.filters/03.strings/index.mdx
@@ -120,6 +120,31 @@ Escapes special characters in a string. The `type` argument controls which style
{# output: Can\'t be here #}
```
+## Regex filters
+
+Three filters cover the most common regex operations:
+
+- `regexMatch(regex)` — returns `true` if the input contains a substring matching the pattern, `false` otherwise.
+- `regexReplace(regex, replacement)` — replaces all non-overlapping matches. Use `$1`, `$2`, … to reference capture groups in the replacement.
+- `regexExtract(regex, group)` — returns the first match or a specific capture group. `group` defaults to `0` (the whole match); returns `null` if there is no match.
+
+```twig
+{{ "hello world" | regexMatch("w[a-z]+") }}
+{# output: true #}
+{{ "2024-01-15" | regexReplace("(\\d{4})-(\\d{2})-(\\d{2})", "$3/$2/$1") }}
+{# output: 15/01/2024 #}
+{{ "order-12345-done" | regexExtract("\\d+") }}
+{# output: 12345 #}
+{{ "2024-01-15" | regexExtract("(\\d{4})-(\\d{2})-(\\d{2})", 1) }}
+{# output: 2024 #}
+```
+
+:::alert{type="warning"}
+Regex filter operations are subject to a **10-second timeout** to prevent ReDoS (catastrophic backtracking). If a pattern takes longer than the limit, the task fails with a timeout error.
+
+Patterns with nested quantifiers such as `(a+)+` applied to large inputs are most likely to trigger this. Use anchored, non-ambiguous patterns to avoid it. The timeout can be adjusted with [`kestra.regex.timeout`](../../../configuration/05.security-and-secrets/index.md#regex-timeout) in your Kestra configuration.
+:::
+
## Worked string filter example
This flow builds a sanitized filename and a display-safe summary from a raw input title:
diff --git a/src/contents/docs/expressions/03.filters/index.mdx b/src/contents/docs/expressions/03.filters/index.mdx
index 45174dffbc2..d673208f47b 100644
--- a/src/contents/docs/expressions/03.filters/index.mdx
+++ b/src/contents/docs/expressions/03.filters/index.mdx
@@ -14,7 +14,7 @@ Use filters when you need to transform a value with the pipe syntax: `{{ value |
- [JSON and structured data](./01.json/index.mdx) — `toJson`, `toIon`, `jq`
- [Numbers and collections](./02.collections/index.mdx) — `abs`, `number`, `first`, `last`, `sort`, `chunk`, `distinct`, and more
-- [Strings](./03.strings/index.mdx) — `lower`, `upper`, `replace`, `slugify`, `base64encode`, and more
+- [Strings](./03.strings/index.mdx) — `lower`, `upper`, `replace`, `slugify`, `base64encode`, `regexMatch`, `regexReplace`, `regexExtract`, and more
- [Dates](./04.dates/index.mdx) — `date`, `dateAdd`, `timestamp`, `timestampMilli`, and precision variants
- [YAML](./05.yaml/index.mdx) — `yaml`, `indent`, `nindent`
diff --git a/src/contents/docs/expressions/04.functions/01.rendering/index.mdx b/src/contents/docs/expressions/04.functions/01.rendering/index.mdx
index 9bb9b1830d3..ec028672350 100644
--- a/src/contents/docs/expressions/04.functions/01.rendering/index.mdx
+++ b/src/contents/docs/expressions/04.functions/01.rendering/index.mdx
@@ -39,8 +39,6 @@ Equivalent to `render(expression, recursive=false)`. Use `renderOnce()` when you
{{ renderOnce(namespace.github.token) }}
```
-`renderOnce()` is the safer choice when you need one extra evaluation pass but do not want recursive expansion to keep walking nested Pebble content.
-
## `fetchContext()`
Outputs the full execution context as a string. Use it in the Debug Expression console to inspect every variable available at that point in the execution:
diff --git a/src/contents/docs/expressions/04.functions/02.data-access/index.mdx b/src/contents/docs/expressions/04.functions/02.data-access/index.mdx
index 53b0b8887f4..5bc66e04608 100644
--- a/src/contents/docs/expressions/04.functions/02.data-access/index.mdx
+++ b/src/contents/docs/expressions/04.functions/02.data-access/index.mdx
@@ -1,7 +1,7 @@
---
title: "Data Access Functions in Kestra Expressions"
h1: "Data Access Functions"
-description: Reference for Kestra's data access functions — secret(), credential(), read(), fileURI(), kv(), encrypt(), and decrypt() — for resolving secrets, files, and stored values at runtime.
+description: Reference for Kestra's data access functions — secret(), env(), credential(), read(), fileURI(), kv(), encrypt(), and decrypt() — for resolving secrets, environment variables, files, and stored values at runtime.
sidebarTitle: Data Access Functions
icon: /src/contents/docs/icons/expression.svg
---
@@ -24,9 +24,64 @@ Arguments:
- `key` — the secret key
- `namespace` — defaults to the flow's namespace; the secret is resolved using that namespace's secret backend, with values inherited from parent namespaces
- `subkey` — optional field to extract when the secret holds a JSON object
+- `full` — when `true`, returns a map with two fields: `value` (the primary secret) and `metadata` (any additional fields exposed by the secret manager). Cannot be combined with `subkey`.
Cross-namespace reads stay within the same tenant. In the Enterprise Edition, a flow may read another namespace's secrets by default; restrict this by configuring `allowedNamespaces` on the target namespace.
+### Retrieving multi-field secrets with `full=true`
+
+Some secret managers store credentials as structured objects — for example, a database credential with a password, username, and domain. Pass `full=true` to retrieve all fields at once:
+
+```twig
+{% set cred = secret('prod-mssql', full=true) %}
+{{ cred.value }} {# primary secret, same as secret('prod-mssql') #}
+{{ cred.metadata.username }} {# additional field from the secret manager #}
+{{ cred.metadata.domain }} {# additional field from the secret manager #}
+```
+
+`cred.value` is always present and holds the primary secret. `cred.metadata` is only present when the secret manager exposes additional fields — for single-value backends (AWS, Azure, GCP, Doppler, internal), the `metadata` key is absent and only `cred.value` is accessible.
+
+| Backend | `cred.metadata` |
+| --- | --- |
+| Delinea | credential fields from the secret template |
+| CyberArk | account fields (username, address, etc.) |
+| BeyondTrust | credential fields |
+| 1Password | item fields |
+| AWS, Azure, GCP, Doppler, internal | absent — use `cred.value` only |
+
+## `env()`
+
+Reads an environment variable from the execution context by name. Use `env()` when the variable name is dynamic — composed at runtime from inputs or outputs — something not possible with the static `envs.varname` dot notation.
+
+```twig
+{{ env('API_HOST') }}
+{{ env('API_HOST', 'localhost') }}
+{{ env('api_url_' ~ inputs.environment) }}
+```
+
+Arguments:
+
+- `name` — the environment variable name, after normalization (lowercase, prefix stripped)
+- `default` — optional value returned when the variable is missing or empty
+
+Environment variables are exposed via the `envs` context map. By default, only variables prefixed with `ENV_` are available; the prefix is stripped and the name lowercased: `ENV_API_URL_PROD` is accessible as `env('api_url_prod')`. The prefix is configurable via `kestra.variables.env-vars-prefix` in the [runtime configuration](../../../configuration/02.runtime-and-storage/index.md).
+
+The key difference from `{{ envs.api_url_prod }}` is that the name argument accepts any expression, allowing the key to be composed dynamically:
+
+```yaml
+inputs:
+ - id: environment
+ type: STRING
+ defaults: prod
+
+tasks:
+ - id: log_url
+ type: io.kestra.plugin.core.log.Log
+ message: "{{ env('api_url_' ~ inputs.environment) }}"
+```
+
+With `ENV_API_URL_PROD` and `ENV_API_URL_DEV` set in the host environment, running the flow with `environment` set to `prod` resolves to the production URL and `dev` to the development URL — without any `if`/`switch` logic in the flow.
+
## `credential()`
In Enterprise Edition, use `credential()` to inject a short-lived token from a managed credential:
@@ -59,6 +114,15 @@ tasks:
`read()` accepts both namespace files and internal-storage URIs, which makes it useful after download or transformation tasks that write files as outputs.
+For files in CSV, JSON, XML, YAML, or any other non-ION format, `read()` returns a `String`. For **binary ION files** — the format used by ION task outputs — `read()` returns `byte[]` to preserve binary fidelity. Pass the result directly to `fromIon()` to deserialize it:
+
+```twig
+{{ fromIon(read(outputs.transform.uri)) }}
+{{ fromIon(read(outputs.transform.uri), allRows=true) }}
+```
+
+Do not perform string operations such as `contains` directly on `read()` when the URI points to an ION task output — use `fromIon()` first.
+
## `fileURI()`
Returns the internal URI of a namespace file without reading its contents. Use `fileURI()` when a task parameter expects a URI rather than inline content:
diff --git a/src/contents/docs/expressions/04.functions/03.parsing/index.mdx b/src/contents/docs/expressions/04.functions/03.parsing/index.mdx
index be4f86be31a..fad9ea5f368 100644
--- a/src/contents/docs/expressions/04.functions/03.parsing/index.mdx
+++ b/src/contents/docs/expressions/04.functions/03.parsing/index.mdx
@@ -21,12 +21,26 @@ Use `fromJson()` when a task output arrives as a serialized JSON string rather t
## `fromIon()`
-Use `fromIon()` when a previous task or serializer produces Ion rather than JSON:
+Use `fromIon()` when a previous task or serializer produces ION rather than JSON. `fromIon()` accepts both text ION strings and binary ION byte arrays, so it works correctly with `read()` regardless of the storage format.
+
+Access a field from the first row:
```twig
{{ fromIon(read(outputs.serialize.uri)).someField }}
```
+Return all rows as a list with `allRows=true`:
+
+```twig
+{{ fromIon(read(outputs.transform.uri), allRows=true) }}
+```
+
+Count rows:
+
+```twig
+{{ fromIon(read(outputs.query.uri), allRows=true) | length }}
+```
+
## `yaml()`
Parses a YAML string into an object so you can access its fields with dot or array notation:
diff --git a/src/contents/docs/expressions/04.functions/04.workflow/index.mdx b/src/contents/docs/expressions/04.functions/04.workflow/index.mdx
index 4e12492d093..a897fd6578f 100644
--- a/src/contents/docs/expressions/04.functions/04.workflow/index.mdx
+++ b/src/contents/docs/expressions/04.functions/04.workflow/index.mdx
@@ -1,59 +1,60 @@
---
title: "Workflow Helper Functions in Kestra Expressions"
h1: "Workflow Helper Functions"
-description: Reference for Kestra's workflow and execution helper functions — errorLogs(), currentEachOutput(), tasksWithState(), iterationOutput(), parentOutput(), and appLink().
+description: Reference for Kestra's workflow and execution helper functions — loopOutputs(), errorLogs(), tasksWithState(), and appLink().
sidebarTitle: Workflow Functions
icon: /src/contents/docs/icons/expression.svg
---
This group is more situational, but it becomes valuable in complex flows where you need to inspect sibling results, build links back into Kestra, or summarize failures.
-## `errorLogs()`
+## `loopOutputs()`
-Prints all error logs from the current execution:
+Extracts a named output from every iteration of a [Loop](../../../05.workflow-components/01.tasks/00.flowable-tasks/index.md#loop) task and returns the values as an ordered list.
+
+**Prerequisite**: the Loop task must declare an `outputs:` block. Without it, `outputs.loop.outputs` does not exist and this function will fail.
```twig
-{{ errorLogs() }}
+{{ loopOutputs(outputs.myLoop.outputs, 'result') }}
```
-It is most useful in `errors` blocks, where you need a compact summary of what failed without manually traversing task state objects.
-
-## `currentEachOutput()`
+The first argument must be the loop task's `.outputs` list. The second argument is the output ID — it must match an output ID declared in the Loop task's `outputs:` block. The function returns one value per iteration in order, with `null` for iterations where that key is missing.
-Use it inside `ForEach` flows to avoid manual `taskrun.value` indexing:
+For a `Loop` over `["a", "b", "c"]` with a declared output `result`:
```twig
-{{ currentEachOutput(outputs.make_data).values.data }}
+{{ loopOutputs(outputs.loop.outputs, 'result') }}
+{# → ["processed a", "processed b", "processed c"] #}
```
-## `tasksWithState()`
-
-Returns a list of task run objects matching the given state. Use it in error handlers or notifications to report which tasks failed:
+To access a single iteration directly, use list index notation:
```twig
-{{ tasksWithState('FAILED') }}
+{{ outputs.loop.outputs[0].outputs.result }} {# first iteration's declared output #}
+{{ outputs.loop.outputs[0].item.value }} {# first iteration's input value #}
+{{ outputs.loop.outputs[0].item.iteration }} {# first iteration's zero-based index #}
```
-Useful for building conditional logic or failure summaries based on task outcomes.
-
-## `iterationOutput()`
+## `errorLogs()`
-Retrieves the output of a specific iteration from a previous task. Both arguments are optional — `taskId` defaults to the current task and `iteration` defaults to the previous iteration:
+Prints all error logs from the current execution:
```twig
-{{ iterationOutput(outputs.myTask).value }}
-{{ iterationOutput(outputs.myTask, 2).value }}
+{{ errorLogs() }}
```
-## `parentOutput()`
+It is most useful in `errors` blocks, where you need a compact summary of what failed without manually traversing task state objects.
-Retrieves the output of a parent task. The optional `index` argument specifies which ancestor to target; omitting it returns the direct parent's output:
+## `tasksWithState()`
+
+Returns a list of task run objects matching the given state. Use it in error handlers or notifications to report which tasks failed:
```twig
-{{ parentOutput() }}
-{{ parentOutput(1) }}
+{{ tasksWithState('FAILED') }}
```
+Useful for building conditional logic or failure summaries based on task outcomes.
+
## `subflow()`
Synchronously runs a subflow and returns its terminal execution result, so you can read the subflow's outputs, state, or labels from within an expression.
@@ -96,6 +97,7 @@ When a user opens the Execute form, Kestra runs the subflow synchronously, reads
**Important constraints:**
- Only valid in an input `expression:` context. Using `subflow()` inside a task or trigger property throws an error, because blocking a worker thread while waiting for a child execution can deadlock a worker under load.
+- Flows referenced by `subflow()` in a `SELECT` or `MULTISELECT` input expression appear in the parent flow's **Dependencies** graph, the same way a `Subflow` task reference does.
- Only available on `WEBSERVER` and `STANDALONE` server types. The function is not registered on other server types.
- The default timeout is `PT1M`. The hard cap is `PT5M` — passing a larger `timeout` value is rejected at runtime. Both limits are configurable; see [configuration reference](../../../configuration/04.plugins-and-execution/index.md#subflow-function-configuration).
- Subflow recursion depth is capped at 3. A subflow whose own inputs call `subflow()` counts against this limit.
diff --git a/src/contents/docs/expressions/04.functions/06.dates/index.mdx b/src/contents/docs/expressions/04.functions/06.dates/index.mdx
index 96c3b19f477..cc19500c8a2 100644
--- a/src/contents/docs/expressions/04.functions/06.dates/index.mdx
+++ b/src/contents/docs/expressions/04.functions/06.dates/index.mdx
@@ -33,6 +33,17 @@ Returns `true` if the date is the Nth occurrence of the given weekday in its mon
{{ isDayWeekInMonth(trigger.date, 'MONDAY', 'FIRST') }}
```
+## `isLastWorkingDay()`
+
+Returns `true` if the date is the last working day of its month. Working days default to Monday–Friday. An optional second argument overrides which days count as working days using a comma-separated list of uppercase day names:
+
+```twig
+{{ isLastWorkingDay(trigger.date) }}
+{{ isLastWorkingDay(trigger.date, 'MONDAY,TUESDAY,WEDNESDAY,THURSDAY') }}
+```
+
+The `date` argument accepts any ISO 8601 date or datetime string. Combine with `isPublicHoliday()` if you also need to exclude public holidays.
+
## `dayOfWeek()`
Returns the uppercase day name such as `MONDAY`:
diff --git a/src/contents/docs/expressions/04.functions/index.mdx b/src/contents/docs/expressions/04.functions/index.mdx
index 98620e16cf2..54697ea4c75 100644
--- a/src/contents/docs/expressions/04.functions/index.mdx
+++ b/src/contents/docs/expressions/04.functions/index.mdx
@@ -15,9 +15,9 @@ Functions are best thought of as helpers that either fetch something, compute so
## Function groups
- [Rendering and debugging](./01.rendering/index.mdx) — `render()`, `renderOnce()`, `fetchContext()`, template inheritance helpers
-- [Data access](./02.data-access/index.mdx) — `secret()`, `credential()`, `read()`, `fileURI()`, `kv()`, `encrypt()`, `decrypt()`
+- [Data access](./02.data-access/index.mdx) — `secret()`, `env()`, `credential()`, `read()`, `fileURI()`, `kv()`, `encrypt()`, `decrypt()`
- [Data parsing](./03.parsing/index.mdx) — `fromJson()`, `fromIon()`, `yaml()`
-- [Workflow helpers](./04.workflow/index.mdx) — `errorLogs()`, `currentEachOutput()`, `tasksWithState()`, `iterationOutput()`, `parentOutput()`, `appLink()`
+- [Workflow helpers](./04.workflow/index.mdx) — `errorLogs()`, `tasksWithState()`, `loopOutputs()`, `appLink()`
- [Utilities](./05.utilities/index.mdx) — `now()`, `uuid()`, `randomInt()`, `http()`, `fileSize()`, `fileExists()`, and more
- [Date and calendar](./06.dates/index.mdx) — `isWeekend()`, `isPublicHoliday()`, `dayOfWeek()`, `monthOfYear()`, and more
diff --git a/src/contents/docs/icons/clevercloud.svg b/src/contents/docs/icons/clevercloud.svg
new file mode 100644
index 00000000000..94f832c24fb
--- /dev/null
+++ b/src/contents/docs/icons/clevercloud.svg
@@ -0,0 +1,11 @@
+
+
diff --git a/src/contents/docs/index.mdx b/src/contents/docs/index.mdx
index 74286236f61..b67ed2c2a86 100644
--- a/src/contents/docs/index.mdx
+++ b/src/contents/docs/index.mdx
@@ -20,6 +20,7 @@ Thanks to plugins and an embedded code editor with G
`).
- `--user` — Basic auth (`user:password`).
-- `--tenant` — Tenant identifier (**EE only**).
-- `--api-token` — API token (**EE only**).
+- `--tenant` — Tenant identifier (**EE and Cloud only**).
+- `--api-token` — API token (**EE and Cloud only**).
**Examples**
@@ -88,16 +88,13 @@ Commands:
plugins handle plugins
server handle servers
flow handle flows
- template handle templates
sys handle systems maintenance
configs handle configs
namespace handle namespaces
auths handle auths
- sys-ee handle kestra ee systems maintenance
tenants handle tenants
migrate handle migrations
backups (EE) handle metadata backups and restore
- server start Kestra servers (see `--flow-path` below for preloading flows)
```
### Preload flows at startup
@@ -216,9 +213,50 @@ kestra flow delete my-namespace my-flow-id
## Migration commands
+### `kestra migrate plan`
+
+Lists all pending database migrations without applying them. Read-only: acquires no lock, writes nothing.
+
+**Options**: `--sql` (print the raw SQL for each SQL-based migration)
+
+```bash
+kestra migrate plan
+kestra migrate plan --sql
+```
+
+---
+
+### `kestra migrate run`
+
+Applies all pending migrations in lexicographic order. Acquires a distributed lock so only one process migrates at a time. Makes a single non-blocking lock attempt; if the lock is already held, exits immediately with code `1`.
+
+```bash
+kestra migrate run
+```
+
+:::alert{type="info"}
+Enterprise Edition users must run this command manually before starting Kestra 2.0 for the first time. By default (`kestra.migration.auto=false`), Kestra EE refuses to start if any pending migrations exist. Open-source Kestra runs migrations automatically on startup.
+:::
+
+---
+
+### `kestra migrate unlock`
+
+Force-releases the migration lock. Use only when `kestra migrate run` exited abnormally and left the lock held.
+
+```bash
+kestra migrate unlock
+```
+
+:::alert{type="warning"}
+On **PostgreSQL, MySQL, and H2**, the lock is session-scoped. `kestra migrate unlock` always exits `0` but does nothing on these backends. The lock releases when the holding process terminates. Kill the hung process instead. On **Elasticsearch**, the command works as expected.
+:::
+
+---
+
### `kestra migrate default-tenant`
-Migrate all resources without tenant to a new tenant (multi-tenant setups).
+Migrate all resources without a tenant to a new tenant (multi-tenant setups).
**Options**: `--tenant-id`, `--tenant-name`, `--dry-run`
@@ -354,7 +392,7 @@ kestra server webserver --no-tutorials
Start a worker.
-**Options**: `-t, --thread` (max threads), `-g, --worker-group` (EE only)
+**Options**: `-t, --thread` (max threads)
```bash
kestra server worker --thread 16
@@ -370,7 +408,7 @@ kestra server local
## Kestra with server components in different services
-Server components can run independently from each other. Each of them communicate through the database.
+Server components run independently from each other. Most communicate through the queue layer; Workers communicate with the Worker Controller over a bidirectional gRPC stream and never connect to the database directly.
Below is an example Docker Compose configuration file running Kestra services with replicas on the PostgreSQL database backend.
@@ -552,10 +590,14 @@ kestra sys state-store migrate
Create a user.
**Inputs**: `username` (required), `password` (optional)
-**Options**: `--groups`, `--tenant`, `--admin`, `--superadmin`, `--if-not-exists`
+**Options**: `--groups`, `--tenant`, `--admin`, `--instance-owner`, `--if-not-exists`
+
+:::alert{type="info"}
+`--superadmin` is a deprecated alias for `--instance-owner` and still works.
+:::
```bash
-kestra auths users create --superadmin --tenant=default admin Admin_password@123
+kestra auths users create --instance-owner --tenant=default admin Admin_password@123
```
### `kestra auths users create-basic-auth`
@@ -574,16 +616,20 @@ Refresh users to update their properties.
kestra auths users refresh
```
-### `kestra auths users set-superadmin`
+### `kestra auths users set-instance-owner`
-Set or remove Superadmin status.
+Set or remove Instance Owner status.
-**Inputs**: `user`, `isSuperAdmin` (true|false)
+**Inputs**: `user`, `instanceOwner` (true|false)
```bash
-kestra auths users set-superadmin alice true
+kestra auths users set-instance-owner alice true
```
+:::alert{type="info"}
+`set-superadmin` is a deprecated alias for `set-instance-owner` and still works.
+:::
+
### `kestra auths users email-replace-username`
Set the username as the email for every user.
@@ -609,7 +655,7 @@ kestra auths users sync-access
Create a metadata backup.
**Inputs**: `type` (`FULL` | `TENANT`)
-**Options**: `--tenant`, `--encryption-key`, `--no-encryption`, `--include-data`
+**Options**: `--tenant`, `--encryption-key`, `--no-encryption`, `--include-data`, `--resources`
```bash
kestra backups create FULL --no-encryption
@@ -620,63 +666,12 @@ kestra backups create FULL --no-encryption
Restore a metadata backup.
**Input**: `uri` (Kestra internal storage URI)
-**Options**: `--encryption-key`, `--to-tenant`
+**Options**: `--encryption-key`, `--to-tenant`, `--resources`
```bash
kestra backups restore kestra:///backups/full/backup-20240917163312.kestra
```
----
-
-## Systems (EE)
-
-### kestra sys-ee restore-flow-listeners
-
-Restores the state-store for FlowListeners. Useful after restoring a flow queue.
-
-**Inputs**
-
-- `--timeout` (option): Timeout in seconds before quitting (default: 60).
-
-**Example Usage**
-
-```bash
-kestra-ee sys-ee restore-flow-listeners --timeout 120
-```
-
----
-
-### kestra sys-ee restore-queue
-
-Sends all data from a repository to Kafka. Useful for restoring all resources after a backup.
-
-**Inputs**
-
-- `--no-recreate` (option): Don't drop and recreate the Kafka topic.
-- `--no-flows` (option): Don't send flows.
-- `--no-templates` (option): Don't send templates.
-
-**Example Usage**
-
-```bash
-kestra-ee sys-ee restore-queue --no-flows
-```
-
----
-
-### kestra sys-ee reset-concurrency-limit
-
-Resets the concurrency limit stored on the Kafka runner.
-
-**Inputs**
-None
-
-**Example Usage**
-
-```bash
-kestra-ee sys-ee reset-concurrency-limit
-```
-
## Tenants (EE)
### `kestra tenants create`
diff --git a/src/contents/docs/kestra-cli/kestractl/index.md b/src/contents/docs/kestra-cli/kestractl/index.md
index 2ef828612d8..0ac13a5ebbd 100644
--- a/src/contents/docs/kestra-cli/kestractl/index.md
+++ b/src/contents/docs/kestra-cli/kestractl/index.md
@@ -74,7 +74,7 @@ kestractl flows list my.namespace --output json
- `executions`: run, inspect, control, and bulk-manage executions.
- `triggers`: list, enable, disable, unlock, delete, manage backfills, and export triggers.
- `namespaces`: list, get, create, update, delete namespaces and manage their plugin defaults.
-- `nsfiles`: list, get, upload, and delete namespace files.
+- `nsfiles`: list, get, upload, search, move, export, manage revisions, and delete namespace files.
- `kv`: list, set (with optional TTL), update, get, and delete key-value pairs. Note: `kv list` requires token auth and returns 401 with basic auth.
- `dashboards`: list, get, create, update, and delete dashboards. Requires Kestra EE.
- `apps`: list, get, deploy, update, and delete apps. Requires Kestra EE.
@@ -403,6 +403,37 @@ kestractl namespaces import-plugin-defaults my.namespace --file defaults.yaml
`--variable` and `--variables-file` replace the namespace's full variable set on each `create` or `update` call. To preserve existing variables, include them in the file or repeat them as `--variable` flags.
:::
+## Namespace files
+
+```bash
+# List files in a namespace
+kestractl nsfiles list my.namespace
+kestractl nsfiles list my.namespace --output json
+
+# Get the content of a specific file
+kestractl nsfiles get my.namespace workflows/my-flow.yml
+
+# Upload files or directories
+kestractl nsfiles upload my.namespace ./assets --path resources --override --fail-fast
+
+# Search files in a namespace
+kestractl nsfiles search my.namespace
+kestractl nsfiles search my.namespace --query flow --output json
+
+# List revisions of a specific file (--path is required)
+kestractl nsfiles revisions my.namespace --path workflows/my-flow.yml
+
+# Move or rename a file or directory
+kestractl nsfiles move my.namespace workflows/old-name.yml workflows/new-name.yml
+kestractl nsfiles move my.namespace old-dir/ new-dir/
+
+# Export all namespace files as a ZIP archive
+kestractl nsfiles export my.namespace --output-file my-namespace.zip
+
+# Delete a file
+kestractl nsfiles delete my.namespace workflows/my-flow.yml
+```
+
## Key-value pairs
```bash
@@ -468,6 +499,9 @@ kestractl dashboards update --file my-dashboard.yaml
# Delete a dashboard
kestractl dashboards delete
+# Show which dashboards are configured as defaults for the home, flow overview, and namespace overview views
+kestractl dashboards defaults
+
# Validate a dashboard or chart definition
kestractl dashboards validate --file my-dashboard.yaml
kestractl dashboards validate-chart --file my-chart.yaml
@@ -954,6 +988,34 @@ Use `--output json` for full plugin metadata (groupId, artifactId, license, vers
| `--from-config` | — | Derive required core plugins from one or more config files (see below) |
| `--output` | `table` | Output format: `table` (space-separated coordinates) or `json` |
+### `kestractl plugins get `
+
+Download a single plugin JAR by its Maven coordinates (`groupId:artifactId:version`) into `--plugins-dir`, without pulling the full compatibility set for a version.
+
+```bash
+kestractl plugins get io.kestra.plugin:plugin-kafka:1.6.0
+```
+
+| Flag | Default | Description |
+|---|---|---|
+| `--plugins-dir` | `./plugins` | Directory to write the downloaded JAR into |
+| `--force-redownload` | `false` | Re-download even if the JAR already exists |
+| `--global-timeout` | `5m` | Maximum time allowed for the download |
+| `--maven-repository` | Maven Central | Custom Maven repository base URL |
+| `--maven-username` | — | Username for Maven basic authentication |
+| `--maven-password` | — | Password for Maven basic authentication |
+
+If the JAR already exists on disk, the download is skipped unless `--force-redownload` is set. For new downloads, the JAR is verified against its published SHA-1 checksum when available. The version must be an exact version number — symbolic aliases like `latest` and `develop` are not supported and produce an error.
+
+Use `--maven-repository`, `--maven-username`, and `--maven-password` to pull from a private registry, or the global `--header` flag for bearer token authentication. For example, to install an Enterprise Edition plugin from a private registry:
+
+```bash
+kestractl plugins get io.kestra.ee:ee-plugin:2.0.0 \
+ --maven-repository https://registry.kestra.io/maven \
+ --maven-username myuser \
+ --maven-password mypassword
+```
+
### `kestractl plugins download [version]`
Download plugins to a local directory. By default, all compatible plugins for the given version are downloaded from Maven Central.
@@ -1017,7 +1079,7 @@ Bundled backends produce no output. Only backends that ship as a separate plugin
```bash
kestractl plugins list 2.0.0 --from-config /etc/kestra/application.yaml
-# → io.kestra.storage:storage-s3:1.4.1
+# → io.kestra.storage:storage-s3:1.4.1 io.kestra.secret:secret-aws-secret-manager:1.4.1
```
If all configured backends are bundled, the command exits cleanly with:
@@ -1079,6 +1141,7 @@ When deploying a standalone worker, you also need to download the core infrastru
- `--username` - Basic auth username (Open Source)
- `--password` - Basic auth password (Open Source)
- `--output` / `-o` - Output format (`table` or `json`)
+- `--header` - Extra HTTP header to include in all requests, in `Key:Value` format (repeatable). Use for bearer token authentication against a private registry: `--header "Authorization:Bearer "`
- `--config` - Custom config file path (default: `~/.kestractl/config.yaml`)
- `--verbose` / `-v` - Verbose output (warning: prints credentials in HTTP requests)
diff --git a/src/contents/docs/oss-vs-paid/index.md b/src/contents/docs/oss-vs-paid/index.md
index 7ff3d2afbad..43b613417a0 100644
--- a/src/contents/docs/oss-vs-paid/index.md
+++ b/src/contents/docs/oss-vs-paid/index.md
@@ -8,73 +8,77 @@ Understand the differences between Kestra's Open-Source and Enterprise Editions,
## Choose the right Kestra edition
-Kestra's Open-Source Edition provides a foundation for workflow automation — it's best suited for solo-developers or small teams exploring workflow orchestration.
+Kestra's Open-Source Edition is a fully functional orchestration platform suitable for individuals, teams, and production deployments that don't require enterprise security, multi-tenancy, or compliance controls.
-The [Enterprise Edition](../07.enterprise/index.mdx) adds enterprise-grade security, scalability, and governance features required by organizations managing complex workflows across multiple teams or environments. It includes advanced authentication and access controls with SSO, SCIM & RBAC, multi-tenancy, high availability, dedicated secrets manager and storage backends per team, dedicated worker groups or on-demand remote task runners, audit logs, service accounts, apps, revision history for every resource, maintenance mode, log shipper, cluster monitoring, backup and restore, dedicated support with SLAs, plus newer safeguards like assets packaging, versioned plugins, read-only secrets, plugin allow-listing, worker isolation, and built-in flow unit tests. In short, everything you need for production deployments with strict compliance or reliability requirements is available in the Enterprise Edition.
+The [Enterprise Edition](../07.enterprise/index.mdx) adds enterprise-grade security, scalability, and governance for organizations managing complex workflows across multiple teams or environments: SSO, SCIM, and RBAC for access control; multi-tenancy and worker isolation for separation; high availability and dedicated worker groups for scale; audit logs, log shipper, and cluster monitoring for observability; asset lineage, versioned plugins, read-only secrets, and allowed-plugin lists for governance; apps, custom blueprints, and policies for productivity; and dedicated support with SLAs. Everything you need for production deployments with strict compliance or reliability requirements.
---
-## Security and Access Control
+## Security and access control
-The Open-Source Edition supports basic authentication, suitable for one-person projects or small teams with shared credentials. In contrast, the Enterprise Edition has an easy way to add collaborators via [invitations](../07.enterprise/03.auth/invitations/index.md) and manage permissions at scale using [SCIM Directory Sync](../07.enterprise/03.auth/scim/index.mdx). It integrates with many identity providers via [Single Sign-On (SSO)](../07.enterprise/03.auth/sso/index.md) and **OpenID Connect (OIDC)**, simplifying user management for large teams.
+The Open-Source Edition supports basic authentication, suitable for one-person projects or small teams with shared credentials. In contrast, the Enterprise Edition has an easy way to add collaborators via [invitations](../07.enterprise/03.auth/invitations/index.md) and manage permissions at scale using [SCIM Directory Sync](../07.enterprise/03.auth/scim/index.mdx). It integrates with many identity providers via [Single Sign-On (SSO)](../07.enterprise/03.auth/sso/index.md) and OpenID Connect (OIDC), simplifying user management for large teams.
-[Role-Based Access Control (RBAC)](../07.enterprise/03.auth/rbac/index.md) lets you define granular permissions at user, group and namespace level, e.g. restricting developer access to specific namespaces while granting auditors read-only access. [Namespace-level secrets management](../07.enterprise/02.governance/secrets-manager/index.md) ensures that sensitive credentials stay isolated between projects. [Service accounts](../07.enterprise/03.auth/service-accounts/index.md) and [API tokens](../07.enterprise/03.auth/api-tokens/index.md) enable secure automation, such as [CI/CD pipelines](../version-control-cicd/index.mdx) deploying workflows without requiring user credentials.
+[Role-Based Access Control (RBAC)](../07.enterprise/03.auth/rbac/index.md) uses a resource and action model — `FLOW: EXECUTE`, `EXECUTION: ACCESS_LOGS`, `TRIGGER: BACKFILL` — so you can grant exactly what a user needs without over-provisioning. Define permissions at user, group, and namespace level, e.g. restricting developer access to specific namespaces while granting auditors read-only access. [Namespace-level secrets management](../07.enterprise/02.governance/secrets-manager/index.md) ensures that sensitive credentials stay isolated between projects. [Service accounts](../07.enterprise/03.auth/service-accounts/index.md) and [API tokens](../07.enterprise/03.auth/api-tokens/index.md) enable secure automation, such as [CI/CD pipelines](../version-control-cicd/index.mdx) deploying workflows without requiring user credentials.
For organizations using external [secrets managers](../07.enterprise/02.governance/secrets-manager/index.md) such as Azure Key Vault or HashiCorp Vault, Enterprise Edition integrates directly with these systems. [SCIM directory sync](../07.enterprise/03.auth/scim/index.mdx) automates user (de)provisioning at scale, reducing administrative overhead when onboarding or offboarding team members.
-Enterprise-only safeguards include [read-only secrets](../07.enterprise/02.governance/secrets-manager/index.md) for least-privilege access and [allowed plugins](../07.enterprise/02.governance/allowed-plugins/index.md) to centrally control which plugins may run.
+Enterprise-only safeguards include [read-only secrets](../07.enterprise/02.governance/secrets-manager/index.md) for least-privilege access, [allowed plugins](../07.enterprise/02.governance/allowed-plugins/index.md) to centrally control which plugins may run, and [Policies](../07.enterprise/02.governance/policies/index.md) to inject or enforce plugin configuration across namespaces without modifying flow YAML.
---
-## Governance and Compliance
+## Governance and compliance
Enterprise Edition provides [audit logs](../07.enterprise/02.governance/06.audit-logs/index.md) that track every user action and resource change, which are critical in highly regulated industries. Logs can be automatically exported to observability platforms such as Datadog or Elasticsearch using the [Log Shipper](../07.enterprise/02.governance/logshipper/index.md).
[Multi-tenancy](../07.enterprise/02.governance/tenants/index.md) allows you to create fully isolated environments, e.g. separate tenants for specific [teams or business units](../14.best-practices/8.business-unit-separation/index.md). Each tenant can use separate secrets managers or dedicated internal storage backends (e.g., AWS S3 for Tenant A, GCS for Tenant B).
-[Worker Group](../07.enterprise/04.scalability/worker-group/index.md) ensures tasks from different tenants run on separate infrastructure, reducing the risk of resource contention or cross-tenant breaches. [Worker Isolation](../07.enterprise/02.governance/worker-isolation/index.md) adds hard isolation policies when you need stricter separation. Encryption safeguards data at rest and in transit, meeting regulatory standards.
+[Worker Groups](../07.enterprise/04.scalability/worker-group/index.md) ensure tasks from different tenants run on separate infrastructure, reducing the risk of resource contention or cross-tenant breaches. [Worker Isolation](../07.enterprise/02.governance/worker-isolation/index.md) adds hard isolation policies when you need stricter separation. Encryption safeguards data at rest and in transit, meeting regulatory standards.
---
-## Scalability and Reliability
+## Scalability and reliability
-The Open-Source Edition runs by default on a single server, which can become a bottleneck for large workloads. Enterprise Edition can use Kafka and Elasticsearch for distributed event processing, enabling horizontal scaling and high throughput. High Availability (HA) architecture eliminates single points of failure — if a worker node fails, tasks automatically reroute to healthy nodes.
+The Open-Source Edition runs by default on a single server, which can become a bottleneck for large workloads. Enterprise Edition can use Kafka (paired with Elasticsearch for the search and read model), Redis, AMQP, or GCP Pub/Sub as the queue backend, enabling horizontal scaling and high throughput. High Availability (HA) architecture eliminates single points of failure — if a worker node fails, tasks automatically reroute to healthy nodes.
-[Worker Groups](../07.enterprise/04.scalability/worker-group/index.md) let you assign tasks to specialized infrastructure. For example, GPU-heavy machine learning workflows can target a worker group with NVIDIA GPUs, while ETL jobs run on cost-optimized spot instances. [Task Runners](../07.enterprise/04.scalability/task-runners/index.md) offload compute-intensive scripts on-demand to Kubernetes or cloud batch services such as Azure Batch, Google Cloud Run or AWS ECS Fargate to prevent resource contention and making it easy to scale in a cost-effective way.
+[Worker Groups](../07.enterprise/04.scalability/worker-group/index.md) let you assign tasks to specialized infrastructure. For example, GPU-heavy machine learning workflows can target a worker group with NVIDIA GPUs, while ETL jobs run on cost-optimized spot instances. [Task Runners](../07.enterprise/04.scalability/task-runners/index.md) offload compute-intensive scripts on-demand to Kubernetes or cloud batch services such as Azure Batch, Google Cloud Run, or AWS ECS Fargate, preventing resource contention and scaling costs with usage.
:::alert{type="info"}
-Please note that Worker Groups are not yet available in Kestra Cloud, only in Kestra Enterprise Edition.
+Worker Groups are available in Kestra Enterprise Edition only, not in Kestra Cloud.
:::
-[Maintenance Mode](../07.enterprise/05.instance/maintenance-mode/index.md) allows safe upgrades: new executions queue while in-progress tasks complete gracefully, avoiding abrupt workflow termination. [Cluster monitoring](../07.enterprise/05.instance/index.mdx) provides real-time visibility into resource usage, helping teams proactively address infrastructure bottlenecks. Additionally, using **Custom Dashboards**, you can create custom views to track specific metrics, logs, or executions. The [Backup and Restore](../10.administrator-guide/backup-and-restore/index.md) eliminates the risk of data loss or corruption during upgrades, allowing you to recover from accidental deletions or system failures.
+The [External Log Data Store](../10.administrator-guide/log-data-store/index.md) (EE) routes execution logs to a dedicated JDBC database or Elasticsearch, separate from the main backend, keeping the main database lean and reducing schema migration time.
+
+[Maintenance Mode](../07.enterprise/05.instance/maintenance-mode/index.md) allows safe upgrades: new executions queue while in-progress tasks complete gracefully, avoiding abrupt workflow termination. [Cluster monitoring](../07.enterprise/05.instance/index.mdx) provides real-time visibility into resource usage, helping teams proactively address infrastructure bottlenecks. **Custom Dashboards** let you create custom views to track specific metrics, logs, or executions. [Backup and Restore](../10.administrator-guide/backup-and-restore/index.md) lets you recover from accidental deletions, data corruption, or failed upgrades.
[Versioned Plugins](../07.enterprise/05.instance/versioned-plugins/index.md) let you pin plugin versions per environment for safe rollouts, while the [Kill Switch](../07.enterprise/05.instance/kill-switch/index.md) can pause risky changes instantly. [Announcements](../07.enterprise/05.instance/announcements/index.md) provide in-product notifications for maintenance or policy updates.
---
-## Productivity and Collaboration
+## Productivity and collaboration
-[Custom Blueprints](../07.enterprise/02.governance/custom-blueprints/index.md) act as reusable workflow templates, e.g. a **standardized** data ingestion pattern that all teams can consistently adopt. **Full-text search across task runs** speeds up navigation — e.g. engineers can quickly find logs for a failed Python script without manually filtering through thousands of executions.
+[Custom Blueprints](../07.enterprise/02.governance/custom-blueprints/index.md) act as reusable workflow templates, e.g. a standardized data ingestion pattern that all teams can consistently adopt. Templated Blueprints extend this further: non-technical users fill in a form and Kestra generates the flow YAML — no YAML editing required. Blueprint libraries can be version-controlled with Git using `PushBlueprints` and `SyncBlueprints`, the same GitOps pattern as flows. **Full-text search across task runs** speeds up navigation — e.g. engineers can quickly find logs for a failed Python script without manually filtering through thousands of executions.
-**Centralized** namespace-level [plugin defaults](../07.enterprise/02.governance/07.namespace-management/index.md) simplify configuration. A [namespace-wide setting](../07.enterprise/02.governance/07.namespace-management/index.md) on a root namespace might **define AWS credentials** that will be automatically inherited, and optionally also enforced, by all child namespaces, eliminating redundant code and allowing admins to centrally govern secrets and plugin configurations.
+[Policies](../07.enterprise/02.governance/policies/index.md) let administrators inject, validate, and enforce plugin configuration across namespaces without touching individual flow YAML. A policy can automatically add credentials to every task of a given plugin type, block flows that use disallowed plugins, or enforce required properties at the namespace level — all without requiring developers to change their flows.
-**Impersonation** lets admins validate permissions by temporarily assuming a user’s role, which significantly helps with troubleshooting access management issues.
+**Impersonation** lets admins assume a user’s role temporarily to validate permissions and troubleshoot access issues.
[Apps](../07.enterprise/04.scalability/apps/index.md) turn workflows into user-friendly interfaces. A finance team can build a self-service tool for expense approvals, where non-technical stakeholders can submit requests via a form. Approved requests automatically trigger downstream tasks to process payments.
-[Assets](../07.enterprise/02.governance/01.assets/index.md) package reusable files and artifacts alongside flows, and [Unit Tests](../07.enterprise/02.governance/unit-tests/index.md) let teams validate flows early to prevent regressions.
+[Reusable Inputs](../05.workflow-components/22.reusable-inputs/index.md) let teams define shared input groups once at the namespace level and reference them across flows — no repeated declarations, and updates propagate automatically on the next execution.
+
+[Asset Lineage](../07.enterprise/02.governance/01.assets/index.md) tracks data dependencies across flows, and [Unit Tests](../07.enterprise/02.governance/unit-tests/index.md) let teams validate flows early to prevent regressions.
---
-## Support and Services
+## Support and services
-Enterprise Edition includes **SLAs with guaranteed response times** for support tickets, which is critical for teams running 24/7-operations. Onboarding support helps customize Kestra to your stack and deployment requirements.
+Enterprise Edition includes **SLAs with guaranteed response times** for support tickets, which is critical for teams running 24/7 operations. Onboarding support helps customize Kestra to your stack and deployment requirements.
Customers’ feature requests are prioritized over those from open-source users. They also get early access to beta features and roadmap previews, allowing teams to plan upgrades around upcoming capabilities. The dedicated customer portal provides direct access to Kestra’s engineering team for architecture reviews or best practices.
---
-## When to Choose Enterprise Edition
+## When to choose Enterprise Edition
**Stick with Open-Source if:**
- You’re a solo developer
@@ -90,6 +94,6 @@ Customers’ feature requests are prioritized over those from open-source users.
---
-## How Upgrading Works
+## How upgrading works
Switching to Enterprise involves adding a license key to your configuration and restarting Kestra — no code changes required. All existing workflows and plugins remain compatible. For hybrid setups, you can run Open-Source and Enterprise instances side-by-side during transition periods.
diff --git a/src/contents/docs/performance/benchmark/index.md b/src/contents/docs/performance/benchmark/index.md
index 0a955805053..b10fc567ef1 100644
--- a/src/contents/docs/performance/benchmark/index.md
+++ b/src/contents/docs/performance/benchmark/index.md
@@ -6,6 +6,10 @@ icon: /src/contents/docs/icons/admin.svg
description: View performance benchmarks for Kestra's orchestration throughput and latency across Open Source and Enterprise editions.
---
+:::alert{type="info"}
+These benchmarks were measured on Kestra 1.3. Updated results for Kestra 2.0 will be published after release.
+:::
+
Kestra is an orchestration platform: you define a flow, and Kestra orchestrates it.
Flows can range from lightweight tasks running in milliseconds to complex scripts in containers that run for tens of minutes.
@@ -188,7 +192,7 @@ tasks:
**Results for Kestra EE**
-
+
| Executions(per minutes) | Tasks (per minutes) | Execution Latency (in seconds) |
|:-------------------------|:--------------------|:--------------------------------|
@@ -331,7 +335,7 @@ With 1.6 KB small-sized messages:
With 16 KB medium-sized messages:
-
+
| Executions(per minutes) | Tasks (per minutes) | Execution Latency (in seconds) |
|:-------------------------|:--------------------|:--------------------------------|
@@ -347,7 +351,7 @@ With 16 KB medium-sized messages:
With 160 KB large-sized messages:
-
+
| Executions(per minutes) | Tasks (per minutes) | Execution Latency (in seconds) |
|:-------------------------|:--------------------|:--------------------------------|
diff --git a/src/contents/docs/performance/performance-tuning/index.md b/src/contents/docs/performance/performance-tuning/index.md
index 3fc7edb9793..8deb78ea58e 100644
--- a/src/contents/docs/performance/performance-tuning/index.md
+++ b/src/contents/docs/performance/performance-tuning/index.md
@@ -6,6 +6,10 @@ icon: /src/contents/docs/icons/admin.svg
description: Tips and best practices for tuning Kestra performance, covering Worker configuration, JDBC backend, and Kafka settings.
---
+:::alert{type="info"}
+This guide covers Kestra 1.3. Some configuration options reference the JDBC queue and Kafka backend, which were replaced by the gRPC worker-controller in Kestra 2.0. An updated guide for 2.0 will be published after release.
+:::
+
Not all workloads are the same, so Kestra is configured to balance throughput (the ability to process a lot of executions in parallel) and latency (the ability to process executions quickly) without using too many resources.
## Tune Kestra for throughput and latency
diff --git a/src/contents/docs/performance/sizing-and-scaling-infrastructure/index.md b/src/contents/docs/performance/sizing-and-scaling-infrastructure/index.md
index a01a15864db..f192f1b0214 100644
--- a/src/contents/docs/performance/sizing-and-scaling-infrastructure/index.md
+++ b/src/contents/docs/performance/sizing-and-scaling-infrastructure/index.md
@@ -6,6 +6,10 @@ icon: /src/contents/docs/icons/admin.svg
description: Guidance on sizing and scaling your Kestra infrastructure, including Executors, Workers, and Schedulers, for optimal performance.
---
+:::alert{type="info"}
+This guide covers Kestra 1.3. Updated sizing guidance for Kestra 2.0 will be published after release.
+:::
+
Kestra is designed to scale from lightweight workflows to enterprise-scale orchestration with thousands of task runs per minute. Choosing the right infrastructure depends on your workload patterns, execution volume, and latency requirements. This page provides practical guidance on how to size your Kestra deployment, how many Executors and Workers you need, and how to scale and tune performance over time.
## Size and scale your Kestra deployment
@@ -13,10 +17,10 @@ Kestra is designed to scale from lightweight workflows to enterprise-scale orche
## Core concepts
Before diving into numbers, it helps to understand how Kestra executes work:
-1. **Executors** orchestrate workflows: they orchestrate workflow logic via flowable tasks, delegate tasks to the right worker nodes, and manage execution state and concurrency.
+1. **Executors** drive the execution state machine: they process flowable tasks, manage concurrency, retries, and pauses, and dispatch runnable tasks to Workers via the Worker Controller.
2. **Workers** run the tasks themselves: from lightweight logging to long-running scripts or container workloads.
-3. **Schedulers** handle triggers such as scheduled events, webhook calls, or polling external resources.
-4. **Webservers** provide the API and UI, they handle user interactions incl. processing execution inputs.
+3. **Schedulers** handle time-based, polling, and realtime triggers. Flow Triggers are evaluated by the Executor; Webhook triggers are handled by the Webserver.
+4. **Webservers** provide the API and UI and handle inbound webhook triggers.
Performance depends on balancing **throughput** (task runs per minute) and **latency** (how quickly executions start and complete) given your infrastructure.
@@ -106,7 +110,7 @@ To improve executor throughput:
## Backend considerations
-- **JDBC/Postgres backend (Enterprise and OSS)**: simpler to operate with low latency for up to ~1,000 task runs/min. [Performance tunin](../../performance/performance-tuning/index.md)g involves adjusting JDBC queue polling intervals and executor threads beyond scaling the infrastructure.
+- **JDBC/Postgres backend (Enterprise and OSS)**: simpler to operate with low latency for up to ~1,000 task runs/min. [Performance tuning](../../performance/performance-tuning/index.md) involves adjusting JDBC queue polling intervals and executor threads beyond scaling the infrastructure.
- **Kafka backend (Enterprise)**: required for higher throughput, real-time triggers, and scaling beyond ~2,000 task runs/min. Ensure enough partitions are allocated (≥ number of Executors/Workers) for full parallelism.
---
diff --git a/src/contents/docs/plugin-developer-guide/03.task/index.md b/src/contents/docs/plugin-developer-guide/03.task/index.md
index d5d5be66372..f323e33f0dd 100644
--- a/src/contents/docs/plugin-developer-guide/03.task/index.md
+++ b/src/contents/docs/plugin-developer-guide/03.task/index.md
@@ -253,7 +253,7 @@ public Output run(RunContext runContext) throws Exception {
Kestra uses a special type to carry data in a flexible way: `Data`.
`Data` can be built via three different types or properties: a URI (which will points to a Kestra internal storage file), a list of map (for defining multiple items), or a map (for a single item). Thanks to this, the task user can pass data to it in a very flexible way, we strongly encourage you to use this type when it fits your needs.
-Here is an example that defines a `Data` attibute of type `Message`, at run time you will need to render this property and map the message from a `Map`. It uses Project Reactor `Flux` under the cover to allow processing items one by one in a reactive manner allowing to process an arbitrary number of items. When couple with our internal storage files, it can process files or billions of items if needed:
+Here is an example that defines a `Data` attribute of type `Message`, at run time you will need to render this property and map the message from a `Map`. It uses Project Reactor `Flux` under the cover to allow processing items one by one in a reactive manner allowing to process an arbitrary number of items. When couple with our internal storage files, it can process files or billions of items if needed:
```java
// property definition
diff --git a/src/contents/docs/plugin-developer-guide/05.condition/index.md b/src/contents/docs/plugin-developer-guide/05.condition/index.md
index 7a393572528..3af4b41a109 100644
--- a/src/contents/docs/plugin-developer-guide/05.condition/index.md
+++ b/src/contents/docs/plugin-developer-guide/05.condition/index.md
@@ -7,7 +7,7 @@ description: Develop custom Condition plugins for Kestra to control flow executi
---
-Here is how you can develop a new [Condition](../../05.workflow-components/07.triggers/index.mdx#conditions).
+Here is how you can develop a new [Condition](../../05.workflow-components/07.triggers/index.mdx).
## Build a condition plugin for Kestra
diff --git a/src/contents/docs/plugin-developer-guide/07.document/index.md b/src/contents/docs/plugin-developer-guide/07.document/index.md
index b2dba1e9375..cef73737050 100644
--- a/src/contents/docs/plugin-developer-guide/07.document/index.md
+++ b/src/contents/docs/plugin-developer-guide/07.document/index.md
@@ -69,7 +69,7 @@ If there are files inside the `src/main/resources/doc/guides` directory, they ar
### Group Icon
-It is possible to provide an icon representing the whole plugin group. If there is a [SVG file](https://www.worg/Graphics/SVG/) `src/main/resources/icons/plugin-icon.svg`, it will be used as the group icon.
+It is possible to provide an icon representing the whole plugin group. If there is a [SVG file](https://www.w3.org/Graphics/SVG/) `src/main/resources/icons/plugin-icon.svg`, it will be used as the group icon.
## Document the plugin sub-groups
@@ -229,7 +229,7 @@ id: example_flow
namespace: example_namespace
tasks:
- id: example_task
- type: io.kestra.core.tasks.log.Log
+ type: io.kestra.plugin.core.log.Log
message: "Hello, World!"
```
diff --git a/src/contents/docs/plugin-developer-guide/09.file-renderer/index.md b/src/contents/docs/plugin-developer-guide/09.file-renderer/index.md
new file mode 100644
index 00000000000..62f0ec776a2
--- /dev/null
+++ b/src/contents/docs/plugin-developer-guide/09.file-renderer/index.md
@@ -0,0 +1,288 @@
+---
+title: Develop a Kestra File Renderer Plugin
+h1: How to Build a Custom File Preview Renderer for Kestra
+sidebarTitle: Develop a File Renderer
+icon: /src/contents/docs/icons/dev.svg
+description: Learn how to implement the FileRenderer interface to add custom file preview support for the Kestra UI Outputs tab.
+---
+
+Kestra previews output files directly in the **Outputs** tab. This preview is powered by `FileRenderer` implementations — one per file type. Implement `FileRenderer` to add preview support for file formats your plugin produces.
+
+## What is a `FileRenderer`
+
+`FileRenderer` is a Java interface (`io.kestra.core.preview.FileRenderer`) annotated with `@Plugin`. Kestra's plugin scanner automatically discovers every `FileRenderer` implementation on the classpath — implement the interface, package it with your plugin, and Kestra picks it up at startup.
+
+Built-in renderers in `io.kestra.plugin.core.preview` cover text, ION, image, and PDF formats. When your plugin produces files in a format not covered by those — such as CSV, Parquet, Avro, or a domain-specific binary — implement a `FileRenderer` to give those files a meaningful preview.
+
+## The `FileRenderer` interface
+
+```java
+@io.kestra.core.models.annotations.Plugin
+public interface FileRenderer extends Plugin {
+
+ boolean supports(String extension);
+
+ FilePreview render(
+ String extension,
+ InputStream inputStream,
+ Optional charset,
+ int maxRows
+ ) throws IOException;
+}
+```
+
+Implement both methods:
+
+- `supports(String extension)` — return `true` if this renderer handles the given extension. The extension is always lowercase with no leading dot (e.g., `"csv"`, `"parquet"`).
+- `render(...)` — read the `InputStream` and return a `FilePreview`. Throw `IllegalArgumentException` at the top of `render()` when `supports(extension)` returns `false`.
+
+## The `FilePreview` return type
+
+```java
+@Getter
+@Builder
+public class FilePreview {
+ private String extension;
+ private Type type;
+ private Object content;
+ @Builder.Default
+ private boolean truncated = false;
+
+ public enum Type {
+ TEXT, LIST, IMAGE, MARKDOWN, PDF
+ }
+}
+```
+
+The `type` field controls how the UI renders `content`:
+
+| Type | Content format | UI rendering |
+|---|---|---|
+| `TEXT` | `String` | Plain text, monospace |
+| `MARKDOWN` | `String` | Rendered markdown |
+| `LIST` | `List