Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/references/errors.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -351,3 +351,23 @@ Each level must be contain only `[a-zA-Z0-9_.-]` characters with maximum length

Limit key can be used only in combination with scope.

## RT0025

The maximum nesting depth for the futures (promises) an invocation awaits on has been reached.

Restate futures can be composed with combinators (e.g. `all`, `race`), and each level
of composition adds one level of nesting. This error is raised when an invocation awaits
on (or suspends on) a future whose nesting depth exceeds the limit configured in
`worker.invoker.max-awaited-future-depth` (default: `1000`).

This is most likely caused by unbounded or runaway recursion in the service code building
ever-deeper combinator futures.

When this limit is reached, the invocation is **paused**. You can resume the
invocation manually after addressing the root cause.

Suggestions:

* Check your service code for recursive or loop-based composition of futures (e.g. repeatedly
wrapping futures in `all`/`race` combinators) that grows without a bound.

18 changes: 18 additions & 0 deletions docs/references/server-config.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2842,6 +2842,24 @@ unit defaults to per second if not specified.
</Expandable>
</ResponseField>

<ResponseField name="max-awaited-future-depth" type="integer" post={['format: uint','env: RESTATE_WORKER__INVOKER__MAX_AWAITED_FUTURE_DEPTH']} default="1000">
Maximum awaited future nesting depth: The maximum nesting depth allowed for the futures (promises) an invocation
awaits on. Restate futures can be composed with combinators (e.g. `all`,
`race`), and each level of composition adds one level of nesting. This limit
bounds how deeply such futures may be nested, guarding against unbounded or
runaway recursion in service code.

When an invocation awaits on (or suspends on) a future whose nesting depth
exceeds this limit, the attempt fails with a \"maximum promise recursion
reached\" error, and the invocation is then handled according to
`on-future-recursion-limit`.

Default: `1000`.

Since v1.7.3

</ResponseField>

<ResponseField name="memory-limit" type="string" post={['minLength: 1','env: RESTATE_WORKER__INVOKER__MEMORY_LIMIT']} default="1.5 GiB">
Non-zero human-readable bytes

Expand Down
30 changes: 30 additions & 0 deletions docs/references/sql-introspection.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,19 @@ To learn how to access the introspection interface, check out the [introspection
| `completion_success_value_utf8` | `Utf8` | The completion success as UTF-8 string, if any. |
| `completion_failure` | `Utf8` | The completion failure, if any. |

## Table: `sys_rules`



| Column name | Type | Description |
|-------------|------|-------------|
| `pattern` | `Utf8` | Rule pattern in canonical display form (e.g. `scope/*/tenant`). |
| `concurrency` | `UInt32` | Concurrency limit imposed by this rule. Null means the rule does not constrain concurrency. |
| `description` | `Utf8` | Free-form description set by the operator. |
| `disabled` | `Boolean` | True when the rule is parked (treated as absent at runtime). |
| `version` | `UInt32` | Per-rule version, bumped on runtime-relevant changes. |
| `last_modified` | `TimestampMillisecond` | Last modification time. |

## Table: `sys_scheduler`


Expand Down Expand Up @@ -147,6 +160,23 @@ To learn how to access the introspection interface, check out the [introspection
| `value` | `Binary` | A binary, uninterpreted representation of the value. You can use the more specific column `value_utf8` if the value is a string. |
| `value_length` | `UInt64` | The byte length of the value. If you are writing a query that only needs to know the length, reading this field will be much more efficient than reading length(value). |

## Table: `sys_user_limits`



| Column name | Type | Description |
|-------------|------|-------------|
| `partition_key` | `UInt64` | Internal column that is used for partitioning. Can be ignored. |
| `scope` | `Utf8` | The scope this counter belongs to. |
| `l1` | `Utf8` | The level-1 key component (null for scope-level counters). |
| `l2` | `Utf8` | The level-2 key component (null for scope-level and L1-level counters). |
| `level` | `Utf8` | The hierarchy level: "Scope", "Level1", or "Level2". |
| `usage` | `UInt32` | Current concurrency usage at this counter. |
| `concurrency_limit` | `UInt32` | The configured concurrency limit (null if unlimited). |
| `rule_pattern` | `Utf8` | The rule pattern that defines the limit (null if unlimited). Resolved from the rule handle; shows "[removed]" if the rule was deleted since the counter was created. |
| `available` | `UInt32` | Available capacity (limit - usage). Null if unlimited. |
| `num_waiters` | `UInt64` | Number of vqueues currently waiting behind this counter. |

## Table: `sys_vqueue_entry_status`


Expand Down
2 changes: 1 addition & 1 deletion docs/schemas/openapi-admin.json

Large diffs are not rendered by default.

11 changes: 11 additions & 0 deletions docs/schemas/restate-server-configuration-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -753,6 +753,7 @@
"in-memory-queue-length-limit": 66049,
"inactivity-timeout": "1m",
"invocation-throttling": null,
"max-awaited-future-depth": 1000,
"memory-limit": "1.5 GiB",
"message-size-warning": "10.0 MiB",
"no-proxy": null,
Expand All @@ -766,6 +767,7 @@
"max-command-batch-size": 32,
"num-timers-in-memory-limit": null,
"rule-book-poll-interval": "30s",
"self-proposal-queue-memory-limit": "64.0 MiB",
"shuffle": {
"connection-retry-policy": {
"factor": 2.0,
Expand Down Expand Up @@ -1532,6 +1534,14 @@
],
"default": null
},
"max-awaited-future-depth": {
"title": "Maximum awaited future nesting depth",
"description": "The maximum nesting depth allowed for the futures (promises) an invocation\nawaits on. Restate futures can be composed with combinators (e.g. `all`,\n`race`), and each level of composition adds one level of nesting. This limit\nbounds how deeply such futures may be nested, guarding against unbounded or\nrunaway recursion in service code.\n\nWhen an invocation awaits on (or suspends on) a future whose nesting depth\nexceeds this limit, the attempt fails with a \"maximum promise recursion\nreached\" error, and the invocation is then handled according to\n`on-future-recursion-limit`.\n\nDefault: `1000`.\n\nSince v1.7.3",
"type": "integer",
"format": "uint",
"default": 1000,
"minimum": 0
},
"memory-limit": {
"title": "Memory limit",
"description": "Global memory budget for the invoker, shared across all partitions on this node.\nThis controls how much memory can be used for in-flight journal entries, state,\nand protocol messages between the invoker and service deployments.\n\nTo effectively disable memory limiting, set this to a very large value.\n\nSince v1.7.0",
Expand Down Expand Up @@ -3085,6 +3095,7 @@
"in-memory-queue-length-limit": 66049,
"inactivity-timeout": "1m",
"invocation-throttling": null,
"max-awaited-future-depth": 1000,
"memory-limit": "1.5 GiB",
"message-size-warning": "10.0 MiB",
"no-proxy": null,
Expand Down
2 changes: 2 additions & 0 deletions docs/schemas/restate.toml
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ max-command-batch-bytes = "1.0 MiB"
trim-delay-interval = "10m"
data-service-memory-limit = "256.0 MiB"
rule-book-poll-interval = "30s"
self-proposal-queue-memory-limit = "64.0 MiB"

[worker.storage]
rocksdb-memory-ratio = 0.49
Expand All @@ -97,6 +98,7 @@ http2-initial-connection-window-size = "5.0 MiB"
http2-max-frame-size = "16.0 KiB"
request-compression-threshold = "4.0 MiB"
request-identity-expiration = "1m"
max-awaited-future-depth = 1000

[worker.snapshots]
num-retained = 1
Expand Down
8 changes: 4 additions & 4 deletions docs/snippets/custom-variables.mdx
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
export const CustomVars = ({name}) => {
const RESTATE_VERSION = "1.7"
const TYPESCRIPT_SDK_VERSION = "1.16.2"
const TYPESCRIPT_SDK_VERSION = "1.16.6"
const JAVA_SDK_VERSION = "2.9.3"
const GO_SDK_VERSION = "1.0.2"
const PYTHON_SDK_VERSION = "1.0.3"
const RUST_SDK_VERSION = "0.11.0"
const GO_SDK_VERSION = "1.0.3"
const PYTHON_SDK_VERSION = "1.0.4"
const RUST_SDK_VERSION = "0.11.1"
const mapping = {
RESTATE_VERSION,
TYPESCRIPT_SDK_VERSION,
Expand Down
2 changes: 1 addition & 1 deletion snippets/go/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ go 1.25.0
require (
github.com/aws/aws-lambda-go v1.50.0
github.com/google/uuid v1.6.0
github.com/restatedev/sdk-go v1.0.2
github.com/restatedev/sdk-go v1.0.3
github.com/restatedev/sdk-go/testing v1.0.0
github.com/restatedev/sdk-go/x/protoc-gen-go-restate v0.26.0
github.com/stretchr/testify v1.11.1
Expand Down
4 changes: 2 additions & 2 deletions snippets/go/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/restatedev/sdk-go v1.0.2 h1:XQYa8mrel+OPc9ObswwGfDHM/3IYw6FOXbfF5q9FgaA=
github.com/restatedev/sdk-go v1.0.2/go.mod h1:wBPSXBOgBvwTQhLqujDSeW1BA480JXh6ZcIervk7tnA=
github.com/restatedev/sdk-go v1.0.3 h1:pDXoFot3eQk5T//2MpTTfwXsMtDgHeoh0grweW8kZxE=
github.com/restatedev/sdk-go v1.0.3/go.mod h1:wBPSXBOgBvwTQhLqujDSeW1BA480JXh6ZcIervk7tnA=
github.com/restatedev/sdk-go/testing v1.0.0 h1:NvRTaGjL6IhCykd97GxUWa1R/HtDbKW7Tg3Yiv9wygg=
github.com/restatedev/sdk-go/testing v1.0.0/go.mod h1:S8XeR6IBENLVzwqCGNe40B6naGfW10PVEtjuiAkhHVI=
github.com/restatedev/sdk-go/x/protoc-gen-go-restate v0.26.0 h1:UIRN0iOoUzeljnYisvSqFbfdpNEVti5RsZtKp9ldTVo=
Expand Down
2 changes: 1 addition & 1 deletion snippets/python/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ version = "0.1.0"
description = "Python code snippets for documentation"
requires-python = ">=3.11"
dependencies = [
"restate-sdk[harness,serde]==1.0.3",
"restate-sdk[harness,serde]==1.0.4",
"hypercorn",
"pydantic",
"requests",
Expand Down
Loading