[INFRA-799] feat: ship RabbitMQ 4.2.9 instead of end-of-life 3.13.6 #304
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -21,6 +21,32 @@ | |
| helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress-nginx --create-namespace | ||
| ``` | ||
|
|
||
| ## Upgrading RabbitMQ from 3.13 to 4.2 | ||
|
|
||
| This chart now ships `rabbitmq:4.2.9-management-alpine`. RabbitMQ 3.x is end-of-life and no longer receives security updates. | ||
|
|
||
| **If you are upgrading an existing install with `rabbitmq.local_setup: true`, do this first.** The chart upgrade restarts the broker StatefulSet against the same volume, and RabbitMQ requires all stable feature flags to be enabled *before* a major upgrade — otherwise the 4.2 node refuses to start and your queues are unreachable until you roll back. | ||
|
|
||
| ```bash | ||
| # 1. While still on 3.13, enable every stable feature flag. | ||
| kubectl -n <namespace> exec <release>-rabbitmq-wl-0 -- rabbitmqctl enable_feature_flag all | ||
|
|
||
| # 2. Confirm nothing stable is left disabled. Only `khepri_db` should remain, | ||
| # and it MUST stay disabled -- a 3.13 node with Khepri enabled cannot be | ||
| # upgraded to 4.x at all and needs a blue-green migration instead. | ||
| kubectl -n <namespace> exec <release>-rabbitmq-wl-0 -- rabbitmqctl list_feature_flags | ||
|
|
||
| # 3. Now run the chart upgrade, then confirm the broker came back. | ||
| kubectl -n <namespace> exec <release>-rabbitmq-wl-0 -- rabbitmqctl status | grep 'RabbitMQ version' | ||
| ``` | ||
|
|
||
| Notes: | ||
|
|
||
| - **Do not jump straight to 4.3.** RabbitMQ does not support a direct 3.13 → 4.3 upgrade ([version upgradability](https://www.rabbitmq.com/docs/upgrade#rabbitmq-version-upgradability)); 4.2 is the supported hop, and a later chart release will move to 4.3. Two further things break on 4.3 but not on 4.2: Celery's control/event queues (fixed in the application by `CELERY_CONTROL_QUEUE_EXCLUSIVE` / `CELERY_EVENT_QUEUE_EXCLUSIVE`), and `x-consumer-timeout` on classic queues. | ||
| - **Downgrades do not work.** A 4.x node will not start on a data directory it has already upgraded, so keep a volume snapshot if you need a way back. | ||
|
Contributor
There was a problem hiding this comment. 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: sed -n '34,51p;321,338p' charts/plane-ce/README.md charts/plane-enterprise/README.mdRepository: makeplane/helm-charts Length of output: 5788 🌐 Web query:
💡 Result: RabbitMQ does not officially support downgrades between major or minor versions [1][2][3]. Because RabbitMQ upgrades frequently involve automatic and irreversible changes to the data directory, downgrading to a previous version is not tested and should not be relied upon [1][2]. If you are performing an upgrade from RabbitMQ 3.x to 4.x, the official guidance regarding your data directory and rollback is as follows: 1. Mandatory Backup: You are strongly advised to back up the node's data directory before beginning any upgrade [1][2][3]. If an upgrade fails or you need to revert, restoring from this backup is the only reliable way to return to the previous state [2]. 2. Unsupported Downgrades: Attempting to start a node with a data directory that has been upgraded to a newer version (e.g., 4.x) using an older version (e.g., 3.x) will likely fail or cause data corruption. 3. Blue-Green Deployment: For production environments requiring high availability and a safe rollback mechanism, the official recommendation is to use a Blue-Green deployment strategy [1][2][3]. This allows you to maintain your original (Blue) cluster running the old version while spinning up a new (Green) cluster with the target version. If issues occur in the new cluster, you can redirect traffic back to the old one without needing to downgrade the data directory [1][4]. 4. Experimental Features: Note that RabbitMQ 3.13 included experimental Khepri support which is incompatible with 4.x; upgrading a node with Khepri enabled will result in failure, and these cases strictly require a migration to a fresh cluster rather than an in-place upgrade [5][6]. In summary, do not attempt to downgrade your existing data directory after it has been upgraded. Always rely on a full backup of your data directory or a Blue-Green deployment strategy to manage risks during major version transitions [1][2][3]. Citations:
🌐 Web query:
💡 Result: RabbitMQ does not officially support downgrades, as they are not tested and should not be relied upon [1]. While downgrades may technically function between certain versions—particularly within the same patch release series—this is not guaranteed, and even some patch releases cannot be downgraded to their immediate predecessor [1]. Because data directory structures can change during an upgrade, a node that has been upgraded to a newer version may write data in a format that an older version cannot read [1]. Consequently, attempting to use an upgraded data directory with an older version of RabbitMQ is high-risk and generally unsupported. To mitigate this risk, the recommended approach is to use a blue-green deployment strategy [1]. In a blue-green setup, the "green" environment (the new version) is deployed alongside the "blue" environment (the old version). This allows you to switch traffic back to the original, unmodified "blue" environment if issues arise, rather than attempting to downgrade the data directory of the upgraded nodes [1]. If you are performing an upgrade and wish to ensure maximum safety, you should always back up the node's data directory before beginning the process [1]. In development or test environments where data persistence is not required, the safest path is often to delete the data directory and perform a fresh installation of the target version [1]. Citations: Correct the rollback warning in both guides. RabbitMQ does not officially support downgrades. If a 4.x node upgrades the data directory, a 3.x node may not start with it. Document restoring a pre-upgrade volume snapshot or using blue-green migration.
📍 Affects 2 files
🤖 Prompt for AI AgentsSource: MCP tools |
||
| - **No queue changes are required.** Existing queues keep their arguments and are re-declared as-is by the application; durable messages survive the restart. Verified end to end on a 3.13.6 → 4.2.9 in-place upgrade with pre-existing queues. | ||
| - **Using an external broker?** If `rabbitmq.local_setup: false` and you point `external_rabbitmq_url` at a managed broker (Amazon MQ, CloudAMQP), this chart does not manage its version — upgrade it on the provider side, following the same feature-flag prerequisite. | ||
|
|
||
| ## Installing Plane | ||
|
|
||
| 1. Open Terminal or any other command-line app that has access to Kubernetes tools on your local system. | ||
|
|
@@ -210,7 +236,7 @@ The default value is `"traefik"`. If you previously relied on the implicit defau | |
| | Setting | Default | Required | Description | | ||
| | ------------------------------ | :-------------------------------: | :------: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ||
| | rabbitmq.local_setup | true | | Plane uses `rabbitmq` as message queuing system. This can be hosted within kubernetes as part of helm chart deployment or can be used as hosted service remotely (e.g. aws mq or similar services). Set this to `true` when you choose to setup stateful deployment of `rabbitmq`. Mark it as `false` when using a remotely hosted service | | ||
| | rabbitmq.image | rabbitmq:4.2.9-management-alpine | | Using this key, user must provide the docker image name to setup the stateful deployment of `rabbitmq`. (must be set when `rabbitmq.local_setup=true`) | | ||
| | rabbitmq.pullPolicy | IfNotPresent | | Using this key, user can set the pull policy for the stateful deployment of `rabbitmq`. (must be set when `rabbitmq.local_setup=true`) | | ||
| | rabbitmq.servicePort | 5672 | | This key sets the default port number to be used while setting up stateful deployment of `rabbitmq`. | | ||
| | rabbitmq.managementPort | 15672 | | This key sets the default management port number to be used while setting up stateful deployment of `rabbitmq`. | | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -473,6 +473,32 @@ ingress: | |
| entryPoints: ['websecure'] | ||
| ``` | ||
|
|
||
| ## Upgrading RabbitMQ from 3.13 to 4.2 | ||
|
|
||
| This chart now ships `rabbitmq:4.2.9-management-alpine`. RabbitMQ 3.x is end-of-life and no longer receives security updates. | ||
|
|
||
| **If you are upgrading an existing install with `services.rabbitmq.local_setup: true`, do this first.** The chart upgrade restarts the broker StatefulSet against the same volume, and RabbitMQ requires all stable feature flags to be enabled *before* a major upgrade — otherwise the 4.2 node refuses to start and your queues are unreachable until you roll back. | ||
|
|
||
| ```bash | ||
| # 1. While still on 3.13, enable every stable feature flag. | ||
| kubectl -n <namespace> exec <release>-rabbitmq-wl-0 -- rabbitmqctl enable_feature_flag all | ||
|
|
||
| # 2. Confirm nothing stable is left disabled. Only `khepri_db` should remain, | ||
| # and it MUST stay disabled -- a 3.13 node with Khepri enabled cannot be | ||
| # upgraded to 4.x at all and needs a blue-green migration instead. | ||
| kubectl -n <namespace> exec <release>-rabbitmq-wl-0 -- rabbitmqctl list_feature_flags | ||
|
|
||
| # 3. Now run the chart upgrade, then confirm the broker came back. | ||
| kubectl -n <namespace> exec <release>-rabbitmq-wl-0 -- rabbitmqctl status | grep 'RabbitMQ version' | ||
| ``` | ||
|
|
||
| Notes: | ||
|
|
||
| - **Do not jump straight to 4.3.** RabbitMQ does not support a direct 3.13 → 4.3 upgrade ([version upgradability](https://www.rabbitmq.com/docs/upgrade#rabbitmq-version-upgradability)); 4.2 is the supported hop, and a later chart release will move to 4.3. Two further things break on 4.3 but not on 4.2: Celery's control/event queues (fixed in the application by `CELERY_CONTROL_QUEUE_EXCLUSIVE` / `CELERY_EVENT_QUEUE_EXCLUSIVE`), and `x-consumer-timeout` on classic queues. | ||
| - **Downgrades do not work.** A 4.x node will not start on a data directory it has already upgraded, so keep a volume snapshot if you need a way back. | ||
| - **No queue changes are required.** Existing queues keep their arguments and are re-declared as-is by the application; durable messages survive the restart. Verified end to end on a 3.13.6 → 4.2.9 in-place upgrade with pre-existing queues. | ||
|
Contributor
There was a problem hiding this comment. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: sed -n '470,505p' charts/plane-enterprise/README.mdRepository: makeplane/helm-charts Length of output: 2738 🌐 Web query:
💡 Result: In RabbitMQ 4.0, classic queue mirroring has been completely removed [1][2]. Classic queues remain supported as a non-replicated queue type, but any policy configurations previously used for mirroring will have no effect after upgrading to 4.0 [1]. To assist with migration and cleanup, RabbitMQ provides command-line tools to identify existing policies that enabled classic queue mirroring: 1. check_if_cluster_has_classic_queue_mirroring_policy: This is a health check command that exits with a non-zero code if any policies in the cluster are still configured to enable classic queue mirroring [3][4]. It can be run as: rabbitmq-queues check_if_cluster_has_classic_queue_mirroring_policy 2. list_policies_with_classic_queue_mirroring: This command lists the specific policies that still contain classic queue mirroring configurations [5]. For replicated messaging requirements, RabbitMQ 4.0 officially supports Quorum Queues and Streams [1][6]. Migration from mirrored classic queues to quorum queues is typically handled via a blue-green deployment strategy or by migrating to a new virtual host within the same cluster using the Federation Plugin [7][8]. Official documentation for these migration procedures is available in the RabbitMQ migration guides [9][7][8]. Citations:
🌐 Web query:
💡 Result: The command rabbitmq-queues check_if_cluster_has_classic_queue_mirroring_policy is a health check used to identify if any policies in a RabbitMQ cluster are configured to enable classic queue mirroring [1]. This command exits with a non-zero status code if such policies are found [1]. It is important to note that classic queue mirroring was deprecated in 2021 and has been completely removed in RabbitMQ 4.0 [1][2]. Consequently, while this check is relevant for older versions or migration scenarios, the feature it checks for is no longer supported in modern versions of RabbitMQ [1]. Citations:
Qualify the “no queue changes” upgrade claim. RabbitMQ 4.x removes classic queue mirroring. Existing mirrored classic queues become non-replicated after the upgrade, which can remove queue high availability. Add a preflight check with 🤖 Prompt for AI AgentsSource: MCP tools |
||
| - **Using an external broker?** If `services.rabbitmq.local_setup: false` and you point `external_rabbitmq_url` at a managed broker (Amazon MQ, CloudAMQP), this chart does not manage its version — upgrade it on the provider side, following the same feature-flag prerequisite. | ||
|
|
||
| ## Installing Plane | ||
|
|
||
| 1. Open Terminal or any other command-line app that has access to Kubernetes tools on your local system. | ||
|
|
@@ -753,7 +779,7 @@ the bundled datastores off. Three things to know before you use it: | |
| | Setting | Default | Required | Description | | ||
| | --------------------------------------- | :-------------------------------: | :------: | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ||
| | services.rabbitmq.local_setup | true | | Plane uses `rabbitmq` as message queuing system. This can be hosted within kubernetes as part of helm chart deployment or can be used as hosted service remotely (e.g. aws mq or similar services). Set this to `true` when you choose to setup stateful deployment of `rabbitmq`. Mark it as `false` when using a remotely hosted service | | ||
| | services.rabbitmq.image | rabbitmq:4.2.9-management-alpine | | Using this key, user must provide the docker image name to setup the stateful deployment of `rabbitmq`. (must be set when `services.rabbitmq.local_setup=true`) | | ||
| | services.rabbitmq.pullPolicy | IfNotPresent | | Using this key, user can set the pull policy for the stateful deployment of `rabbitmq`. (must be set when `services.rabbitmq.local_setup=true`) | | ||
| | services.rabbitmq.servicePort | 5672 | | This key sets the default port number to be used while setting up stateful deployment of `rabbitmq`. | | ||
| | services.rabbitmq.managementPort | 15672 | | This key sets the default management port number to be used while setting up stateful deployment of `rabbitmq`. | | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🔎 Supported by static analysis
🏁 Script executed:
Repository: makeplane/helm-charts
Length of output: 29042
🌐 Web query:
RabbitMQ 4.2 upgrade feature flags khepri_db 3.13 4.2 4.3 stable feature flags post-upgrade💡 Result:
In RabbitMQ 4.2, the khepri_db feature flag is considered stable [1]. While Khepri is the default metadata store for new deployments starting with RabbitMQ 4.2.0, existing clusters upgraded to 4.2.x will continue to use their previous metadata store (typically Mnesia) until the khepri_db feature flag is explicitly enabled by an administrator [2][3]. For RabbitMQ 4.2 and subsequent upgrades: 1. Feature Flag Best Practices: After a successful version upgrade to 4.2.x, it is standard practice to enable all stable feature flags using the command rabbitmqctl enable_feature_flag all [4][1]. Running this command will also enable khepri_db if it was not already active, transitioning the cluster to Khepri [1]. 2. Upgrade Path Requirements: - RabbitMQ 4.2 requires all feature flags introduced in the 3.13 series to be enabled prior to upgrading to 4.2 [5][6]. - RabbitMQ 4.3 requires all feature flags from 4.2 (including khepri_db) to be enabled [6]. - If you are upgrading from 3.13.x, you must follow the supported path: upgrade to 4.2.x first, then to 4.3.x [7]. 3. Khepri Specifics: - Once Khepri is enabled, it cannot be disabled; reverting to Mnesia is not supported [8]. - If Khepri was enabled on a 3.13.x cluster, that cluster cannot be upgraded in-place to 4.0 or higher; a blue-green deployment is required in such cases [5][8]. - For standard upgrades where Khepri is not yet enabled, you may enable it after the nodes are running on the new version [9][2]. As of RabbitMQ 4.3.0, Khepri is the only supported metadata store [10]. Always ensure your cluster is healthy and has no pending synchronizations before enabling feature flags or performing version upgrades [7].
Citations:
Complete the post-upgrade RabbitMQ feature-flag step in both guides.
After RabbitMQ 4.2 starts, run
rabbitmqctl enable_feature_flag all, then runrabbitmqctl list_feature_flagsand confirm thatkhepri_dbis enabled. The current pre-upgrade step correctly leaveskhepri_dbdisabled on 3.13, but the guides never enable it on 4.2. RabbitMQ 4.3 requires the 4.2 feature flags, includingkhepri_db; without this step, the next upgrade can fail. Add these commands after the status check incharts/plane-ce/README.mdandcharts/plane-enterprise/README.md.📍 Affects 2 files
charts/plane-ce/README.md#L39-L40(this comment)charts/plane-enterprise/README.md#L326-L327🤖 Prompt for AI Agents
Source: MCP tools