[GSoC 2026] fix(settings): single, env-driven definition for CHATBOT_QUEUE

[berardifra]

Description

Fixes the duplicate CHATBOT_QUEUE definition reported by @Aditya30ag in #3910 — thanks for
catching it, the analysis in the issue was exactly right.

@Aditya30ag also proposed a fix in #3911, which @mlodic decided to supersede with this one. The
settings change there was correct; this PR additionally covers the worker entrypoint, the env
template and the regression tests.

CHATBOT_QUEUE was assigned in two settings modules:

module	value
intel_owl/settings/chatbot.py:19	secrets.get_secret("CHATBOT_QUEUE", "chatbot") (env-driven)
intel_owl/settings/celery.py:21	"chatbot" (hardcoded literal)

Both are wildcard-imported by intel_owl/settings/__init__.py (.chatbot at line 76, .celery at
line 79). The last wildcard wins, so the literal shadowed the env read and CHATBOT_QUEUE in the
environment never reached settings.CHATBOT_QUEUE. Reproduced before the fix:

$ docker exec -e CHATBOT_QUEUE=my_custom_queue -i intelowl_uwsgi python manage.py shell
intel_owl.settings.chatbot.CHATBOT_QUEUE : my_custom_queue
settings.CHATBOT_QUEUE                   : chatbot          # <- the literal wins

The fix

The surviving definition is the env-driven one, moved into settings/celery.py:

CHATBOT_QUEUE = get_secret("CHATBOT_QUEUE") or "chatbot"

settings/celery.py already owns DEFAULT_QUEUE / BROADCAST_QUEUE / CONFIG_QUEUE and the
CELERY_QUEUES loop that consumes them, and it is the lower-level module (it carries
# this module must run before the others and imports only ._util and .aws). Keeping the
definition in chatbot.py instead would have forced celery.py to from .chatbot import CHATBOT_QUEUE, inverting that layering and pulling .cache into the module that must run first.

The fix is order-independent by construction: with a single assignment, no arrangement of the
wildcard imports in __init__.py can change the resulting value — which is the property the issue
asks for.

Trade-off worth flagging: the lookup moves from intel_owl.secrets.get_secret (AWS Secrets Manager
fallback) to settings/_util.get_secret (os.environ.get), matching the sibling BROKER_URL and
CELERY_QUEUES. A queue name is not a secret, and the AWS fallback for this key never worked
anyway since the value was being overwritten.

The other half of the bug

Making the setting configurable is not enough on its own: docker/entrypoints/celery_chatbot.sh
hardcoded the consumer side (queues="chatbot,broadcast,config", or chatbot.fifo,config.fifo
under SQS), and CHATBOT_QUEUE was missing from docker/env_file_app_template. A custom queue name
would therefore have published chat turns to a queue no worker drains, hanging the chat. The
entrypoint now reads the same variable, with ${CHATBOT_QUEUE:-chatbot} matching the or fallback
on the Python side so both ends agree on unset and on blank. test.ollama.override.yml and
ci.ollama.override.yml inherit the entrypoint from ollama.override.yml, so they are covered.

api_app/chatbot_manager/health.py needs no change — it already derives the queue from
settings.CHATBOT_QUEUE, and it remains the detector for a producer/consumer mismatch.

Audit: is any other setting defined twice?

No. CHATBOT_QUEUE was the only name assigned in two wildcard-imported settings modules. The
other 15 name collisions (DEBUG, CACHES, AWS_SQS, get_secret, WEB_CLIENT_URL, …) are
re-exports of the same object via from .X import Y, so every import order yields the same value.

Tests

Two new files under tests/intel_owl/, one per concern. The first two tests below fail on the code
before this PR:

• test_settings_chatbot_queue.py — test_custom_chatbot_queue_reaches_settings_and_task_routing
  boots Django in a subprocess with
  CHATBOT_QUEUE=my_custom_queue and asserts the value reaches settings.CHATBOT_QUEUE
  (AssertionError: 'chatbot' != 'my_custom_queue' before the fix), lands in CELERY_QUEUES, is
  the routing target of process_chat_message via get_queue_name, and that a queue is actually
  declared for it. A second case pins the fallback for unset and blank values. The subprocess is
  necessary: settings are read at import time, so override_settings would assign the value the
  test is trying to prove is computed, and reloading the settings package in-process corrupts it
  for the rest of the suite.
• test_settings_shadowing.py — test_no_setting_is_assigned_in_two_wildcard_imported_modules is
  an AST guard over intel_owl/settings/: it fails if any setting is ever again assigned in two of
  those modules (before the fix it reports
  {'CHATBOT_QUEUE': ['celery.py:21', 'chatbot.py:19']}). It audits __init__.py's own
  assignments too — TEST_RUNNER and INSTALLED_APPS are set before the wildcard block and a
  submodule shadowing one of them is the same bug in the opposite direction. It distinguishes
  assignment from re-export, so the 15 benign collisions above do not trip it.
• test_settings_chatbot_queue.py — test_worker_entrypoint_derives_its_queue_from_the_setting
  closes the gap the fix itself opens: with both layers reading CHATBOT_QUEUE, the default queue
  name now exists in two places, and letting them drift reproduces this very bug one layer down. It
  extracts ${CHATBOT_QUEUE:-…} from the entrypoint, compares it with the settings fallback, and
  fails if either side hardcodes the queue again.

Both guards were verified by mutation rather than assumed: flipping the shell default to chat
fails with 'chat' != 'chatbot', and assigning TEST_RUNNER in a submodule (on a throwaway copy of
the settings package) is reported as {'TEST_RUNNER': ['__init__.py:7', 'websocket.py:20']}.

$ docker exec intelowl_uwsgi python manage.py test tests.api_app.chatbot_manager tests.intel_owl --keepdb
Ran 206 tests in 14.083s

OK

Ruff (check + format --check) and shellcheck are clean on the changed files.

Type of change

• Bug fix (non-breaking change which fixes an issue).
• New feature (non-breaking change which adds functionality).
• Breaking change (fix or feature that would cause existing functionality to not work as expected).
• Chore (refactoring, dependency updates, CI/CD changes, code cleanup, docs-only changes).

Checklist

• I have read and understood the rules about how to Contribute to this project
• The pull request is for the branch develop
• I have inserted the copyright banner at the start of the file
• Please avoid adding new libraries as requirements whenever it is possible — no new dependencies.
• Linters (Ruff) gave 0 errors.
• I have added tests for the feature/bug I solved (see tests folder). All the tests (new and old ones) gave 0 errors.
• After you had submitted the PR, if DeepSource, Django Doctors or other third-party linters have triggered any alerts during the CI checks, I have solved those alerts.
• I have addressed raised Copilot issues.
• I have reviewed and verified any LLM-generated code included in this PR. LLM assistance was used while working on this change, and every line was reviewed and verified locally (tests run before and after the fix, plus a manual reproduction in the running stack).

Refs #3910

[berardifra]

Moved to draft: #3911 by @Aditya30ag was already open for the same issue (#3910) and I only noticed after opening
this. Findings posted there. @mlodic, happy to close this in favour of #3911 — or to un-draft it if you prefer
this one.

[mlodic]

we'll merge this once ready

[berardifra]

Ready. Addressed the coupling the fix itself introduced: with both Django and the worker entrypoint reading
CHATBOT_QUEUE, the default queue name lived in two layers, so a new test extracts ${CHATBOT_QUEUE:-…}
from celery_chatbot.sh and asserts it matches the settings fallback. Both guards verified by mutation, not
assumed. Tests split one file per concern.

Note the red linters check is not this diff: Perform ShellCheck Analysis exits 127 because the CICDToolbox
pipeline.sh URL now 404s and curl -s pipes the error body into bash. It breaks every PR right now —
#3912 / #3913. Since backend-tests has needs: linters, no test job has run here; locally
tests.api_app.chatbot_manager + tests.intel_owl are 206 green, ruff and shellcheck clean.

[mlodic]

can you please pull from develop and try again? I pushed the fix for the shellcheck URL

[berardifra]

can you please pull from develop and try again? I pushed the fix for the shellcheck URL

Done. Thanks