📝 Description
When the watcher starts, InitQueues rebuilds its in-memory picture of every
repository's queue from the PipelineRuns in the cluster. To decide whether a
run is occupying a slot it reads one thing: the Pipelines-as-Code state
annotation (pkg/queue/queue_manager.go:240). It never asks Tekton whether
the run already finished.
A PipelineRun that completed, but whose annotation never advanced from
started to completed, is rebuilt as a running run. It occupies a slot, and
since only a running run finishing releases a slot, that slot is gone until
the next restart, which rebuilds the same wrong picture again.
The annotation lags behind reality whenever the watcher fails or dies between
a run finishing and the final annotation write. The provider-detection issue
(see companion issue) is one concrete way to get there.
A unit test confirms it: a PipelineRun with a Succeeded condition and a stale
started annotation comes back from InitQueues inside the running queue.
Found during a post-merge review of #2890.
🛠️ Suggested fix
During the rebuild, skip any PipelineRun that Tekton reports as done or
cancelled, whatever its annotation says.
🧪 Testing Strategy
📝 Description
When the watcher starts,
InitQueuesrebuilds its in-memory picture of everyrepository's queue from the PipelineRuns in the cluster. To decide whether a
run is occupying a slot it reads one thing: the Pipelines-as-Code state
annotation (
pkg/queue/queue_manager.go:240). It never asks Tekton whetherthe run already finished.
A PipelineRun that completed, but whose annotation never advanced from
startedtocompleted, is rebuilt as a running run. It occupies a slot, andsince only a running run finishing releases a slot, that slot is gone until
the next restart, which rebuilds the same wrong picture again.
The annotation lags behind reality whenever the watcher fails or dies between
a run finishing and the final annotation write. The provider-detection issue
(see companion issue) is one concrete way to get there.
A unit test confirms it: a PipelineRun with a Succeeded condition and a stale
startedannotation comes back fromInitQueuesinside the running queue.Found during a post-merge review of #2890.
🛠️ Suggested fix
During the rebuild, skip any PipelineRun that Tekton reports as done or
cancelled, whatever its annotation says.
🧪 Testing Strategy