Repository navigation
[Bug] Node reports healthy status while block synchronization is stalled #1317
Description
Activity
I have analyzed this issue. The current health check likely only monitors the process status or API responsiveness but ignores the Block Height progression.
To fix this, we should implement a 'Sync Progress Monitor' in the health check logic.
Logic: Store the last_block_height and last_update_timestamp.
Check: If current_block_height == last_block_height for more than X minutes (e.g., 5 mins) while the node is in 'Syncing' mode, the health check should return Unhealthy (Stalled).
Suggested Implementation:
Update the /health endpoint to query the execution engine for the latest block.
If the delta between the current time and the last block timestamp exceeds a threshold, mark the node as degraded.
Describe the bug
The node can report a healthy and running status even when block synchronization is no longer progressing.
The process remains active, health checks return OK, and APIs continue to respond normally, but the block height does not increase and synchronization is effectively stalled.
This makes it difficult for operators to detect that the node is no longer contributing to the network.
To Reproduce
Expected behavior
When a node reports a healthy status:
Actual behavior
The node appears healthy while synchronization is effectively stalled.
Impact
Operators may assume the node is functioning correctly while it is no longer participating in the network.
This can result in: