### Description Add new 'rate-limited' state to indicate crawl has been rate limited but is still running. ### Context _No response_
Description
Add new 'rate-limited' state to indicate crawl has been rate limited but is still running.
Context
No response