Description
The events with 400 and 404 (can be extended with dlq_custom_codes) bulk status responses received from ES go into the DLQ. However, to answer "what is happening" in ES side, we either need to take a look at ES logs/diagnostics and cannot be seen in Logstash side. It is always difficult to match Logstash logs with ES logs when analyzing the cases related to DLQ.
Thus, we need to have a visibility on the events going to DLQ (rejected by ES). The events should be logged in the way to tell what error description is actually. This would answer us what is going on in ES side (mapping, etc..) to have a high level understanding.
Description
The events with 400 and 404 (can be extended with
dlq_custom_codes) bulk status responses received from ES go into the DLQ. However, to answer "what is happening" in ES side, we either need to take a look at ES logs/diagnostics and cannot be seen in Logstash side. It is always difficult to match Logstash logs with ES logs when analyzing the cases related to DLQ.Thus, we need to have a visibility on the events going to DLQ (rejected by ES). The events should be logged in the way to tell what error description is actually. This would answer us what is going on in ES side (mapping, etc..) to have a high level understanding.