support giving up on failed individual bulk requests after a number of retries

[thethumbler]

This commit adds a new config parameter retry_max_failures that controls the maximum number of retries that should be attempted for failed individual bulk requests (0 means infinity).

Resolves #795 #806

[ThaSami]

any updates on this?

[andsel]

Thanks @thethumbler for your contribution.
This proposal misses some test for the behavior it introduces.

From a user perspective the drop of events once it reaches the retry_max_failures is not optimal, maybe considering the insertion on DLQ could be an viable path.

[andsel]

The feature we could implement is to drop the events when retry_max_failures is reached, but if the option is configured a WARN log should be present to make the user aware of the event drops, without pushing to the DLQ.
This feature should be well documented with a warning box that when enabled it could generate message loss. Maybe a specific metric could also be used to track the number of dropped events due to retry_max_failure overtake.