AMQP input fixes and improvements

[bernd]

I am experimenting with some fixes and improvements for our AMQP input.

• Fix executor leak by removing the "AMQP Executor" named binding and manually start and stop an executor.
• Replace the bytes-per-second refresh runnable with a memorized supplier to avoid the scheduling overhead.
• Expose AMQP some client metrics into our metrics registry.
• Provide a AMQP connection name to help with debugging connection issues.
• Add a BlockedListeter to AMQP connections so we can log when the client is getting blocked by the broker.
• Use a custom executor service for the AMQP client that uses "parallelQueues" number of threads instead of relying on the default executor that uses a thread count of num-cpus per AMQP input.

[boosty]

Hi @bernd! The changes make a lot of sense. Thanks for working on this!

Besides some general smoke testing, I tested one case in particular not to cause a regression:

If Graylog stands up first - it fails to start the input and never retries

This bug was a motivation for previous work in #15401 and was reported via #12447 (comment).

According to my local testing, it still seems fine after your refactoring. An initial connection is being retried and succeeds once RabbitMQ is up.

[boosty]

@bernd Thanks again for working on this!

I now added a few more changes to this PR:

• Support passive queue declaration. This new option will allow users to create a queue in RabbitMQ using any available setting (e.g. x-max-length). This will avoid errors when Graylog tries to create a queue using incompatible settings, as described in these issues:
  • AMQP - RabbitMQ inputs fail with inequivalent arg 'x-queue-type' errors #6827
  • GELF AMQP Input not support some RabbitMQ settings such as transient and/or quorum queues #7747
• Clarify behavior of parallelQueues option in the input form. The support for a %d placeholder in the queue name was not documented, and neither was the difference in behavior when using it.
• Prevent connection leak on initial connection failures. Prior to this change, the retry behavior could lead to accumulated connections in RabbitMQ, e.g., on a queue creation error.
• Improve error message in IOException cases, e.g., on a queue creation error.

I tested everything and think this is ready for review.