Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@username and @password are not present as config_param #527

Open
Gaurav-Puri101187 opened this issue Feb 19, 2025 · 0 comments
Open

@username and @password are not present as config_param #527

Gaurav-Puri101187 opened this issue Feb 19, 2025 · 0 comments

Comments

@Gaurav-Puri101187
Copy link

Gaurav-Puri101187 commented Feb 19, 2025

Describe the bug

out_rdkafka2.rb is using the username and password to set values for rdkafka, sasl.username and sasl.password as we can see set rdkafka's username and password but there is no config_param to pass the same from fluent configuration. Which also makes it difficult to substitute these params with the environment variables.

A workaround for this is something like rdkafka_options "#{'{ "security.protocol": "sasl_ssl", "sasl.mechanisms": "PLAIN", "sasl.username": "$ConnectionString", "sasl.password": "%s" }' % [ENV['FLUENTD_SECRET']]}"

A proper solution would be to include username and password as config_param which allows to easily pass these values separately.

To Reproduce

 <match **>
              @type rdkafka2

              #general settings
              brokers "#{ENV['EVENT_HUB_BROKERS']}"
              default_topic "#{ENV['EVENT_HUB_TOPIC']}"

              # timeout settings check again for eventhub
              required_acks -1
              ack_timeout 30

               # sasl settings
               username "#{ENV['username']}" -- value not being used
               password "#{ENV['pass']}" -- value not bein used
               

              <format>
                   @type 'json'
              </format>
..........

Expected behavior

username and password values specified should be used, so that no need to do

rdkafka_options "#{'{ "security.protocol": "sasl_ssl", "sasl.mechanisms": "PLAIN", "sasl.username": "$ConnectionString", "sasl.password": "%s" }' % [ENV['FLUENTD_SECRET']]}"

Your Environment

- Fluentd version: 1.17.1
- fluent-plugin-kafka version: 0.19.3
- Operating system: Linux

Your Configuration

<match **>
@type rdkafka2

          #general settings
          brokers "#{ENV['EVENT_HUB_BROKERS']}"
          default_topic "#{ENV['EVENT_HUB_TOPIC']}"

          # timeout settings check again for eventhub
          required_acks -1
          ack_timeout 30

           # sasl settings
           username "#{ENV['username']}" -- value not being used
           password "#{ENV['pass']}" -- value not bein used
           

          <format>
               @type 'json'
          </format>

..........

Your Error Log

Config params not getting used

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Triage
Development

No branches or pull requests

1 participant