kafka input

[jrametta]

Hi There! Is it possible to have kafka input only in gnmic config such as below? If so, what are the appropriate command line args to execute?

inputs:
  kafka-input:
    address: clab-fgw1-kafka-server:9002
    type: kafka
    topics: telemetry
    outputs:
      - prom-output

outputs:
  prom-output:
    type: prometheus
    listen: "gnmic:9804"
    service-registration:
      address: consul-agent:8500

tried with subscribe (but of course there are no subscriptions)

docker run --rm  -v foo.yaml:/etc/gnmic.conf  ghcr.io/karimra/gnmic:latest    sub --skip-verify
Error: no subscriptions configuration found

thanks!

[karimra]

Hi, you are not doing anything wrong, subscribe is the command to use.
I will remove the requirement for subscriptions: to be set when there is at least one input configured.

[karimra]

Another consideration is the format of the messages that gnmic will read from kafka.
Right now it supports proto the actual gNMI proto bytes or event the event msg format defined in the gnmic repo.

[karimra]

@jrametta can you give the last version (v0.25.0) a try, it should allow you to run subscribe command without subscriptions defined in the config file if there are inputs configured.

About other formats, what do you have in mind ?
I thought about adding a template field where the user can set a gotemplate that transforms whatever is received from kafka/nats to an event message.

[jrametta]

Hi @karimra, with v0.25.0 I'm now getting Error: failed reading targets config: failed reading targets config: no targets found. Also, as a test I added a target and subscription config so gnmic does not exit, but am not seeing any connection to the kafka address. Any ideas? thanks.

[karimra]

@jrametta I see the same issue, sorry about that.
Let me try to fix it, will get back to you asap.

[karimra]

please give v0.25.1 a try and let me know how it goes.

[jrametta]

Awesome! With this fix, I'm now able to send telemetry data into kafka and collect it on the other side. This really helps to showcase what a network observability platform can look like. As far as message format is concerned, we have other software services that expect json but having a templated function seems like the way to go. thanks!