Rework of PR# 183 feat: Add support for deterministic listener ports (based on broker ID) #191
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Rework of the PR #183.
The main difference from the base pull request is that, instead of closing the upstream listener, the target connection address (broker address) is changed for new connections.
Smoke test logs:
INFO[2025-01-26T15:47:45+01:00] Starting kafka-proxy version v0.3.12-4-g93770ec
INFO[2025-01-26T15:47:45+01:00] Bootstrap server 172.19.255.200:9094 advertised as 0.0.0.0:30000
INFO[2025-01-26T15:47:45+01:00] Listening on 0.0.0.0:30000 ([::]:30000) for remote 172.19.255.200:9094
INFO[2025-01-26T15:47:45+01:00] Ready for new connections
INFO[2025-01-26T15:51:17+01:00] New connection for 172.19.255.200:9094
INFO[2025-01-26T15:51:17+01:00] Starting dynamic listener for broker 172.19.255.201:9094
INFO[2025-01-26T15:51:17+01:00] Listening on 0.0.0.0:30001 ([::]:30001) for remote 172.19.255.201:9094 broker 0
INFO[2025-01-26T15:51:17+01:00] Dynamic listener 0.0.0.0:30001 for broker 172.19.255.201:9094 brokerId 0 advertised as 0.0.0.0:30001
INFO[2025-01-26T15:51:17+01:00] Starting dynamic listener for broker 172.19.255.203:9094
INFO[2025-01-26T15:51:17+01:00] Listening on 0.0.0.0:30003 ([::]:30003) for remote 172.19.255.203:9094 broker 2
INFO[2025-01-26T15:51:17+01:00] Dynamic listener 0.0.0.0:30003 for broker 172.19.255.203:9094 brokerId 2 advertised as 0.0.0.0:30003
INFO[2025-01-26T15:51:17+01:00] Starting dynamic listener for broker 172.19.255.202:9094
INFO[2025-01-26T15:51:17+01:00] Listening on 0.0.0.0:30002 ([::]:30002) for remote 172.19.255.202:9094 broker 1
INFO[2025-01-26T15:51:17+01:00] Dynamic listener 0.0.0.0:30002 for broker 172.19.255.202:9094 brokerId 1 advertised as 0.0.0.0:30002
INFO[2025-01-26T15:51:17+01:00] New connection for 172.19.255.203:9094 brokerId 2
INFO[2025-01-26T15:51:17+01:00] Client closed local connection on 127.0.0.1:30003 from 127.0.0.1:57362 (172.19.255.203:9094)
INFO[2025-01-26T15:51:17+01:00] Client closed local connection on 127.0.0.1:30000 from 127.0.0.1:59330 (172.19.255.200:9094)
INFO[2025-01-26T15:53:42+01:00] New connection for 172.19.255.200:9094
INFO[2025-01-26T15:53:42+01:00] Starting dynamic listener for broker 172.19.255.205:9094
INFO[2025-01-26T15:53:42+01:00] Listening on 0.0.0.0:30005 ([::]:30005) for remote 172.19.255.205:9094 broker 4
INFO[2025-01-26T15:53:42+01:00] Dynamic listener 0.0.0.0:30005 for broker 172.19.255.205:9094 brokerId 4 advertised as 0.0.0.0:30005
INFO[2025-01-26T15:53:42+01:00] Starting dynamic listener for broker 172.19.255.204:9094
INFO[2025-01-26T15:53:42+01:00] Listening on 0.0.0.0:30004 ([::]:30004) for remote 172.19.255.204:9094 broker 3
INFO[2025-01-26T15:53:42+01:00] Dynamic listener 0.0.0.0:30004 for broker 172.19.255.204:9094 brokerId 3 advertised as 0.0.0.0:30004
INFO[2025-01-26T15:53:42+01:00] New connection for 172.19.255.203:9094 brokerId 2
INFO[2025-01-26T15:53:42+01:00] Client closed local connection on 127.0.0.1:30003 from 127.0.0.1:56460 (172.19.255.203:9094)
INFO[2025-01-26T15:53:42+01:00] Client closed local connection on 127.0.0.1:30000 from 127.0.0.1:60708 (172.19.255.200:9094)
INFO[2025-01-26T15:54:03+01:00] New connection for 172.19.255.200:9094
INFO[2025-01-26T15:54:04+01:00] New connection for 172.19.255.203:9094 brokerId 2
INFO[2025-01-26T15:54:04+01:00] Client closed local connection on 127.0.0.1:30003 from 127.0.0.1:39414 (172.19.255.203:9094)
INFO[2025-01-26T15:54:04+01:00] Client closed local connection on 127.0.0.1:30000 from 127.0.0.1:50600 (172.19.255.200:9094)
.....
INFO[2025-01-26T15:56:28+01:00] New connection for 172.19.255.200:9094
INFO[2025-01-26T15:56:28+01:00] Starting dynamic listener for broker 172.19.255.207:9094
INFO[2025-01-26T15:56:28+01:00] Broker address changed listener 0.0.0.0:30005 for new address 172.19.255.207:9094 old address 172.19.255.205:9094 brokerId 4 advertised as 0.0.0.0:30005
INFO[2025-01-26T15:56:28+01:00] Starting dynamic listener for broker 172.19.255.206:9094
INFO[2025-01-26T15:56:28+01:00] Broker address changed listener 0.0.0.0:30004 for new address 172.19.255.206:9094 old address 172.19.255.204:9094 brokerId 3 advertised as 0.0.0.0:30004
INFO[2025-01-26T15:56:28+01:00] New connection for 172.19.255.203:9094 brokerId 2
INFO[2025-01-26T15:56:28+01:00] Client closed local connection on 127.0.0.1:30003 from 127.0.0.1:37454 (172.19.255.203:9094)
INFO[2025-01-26T15:56:28+01:00] Client closed local connection on 127.0.0.1:30000 from 127.0.0.1:60850 (172.19.255.200:9094)
INFO[2025-01-26T15:56:56+01:00] New connection for 172.19.255.200:9094
INFO[2025-01-26T15:56:56+01:00] New connection for 172.19.255.203:9094 brokerId 2
INFO[2025-01-26T15:56:56+01:00] Client closed local connection on 127.0.0.1:30003 from 127.0.0.1:58096 (172.19.255.203:9094)
INFO[2025-01-26T15:56:56+01:00] Client closed local connection on 127.0.0.1:30000 from 127.0.0.1:53848 (172.19.255.200:9094)
INFO[2025-01-26T15:57:58+01:00] New connection for 172.19.255.200:9094
INFO[2025-01-26T15:57:58+01:00] New connection for 172.19.255.206:9094 brokerId 3
INFO[2025-01-26T15:57:58+01:00] Client closed local connection on 127.0.0.1:30004 from 127.0.0.1:56078 (172.19.255.206:9094)
INFO[2025-01-26T15:57:58+01:00] Client closed local connection on 127.0.0.1:30000 from 127.0.0.1:59014 (172.19.255.200:9094)
INFO[2025-01-26T15:58:08+01:00] New connection for 172.19.255.200:9094
INFO[2025-01-26T15:58:08+01:00] New connection for 172.19.255.206:9094 brokerId 3
INFO[2025-01-26T15:58:08+01:00] New connection for 172.19.255.203:9094 brokerId 2
INFO[2025-01-26T15:58:08+01:00] Client closed local connection on 127.0.0.1:30003 from 127.0.0.1:58384 (172.19.255.203:9094)
INFO[2025-01-26T15:58:08+01:00] Client closed local connection on 127.0.0.1:30004 from 127.0.0.1:48404 (172.19.255.206:9094)
INFO[2025-01-26T15:58:08+01:00] Client closed local connection on 127.0.0.1:30000 from 127.0.0.1:38436 (172.19.255.200:9094)
INFO[2025-01-26T15:58:20+01:00] New connection for 172.19.255.200:9094
INFO[2025-01-26T15:58:20+01:00] New connection for 172.19.255.207:9094 brokerId 4
INFO[2025-01-26T15:58:21+01:00] Client closed local connection on 127.0.0.1:30000 from 127.0.0.1:59006 (172.19.255.200:9094)
INFO[2025-01-26T15:58:21+01:00] Client closed local connection on 127.0.0.1:30005 from 127.0.0.1:46390 (172.19.255.207:9094)
INFO[2025-01-26T15:58:24+01:00] New connection for 172.19.255.200:9094
INFO[2025-01-26T15:58:24+01:00] New connection for 172.19.255.207:9094 brokerId 4
INFO[2025-01-26T15:58:24+01:00] Client closed local connection on 127.0.0.1:30000 from 127.0.0.1:59022 (172.19.255.200:9094)
INFO[2025-01-26T15:58:24+01:00] Client closed local connection on 127.0.0.1:30005 from 127.0.0.1:46400 (172.19.255.207:9094)