Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-33265] Support source parallelism setting for Kafka connector #134

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

X-czh
Copy link

@X-czh X-czh commented Nov 16, 2024

What is the purpose of the change

Support source parallelism setting for Kafka & Upsert-Kafka connector.

Brief change log

  • Add new option scan.parallelism support for Kafka & Upsert-Kafka connector.

Verifying this change

  • Added UTs for parallelism option setting of Kafka & Upsert-Kafka connector.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): (yes / no) no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no) yes
  • The serializers: (yes / no / don't know) no
  • The runtime per-record code paths (performance sensitive): (yes / no / don't know) no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / no / don't know) no
  • The S3 file system connector: (yes / no / don't know) no

Documentation

  • Does this pull request introduce a new feature? (yes / no) yes
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented) docs

Copy link

boring-cyborg bot commented Nov 16, 2024

Thanks for opening this pull request! Please check out our contributing guidelines. (https://flink.apache.org/contributing/how-to-contribute.html)

@X-czh
Copy link
Author

X-czh commented Nov 16, 2024

@BIOINSu @libenchao Could you help take a review when you have time?

Copy link
Contributor

@RocMarshal RocMarshal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @X-czh for the contribution.
Looks good to me on the whole.
I left a few of comments.
Please let me know what's your opinon~

@X-czh
Copy link
Author

X-czh commented Nov 29, 2024

@RocMarshal Thanks for the comments, I've addressed them in a new commit.

@X-czh X-czh requested a review from RocMarshal November 29, 2024 14:03
Copy link
Contributor

@RocMarshal RocMarshal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick update~
LGTM +1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants