-
Notifications
You must be signed in to change notification settings - Fork 117
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(router): enable using redis clusters for rate limiting and apq #1499
Merged
df-wg
merged 15 commits into
main
from
dave/eng-6144-verify-use-of-redis-cluster-mode-for-apq
Jan 31, 2025
Merged
feat(router): enable using redis clusters for rate limiting and apq #1499
df-wg
merged 15 commits into
main
from
dave/eng-6144-verify-use-of-redis-cluster-mode-for-apq
Jan 31, 2025
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
df-wg
force-pushed
the
dave/eng-6144-verify-use-of-redis-cluster-mode-for-apq
branch
from
January 8, 2025 17:10
f26bd87
to
411b670
Compare
df-wg
commented
Jan 8, 2025
df-wg
commented
Jan 8, 2025
router/internal/persistedoperation/operationstorage/redis/rdcloser_test.go
Outdated
Show resolved
Hide resolved
Router image scan passed✅ No security vulnerabilities found in image:
|
StarpTech
requested changes
Jan 9, 2025
router/internal/persistedoperation/operationstorage/redis/rdcloser.go
Outdated
Show resolved
Hide resolved
router/internal/persistedoperation/operationstorage/redis/rdcloser.go
Outdated
Show resolved
Hide resolved
router/internal/persistedoperation/operationstorage/redis/rdcloser.go
Outdated
Show resolved
Hide resolved
StarpTech
requested changes
Jan 9, 2025
router/internal/persistedoperation/operationstorage/redis/rdcloser.go
Outdated
Show resolved
Hide resolved
df-wg
force-pushed
the
dave/eng-6144-verify-use-of-redis-cluster-mode-for-apq
branch
from
January 22, 2025 07:13
a152f50
to
fbe58fc
Compare
chensunny
reviewed
Jan 26, 2025
chensunny
reviewed
Jan 26, 2025
df-wg
force-pushed
the
dave/eng-6144-verify-use-of-redis-cluster-mode-for-apq
branch
from
January 27, 2025 08:35
de3fe03
to
6c94921
Compare
df-wg
force-pushed
the
dave/eng-6144-verify-use-of-redis-cluster-mode-for-apq
branch
from
January 30, 2025 08:12
d9388bb
to
27b8c6d
Compare
StarpTech
reviewed
Jan 31, 2025
router/internal/persistedoperation/operationstorage/redis/rdcloser.go
Outdated
Show resolved
Hide resolved
StarpTech
approved these changes
Jan 31, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
df-wg
deleted the
dave/eng-6144-verify-use-of-redis-cluster-mode-for-apq
branch
January 31, 2025 09:04
james-braund-cabiri
added a commit
to cabiri-io/cosmo
that referenced
this pull request
Jan 31, 2025
* feat: expose type data and record subgraphs for enums (wundergraph#1495) * chore(release): Publish [skip ci] - [email protected] - [email protected] - @wundergraph/[email protected] - [email protected] - @wundergraph/[email protected] - [email protected] * feat: improve rate limit responses (add code, hide stats) (wundergraph#1497) * chore(release): Publish [skip ci] - [email protected] * fix: provider should be specified in the config.yaml (wundergraph#1397) * fix: update the timeouts for clickhouse and platform service (wundergraph#1500) * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] * fix: add edfs to the demo environment (wundergraph#1505) * docs(CONTRIBUTING): fixup minor mistake in CONTRIBUTING.md under Go workspace (wundergraph#1502) Co-authored-by: Dustin Deus <[email protected]> * fix: full demo broken in main branch (wundergraph#1508) * feat(router): optionally add jitter to config polling interval (wundergraph#1506) Co-authored-by: Dustin Deus <[email protected]> * chore(release): Publish [skip ci] - [email protected] * fix(router): remove wildcard from router graphql path (wundergraph#1509) * fix: use gauge for server.uptime metric (wundergraph#1510) Co-authored-by: Ludwig <[email protected]> * feat: cache warmer (wundergraph#1501) Co-authored-by: Ludwig <[email protected]> Co-authored-by: starptech <[email protected]> * chore(release): Publish [skip ci] - [email protected] - @wundergraph/[email protected] - [email protected] - @wundergraph/[email protected] - [email protected] - [email protected] - [email protected] - @wundergraph/[email protected] - [email protected] * fix(cache warmup): consider only po of the last 7 days (wundergraph#1513) * chore(release): Publish [skip ci] - [email protected] * fix(cache operation): swallow cache errors and other improvements (wundergraph#1515) * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] - [email protected] * feat: add variables remapping support (wundergraph#1516) Co-authored-by: starptech <[email protected]> * chore(release): Publish [skip ci] - [email protected] * fix(router): write proper line endings and header for multipart (wundergraph#1517) * chore(release): Publish [skip ci] - [email protected] * feat(router): optimize playground delivery, add concurrency_limit to config (wundergraph#1519) * fix(router): enable health checks during startup (wundergraph#1529) * feat: improve cache warmer (wundergraph#1530) Co-authored-by: Ludwig <[email protected]> * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] * fix: remove semaphore from ResolveGraphQLSubscription (wundergraph#1532) * chore(release): Publish [skip ci] - [email protected] * feat: add compatibility handshake between router and execution config (wundergraph#1534) * chore(release): Publish [skip ci] - [email protected] - @wundergraph/[email protected] - @wundergraph/[email protected] - [email protected] - [email protected] - @wundergraph/[email protected] - [email protected] * feat: also add handshake for static execution configs (wundergraph#1535) * chore(router): bump demo library to pickup subscription fix (wundergraph#1518) * feat(router): add interface for trace propagation (wundergraph#1526) * chore(release): Publish [skip ci] - [email protected] * fix: adding/removing directive is not picked up by wgc subgraph check (wundergraph#1494) * chore(deps): upgrade ristretto to v2 (wundergraph#1538) * feat: add normalizedQuery to query plan and request info to trace (wundergraph#1536) Co-authored-by: df-wg <[email protected]> * fix: add copy button to subgraph routing url (wundergraph#1543) Co-authored-by: Dustin Deus <[email protected]> * fix: webhooks shot when schema is unchanged (wundergraph#1542) * fix: trim the inputs of group mappers (wundergraph#1541) * fix: subgraphs search functionality (wundergraph#1540) * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] - [email protected] * fix: increase max concurrent resolvers (wundergraph#1544) * refactor(router): redesign JWK authentication logic (wundergraph#1498) * chore(release): Publish [skip ci] - [email protected] * fix: increase the test timeout value to prevent failures on slower machines (wundergraph#1547) * fix: reduce the breaking change retention duration (wundergraph#1550) * fix: change the defaults of breaking-change-retention (wundergraph#1551) * feat(router): enable starting the router without subgraphs (wundergraph#1533) * fix(router): parse accept header per rfc 9110 (wundergraph#1549) * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] * feat(router): enable using redis clusters for rate limiting and apq (wundergraph#1499) * fix: json schema for traffic shaping subgraphs (wundergraph#1552) * chore: Update aws-lambda-router customisation after upstream sync --------- Co-authored-by: Nithin Kumar B <[email protected]> Co-authored-by: hardworker-bot <[email protected]> Co-authored-by: Jens Neuse <[email protected]> Co-authored-by: Alessandro Pagnin <[email protected]> Co-authored-by: Suvij Surya <[email protected]> Co-authored-by: endigma <[email protected]> Co-authored-by: Dustin Deus <[email protected]> Co-authored-by: Ludwig <[email protected]> Co-authored-by: Sergiy 🇺🇦 <[email protected]> Co-authored-by: df-wg <[email protected]> Co-authored-by: Aenimus <[email protected]>
james-braund-cabiri
added a commit
to cabiri-io/cosmo
that referenced
this pull request
Feb 4, 2025
* feat: expose type data and record subgraphs for enums (wundergraph#1495) * chore(release): Publish [skip ci] - [email protected] - [email protected] - @wundergraph/[email protected] - [email protected] - @wundergraph/[email protected] - [email protected] * feat: improve rate limit responses (add code, hide stats) (wundergraph#1497) * chore(release): Publish [skip ci] - [email protected] * fix: provider should be specified in the config.yaml (wundergraph#1397) * fix: update the timeouts for clickhouse and platform service (wundergraph#1500) * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] * fix: add edfs to the demo environment (wundergraph#1505) * docs(CONTRIBUTING): fixup minor mistake in CONTRIBUTING.md under Go workspace (wundergraph#1502) Co-authored-by: Dustin Deus <[email protected]> * fix: full demo broken in main branch (wundergraph#1508) * feat(router): optionally add jitter to config polling interval (wundergraph#1506) Co-authored-by: Dustin Deus <[email protected]> * chore(release): Publish [skip ci] - [email protected] * fix(router): remove wildcard from router graphql path (wundergraph#1509) * fix: use gauge for server.uptime metric (wundergraph#1510) Co-authored-by: Ludwig <[email protected]> * feat: cache warmer (wundergraph#1501) Co-authored-by: Ludwig <[email protected]> Co-authored-by: starptech <[email protected]> * chore(release): Publish [skip ci] - [email protected] - @wundergraph/[email protected] - [email protected] - @wundergraph/[email protected] - [email protected] - [email protected] - [email protected] - @wundergraph/[email protected] - [email protected] * fix(cache warmup): consider only po of the last 7 days (wundergraph#1513) * chore(release): Publish [skip ci] - [email protected] * fix(cache operation): swallow cache errors and other improvements (wundergraph#1515) * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] - [email protected] * feat: add variables remapping support (wundergraph#1516) Co-authored-by: starptech <[email protected]> * chore(release): Publish [skip ci] - [email protected] * fix(router): write proper line endings and header for multipart (wundergraph#1517) * chore(release): Publish [skip ci] - [email protected] * feat(router): optimize playground delivery, add concurrency_limit to config (wundergraph#1519) * fix(router): enable health checks during startup (wundergraph#1529) * feat: improve cache warmer (wundergraph#1530) Co-authored-by: Ludwig <[email protected]> * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] * fix: remove semaphore from ResolveGraphQLSubscription (wundergraph#1532) * chore(release): Publish [skip ci] - [email protected] * feat: add compatibility handshake between router and execution config (wundergraph#1534) * chore(release): Publish [skip ci] - [email protected] - @wundergraph/[email protected] - @wundergraph/[email protected] - [email protected] - [email protected] - @wundergraph/[email protected] - [email protected] * feat: also add handshake for static execution configs (wundergraph#1535) * chore(router): bump demo library to pickup subscription fix (wundergraph#1518) * feat(router): add interface for trace propagation (wundergraph#1526) * chore(release): Publish [skip ci] - [email protected] * fix: adding/removing directive is not picked up by wgc subgraph check (wundergraph#1494) * chore(deps): upgrade ristretto to v2 (wundergraph#1538) * feat: add normalizedQuery to query plan and request info to trace (wundergraph#1536) Co-authored-by: df-wg <[email protected]> * fix: add copy button to subgraph routing url (wundergraph#1543) Co-authored-by: Dustin Deus <[email protected]> * fix: webhooks shot when schema is unchanged (wundergraph#1542) * fix: trim the inputs of group mappers (wundergraph#1541) * fix: subgraphs search functionality (wundergraph#1540) * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] - [email protected] * fix: increase max concurrent resolvers (wundergraph#1544) * refactor(router): redesign JWK authentication logic (wundergraph#1498) * chore(release): Publish [skip ci] - [email protected] * fix: increase the test timeout value to prevent failures on slower machines (wundergraph#1547) * fix: reduce the breaking change retention duration (wundergraph#1550) * fix: change the defaults of breaking-change-retention (wundergraph#1551) * feat(router): enable starting the router without subgraphs (wundergraph#1533) * fix(router): parse accept header per rfc 9110 (wundergraph#1549) * chore(release): Publish [skip ci] - [email protected] - [email protected] - [email protected] * feat(router): enable using redis clusters for rate limiting and apq (wundergraph#1499) * fix: json schema for traffic shaping subgraphs (wundergraph#1552) * fix: subgraph timeout can't be bigger than global timeout (wundergraph#1548) * fix: error when graph token is not set when cache warmup is enabled (wundergraph#1554) * chore(release): Publish [skip ci] - [email protected] * fix: incorrect graphql endpoint in playground (wundergraph#1562) * chore(release): Publish [skip ci] - @wundergraph/[email protected] - [email protected] * fix: update vulnerable packages (wundergraph#1560) * fix: synchronize go mod versions (wundergraph#1564) * chore: reduce verbose logging for failed tests (wundergraph#1565) * fix: Add missing config mapping, bump aws-lambda-router version * fix: Repair PNPM lockfile after merge --------- Co-authored-by: Nithin Kumar B <[email protected]> Co-authored-by: hardworker-bot <[email protected]> Co-authored-by: Jens Neuse <[email protected]> Co-authored-by: Alessandro Pagnin <[email protected]> Co-authored-by: Suvij Surya <[email protected]> Co-authored-by: endigma <[email protected]> Co-authored-by: Dustin Deus <[email protected]> Co-authored-by: Ludwig <[email protected]> Co-authored-by: Sergiy 🇺🇦 <[email protected]> Co-authored-by: df-wg <[email protected]> Co-authored-by: Aenimus <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Previously, users were only able to provide a single redis instance to Cosmo for both APQ/Rate Limiting. This didn't take advantage of Redis' built-in cluster mode, which takes care of horizontal scaling for the users.
This PR enables that. In order to do it, users can provide a list of their cluster URLs instead of the singular URL they used to provide, in the now renamed
urls
field. In order to opt in to cluster mode, users have to setcluster_enabled: true
in their configuration for both APQ and Rate LimitingWarning
As part of the preparations for Cosmo V1, targeted for release in Q1 2025, this pull request introduces essential changes to enhance long-term stability and maintainability. While we strive to minimize breaking changes, they are sometimes necessary to lay the foundation for a more robust and scalable system.
Before:
After
Migration Path:
[ ] Rename
storage_providers.redis.url
tostorage_providers.redis.urls
, andrate_limit.storage.url
torate_limit.storage.urls
, as the first value of a listChecklist