Releases: planetscale/airbyte-source
Releases · planetscale/airbyte-source
v1.39.0
Changelog
- 0c0cb75 Merge pull request #122 from planetscale/add-log
- 3612a67 Use SWITCH statement instead
- ccee5e1 Use query type
- 368bdc2 Use date without time for date
- 25d1b3b strip all whitespaces before comparing in test
- 10c7b71 Add test
- ec71b67 Merge branch 'main' of github.com:planetscale/airbyte-source into add-log
- 7deb2ef Add log indicating whether we are looking for a stop position or a COPY COMPLETED
v1.38.0
Changelog
- af3d135 Merge pull request #121 from planetscale/more-improvements
- 67483b2 Compare space-trimmed logs in test for consistency
- 8e916b0 Merge branch 'main' into more-improvements
- 0991c12 Add tests to test correct LastPK passed when full copy, and not when incremental
- 6f8ad97 Update spaces in example test log output
- 8feba64 Add test for max retries
- 477af45 Add err to test mocks
- ae9b76a Update expected log format in tests
- 95ac56f Add configurable amount of retries
v1.37.0
Changelog
- e84e4ba Merge pull request #120 from planetscale/use-vstream
- 6689561 Add test for copy catchup
- 02057d4 Clean up protos
- 787467c Address some PR comments
- cbf56ba Remove extra spaces
- 7026de9 Remove unused method
- 2f4559e Handle case of no new VGTID found
- 0a6aed6 Clean up some more logs
- 3fffdc9 Simplify var names and logic
- 2c22667 Remove unused method
- 317637e Remove unused mock types
- 375dbc8 Finish tests
- 4b8c976 Update test testing stopping at next VGtid after stop position
- 6c4e381 Remove some noisy logs
- 45f45d5 Handle cases of copy phase and first VGTID is past stop position, and incremental (running) phase with no new rows
- 2238dcc Convert more tests
- 5b061ec Update mock types and modify first few tests
- 3c744d6 re run proto
- 0baf971 Clean up logs more
- a873826 Refactor and reuse some logic, add helpful logs
- 772b9ae Undo changes to makefile
- 00a549f Get local sync working
- 06f5d6d Get getLatestCursor working locally
- d77e959 Reset protos
- c47c820 Try modifying getLatestCursorPosition to use vtgate
- 21918b2 Merge branch 'use-vstream' of github.com:planetscale/airbyte-source into use-vstream
- 3f0fede Run tidy
- b2120e3 Update Makefile
- 309f337 Update proto/vtgateservice.proto
- 7a6e7a4 Add vtgate service
- f885974 Split out
- b1853c3 Adding VStream
v1.36.0
Changelog
- 907e108 Merge pull request #116 from planetscale/exit-earlier
- 51f5ca5 Add test for making sure long-running case no longer occurs
- cca64f0 Merge pull request #118 from planetscale/nickvanw/test-exit-earlier
- 9120be2 use older version of go
- 95ae6bf ruby dev
- 431e510 bump go container
- d3fd37c another openssl
- f957a81 openssl-dev
- f70f280 wip
- d943096 Keep old behavior of returning cursor that is after synced-up-to-cursor
- f83f7c5 Use real GTID in tests
- 83d0e65 Squash dockerfile commits
- 8493abd Try to fix openssl issue
- d8485ac Flush records and exit if stop cursor passed
Experimental: End sync as soon as current VGTID is equal to, or after desired stop VGTID
Experimental: End sync as soon as current VGTID is equal to, or after desired stop VGTID
Pre-release
Pre-release
Main change
This release is intended to be upgraded to "latest release" after verifying in production that it works as intended:
- If we reach a current VGTID that is equal to, or after, the desired stop VGTID, end the sync and flush records to Airbyte
How this is different
This differs from the implementations prior to 1.35.0, where the sync will look for a current VGTID that is exactly the desired stop VGTID.
However, sometimes because writes to the database continue to increase the current VGTID, by the time we request records from VStream, the current VGTID has already advanced past the desired stop VGTID. Therefore, the loop does not end.
Instead, the connector retries again with a new stop VGTID, and a new sync request. The new stop VGTID and new current VGTID will only be equal at some point where writes have slowed down, and no writes occur between the time of 1) fetching stop VGTID and 2) fetching current VGTID.
PRs
- Fix
license_finderby @nickvanw in #118 - Exit sync once stop cursor is passed by @notfelineit in #116
Full Changelog: v1.34.0...v1.35.0-experiemental
Contributors
nickvanw and notfelineit
Assets 2
v1.34.0
v1.33.0
Changelog
- df441e9 Merge pull request #114 from planetscale/add-more-logging
- 9a76cc8 update test
- 5a8f79a Merge branch 'add-more-logging' of github.com:planetscale/airbyte-source into add-more-logging
- 9247a75 Update test
- 5de42f2 try that
- 0559292 Merge branch 'add-more-logging' of github.com:planetscale/airbyte-source into add-more-logging
- 87f3f83 Move record logging to outer function
- 87df2d0 try that
- 9b55b09 try using ruby wolfi image
- 3cf059b Try openssl-dev
- 3d262e0 Typo use 3.4
- 55f8e14 Try ruby 4
- 931b3ca try openssl later version
- 6ed29c0 Try 3.3
- 414a232 Try downloading openssl
- cc7c6c2 Log results synced