do not remove publications of slot defined in manifest #2868
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With #2684 we introduced proper stream removal incl. database objects such as slots and publications which helped in cases when
a. users decided to abandon/pause the stream idea after a testing phase
b. the removal of the slot became necessary due to problems with WAL consumption and growing disk space.
Another case might be that users want to migrate to self-deploy the stream resource because it offers more flexibility compared to what our operator will ever offer. Of course, they could turn streams off and on again which would mean a loss of decoded events. If they do not want to lose any event data, they have to
a. define the slot with the exact same name
fes_<dbname>_<appId>
in the manifestb. add parameter
wal_level: logical
topostgresql
sectionc. create a fes_user replication user.
However, the current implementation only persist the slots, not the publication, making the migration impossible. The problem lies in the initialization of publications to sync. For every database we create an empty struct to allow for later removal. When the following spec.Streams loop is skipped
syncPublication
is later called with an emptydatabaseSlotList
which will then remove the publications.This PR suggests to set the slotName in the databaseSlotList when it is specified in the manifest. I could do an additional check if the plugin is also pgoutput, but maybe that's not really necessary (nobody should switch to wal2json). In the syncPublication code I am simply using the current publication tables when the new table list is empty. One cannot define publications in the manifest.