Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhancement: Parallel upload/copy of backups #138

Closed
MaxJa4 opened this issue Aug 5, 2022 · 1 comment · Fixed by #152
Closed

Enhancement: Parallel upload/copy of backups #138

MaxJa4 opened this issue Aug 5, 2022 · 1 comment · Fixed by #152
Labels
enhancement New feature or request

Comments

@MaxJa4
Copy link
Contributor

MaxJa4 commented Aug 5, 2022

Here lies an opportunity for adding a real world feature on top of this refactoring: as copying is a read only operation, we could now run all of the operations in parallel and considerably speed up execution times for people that back up to multiple backends. Do you think we should add this here?

In case you're interested, here's an example how it can be done:

g := new(errgroup.Group)
for _, container := range containersWithCommand {
c := container
g.Go(func() error {
cmd, ok := c.Labels[label]
if !ok && label == "docker-volume-backup.archive-pre" {
cmd, _ = c.Labels["docker-volume-backup.exec-pre"]
} else if !ok && label == "docker-volume-backup.archive-post" {
cmd, _ = c.Labels["docker-volume-backup.exec-post"]
}
s.logger.Infof("Running %s command %s for container %s", label, cmd, strings.TrimPrefix(c.Names[0], "/"))
stdout, stderr, err := s.exec(c.ID, cmd)
if s.c.ExecForwardOutput {
os.Stderr.Write(stderr)
os.Stdout.Write(stdout)
}
if err != nil {
return fmt.Errorf("runLabeledCommands: error executing command: %w", err)
}
return nil
})
}

Originally posted by @m90 in #135 (comment)

@MaxJa4
Copy link
Contributor Author

MaxJa4 commented Aug 5, 2022

Just to keep track and not forget that!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants