-
Notifications
You must be signed in to change notification settings - Fork 107
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
3130 site bug issue with the page self hostedlatestbackup and restore #3626
Changes from 5 commits
4353a4c
827ad45
e59b15a
628d66f
32f20f2
6a2e900
5d15a86
d35765f
f3df11d
cdbab1c
18f3315
c9c519f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,188 @@ | ||
--- | ||
title: Logical backup with pg_dump and pg_restore | ||
excerpt: Back up and restore a hypertable or an entire database using native PostgreSQL commands | ||
keywords: [backups, restore] | ||
tags: [recovery, logical backup, pg_dump, pg_restore] | ||
--- | ||
|
||
# Logical backup with `pg_dump` and `pg_restore` | ||
|
||
You backup and restore each self-hosted PostgreSQL database with TimescaleDB enabled using the native | ||
PostgreSQL [`pg_dump`][pg_dump] and [`pg_restore`][pg_restore] commands. This also works for compressed hypertables, | ||
you don't have to decompress the chunks before you begin. | ||
|
||
If you are using `pg_dump` to backup regularly, make sure you keep | ||
track of the versions of PostgreSQL and TimescaleDB you are running. For more | ||
information, see [Versions are mismatched when dumping and restoring a database][troubleshooting-version-mismatch]. | ||
|
||
This page shows you how to: | ||
|
||
- [Back up and restore an entire database][backup-entire-database] | ||
- [Back up and restore individual hypertables][backup-individual-tables] | ||
|
||
You can also [upgrade between different versions of TimescaleDB][timescaledb-upgrade]. | ||
|
||
## Prerequisites | ||
|
||
- A source database to backup from, and a target database to restore to. | ||
- Install the PostgreSQL client tools on your migration machine. | ||
|
||
This includes `psql`, and `pg_dump`. | ||
|
||
## Back up and restore an entire database | ||
|
||
You backup and restore an entire database using `pg_dump` and `psql`. in Terminal: | ||
|
||
<Procedure> | ||
|
||
In Terminal: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "In terminal" is repeated. Also, let's use keep "terminal" lower case. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Dearie me, thank you. Updated. |
||
|
||
1. **Set your connection strings** | ||
|
||
These variables hold the connection information for the source database to backup from and | ||
the target database to restore to: | ||
|
||
```bash | ||
export SOURCE=postgres://<user>:<password>@<source host>:<source port>/<db_name> | ||
export TARGET=postgres://<user>:<password>@<source host>:<source port> | ||
``` | ||
|
||
1. **Backup your database** | ||
|
||
```bash | ||
pg_dump -d "$SOURCE" \ | ||
-Fc -f <db_name>.bak | ||
``` | ||
You may see some errors while `pg_dump` is running. See [Troubleshooting self-hosted TimescaleDB][troubleshooting] | ||
to check if they can be safely ignored. | ||
|
||
1. **Restore your database from the backup** | ||
|
||
1. Connect to your target database: | ||
```bash | ||
psql -d "$TARGET" | ||
``` | ||
|
||
1. Create a new database and enable TimescaleDB: | ||
|
||
```sql | ||
CREATE DATABASE <restoration database>; | ||
\c <restoration database> | ||
CREATE EXTENSION IF NOT EXISTS timescaledb; | ||
``` | ||
|
||
1. Put your database in the right state for restoring: | ||
|
||
```sql | ||
SELECT timescaledb_pre_restore(); | ||
``` | ||
|
||
1. Restore the database: | ||
|
||
```sql | ||
\! pg_restore -Fc -d <restoration database> <db_name>.bak | ||
billy-the-fish marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
|
||
1. Return your database to normal operations: | ||
|
||
```sql | ||
SELECT timescaledb_post_restore(); | ||
``` | ||
Do not use `pg_restore` with the `-j` option. This option does not correctly restore the | ||
TimescaleDB catalogs. | ||
|
||
</Procedure> | ||
|
||
|
||
## Back up and restore individual hypertables | ||
|
||
`pg_dump` provides flags that allow you to specify tables or schemas | ||
to back up. However, using these flags means that the dump lacks necessary | ||
information that TimescaleDB requires to understand the relationship between | ||
them. Even if you explicitly specify both the hypertable and all of its | ||
constituent chunks, the dump would still not contain all the information it | ||
needs to recreate the hypertable on restore. | ||
|
||
To backup individual hypertables, backup the database schema, then backup only the tables | ||
you need. You also use this method to backup individual plain tables. | ||
|
||
<procedure> | ||
In Terminal: | ||
|
||
1. **Set your connection strings** | ||
|
||
These variables hold the connection information for the source database to backup from and | ||
the target database to restore to: | ||
|
||
```bash | ||
export SOURCE=postgres://<user>:<password>@<source host>:<source port>/<db_name> | ||
export TARGET=postgres://<user>:<password>@<source host>:<source port>/<db_name> | ||
``` | ||
|
||
1. **Backup the database schema and individual tables** | ||
|
||
1. Back up the hypertable schema: | ||
|
||
```bash | ||
pg_dump -s -d $SOURCE --table conditions -N _timescaledb_internal | \ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I couldn't' t understand the reason for adding There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. :-), i would never have got that one. Removed. |
||
grep -v _timescaledb_internal > schema.sql | ||
``` | ||
|
||
1. Backup hypertable data to a CSV file: | ||
|
||
For each hypertable to backup: | ||
```bash | ||
psql -d $SOURCE \ | ||
-c "\COPY (SELECT * FROM <table-name>) TO <table-name>.csv DELIMITER ',' CSV" | ||
``` | ||
|
||
1. **Restore the schema to the target database** | ||
|
||
```bash | ||
psql -d $TARGET < schema.sql | ||
``` | ||
|
||
1. **Restore hypertables from the backup** | ||
|
||
For each hypertable to backup: | ||
1. Recreate the hypertable: | ||
|
||
```bash | ||
psql -d $TARGET -c "SELECT create_hypertable(<table-name>, <partition>)" | ||
``` | ||
When you [create the new hypertable][create_hypertable], you do not need to use the | ||
same parameters as existed in the source database. This | ||
can provide a good opportunity for you to re-organize your hypertables if | ||
you need to. For example, you can change the partitioning key, the number of | ||
partitions, or the chunk interval sizes. | ||
|
||
1. Restore the data: | ||
|
||
```bash | ||
psql -d $TARGET -c "\COPY <table-name> FROM <table-name>.csv CSV" | ||
``` | ||
|
||
The standard `COPY` command in PostgreSQL is single threaded. If you have a | ||
lot of data, you can speed up the copy using the [timescaledb-parallel-copy][parallel importer]. | ||
|
||
</procedure> | ||
|
||
Best practice is to backup and restore a database at a time. However, if you have superuser access to | ||
PostgreSQL instance with TimescaleDB installed, you can use `pg_dumpall` to backup all PostgreSQL databases in a | ||
cluster, including global objects that are common to all databases, namely database roles, tablespaces, | ||
and privilege grants. You restore the PostgreSQL instance using `psql`. For more information, see the | ||
[PostgreSQL documentation][postgres-docs]. | ||
|
||
|
||
[parallel importer]: https://github.com/timescale/timescaledb-parallel-copy | ||
[pg_dump]: https://www.postgresql.org/docs/current/static/app-pgdump.html | ||
[pg_restore]: https://www.postgresql.org/docs/current/static/app-pgrestore.html | ||
[timescaledb_pre_restore]: /api/:currentVersion:/administration/#timescaledb_pre_restore | ||
[timescaledb_post_restore]: /api/:currentVersion:/administration/#timescaledb_post_restore | ||
[timescaledb-upgrade]: /self-hosted/:currentVersion:/upgrades/ | ||
[troubleshooting]: /self-hosted/:currentVersion:/troubleshooting/ | ||
[troubleshooting-version-mismatch]: /self-hosted/:currentVersion:/troubleshooting/#versions-are-mismatched-when-dumping-and-restoring-a-database | ||
[postgres-docs]: https://www.postgresql.org/docs/17/backup-dump.html#BACKUP-DUMP-ALL | ||
[backup-entire-database]: /self-hosted/:currentVersion:/backup-and-restore/logical-backup/#back-up-and-restore-an-entire-database | ||
[backup-individual-tables]: /self-hosted/:currentVersion:/backup-and-restore/logical-backup/#back-up-and-restore-individual-hypertables | ||
[create_hypertable]: /api/:currentVersion:/hypertable/create_hypertable/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An extra line seems odd
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, updated.