Compression process deadlocks with drop hypertable.

[simfleischman]

Relevant system information:

• OS: Mac OS, Linux
• PostgreSQL version (output of postgres --version): 12.4
• TimescaleDB version (output of \dx in psql): 1.7.3
• Installation method: Homebrew on Mac, Nix on Linux

Describe the bug
The compression process and the drop table (for a hypertable) command deadlock.

To Reproduce
Steps to reproduce the behavior:

1. Create a hypertable with a compression policy
2. Write a migration that copies the data into a new hypertable and drops the old one
3. Run the migration at the same time that the compression runs. This happens more easily when starting up the database then immediately running the migration, so that the compression runs.

Expected behavior
The drop table succeeds.

Actual behavior
Here is the relevant log from a deadlock we encountered.

2020-10-06 23:45:34.390 GMT [36920] LOG:  process 36920 still waiting for AccessExclusiveLock on relation 18547 of database 16919 after 1001.008 ms
2020-10-06 23:45:34.390 GMT [36920] DETAIL:  Process holding the lock: 36927. Wait queue: 36920.
2020-10-06 23:45:42.520 GMT [36927] ERROR:  deadlock detected
2020-10-06 23:45:42.520 GMT [36927] DETAIL:  Process 36927 waits for AccessExclusiveLock on relation 18547 of database 16919; blocked by process 36920.
	Process 36920 waits for AccessExclusiveLock on relation 18547 of database 16919; blocked by process 36927.
	Process 36927: -- migration
	Process 36920: <command string not enabled>
2020-10-06 23:45:42.532 GMT [36920] LOG:  process 36920 acquired AccessExclusiveLock on relation 18547 of database 16919 after 9142.986 ms
2020-10-06 23:45:42.727 GMT [36920] LOG:  completed compressing chunk _timescaledb_internal._hyper_1_16_chunk
2020-10-06 23:45:42.737 GMT [36920] LOG:  job 1000 completed compressing chunk

Screenshots
The relation 18547 is from the _timescaledb_internal._hyper_1_16_chunk, and this chunk belongs to the hypertable that is being dropped.

db=> select 18547::regclass;
                regclass                 
-----------------------------------------
 _timescaledb_internal._hyper_1_16_chunk

Additional context
The problem occurred in migration testing where the database starting up seems to trigger the compression process at the same time that we run the migrations.

[simfleischman]

We initially thought it was related to #902 since it was a migration with a large of amount of inserts (20M rows).

However, upon investigation we found the deadlock was on a chunk table from the hypertable being dropped, not the hypertable(s) that we were inserting into.

Workaround: disable compression on the hypertables being dropped at the beginning of the migration.

[NunoFilipeSantos]

@simfleischman can you please try against our latest TimescaleDB version (2.4.2) and get back to us?

[simfleischman]

We have no timeline for updating but when we do I will report back. Thanks

[NunoFilipeSantos]

Thank you for letting us know.
Closing for now. Feel free to reopen it if you can reproduce it with the latest version.