Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: SEGFAULT when using COPY together with pgaudit extension #7667

Open
TinT1 opened this issue Feb 7, 2025 · 4 comments
Open

[Bug]: SEGFAULT when using COPY together with pgaudit extension #7667

TinT1 opened this issue Feb 7, 2025 · 4 comments
Labels

Comments

@TinT1
Copy link

TinT1 commented Feb 7, 2025

What type of bug is this?

Crash

What subsystems and features are affected?

Query executor, Platform/OS

What happened?

We are operating TimescaleDB (2.17.2) in Kubernetes and we noticed segfaults when using COPY command on hypertable with pgaudit (17.0) enabled.
You can find logs and steps to reproduce below.

Thank you

TimescaleDB version affected

2.17.2

PostgreSQL version used

17.2

What operating system did you use?

Ubuntu 22.04.5 LTS

What installation method did you use?

Docker

What platform did you run on?

On prem/Self-hosted

Relevant log output and stack trace

## postgres log
server closed the connection unexpectedly
	This probably means the server terminated abnormally
	before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
The connection to the server was lost. Attempting reset: Failed.

## gdb log
Program received signal SIGSEGV, Segmentation fault.
0x0000ffff8fde3c40 in ?? () from /usr/lib/postgresql/17/lib/pgaudit.so
(gdb) bt
#0  0x0000ffff8fde3c40 in ?? () from /usr/lib/postgresql/17/lib/pgaudit.so
#1  0x0000ffff8fde40f8 in ?? () from /usr/lib/postgresql/17/lib/pgaudit.so
#2  0x0000fffeacbdd458 in copy_constraints_and_check (pstate=0xaaaaeeec8b18, rel=0xfffeacaa65c8,
    attnums=<optimized out>) at /build/timescaledb/src/copy.c:1228
#3  0x0000fffeacbec894 in timescaledb_DoCopy (ht=0xaaaaeee70d60, processed=<synthetic pointer>,
    queryString=0xaaaaeed972a8 "COPY conditions FROM '/home/postgres/conditions.csv' DELIMITER ',';",
    stmt=0xaaaaeed97de0) at /build/timescaledb/src/copy.c:1307
#4  process_copy (args=0xfffff671b9b8) at /build/timescaledb/src/process_utility.c:590
#5  0x0000fffeacbf34a4 in process_ddl_command_start (args=0xfffff671b9b8)
    at /build/timescaledb/src/process_utility.c:4398
#6  timescaledb_ddl_command_start (pstmt=0xaaaaeed97ee0, query_string=<optimized out>,
    readonly_tree=<optimized out>, context=PROCESS_UTILITY_TOPLEVEL, params=0x0,
    queryEnv=<optimized out>, dest=0xaaaaeed98448, completion_tag=0xfffff671bd48)
    at /build/timescaledb/src/process_utility.c:4619

How can we reproduce the bug?

docker run --name timescaledb -d -p 5432:5432 -e POSTGRES_PASSWORD=password -it timescale/timescaledb-ha:pg17.2-ts2.17.2

docker exec -it timescaledb bash -c "echo \"shared_preload_libraries = 'timescaledb,pgaudit'\" >> /home/postgres/pgdata/data/postgresql.conf"

docker exec -it timescaledb bash -c "echo '2025-02-07 10:30:00+00,New York,Device1,22.5,60.0' >> /home/postgres/conditions.csv"

docker container restart timescaledb

PGPASSWORD=password psql -h localhost -U postgres

CREATE EXTENSION pgaudit;
set pgaudit.log = 'all, -misc';

CREATE TABLE conditions (
   time        TIMESTAMPTZ       NOT NULL,
   location    TEXT              NOT NULL,
   device      TEXT              NOT NULL,
   temperature DOUBLE PRECISION  NULL,
   humidity    DOUBLE PRECISION  NULL
);

SELECT create_hypertable('conditions', by_range('time'));
COPY conditions FROM '/home/postgres/conditions.csv' DELIMITER ',';
@TinT1 TinT1 added the bug label Feb 7, 2025
@TinT1
Copy link
Author

TinT1 commented Feb 7, 2025

Hi,

I would also like to add that newer TimescaleDB (2.18.0) version fails with segfault even when using select 1;
You can reproduce it the same way, just run select 1; after set pgaudit.log = 'all, -misc';

Program received signal SIGSEGV, Segmentation fault. 0x0000ffff96393f88 in ?? () from /usr/lib/postgresql/17/lib/pgaudit.so (gdb) bt #0 0x0000ffff96393f88 in ?? () from /usr/lib/postgresql/17/lib/pgaudit.so #1 0x0000ffff963940f8 in ?? () from /usr/lib/postgresql/17/lib/pgaudit.so #2 0x0000aaaad37399d0 in InitPlan (eflags=32, queryDesc=0xaaaae5226328) at executor/./build/../src/backend/executor/execMain.c:850 #3 standard_ExecutorStart (queryDesc=queryDesc@entry=0xaaaae5226328, eflags=32) at executor/./build/../src/backend/executor/execMain.c:263 #4 0x0000fffeb30f2168 in capture_ExecutorStart (queryDesc=0xaaaae5226328, eflags=<optimized out>) at /build/timescaledb/tsl/src/hypercore/attr_capture.c:220 #5 0x0000aaaad393f0e0 in ExecutorStart (eflags=0, queryDesc=<optimized out>) at executor/./build/../src/backend/executor/execMain.c:137 #6 PortalStart (portal=portal@entry=0xaaaae527e108, params=params@entry=0x0, eflags=eflags@entry=0, snapshot=snapshot@entry=0x0) at tcop/./build/../src/backend/tcop/pquery.c:517 #7 0x0000aaaad393f898 in exec_simple_query ( query_string=query_string@entry=0xaaaae51f1188 "select 1;") at tcop/./build/../src/backend/tcop/postgres.c:1239 #8 0x0000aaaad39417b0 in PostgresMain (dbname=<optimized out>, username=<optimized out>) at tcop/./build/../src/backend/tcop/postgres.c:4767

@zilder
Copy link
Contributor

zilder commented Feb 14, 2025

Hi @TinT1,

Thanks for the bug report! I was able to reproduce it. The issue here is that both timescaledb and pgaudit are using the ExecutorStart hook, and normally the calls should be chained, but it doesn't happen here. I'm working on a fix.

@svenklemm
Copy link
Member

@TinT1
Copy link
Author

TinT1 commented Feb 25, 2025

Hi @zilder

Unfortunately, we are still encountering the same issue after upgrading to TimescaleDB 2.18.2, which should include the fix for #7712 .

I was able to reproduce the issue using the same steps, but with the timescale/timescaledb-ha:pg17.2-ts2.18.2 image.

Should we reopen the ticket? Did you manage to reproduce on 2.18.2 ?

Thanks.

gdb log

Program received signal SIGSEGV, Segmentation fault.
0x0000ffff85b03c40 in ?? () from /usr/lib/postgresql/17/lib/pgaudit.so
(gdb) bt
#0  0x0000ffff85b03c40 in ?? () from /usr/lib/postgresql/17/lib/pgaudit.so
#1  0x0000ffff85b040f8 in ?? () from /usr/lib/postgresql/17/lib/pgaudit.so
#2  0x0000fffea28ee738 in copy_constraints_and_check (pstate=0xaaab05566838, rel=0xfffea27652e8, attnums=<optimized out>) at /build/timescaledb/src/copy.c:1266
#3  0x0000fffea28fe48c in timescaledb_DoCopy (ht=0xaaab05572700, processed=<synthetic pointer>, queryString=0xaaab05462cc8 "COPY conditions FROM '/home/postgres/conditions.csv' DELIMITER ',';", stmt=0xaaab05463800)
    at /build/timescaledb/src/copy.c:1345
#4  process_copy (args=0xffffeb7ff288) at /build/timescaledb/src/process_utility.c:855
#5  0x0000fffea2905c74 in process_ddl_command_start (args=0xffffeb7ff288) at /build/timescaledb/src/process_utility.c:4877
#6  timescaledb_ddl_command_start (pstmt=0xaaab05463900, query_string=<optimized out>, readonly_tree=<optimized out>, context=PROCESS_UTILITY_TOPLEVEL, params=0x0, queryEnv=<optimized out>, dest=0xaaab05463e68,
    completion_tag=0xffffeb7ff618) at /build/timescaledb/src/process_utility.c:5100
#7  0x0000aaaae708e06c in ProcessUtility (qc=<optimized out>, dest=<optimized out>, queryEnv=<optimized out>, params=<optimized out>, context=PROCESS_UTILITY_TOPLEVEL, readOnlyTree=<optimized out>, queryString=<optimized out>,
    pstmt=<optimized out>) at tcop/./build/../src/backend/tcop/utility.c:519
#8  PortalRunUtility (portal=portal@entry=0xaaab054e5c28, pstmt=pstmt@entry=0xaaab05463900, isTopLevel=isTopLevel@entry=true, setHoldSnapshot=setHoldSnapshot@entry=false, dest=dest@entry=0xaaab05463e68, qc=qc@entry=0xffffeb7ff618)
    at tcop/./build/../src/backend/tcop/pquery.c:1158
#9  0x0000aaaae708e1f0 in PortalRunMulti (portal=portal@entry=0xaaab054e5c28, isTopLevel=isTopLevel@entry=true, setHoldSnapshot=setHoldSnapshot@entry=false, dest=dest@entry=0xaaab05463e68, altdest=altdest@entry=0xaaab05463e68,
    qc=qc@entry=0xffffeb7ff618) at tcop/./build/../src/backend/tcop/pquery.c:1315
#10 0x0000aaaae708e844 in PortalRun (portal=portal@entry=0xaaab054e5c28, count=count@entry=9223372036854775807, isTopLevel=isTopLevel@entry=true, run_once=run_once@entry=true, dest=dest@entry=0xaaab05463e68,
    altdest=altdest@entry=0xaaab05463e68, qc=qc@entry=0xffffeb7ff618) at tcop/./build/../src/backend/tcop/pquery.c:791
#11 0x0000aaaae708f950 in exec_simple_query (query_string=query_string@entry=0xaaab05462cc8 "COPY conditions FROM '/home/postgres/conditions.csv' DELIMITER ',';") at tcop/./build/../src/backend/tcop/postgres.c:1278
#12 0x0000aaaae70917b0 in PostgresMain (dbname=<optimized out>, username=<optimized out>) at tcop/./build/../src/backend/tcop/postgres.c:4767
#13 0x0000aaaae7085cdc in BackendMain (startup_data=<optimized out>, startup_data_len=<optimized out>) at tcop/./build/../src/backend/tcop/backend_startup.c:105
#14 0x0000aaaae6fd7f90 in postmaster_child_launch (child_type=<optimized out>, startup_data=0xffffeb7ffbc0 "", startup_data_len=4, client_sock=0xffffeb7ffbd8) at postmaster/./build/../src/backend/postmaster/launch_backend.c:277
#15 0x0000aaaae6fe4c7c in postmaster_child_launch (client_sock=0xffffeb7ffbd8, startup_data_len=4, startup_data=0xffffeb7ffbc0 "", child_type=B_BACKEND) at postmaster/./build/../src/include/lib/ilist.h:317
#16 BackendStartup (client_sock=0xffffeb7ffbd8) at postmaster/./build/../src/backend/postmaster/postmaster.c:3593
#17 ServerLoop () at postmaster/./build/../src/backend/postmaster/postmaster.c:1674
#18 0x0000aaaae6fd9bf8 in PostmasterMain (argc=<optimized out>, argv=<optimized out>) at postmaster/./build/../src/backend/postmaster/postmaster.c:1372
#19 0x0000aaaae6c57e90 in main (argc=1, argv=0xaaab053c86c0) at main/./build/../src/backend/main/main.c:197

@svenklemm svenklemm reopened this Feb 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants