Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migration failed while update for schema containing composite primary key #750

Closed
tanyaranjan1995 opened this issue Apr 18, 2024 · 0 comments · Fixed by #883 · May be fixed by #879
Closed

Migration failed while update for schema containing composite primary key #750

tanyaranjan1995 opened this issue Apr 18, 2024 · 0 comments · Fixed by #883 · May be fixed by #879

Comments

@tanyaranjan1995
Copy link

tanyaranjan1995 commented Apr 18, 2024 •
edited
Loading

Issue: Migration failed while update for schema containing composite primary key
Version: Running pgcopydb version 0.15 from "/usr/bin/pgcopydb"

Repro steps:

  1. Initiate migration with: pgcopydb clone --follow --drop-if-exists --restart --plugin test_decoding --table-jobs 4 --index-jobs 4
    Source database containing:
(id integer,
	name text,
 constraint test_pk PRIMARY KEY(id,name)
);
insert into test_002 (id,name) values (1,'postgres');
insert into test_002 (id,name) values (2,'oracle');
insert into test_002 (id,name) values (3,'mysql');
insert into test_002 (id,name) values (4,'couchbase');
insert into test_002 (id,name) values (5,'redis');
  1. During the cutover phase executed the following commands-
Insert into test_002 (id,name) values (8,'AWS RDS12');
Insert into test_002 (id,name) values (9,'AWS Aurora123');

Executing the above statements was successful.

Migration failed when executing-

update test_002
set id = id
where 1 = 1

Error Logs:

15:22:14.907 8682 INFO   Reported write_lsn A/20000848, flush_lsn A/20000818, replay_lsn A/200007E0
15:22:24.911 8682 INFO   Reported write_lsn A/20000880, flush_lsn A/20000848, replay_lsn A/20000818
15:22:30.588 8683 ERROR  Failed to parse decoding message for UPDATE on table public.test_002: SET clause columns not found
15:22:30.589 8683 ERROR  Failed to parse UPDATE new-tuple columns for logical message table public.test_002: UPDATE: id[integer]:1 name[text]:'postgres'
15:22:30.589 8683 ERROR  Failed to parse test_decoding UPDATE message: table public.test_002: UPDATE: id[integer]:1 name[text]:'postgres'
15:22:30.589 8683 ERROR  Failed to parse test_decoding message, see above for details
15:22:30.589 8683 ERROR  Failed to parse JSON message: {"action":"U","xid":"0","lsn":"A/20000880","timestamp":"2024-04-18 09:52:31.251981+0530","message":"table public.test_002: UPDATE: id[integer]:1 name[text]:'postgres'"}
15:22:30.589 8683 ERROR  Failed to transform JSON messages from input stream, see above for details
15:22:30.589 8683 INFO   Transform process has terminated
15:22:30.744 8328 ERROR  Subprocess transform with pid 8683 has exited with error code 12
15:22:30.744 8684 ERROR  Postgres query was interrupted: select pg_replication_origin_progress($1, $2)
15:22:30.745 8684 ERROR  Failed to fetch progress of replication origin for "pgcopydb"
15:22:30.745 8684 ERROR  Failed to retrieve origin progress, see above for details
15:22:30.745 8684 WARN   Skipping sentinel replay_lsn update: failed to find a durable LSN matching current flushLSN
15:22:30.745 8684 ERROR  Pipeline sync was interrupted
15:22:30.745 8684 ERROR  Failed to sync the pipeline, see previous error for details
15:22:30.746 8684 ERROR  Failed to read SQL lines from input stream, see above for details
15:22:30.746 8684 INFO   Apply process has terminated
15:22:30.751 8682 INFO   Reported write_lsn A/20000AE8, flush_lsn A/20000AE8, replay_lsn A/20000818
15:22:30.886 8682 INFO   Streamed up to write_lsn A/20000AE8, flush_lsn A/20000AE8, stopping: endpos is 0/0
15:22:30.886 8682 INFO   Prefetch process has terminated
15:22:30.906 8328 ERROR  Subprocess catchup with pid 8684 has exited with error code 12
15:22:31.057 8328 ERROR  Some sub-process exited with errors, see above for details
15:22:31.057 8328 ERROR  Failed to follow changes from source, see above for details
15:22:31.134 8260 ERROR  follow process 8328 has terminated [12]

Complete Error Logs:
pgcopydbLogsCPK.txt

Additional Information:
This issue is in wal2json as well.
Works via pg_dump/restore.
hanefi pushed a commit to hanefi/pgcopydb that referenced this issue Sep 2, 2024
@rimbi @hanefi
Fixes online migration with composite primary key
8122662 
Refactor code to skip executing empty UPDATE statements

The logical_message_metadata_should_skip_statement function has been
added to check if the logical message metadata should be skipped. This
is necessary due to a bug in the wall2json output plugin that creates an
UPDATE statement with an empty SET clause and places the actual SET
clause in the WHERE clause. The function checks for this specific
pattern and returns true if found, indicating that the statement should
be skipped during execution.

In passing fixes a double free bug.

Fixes: dimitri#750
@hanefi hanefi mentioned this issue Sep 2, 2024
Open
@arajkumar arajkumar mentioned this issue Oct 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@tanyaranjan1995