Pass errors to the sink trait instead of dropping the batch

[passcod]

Background: this is something I did in my branch while implementing a custom Sink. My target database had types that pg_replicate didn't handle at the time, as well as some custom types which pg_replicate can't handle at all. Debugging this efficiently was hindered by pg_replicate crashing immediately when it encountered a type it couldn't handle.

More recently the unknown_types_to_bytes feature has been introduced which probably renders this change unnecessary at least for decoding errors, but I want to propose it anyway; however I understand if it's rejected.

• Change the Sink trait to accept Results in the write_table_row, write_table_rows, and write_cdc_event methods.
• The pipeline, instead of erroring early on encountering an error while processing an event/row batch, and dropping the events/rows, now passes the entire results directly to the trait.
• The trait implementation is then responsible for handling those errors as it wishes, including ignoring errored rows.

[imor]

Thanks for the PR, I'll review tomorrow.

[imor]

unknown_types_to_bytes was kind of a stopgap solution and would probably be removed. Now that we've switched the table copies as well the CDC stream to the text format the fallback type is a string. So any unsupported type would become a string with its value being whatever Postgres's text format output for that type is. But I actually liked the fail early approach because it surfaced the error instead of silently converting its type to string.

I think a much more useful feature would be to add the ability for the consumers of pg_replicate to add support for their custom types in their own crates. Do you think this will work for you with your custom types?

[passcod]

Absolutely. That sounds like a much better solution. I'll close this then.