Support postgres backend

[mats16]

Feature request

Is your feature request related to a problem? Please describe.

I want to use a postgresql for backends instead of BigQuery.

Describe the solution you'd like

Logflare would support postgresql for backends.

Additional context

If already supported, please show me how to run Logflare with postgres.

[Ziinc]

Hi @mats16 , this is already in the works. However, it would be great to give us a view on how you would ideally like this feature to be implemented, so that we can improve on our initial iteration.

[mats16]

Hi @Ziinc , I want to run Logflare on AWS using Aurora Serverless v2 or Redshift Serverless. Both are compatible with Postgres.

This is just idea...
I think It would be nice if the write and read modules were explicitly separate and separately extensible.
If you use Postgres for the backend, you will want to set up separate read and write instances.
If you use Redshift for the backend, you will want to use Kinesis to write data streamly.

[ikkz]

Hi @mats16 , this is already in the works. However, it would be great to give us a view on how you would ideally like this feature to be implemented, so that we can improve on our initial iteration.

@Ziinc somewhere I can see the progress of this work? Thanks!

[Ziinc]

Thanks for the input, we will definitely consider extending further into the PG ecosystem.
@ikkz in terms of monitoring progress, the best way is to watch for PRs relating to the postgres backend. We are focused on integration with the supabase stack first before making the PG backend available to all customers of the Logflare service. This is because it will leverage our v2 ingestion pipeline, and as it is rather untested at production scale, it would only be done gradually.

I will leave this issue open as a catchall for further feedback.

[encima]

Postgres backend is available!

Shall we close this, @Ziinc (I do not have permissions to)

[Ziinc]

👍 Please note that it is only headless at the moment, so UI searching is not supported for the PG backend.

[Ziinc]

issue tracking the ui search for PG backend is here #1895