Large number of database connections

[YaBatman]

Edgy version: 0.24.2
Python version: 3.12

Hi there!
I'm using EDGE in conjunction with FastAPI and PostgreSQL, as well as pgbouncer for database connection pooling. When initializing the Registry, we specify poolclass=NullPool, since we're using pgbouncer to manage connections.

However, during production and load testing, we've noticed that EDGY is generating a high number of database connections, which is causing some issues. We'd like to find a way to reduce the number of connections so that we can optimize our system's performance.

We were wondering if there are any solutions available that could help us achieve this goal. For example, could we use decorators to ensure that all EDGY calls within an endpoint are executed within the same session? This would allow us to minimize the number of connection requests and improve the overall performance of our application.

[tarsil]

@devkral I believe you are better to handle this scenario

[devkral]

If you don't get warnings it seems your Database objects are referenced and dereferenced all the time.
you may want to use the lifespan asgi wrapper for flask, in case you don't use it.

Otherwise check the ref count of the Database objects. It should not always drop to 0.

Please ignore this. This was wrong. It assumed that the destruction of pools causing new connections what is not the case with NullPool

[devkral]

Sry I was wrong. You use the NullPool that implied something other what I was suggesting.

You can maybe reduce the connections by using a normal Pool, databasez makes them async and threadsafe.

You probably trigger implicit loads of values by accessing uninitialized attributes on your models.

[YaBatman]

But each edge query brings a new flavor to the pool of databases. However, I would like to limit this behavior and have the API request use only one connection at a time. With a large amount of traffic, this can cause too many connections to be opened.

In any case, I will try to review all our code to see if there is any unobvious usage of edge cases.

[devkral]

then eliminate the implicit loads when non-existing fields are accessed (sry we need this for deferred loading).

You can use the contextvariable MODEL_GETATTR_BEHAVIOR for this.
Set it to "passdown" and iron out the now appearing errors.

If you check a model instance for an attribute check it's dict instead of using hasattr/getatttr. This prevents triggering the load.

You can also set the model instance variable _loaded_or_deleted to True for archiving the same like MODEL_GETATTR_BEHAVIOR but you sacrifices deferred loading this way.

[devkral]

I will add a debugging guide

[devkral]

the debugging guide is here:

https://edgy.dymmond.com/debugging

Could you solve the problem?

[devkral]

can you try main? I fixed a bug which prevented subloops to close.