PSA: Migrating to Postgresql

[EricTheMagician]

For anyone actively using DriveFS, it will be migrated to postgres from mongodb in the coming weeks.

[BerriJ]

Interesting. Why so?

Is it for performance?

Thanks a lot for all the effort you are putting into this Project.

[EricTheMagician]

It was originally motivated by performance and memory issues.
MongoDB on my system was a huge memory hog, and that was something I wanted to tackle by switching databases.

The other was to reduce overall memory usage, especially for very large amounts files/folders by not having all the required information in memory, and falling back to the database, when needed.

As for performance, I did expect a small loss compared to having the data stored in memory, but it's quite significant. I primarily use it through rclone and I have noticed it hanging from time to time with ls. By hanging, it would last a few seconds, sometimes alot more, on the order of a minute / did I code this incorrectly moments. fixed, performs reasonably well.

[mcadam]

Would this work with stateless container if everything is in database now and not in memory anymore? As in if I run multiple times the program connecting to the same database it would work?
I am running it in a cluster to mount the drive in multiple places and currently have multiple databases so would be nice to cut on that :)

[EricTheMagician]

It's still not stateless as some temporary states are still being stored. I still have some occasional crashes, so I want to fix those before pushing it to the master branch.

I'm currently traveling for work, and so I won't have too much time to work on this until august.
So stay tuned.

[mcadam]

Hi, any updates on that one? :)