Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1392 : Add support for SQLite VFS and open mode flags #1393

Draft
wants to merge 65 commits into
base: dev
Choose a base branch
from

Conversation

ancientjpeg
Copy link

@ancientjpeg ancientjpeg commented Feb 1, 2025

Addresses #1392

Changelog

  • Add vfs_mode enum class to simplify the sqlite_orm API for using different VFS systems, as compared to the raw string passed in sqlite3.h
  • Add open_mode enum class to support the open flags passed by sqlite3_open_v2. Only two values have been added so far, but this addition makes it very easy to add more in the future.
  • Add platform definitions to detect operating system
    • Necessary for exposing platform-specific VFS types
  • Add tests

More Info

https://www.sqlite.org/vfs.html - VFS options and description
https://www.sqlite.org/c3ref/open.html - open flag options

@fnc12
Copy link
Owner

fnc12 commented Feb 1, 2025

hey @ancientjpeg . Thank for this PR. Please work in dev branch not master

@ancientjpeg ancientjpeg force-pushed the feature/1392-support-sqlite-vfs branch from 13c0562 to 42d717b Compare February 1, 2025 08:57
@ancientjpeg ancientjpeg changed the base branch from master to dev February 1, 2025 08:57
@ancientjpeg
Copy link
Author

Just rebased. Thanks and apologies again.

@ancientjpeg ancientjpeg changed the title #1392 : Add support for SQLite VFS #1392 : Add support for SQLite VFS and open mode flags Feb 3, 2025
Copy link
Author

@ancientjpeg ancientjpeg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fnc12 I've added enums to allow for full control over what open mode/VFS options users actually have access to. I'm happy with the state this PR is in, but I have some preliminary evidence that using some VFS modes like unix-dotfile may interfere with the ability to set certain journal_modes. This is detectable by just checking journal_mode after you've set it, but if you'd like me to brainstorm a layer of protection for that I'd be happy to.

dev/open_mode.h Outdated Show resolved Hide resolved
Comment on lines +662 to +664
bool readonly() const {
sqlite3* db = this->connection->get();
return sqlite3_db_readonly(db, "main");
}
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At first I was a little nervous about using the "main" DB name here but then I realized that's used in other places in the lib. I figure this indicates there's no near-future plans for attachment support.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From where exactly do you draw the conclusion about "main"?
Currently it's only the backup functionality that literally uses "main".

At least dealing with the "temp" DB is in the works, upon which the attachment functionality could be built.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I drew the conclusion from the backup code, as it was the only other sqlite3 method I could find in sqlite_orm that was being called with a database name parameter. How would you like me structure this in order to make it more compatible with future attempts to implement ATTACH? My first assumption would be to pass in the database name as one of the storage_options, but let me know what approach you think is best.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can just leave it as is.

@ancientjpeg ancientjpeg marked this pull request as ready for review February 4, 2025 19:56
Copy link
Collaborator

@trueqbit trueqbit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ancientjpeg for your contribution! Here come my 2 cents...

dev/open_mode.h Show resolved Hide resolved
dev/open_mode.h Outdated Show resolved Hide resolved
dev/open_mode.h Show resolved Hide resolved
dev/storage.h Outdated Show resolved Hide resolved
Comment on lines +662 to +664
bool readonly() const {
sqlite3* db = this->connection->get();
return sqlite3_db_readonly(db, "main");
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From where exactly do you draw the conclusion about "main"?
Currently it's only the backup functionality that literally uses "main".

At least dealing with the "temp" DB is in the works, upon which the attachment functionality could be built.

dev/vfs_mode.h Outdated Show resolved Hide resolved
tests/open_v2_tests.cpp Show resolved Hide resolved
@@ -41,6 +49,8 @@ namespace sqlite_orm {
}

const std::string filename;
const vfs_mode vfs;
const sqlite_orm::open_mode open;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to call the member variable openFlags or openMode.
Also there's no need to qualify it with the namespace.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opted to just move the entire options structure in here instead—this avoid some errors that GCC throws. See this comment for more details.

dev/storage.h Outdated Show resolved Hide resolved
dev/connection_holder.h Outdated Show resolved Hide resolved
@ancientjpeg ancientjpeg marked this pull request as draft February 5, 2025 17:32
@ancientjpeg
Copy link
Author

ancientjpeg commented Feb 5, 2025

@trueqbit Still working through your changes, but something occurred to me while refactoring. I had some of those unnecessary sqlite_orm:: qualifiers on the mode types because I actually got a couple compile errors complaining about ambiguous symbols. Would it be alright if I appended _t to these flag types? I.e.,

enum class vfs_mode_t;

Then it'd be a lot clearer to re-use the same verbiage for defining variables, e.g.

vfs_mode_t vfs_mode;
open_mode_t open_mode;

struct storage_options {
  vfs_mode_t vfs_mode;
  open_mode_t open_mode;
};

Let me know what you think of that.

Edit: I ended up implementing this because GCC complains of ambiguity in cases such as const open_mode open_mode. I prefer the look of const open_mode_t open_mode to const open_mode open_flags as I feel the latter might sow confusion, but I'm happy to change it if it feels more in line with the rest of the lib.

@ancientjpeg ancientjpeg force-pushed the feature/1392-support-sqlite-vfs branch from b96a21f to 319c614 Compare February 5, 2025 21:29
@trueqbit
Copy link
Collaborator

trueqbit commented Feb 5, 2025

@trueqbit Still working through your changes, but something occurred to me while refactoring. I had some of those unnecessary sqlite_orm:: qualifiers on the mode types because I actually got a couple compile errors complaining about ambiguous symbols. Would it be alright if I appended _t to these flag types? I.e.,

enum class vfs_mode_t;

Then it'd be a lot clearer to re-use the same verbiage for defining variables, e.g.

vfs_mode_t vfs_mode;
open_mode_t open_mode;

struct storage_options {
  vfs_mode_t vfs_mode;
  open_mode_t open_mode;
};

Let me know what you think of that.

Hmm, that's a funny problem as both are public :)
However, I do not recommend changing the enum type's name. Firstly, it is not recommended in the coding guidelines, and we do not use it in sqlite_orm with one exception.

I personally like camel case for (member) variables, but I agree that's debatable.
A good choice in any case for the variable name is vfsOption, modeOption or vfs_option, mode_option as "option" conveys that fact that a specific thing is selected .

While we are at it, I also suggest renaming vfs_mode -> vfs_object, as SQLite calls the feature Virtual File System Objects.

@ancientjpeg
Copy link
Author

@trueqbit Great point about using option in the variable name, I think I'll go with that. I'll also change the members and parameters to camelCase if that's the preferred styling for this library—I had just assumed snake_case was the go-to given the external API is style that way, but now that I look closer I see a lot of parameter names are camelCase.

I'll be away from my development system until next week, at which time I'll wrap up these changes. Thanks for the feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants