Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for OpenEphemeral bytecode #768

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

diegoreis42
Copy link
Contributor

@diegoreis42 diegoreis42 commented Jan 23, 2025

First step to close #741. It allows creating temporary table operations and improve flexibility in handling in-memory data. Since is my first time working on core code, every feedback is welcome. @jussisaurio @pereman2 @PThorpe92 @penberg

@penberg
Copy link
Collaborator

penberg commented Jan 26, 2025

Hey @diegoreis42, looks very sensible to me!

core/vdbe/mod.rs Outdated
@@ -260,12 +269,19 @@ impl ProgramState {
}

macro_rules! must_be_btree_cursor {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This macro will probably have to be renamed?

Copy link
Contributor Author

@diegoreis42 diegoreis42 Jan 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

must_be_general_cursor maybe?

@diegoreis42 diegoreis42 marked this pull request as ready for review January 29, 2025 14:50
cursors
.get_mut(*cursor_id)
.unwrap()
.replace(Cursor::new_ephemeral(EphemeralCursor::new()));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not so sure what should be returned here.

@diegoreis42 diegoreis42 changed the title wip: Add support for OpenEphemeral bytecode Add support for OpenEphemeral bytecode Jan 31, 2025
use crate::{types::OwnedValue, Result};

pub struct EphemeralCursor {
table: Rc<RefCell<EphemeralTable>>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't need to be a Rc<RefCell<T>> right? Just table: EphemeralTable

}
}
SeekKey::IndexKey(index_key) => {
// Seek by index key (ignoring row ID)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should support the key being either a rowid or an arbitrary key. From SQLite docs:

Open a new cursor P1 to a transient table. The cursor is always opened read/write even if the main database is read-only. The ephemeral table is deleted automatically when the cursor is closed.
If the cursor P1 is already opened on an ephemeral table, the table is cleared (all content is erased).

P2 is the number of columns in the ephemeral table. The cursor points to a BTree table if P4==0 and to a BTree index if P4 is not 0. If P4 is not NULL, it points to a KeyInfo structure that defines the format of keys in the index.

So I think EphemeralTable should be something like

pub struct EphemeralTable<Key: Ord> {
    pub rows: BTreeMap<Key, Vec<OwnedValue>>,
    pub next_rowid: u64,                      // generate rowids
    pub columns: Vec<Column>,                 // columns
}

OR define it as an enum:

enum Ephemeral {
  Table(EphemeralTable)
  Index(EphemeralIndex)
}

and have separate definitions.

Either way, searching with an index key should be O(log n) in an ephemeral index, instead of O(n) as here.

Take the logical AND of the values in registers P1 and P2 and write the result into register P3. If either P1 or P2 is 0 (false) then the result is 0 even if the other input is NULL. A NULL and true or two NULLs give a NULL output.
Take the logical OR of the values in register P1 and P2 and store the answer in register P3. If either P1 or P2 is nonzero (true) then the result is 1 (true) even if the other input is NULL. A NULL and false or two NULLs give a NULL output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Count distinct is not supported
4 participants