vector_top_k() fails with query vector from CTE

[rhashimoto]

I'm getting a vector_top_k.xBestIndex malfunction error when using a query vector defined in a CTE.

My use case is that I want to return the distance with nearest neighbor data. vector_top_k() only returns id and not distance, so I need to call the distance function. I didn't want to repeat the query vector for both vector_distance_cos() and vector_top_k(), so I tried to define it in a CTE.

Here's a simple reproduction with the movies example:

-- This is the example table and index definition.
CREATE TABLE movies (
  title    TEXT,
  year     INT,
  embedding F32_BLOB(4) -- 4-dimensional f32 vector
);
INSERT INTO movies (title, year, embedding)
VALUES
  ('Napoleon', 2023, vector32('[0.800, 0.579, 0.481, 0.229]')),
  ('Black Hawk Down', 2001, vector32('[0.406, 0.027, 0.378, 0.056]')),
  ('Gladiator', 2000, vector32('[0.698, 0.140, 0.073, 0.125]')),
  ('Blade Runner', 1982, vector32('[0.379, 0.637, 0.011, 0.647]'));
CREATE INDEX movies_idx ON movies(libsql_vector_idx(embedding));

-- This is the CTE that fails.
WITH common AS (SELECT vector32('[0.064, 0.777, 0.661, 0.687]') AS q)
SELECT title, year, vector_distance_cos(embedding, q)
FROM vector_top_k('movies_idx', q, 3)
JOIN movies ON movies.rowid = id
JOIN common
WHERE year >= 2020;

-- This similar CTE works without vector_top_k().
WITH common AS (SELECT vector32('[0.064, 0.777, 0.661, 0.687]') AS q)
SELECT title, year, vector_distance_cos(embedding, q)
FROM movies, common
WHERE year >= 2020;

Here's the failing statement on v0.24.31:

libsql> WITH common AS (SELECT vector32('[0.064, 0.777, 0.661, 0.687]') AS q)
SELECT title, year, vector_distance_cos(embedding, q)
FROM vector_top_k('movies_idx', q, 3)
JOIN movies ON movies.rowid = id
JOIN common
WHERE year >= 2020;

   ...>    ...>    ...>    ...>    ...> Parse error: vector_top_k.xBestIndex malfunction
libsql>