Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Last note per pubkey algo feed #621

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from
Draft

Last note per pubkey algo feed #621

wants to merge 6 commits into from

Conversation

jb55
Copy link
Contributor

@jb55 jb55 commented Dec 29, 2024

This is damus' first algo feed.

What's left to do

Optional

  • Switch over to token parser combinator

I noticed I was missing many parts of the codebase related to column serialization. In an effort to have a more direct parsing style, I created a new token parser combinator.

The algo

algos: introduce last_n_per_pubkey_from_tags

This function creates filters for the base our first algo in Damus:

Called "last N note per pubkey". I don't have a better name for it.

This function generates a query in the form:

[
  {"authors": ["author_a"], "limit": 1, "kinds": [1]
, {"authors": ["author_b"], "limit": 1, "kinds": [1]
, {"authors": ["author_c"], "limit": 1, "kinds": [1]
, {"authors": ["author_c"], "limit": 1, "kinds": [1]
  ...
]

Due to an unfortunate restriction currently in nostrdb and strfry, we
can only do about 16 to 20 of these at any given time. I have made
this limit configurable in strfry[1]. I just need to do the same in
nostrdb now.

[1] https://github.com/hoytech/strfry/pull/133

jb55 added 5 commits December 26, 2024 09:38
This function creates filters for the base our first algo in Damus:

Called "last N note per pubkey". I don't have a better name for it.

This function generates a query in the form:

[
  {"authors": ["author_a"], "limit": 1, "kinds": [1]
, {"authors": ["author_b"], "limit": 1, "kinds": [1]
, {"authors": ["author_c"], "limit": 1, "kinds": [1]
, {"authors": ["author_c"], "limit": 1, "kinds": [1]
  ...
]

Due to an unfortunate restriction currently in nostrdb and strfry, we
can only do about 16 to 20 of these at any given time. I have made
this limit configurable in strfry[1]. I just need to do the same in
nostrdb now.

[1] hoytech/strfry#133

Changelog-Added: Add last_n_per_pubkey_from_tags algo function
Signed-off-by: William Casarin <[email protected]>
In an attempt to make our deck serializer more localized,
comprehensible, and less error-prone, we introduce a new parser
combinator based around string tokens.

This replaces the Selection-based intermediary types so that we have a
more direct serialization style.
This reduces the number of things we have to update in our token parser
and serializer. For payloads, we we have to handle the payload cases
different, but we now have a structure that can deal with that
efficiently.

Signed-off-by: William Casarin <[email protected]>
This simplifies token serialization

Signed-off-by: William Casarin <[email protected]>
@jb55 jb55 linked an issue Jan 6, 2025 that may be closed by this pull request
1 task
@jb55 jb55 mentioned this pull request Jan 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Last note per pubkey algo feed
1 participant