-
-
Notifications
You must be signed in to change notification settings - Fork 394
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
filter: add API for registering Python filters #1237
Conversation
def __init__(self): | ||
pass | ||
|
||
def check(self, src: FilterSource, attr_values: List[str]): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filter.check()
is equivalent to git_filter->check()
data: bytes, | ||
src: FilterSource, | ||
write_next: Callable[[bytes], None] | ||
): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filter.write()
is equivalent to git_writestream->write()
(i.e git_filter->stream->write
).
write_next
writes to the next stream in the linked-list of git_writestream
's (where there is one stream for each filter in the chain)
def close( | ||
self, | ||
write_next: Callable[[bytes], None] | ||
): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Filter.close()
is equivalent to git_writestream->close()
(i.e. git_filter->stream->close()
)
Hi @pmrowla , thanks for the contribution. |
Thanks for the review, I'll make the suggested changes and take a look into the pypy issue. I'll likely keep the PR marked as draft for now, I'm still testing to make sure this works with real-world LFS filtering so it's still possible this PR may change a bit. |
4b5a6c6
to
d9c72e0
Compare
After some investigation, it looks like pypy and cffi don't always mix well if an ffi callback is run outside of the main thread, which is what was causing the pypy tests to hang (since the libgit2 filtering can be done in a separate thread). I've rewritten the underlying filter/writestream code to done entirely in the _pygit2 C-extension (so this PR no longer uses cffi at all) |
ebbc5d4
to
522a716
Compare
The flaky pypy tests should now be taken care of. The issue was a difference |
4bacc1e
to
ac8f335
Compare
@jdavid we'd appreciate it if we could get a release for this pushed whenever you can, thanks! |
yes, this week, just have to finish the ongoing PRs |
The release is done |
pygit2.filter
API for registering pure-pythongit_filter_*
filters.pygit2.BlobIO()
wrapper for reading raw and filtered blob contents.BlobIO
will stream buffered content (instead of loading the entire blob into memory) if the underlying libgit2 filter(s) support the streaming filter API.Repository.get_attr(..., commit=...)
to support attr lookup by commit IDFilter.check()
but it is useful for anyone implementing filters to lookup commit specific attributes before actually running the filter (i.e. to pre-fetch LFS objects in a batch rather than streaming them one by one upon checkout)This allows the user to subclass
pygit2.Filter
and then register their own Python smudge/clean filters.In libgit2, you normally need to define both a
git_filter
and agit_writestream
implementation for your filter. This PR abstracts this out into a singlepygit2.Filter
class so the user does not need to work directly with thegit_writestream
linked list chain that is invoked when libgit2 does filtering.