Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Import iter_subproc() #23

Merged
merged 16 commits into from
Jun 14, 2024
Merged

Import iter_subproc() #23

merged 16 commits into from
Jun 14, 2024

Conversation

mih
Copy link
Member

@mih mih commented Jun 11, 2024 •
edited
Loading

TODO:

  • Bring in iterable_subprocess
  • Adapt to match standards
  • doctest examples for iterable_subprocess
  • Bring in iter_subproc

@codecov-commenter
Copy link

codecov-commenter commented Jun 11, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 98.42932% with 6 lines in your changes missing coverage. Please review.

Project coverage is 98.61%. Comparing base (4d4823f) to head (4b44891).

Files Patch % Lines
...tasalad/iterable_subprocess/iterable_subprocess.py 94.33% 3 Missing and 3 partials ⚠️
Additional details and impacted files 
@@             Coverage Diff             @@
##              main      #23      +/-   ##
===========================================
- Coverage   100.00%   98.61%   -1.39%     
===========================================
  Files            3        8       +5     
  Lines           78      432     +354     
  Branches        13       71      +58     
===========================================
+ Hits            78      426     +348     
- Misses           0        3       +3     
- Partials         0        3       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mih mih force-pushed the itersubproc branch 3 times, most recently from cf9d316 to 8ded4b9 Compare June 12, 2024 08:39
This was referenced Jun 12, 2024
Closed
Closed
mih added 16 commits June 14, 2024 07:30
@mih
ci: run code linter via hatch
43a66d0 
Previously a 3rd-party GitHub action was used. Now we do automatically
what a developer would run locally to minimize surprises and friction.
@mih
feat: iterable_subprocess() utility
284ea58 
This is a code import from datalad-next@0f4829f4b69b826057c2881f0318329480cc9f38

This 3rd-party implementation had been originally introduced in
datalad-next V1.1. It is a context manager that feeds input to
subprocesses via iterables, and also exposes their output as an
iterable. The implementation is based on
https://github.com/uktrade/iterable-subprocess, and this fork
has been modified to work homogeneously on the Windows platform
too.

This code is the foundation for a future code import of
`iter_subproc()`, the core subprocess execution utility of
datalad-next.

For documentation/compliance purposes, the code import include the
original license, README, and `pyproject.toml` metadata.
@mih
feat(CommandError): report stderr in __str__()
ff7244e 
A truncated string with the error output is included, in an effort to
relay possible error messages. For byte strings an attempt is made to
decode it using the system's default encoding (see rationale in the
comments on why that and why not more).

The tests are extended with a few illustrative cases.

This change should provide the necessary error reporting behavior for
the `iterable_subprocess` tests to pass.
@mih
rf(iterable_subprocess): adapt to use datasalad's CommandError
3604138 
This is no more than using the `returncode` argument, instead of `code`.
@mih
style: basic, initial ruff configuration (annotated)
bda20bc
@mih
style: apply ruff rules
71c0c62 
This includes manually implementing suggestions, case-by-case
whitelisting exceptions to rules, and also automated fixes and
reformating.
@mih
rf(CommandError): remove duplicate conditional in __str__()
8c161c8 
This created a branch that could never be reached in practice.
@mih
doc(iterable_subprocess): reformat comment into rendered docs
c87ef27 
Also add this documentation to the main index.
@mih
test(iterable_subprocess): add type annotations to main API
539648f 
This involved "littering" the code with a number if `TYPE_CHECKING`
conditionals to document a number of assert regarding various type
changes at runtime.
@mih
style(version): match tag_format config with actual practice
bc629a0 
Add `v` prefix
@mih
doc: tune synopsis of runners package
7e845c6
@mih
doc(iterable_subprocess): add doc-tested examples
2fce12b
@mih
feat: iter_subproc() utility
484a7af 
This is a code import from datalad-next@0f4829f4b69b826057c2881f0318329480cc9f38

`iter_subproc()` is a convenience wrapper around `iterable_subprocess()` that
slightly tunes its API for the purpose of a somewhat more homogeneous
integration with other iterators in the DataLad ecosystem.
@mih
rf(iter_subproc): adapt to meet standard and API of datasalad
44fd682 
For now, `COPY_BUFSIZE` import and definition are included in the
module. At least until a second consumer exists.
@mih
style(iter_subproc): apply ruff rules
1d4add7
@mih
rf(iter_subproc): rename argument input -> inputs
4b44891 
The original names shadowed a built-in. While this did not cause
an actual problem, it is confusing.

The plural form also matches better, because `inputs` is an
iterable.
@codeclimate CodeClimate
Copy link

codeclimate bot commented Jun 14, 2024

Code Climate has analyzed commit 4b44891 and detected 4 issues on this pull request.

Here's the issue category breakdown:

Category Count
Complexity 4

View more on Code Climate.

@mih mih merged commit e03bca7 into main Jun 14, 2024
8 checks passed
@mih mih deleted the itersubproc branch June 14, 2024 05:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Import candidate: iter_subproc
2 participants
@mih @codecov-commenter