Skip to content

Commit

Permalink
Release the GIL when reading from BytesIO objects. (#291)
Browse files Browse the repository at this point in the history 
* Add special case to release the GIL when reading from a BytesIO object.

* Fix tests.

* Disable flakey test on underpowered machines.

* Decrease test flakiness.

* Disable flaky test on CI.

* Skip on CI properly.

* Ensure that we can't read past the end of a BytesIO buffer.

* Use one of virtual or override.

* Apply suggestions from code review

Co-authored-by: David Rubinstein <[email protected]>

* Simplify read math for PythonMemoryViewInputStream.

* Fix type annotations in docs.

* Appease the linter.

---------

Co-authored-by: David Rubinstein <[email protected]>
  • Loading branch information
@psobot @drubinstein
psobot and drubinstein authored Feb 27, 2024
1 parent b35f6e9 commit cd686d5
Show file tree
Hide file tree
Showing 9 changed files with 234 additions and 44 deletions.
8 changes: 4 additions & 4 deletions docs/reference/pedalboard.io.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -252,7 +252,7 @@ <h1>The <code class="docutils literal notranslate"><span class="pre">pedalboard.
<dt class="sig sig-object py" id="pedalboard.io.AudioFile">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pedalboard.io.</span></span><span class="sig-name descname"><span class="pre">AudioFile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">filename</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.12)"><span class="pre">str</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mode</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Literal" title="(in Python v3.12)"><span class="pre">Literal</span></a><span class="p"><span class="pre">[</span></span><span class="s"><span class="pre">'r'</span></span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">'r'</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pedalboard.io.AudioFile" title="Permalink to this definition">#</a></dt>
<dt class="sig sig-object py">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pedalboard.io.</span></span><span class="sig-name descname"><span class="pre">AudioFile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file_like</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.BinaryIO" title="(in Python v3.12)"><span class="pre">BinaryIO</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mode</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Literal" title="(in Python v3.12)"><span class="pre">Literal</span></a><span class="p"><span class="pre">[</span></span><span class="s"><span class="pre">'r'</span></span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">'r'</span></span></em><span class="sig-paren">)</span></dt>
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pedalboard.io.</span></span><span class="sig-name descname"><span class="pre">AudioFile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file_like</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Union" title="(in Python v3.12)"><span class="pre">Union</span></a><span class="p"><span class="pre">[</span></span><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.BinaryIO" title="(in Python v3.12)"><span class="pre">BinaryIO</span></a><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#memoryview" title="(in Python v3.12)"><span class="pre">memoryview</span></a><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">mode</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Literal" title="(in Python v3.12)"><span class="pre">Literal</span></a><span class="p"><span class="pre">[</span></span><span class="s"><span class="pre">'r'</span></span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">'r'</span></span></em><span class="sig-paren">)</span></dt>
<dt class="sig sig-object py">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pedalboard.io.</span></span><span class="sig-name descname"><span class="pre">AudioFile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">filename</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.12)"><span class="pre">str</span></a></span></em>, <em class="sig-param"><span class="n"><span class="pre">mode</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Literal" title="(in Python v3.12)"><span class="pre">Literal</span></a><span class="p"><span class="pre">[</span></span><span class="s"><span class="pre">'w'</span></span><span class="p"><span class="pre">]</span></span></span></em>, <em class="sig-param"><span class="n"><span class="pre">samplerate</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Optional" title="(in Python v3.12)"><span class="pre">Optional</span></a><span class="p"><span class="pre">[</span></span><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.12)"><span class="pre">float</span></a><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">num_channels</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.12)"><span class="pre">int</span></a></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">1</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">bit_depth</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/functions.html#int" title="(in Python v3.12)"><span class="pre">int</span></a></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">16</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">quality</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Optional" title="(in Python v3.12)"><span class="pre">Optional</span></a><span class="p"><span class="pre">[</span></span><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Union" title="(in Python v3.12)"><span class="pre">Union</span></a><span class="p"><span class="pre">[</span></span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.12)"><span class="pre">str</span></a><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/functions.html#float" title="(in Python v3.12)"><span class="pre">float</span></a><span class="p"><span class="pre">]</span></span><span class="p"><span class="pre">]</span></span></span><span class="w"> </span><span class="o"><span class="pre">=</span></span><span class="w"> </span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span></dt>
<dt class="sig sig-object py">
Expand All @@ -270,8 +270,8 @@ <h1>The <code class="docutils literal notranslate"><span class="pre">pedalboard.
the sample rate of the file.</p></li>
<li><p>A file-like object can be provided to <a class="reference internal" href="#pedalboard.io.AudioFile" title="pedalboard.io.AudioFile"><code class="xref py py-class docutils literal notranslate"><span class="pre">AudioFile</span></code></a>, allowing for reading and
writing to in-memory streams or buffers. The provided file-like object must be seekable
and must be opened in binary mode (i.e.: <code class="docutils literal notranslate"><span class="pre">io.BinaryIO</span></code> instead of <code class="docutils literal notranslate"><span class="pre">io.StringIO</span></code>,
if using the <cite>io</cite> package).</p></li>
and must be opened in binary mode (i.e.: <code class="docutils literal notranslate"><span class="pre">io.BinaryIO</span></code> instead of <code class="docutils literal notranslate"><span class="pre">io.StringIO</span></code>.).
A <a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#memoryview" title="(in Python v3.12)"><code class="xref py py-class docutils literal notranslate"><span class="pre">memoryview</span></code></a> object may also be provided when reading audio.</p></li>
</ul>
</dd>
</dl>
Expand Down Expand Up @@ -421,7 +421,7 @@ <h1>The <code class="docutils literal notranslate"><span class="pre">pedalboard.
<dt class="sig sig-object py" id="pedalboard.io.ReadableAudioFile">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pedalboard.io.</span></span><span class="sig-name descname"><span class="pre">ReadableAudioFile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">filename</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#str" title="(in Python v3.12)"><span class="pre">str</span></a></span></em><span class="sig-paren">)</span><a class="headerlink" href="#pedalboard.io.ReadableAudioFile" title="Permalink to this definition">#</a></dt>
<dt class="sig sig-object py">
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pedalboard.io.</span></span><span class="sig-name descname"><span class="pre">ReadableAudioFile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file_like</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.BinaryIO" title="(in Python v3.12)"><span class="pre">BinaryIO</span></a></span></em><span class="sig-paren">)</span></dt>
<em class="property"><span class="pre">class</span><span class="w"> </span></em><span class="sig-prename descclassname"><span class="pre">pedalboard.io.</span></span><span class="sig-name descname"><span class="pre">ReadableAudioFile</span></span><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">file_like</span></span><span class="p"><span class="pre">:</span></span><span class="w"> </span><span class="n"><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.Union" title="(in Python v3.12)"><span class="pre">Union</span></a><span class="p"><span class="pre">[</span></span><a class="reference external" href="https://docs.python.org/3/library/typing.html#typing.BinaryIO" title="(in Python v3.12)"><span class="pre">BinaryIO</span></a><span class="p"><span class="pre">,</span></span><span class="w"> </span><a class="reference external" href="https://docs.python.org/3/library/stdtypes.html#memoryview" title="(in Python v3.12)"><span class="pre">memoryview</span></a><span class="p"><span class="pre">]</span></span></span></em><span class="sig-paren">)</span></dt>
<dd><p>A class that wraps an audio file for reading, with native support for Ogg Vorbis,
MP3, WAV, FLAC, and AIFF files on all operating systems. Other formats may also
be readable depending on the operating system and installed system libraries:</p>
Expand Down
2 changes: 1 addition & 1 deletion docs/searchindex.js
View file

Large diffs are not rendered by default.

25 changes: 17 additions & 8 deletions pedalboard/io/AudioFileInit.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ Unlike a typical ``open`` call:
the sample rate of the file.
- A file-like object can be provided to :class:`AudioFile`, allowing for reading and
writing to in-memory streams or buffers. The provided file-like object must be seekable
and must be opened in binary mode (i.e.: ``io.BinaryIO`` instead of ``io.StringIO``,
if using the `io` package).
and must be opened in binary mode (i.e.: ``io.BinaryIO`` instead of ``io.StringIO``.).
A :class:`memoryview` object may also be provided when reading audio.
Examples
Expand Down Expand Up @@ -155,15 +155,24 @@ inline void init_audio_file(
"__new__",
[](const py::object *, py::object filelike, std::string mode) {
if (mode == "r") {
if (!isReadableFileLike(filelike)) {
if (!isReadableFileLike(filelike) &&
!tryConvertingToBuffer(filelike)) {
throw py::type_error(
"Expected either a filename or a file-like object (with "
"read, seek, seekable, and tell methods), but received: " +
filelike.attr("__repr__")().cast<std::string>());
"Expected either a filename, a file-like object (with "
"read, seek, seekable, and tell methods) or a memory view, "
"but received: " +
py::repr(filelike).cast<std::string>());
}

return std::make_shared<ReadableAudioFile>(
std::make_unique<PythonInputStream>(filelike));
if (std::optional<py::buffer> buf =
tryConvertingToBuffer(filelike)) {
return std::make_shared<ReadableAudioFile>(
std::make_unique<PythonMemoryViewInputStream>(*buf,
filelike));
} else {
return std::make_shared<ReadableAudioFile>(
std::make_unique<PythonInputStream>(filelike));
}
} else if (mode == "w") {
throw py::type_error(
"Opening an audio file-like object for writing requires "
Expand Down
9 changes: 5 additions & 4 deletions pedalboard/io/PythonFileLike.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,9 @@ inline void raise() {
class PythonFileLike {
public:
PythonFileLike(py::object fileLike) : fileLike(fileLike) {}
virtual ~PythonFileLike() {}

std::string getRepresentation() {
virtual std::string getRepresentation() {
py::gil_scoped_acquire acquire;
if (PythonException::isPending())
return "<__repr__ failed>";
Expand All @@ -72,7 +73,7 @@ class PythonFileLike {
}
}

bool isSeekable() noexcept {
virtual bool isSeekable() noexcept {
py::gil_scoped_acquire acquire;

if (!PythonException::isPending()) {
Expand All @@ -88,7 +89,7 @@ class PythonFileLike {
return false;
}

juce::int64 getPosition() noexcept {
virtual juce::int64 getPosition() noexcept {
py::gil_scoped_acquire acquire;

if (!PythonException::isPending()) {
Expand All @@ -104,7 +105,7 @@ class PythonFileLike {
return -1;
}

bool setPosition(juce::int64 pos) noexcept {
virtual bool setPosition(juce::int64 pos) noexcept {
py::gil_scoped_acquire acquire;

if (!PythonException::isPending()) {
Expand Down
Loading

0 comments on commit cd686d5

Please sign in to comment.