py-js: python3 objects for max

Simple (and extensible) python3 externals for MaxMSP.

Cross-platform: currently builds 'natively' on macOS (x86_64 or arm64) and Windows (with MSVC).

repo - https://github.com/shakfu/py-js

Overview

This project started out as an attempt (during a covid-19 lockdown) to develop a basic python3 external for Max/MSP. It then evolved into an umbrella project for exploring different ways of using python3 in Max/MSP.

Along the way, a number of externals have been developed for use in a live Max environment (click on the links to see more detailed documentation):

Python3 Externals:

name	sdk	lang	description
py	max-sdk	c	well-featured, many packaging options + cython api
pyjs	max-sdk	c	js-friendly -- written as a Max javascript-extension
mamba	max-sdk	c	single-header c library to nest a python3 interpreter in any external
krait	max-sdk	c++	single-header c++ library to nest a python3 interpreter in any external
cobra	max-sdk	c	python3 external providing deferred and clocked function execution
mxpy	max-sdk	c	a translation of pdpython into Max
zedit	max-sdk	c	a web-based python editor using codemirror and the mongoose embedded webserver.
pymx [2]	min-devkit	c++	concise, modern, using pybind11

[1] pymx has been moved to its own github project because it uses the min-devkit sdk.

Alternative Python Implementation Externals:

name	sdk	lang	description
pktpy	max-sdk	c++	uses the pocketpy single-header c++ library
mpy [2]	max-sdk	c	a proof-of-concept embedding micropython

[2] mpy is not enabled by default since it is still in early stages and more of a proof-of-concept to embed micropython in an external. To build it use the -DBUILD_MICROPYTHON_EXTERNAL option with cmake.

ZeroMQ-related Externals:

name	sdk	lang	description
jmx	max-sdk	c	explore how to embed a jupyter client or kernel in an external
zpy	max-sdk	c	uses zeromq for 2way-comms with an external python process
zthread	max-sdk	c	exploration of zeromq and Max threads

Note: zeromq externals are not enabled by default since they require zeromq libraries to be installed. To build them use the -DBUILD_ZEROMQ_EXTERNALS option with cmake.

The common objective in these externals is to help use and distribute python code and libraries in Max applications. Many can be considered experimental, with 80% of development time going to the first two externals (py and pyjs). Please see below for an overview and feature comparison.

At the time of this writing, and since the switch to max-sdk-base, the project has the following compatibility profile:

• macOS: both x86_64 and Apple Silicon compatible. Note that the project intentionally only produces 'native' (x86_64 xor arm64) externals with no current plans for 'fat' or universal externals to serve both architectures. You can download codesigned, notarized x86_64-based and arm64-based python3 externals from the releases section.

• Windows: windows support was provided relatively recently, with currently all Python3 externals and also the pktpy projects building without issues on Windows. The only caveat is that as of this writing python3 externals are dynamically linked to the local Python3 .dll and are therefore not relocatable. One idea to overcome this constraint is to include the external's dependencies in the 'support' folder. This will hopefully be addressed in future iterations. The pktpy external, however, is fully portable and self-contained.

This README will mostly cover the first two mature externals (py.mxo and pyjs.mxo) and their many build variations available via a custom python-based build system which was specifically developed to cater for different scenerios of packaging and deploying the externals in Max packages and standalones.

If you are interested in any of the other subprojects, please look into the respective folder in the py-js/source/projects section.

The Quickstart section below covers general setup for all of the externals, and will get you up and running with the py and pyjs externals. The Building Experimental Externals using Cmake section provides additional info to build the other remaining externals, and the Building self-contained Python3 Externals for Packages and Standalones section covers more advanced building and deployment scenarios.

Please feel free to ask questions or make suggestions via the project's github issue tracker.

Quickstart

This repo has a git submodule dependency with max-sdk-base. This is quite typical for Max externals.

This means you should git clone as follows:

git clone https://github.com/shakfu/py-js.git
git submodule init
git submodule update

or more concisely:

git clone --recursive https://github.com/shakfu/py-js.git

macOS

As mentioned earlier, the py and pyjs objects are the most mature and best documented of the collection. Happily, there is also no need to compile them as they are available for download, fully codesigned and notarized, from the releases section.

If you'd rather build them or any of the other externals yourself then the process is straightforward:

1. You should have a modern python3 cpython implementation installed on your Mac: preferably either from python.org or from Homebrew. Note that even system python3 provided by Apple will work in a number of cases. Python versions from 3.8 to 3.13 are tested and known to work.

2. Make sure you also have Xcode installed.

3. Git clone the py-js repo as per the above method to a path without a space and without possible icloud syncing (i.e don't clone to $HOME/Documents/Max 8/Packages) [?] and run the following in the cloned repo:

   make setup

   The above will initialize and update the required git submodules and symlink the repo to $HOME/Documents/Max 8/Packages/py-js to install it as a Max Package and enable you to test the externals and run the patches.

   [?] It is possible to install py-js directly into $HOME/Documents/Max 8/Packages, but it requires moving the place of compilation to a location in your filesystem that is not exposed to errors due to icloud syncing or spaces in the path. This split is possible, but it is not recommended for the purposes of this quickstart.

4. Install cython via pip3 install cython, required for translating the cython-based api.pyx, which wraps the the Max c-api, to c.

5. To build only the py and pyjs externals, type the following in the root directory of the py-js project (other installation options are detailed below):

   make

Note that typing make here is the same as typing make default or make all. This will create two externals py.mxo and pyjs.mxo in your externals folder. These are relatively small in size and are linked to your system python3 installation. This has the immediate benefit that you have access to your curated collection of existing python packages. The tradeoff is that these externals are dynamically linked with local dependencies and therefore not usable in standalones and relocatable Max packages.

No worries, if you need portable relocatable python3 externals for your package or standalone then make sure to read the Building self-contained Python3 Externals for Packages and Standalones section

Open up any of the patch files in the patchers directory of the repo or the generated Max package, and also look at the .maxhelp patchers to understand how the py and the pyjs objects work.

Windows

Since Windows support still is relatively new, no releases have been made pending further testing.

Currently, the externals which are enabled by default in this project can be built with only a few requirements:

1. Install Visual Studio Community Edition or use the commercial versions as you like.

2. Install Python3 for Windows from python.org

3. (Optional) since Visual Studio has its captive cmake, you can use that, but it is preferable to install cmake independently.

After installation of the above you can build the externals inside your Documents/Max 8/Packages folder as follows:

git clone --recursive https://github.com/shakfu/py-js
cd py-js
mkdir build
cd build
cmake .. -DCMAKE_BUILD_TYPE=Release
cmake --build . --config Release

Open one of the .maxhelp files or any of the files in the patchers folders to see how things work.

Building Experimental Externals using Cmake

You can also use cmake to build all externals using similar methods to the max-sdk.

First make sure you have completed the Quickstart section above. Next you will install cmake if necessary and a couple of additional dependencies for some of the subprojects. Of course, skip what is already installed:

brew install cmake zmq czmq

Now you can build all externals (including py and pyjs) in one shot using cmake:

make projects

After doing the above, the recommended iterative development workflow is to make changes to the source code in the respective project and then cd py-js/build and cmake --build .. This will cause cmake to only build modified projects efficiently.

Note that for some of the less developed externals and more experimental features please don't be surprised if Max seg-faults (especially if you start experimenting with the cython wrapped api module which operates on the c-level of the Max SDK).

Also note that for py and pyjs externals the cmake build method described does not yet create self-contained python externals which can be used in Max Packages and Standalones. The Building self-contained Python3 Externals for Packages and Standalones section addresses this requirement.

Visit the project-specific links above for more detailed documentation.

Related projects

• relocatable-python: A tool for building standalone relocatable Python.framework bundles. (used in this project)

• python-build-standalone: Produce redistributable builds of Python. (Interesting but not used in this project)

• Python Apple Support: A meta-package for building a version of Python that can be embedded into a macOS, iOS, tvOS or watchOS project. (directly inspired static linking approach)

• python-cmake-buildsystem: A cmake buildsystem for compiling Python. Not currently used in this project, but may be used in the future.

• py2max : using python3 with Max in an offline capacity to generate max patches.

• maxutils : scripts and utilities to help with codesigning and notarization of Max standalones and externals.

• pocketpy: C++17 header-only Python interpreter for game engines.

• micropython: a lean and efficient Python implementation for microcontrollers and constrained systems

Prior Art and Thanks

I was motivated to start this project because I found myself recurrently wanting to use some python libraries or functions in Max.

Looking around for a python max external I found the following:

• Thomas Grill's py/pyext – Python scripting objects for Pure Data and Max is the most mature Max/Python implementation and when I was starting this project, it seemed very promising but then I read that the 'available Max port is not actively maintained.' I also noted that it was written in C++ and that it needed an additional c++ flext layer to compile. I was further dissuaded from diving in as it supported, at the time, only python 2 which seemed difficult to swallow considering Python2 is basically not developed anymore. Ironically, this project has become more active recently, and I finally was persuaded to go back and try to compile it and finally got it running. I found it to be extremely technically impressive work, but it had probably suffered from the burden of having to maintain several moving dependencies (puredata, max, python, flext, c++). The complexity probably put off some possible contributors which would have made the maintenance of the project easier for Thomas. In any case, it's an awesome project and it would be great if this project could somehow help py/ext in some way or the other.

• max-py -- Embedding Python 2 / 3 in MaxMSP with pybind11. This looks like a reasonable effort, but only 9 commits and no further commits for 2 years as of this writing.

• nt.python_for_max -- Basic implementation of python in max using a fork of Graham Wakefield's old c++ interface. Hasn't really been touched in 3 years.

• net.loadbang.jython -- A jython implementation for Max which uses the MXJ java interface. It's looks like a solid effort using Jython 2.7 but the last commit was in 2015.

Around the time of the beginning of my first covid-19 lockdown, I stumbled upon Iain Duncan's Scheme for Max project, and I was quite inspired by his efforts and approach to embed a scheme implementation into a Max external.

So it was decided, during a period with less distractions than usual, to try to make a minimal python3 external, learn the max sdk, the python c-api, and also how to write more than a few lines of c that didn't crash.

It's been an education and I have come to understand precisely a quote I remember somewhere about the c language: that it's "like a scalpel". I now understand this to mean that in skilled hands it can do wonders, otherwise you almost always end up killing the patient.

Thanks to Luigi Castelli for his help with Max/MSP questions, to Stefan Behnel for his help with Cython questions, and to Iain Duncan for providing the initial inspiration and for saving me time with some great implementation ideas.

Thanks to Greg Neagle for zeroing in on the relocatability problem and sharing his elegant solution for Python frameworks via his relocatable-python project on Github.