Skip to content

Commit

Permalink
Rebrand funcx/funcx-endpoint as globus-compute-sdk/globus-compute-end…
Browse files Browse the repository at this point in the history
…point. (globus#1067)

Summary of changes:
  Redo directory structure (compute_endpoint, compute_sdk, compute_funcx/... for new funcX wrapper)
  Rename all classes with FuncX in name (FuncXClient, FuncXExecutor, FuncXTask etc)
  Rename most funcX related internal variable names such as funcx_client, fxc, fxe, etc
  Rename some but not all constants with FUNCX in name, except for user-facing ones (see doc for details)
  Update all readthedocs pages to use Globus Compute in place of funcX
  Create funcx funcx-endpoint packages under compute_funcx/sdk and /endpoint that are simply wrappers around globus-compute-sdk/-endpoint

Detailed change list doc:
   https://docs.google.com/document/d/13gP589fSbH_4jS5Ebz9cotHEOpFl5TN60VDNn6KpzvY

Messages of selected individual commits pre-squash:

  Rename dirs to compute_endpoint and compute_sdk

  Rename FuncXTask to ComputeTask

  Update docs to reflect Globus Compute changes

  Update package/deploy files for compute branding

  Modify imports for compute packages

  Rename FuncXClient to Client

  Rename FuncXExecutor to Executor

  Rename FuncxWebClient to WebClient

  Rename FuncXTask to ComputeTask

  Rename FuncXSerializer to ComputeSerializer

  Rename FuncxScopes to ComputeScopes

  Rename funcx_manager to manager

  Rename FuncXWorker to Worker

  Rename funcx_endpoint/ tests dir to endpoint/

  Rename FuncXFuture to ComputeFuture

  Update exception names for compute branding

  Rename get_funcx_web_client to get_web_client

  Rename funcx-endpoint to globus-compute-endpoint

  Rename funcx-worker to globus-compute-worker

  Rename funcx-manager to globus-compute-manager

  Update test config for compute branding

  Update misc. docstring/comments for compute branding

  Rename FuncxScopeBuilder to ComputeScopeBuilder

  Rename FuncxConsoleFormatter to ComputeConsoleFormatter

  setup and init files for funcx

  funcx and -endpoint to compute import redirects

  move images and use new logo

  tox files for publishing to pypi

  add release readme for funcx

  Renamed test URLs like FUNCX_INTEGRATION_TEST_WEB_URL to COMPUTE_INTEGRATION_TEST_WEBV_URL

  catch funcx_endpoint import errors in config and suggest fix

  import deprecation warnings

add wrappers release.sh

remove search client in loginmanagerprotocol

refactor import modification from suggestions

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

revert action provider url/scope to old one until ecs is ready

Change versions to 2.0.1 for globus-compute* and 2.0.0a4 for wrappers

change deprecation notice color
  • Loading branch information
LeiGlobus authored Apr 7, 2023
1 parent 71a350c commit 9706b5f
Show file tree
Hide file tree
Showing 232 changed files with 2,900 additions and 1,760 deletions.
6 changes: 3 additions & 3 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ A clear and concise description of what the bug is.

**To Reproduce**
Steps to reproduce the behavior, for e.g:
1. Install funcx==0.2.2 and funcx-endpoint==0.2.2 with Python 3.7 on cluster
1. Install globus-compute-sdk==2.0.0 and globus-compute-endpoint==2.0.0 with Python 3.7 on cluster
2. Run a test script
3. Wait 5 mins
4. See error
Expand All @@ -25,8 +25,8 @@ A clear and concise description of what you expected to happen.
- OS & Container technology: [e.g. ubuntu, centos & singularity, docker] @ endpoint
- Python version @ client
- Python version @ endpoint
- funcx version @ client
- funcx-endpoint version @ endpoint
- globus-compute-sdk version @ client
- globus-compute-endpoint version @ endpoint

**Distributed Environment**
- Where are you running the funcX script from? [e.g. Laptop/Workstation, Login node, Compute node]
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,11 +22,11 @@ jobs:
run: pre-commit run -a
- name: mypy (sdk)
run: |
cd funcx_sdk
cd compute_sdk
tox -e mypy
- name: mypy (endpoint)
run: |
cd funcx_endpoint
cd compute_endpoint
tox -e mypy
# ensure docs can build, imitating the RTD build as best we can
Expand All @@ -50,8 +50,8 @@ jobs:
- name: install requirements
run: |
python -m pip install --upgrade setuptools
python -m pip install './funcx_sdk'
python -m pip install './funcx_endpoint'
python -m pip install './compute_sdk'
python -m pip install './compute_endpoint'
python -m pip install safety
- name: run safety check
run: safety check
Expand All @@ -72,7 +72,7 @@ jobs:
- run: python -m pip install tox
- name: run tests
run: |
cd funcx_sdk
cd compute_sdk
tox -e py
test-endpoint:
Expand All @@ -97,7 +97,7 @@ jobs:
- run: python -m pip install tox
- name: run tests
run: |
cd funcx_endpoint
cd compute_endpoint
tox -e py -- --log-cli-level=ERROR
- name: Collect Docker Logs
if: failure()
Expand All @@ -122,7 +122,7 @@ jobs:
run: echo "##[set-output name=imagetag;]$(echo ${GITHUB_REF##*/})"
id: extract_tag_name

- name: Build funcX-endpoint Image for selected python version
- name: Build globus-compute-endpoint Image for selected python version
uses: elgohr/Publish-Docker-Github-Action@master
env:
PYTHON_VERSION: ${{ matrix.python }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/daily.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ jobs:
- name: install requirements
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install './funcx_sdk'
python -m pip install './compute_sdk'
python -m pip install safety
- name: run safety check
run: safety check
Expand All @@ -50,7 +50,7 @@ jobs:
- name: install requirements
run: |
python -m pip install --upgrade pip setuptools wheel
python -m pip install './funcx_endpoint'
python -m pip install './compute_endpoint'
python -m pip install safety
- name: run safety check
run: safety check
Expand Down
2 changes: 1 addition & 1 deletion .isort.cfg
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
[settings]
profile = black
known_first_party = funcx,funcx_endpoint
known_first_party = globus-compute-sdk, globus-compute-endpoint
4 changes: 2 additions & 2 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@ build:
python:
install:
- method: pip
path: ./funcx_sdk
path: ./compute_sdk
extra_requirements:
- docs
- method: pip
path: ./funcx_endpoint
path: ./compute_endpoint
10 changes: 5 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,14 @@ to fill out the fragment.
## Installing Testing Requirements

Testing requirements for each of the two packages in this repository
(funcx-sdk and funcx-endpoint) are specified as installable extras.
(globus-compute-sdk and globus-compute-endpoint) are specified as installable extras.

To install the funcx-sdk test requirements
To install the globus-compute-sdk test requirements

cd funcx_sdk
cd compute_sdk
pip install '.[test]'

To install the funcx-endpoint test requirements
To install the globus-compute-endpoint test requirements

cd funcx_endpoint
cd compute_endpoint
pip install '.[test]'
20 changes: 10 additions & 10 deletions Dockerfile-endpoint
Original file line number Diff line number Diff line change
Expand Up @@ -14,26 +14,26 @@ RUN apt-get update && apt-get upgrade -y

RUN if [ -n "$pip_conf" ]; then echo "$pip_conf" > "/etc/pip.conf"; fi

RUN mkdir /opt/funcx
RUN mkdir /opt/compute
RUN python -m pip install -U pip
RUN python -m pip install kubernetes
RUN python -m pip install --no-binary :all: --force-reinstall pyzmq

COPY funcx_sdk /opt/funcx/funcx_sdk/
WORKDIR /opt/funcx/funcx_sdk
COPY compute_sdk /opt/compute/compute_sdk/
WORKDIR /opt/compute/compute_sdk
RUN python -m pip install .

COPY funcx_endpoint /opt/funcx/funcx_endpoint
WORKDIR /opt/funcx/funcx_endpoint
COPY compute_endpoint /opt/compute/compute_endpoint
WORKDIR /opt/compute/compute_endpoint
RUN python -m pip install .

# Undo local-dev ministrations
RUN if [ -n "$pip_conf" ]; then rm -f "/etc/pip.conf"; fi
RUN if [ -n "$apt_proxy_url" ]; then rm -f "/etc/apt/apt.conf.d/01proxy"; fi

RUN useradd -m funcx
RUN mkdir -p /home/funcx/.kube
USER funcx
WORKDIR /home/funcx
RUN useradd -m compute
RUN mkdir -p /home/compute/.kube
USER compute
WORKDIR /home/compute
COPY helm/boot.sh .
ENV HOME /home/funcx
ENV HOME /home/compute
6 changes: 3 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
.venv-docs:
python -m venv .venv-docs
.venv-docs/bin/pip install -U pip setuptools
.venv-docs/bin/pip install './funcx_sdk[docs]' './funcx_endpoint'
.venv-docs/bin/pip install './compute_sdk[docs]' './compute_endpoint'

.PHONY: lint docs
lint:
pre-commit run -a
cd funcx_sdk; tox -e mypy; cd ..
cd funcx_endpoint; tox -e mypy; cd ..
cd compute_sdk; tox -e mypy; cd ..
cd compute_endpoint; tox -e mypy; cd ..

docs: .venv-docs
# clean the build dir before rebuilding
Expand Down
24 changes: 12 additions & 12 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
funcX - Fast Function Serving
Globus Compute - Fast Function Serving
=============================
|licence| |build-status| |docs| |launch| |NSF-2004894| |NSF-2004932|

funcX is a high-performance function-as-a-service (FaaS) platform that enables
intuitive, flexible, efficient, scalable, and performant remote function execution
on existing infrastructure including clouds, clusters, and supercomputers.
Globus Compute (formerly funcX) is a high-performance function-as-a-service (FaaS)
platform that enables intuitive, flexible, efficient, scalable, and performant remote
function execution on existing infrastructure including clouds, clusters, and supercomputers.

.. |licence| image:: https://img.shields.io/badge/License-Apache%202.0-blue.svg
:target: https://github.com/funcx-faas/funcX/blob/master/LICENSE
Expand All @@ -26,36 +26,36 @@ on existing infrastructure including clouds, clusters, and supercomputers.
:alt: NSF award info


.. image:: docs/_static/logo.png
.. image:: docs/_static/images/globus-300x300-blue.png
:target: https://www.funcx.org
:width: 200

Website: https://www.funcx.org

Documentation: https://funcx.readthedocs.io/en/latest/
Documentation: https://globus-compute.readthedocs.io/en/latest/

Quickstart
==========

funcX is in a beta state with version `0.3.7` currently available on PyPI.
Globus Compute is currently available on PyPI.

To install funcX, please ensure you have python3.6+.::
To install Globus Compute, please ensure you have python3.7+.::

$ python3 --version

Install using Pip::

$ pip install funcx
$ pip install globus-compute-sdk

To use our example notebooks you will need Jupyter.::

$ pip install jupyter

.. note:: The funcX client is supported on MacOS, Linux and Windows.
The funcx-endpoint is only supported on Linux.
.. note:: The Globus Compute client is supported on MacOS, Linux and Windows.
The Globus Compute endpoint is only supported on Linux.

Documentation
=============

Complete documentation for funcX is available `here <https://funcx.readthedocs.io>`_
Complete documentation for Globus Compute is available `here <https://funcx.readthedocs.io>`_

12 changes: 6 additions & 6 deletions RELEASING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Releasing

Releases of `funcx` and `funcx-endpoint` are always done with a single version
Releases of `globus-compute-sdk` and `globus-compute-endpoint` are always done with a single version
number, even when only one package has changes.

The process is partially automated with tools to help along the way.
Expand All @@ -17,15 +17,15 @@ You will also need the following credentials:

- a configured GPG key in `git` in order to create signed tags
- pypi credentials for use with `twine` (e.g. a token in `~/.pypirc`) valid for
publishing `funcx` and `funcx-endpoint`
publishing `globus-compute-sdk` and `globus-compute-endpoint`

## Procedure

1. Bump versions of both packages to a new latest version number by removing
the `-dev` suffix

```bash
$EDITOR funcx_sdk/funcx/version.py funcx_endpoint/funcx_endpoint/version.py
$EDITOR compute_sdk/globus_compute_sdk/version.py compute_endpoint/globus_compute_endpoint/version.py
```

2. Update the changelog by running `scriv collect --edit`
Expand All @@ -35,7 +35,7 @@ $EDITOR funcx_sdk/funcx/version.py funcx_endpoint/funcx_endpoint/version.py

```bash
git add changelog.d/ docs/changelog.rst
git add funcx_sdk/funcx/version.py funcx_endpoint/funcx_endpoint/version.py
git add compute_sdk/globus_compute_sdk/version.py compute_endpoint/globus_compute_endpoint/version.py
git commit -m 'Bump versions and changelog for release'
git push
```
Expand All @@ -50,8 +50,8 @@ git push
then commit and push, e.g.

```bash
$EDITOR funcx_sdk/funcx/version.py funcx_endpoint/funcx_endpoint/version.py
git add funcx_sdk/funcx/version.py funcx_endpoint/funcx_endpoint/version.py
$EDITOR compute_sdk/globus_compute_sdk/version.py compute_endpoint/globus_compute_endpoint/version.py
git add compute_sdk/globus_compute_sdk/version.py compute_endpoint/globus_compute_endpoint/version.py
git commit -m 'Bump versions for dev'
git push
```
4 changes: 2 additions & 2 deletions changelog.d/scriv.ini
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@
format = rst
rst_header_chars = -^
output_file = docs/changelog.rst
entry_title_template = funcx & funcx-endpoint v{{ version }}
version = literal: funcx_sdk/funcx/version.py: __version__
entry_title_template = globus-compute-sdk & globus-compute-endpoint v{{ version }}
version = literal: compute_sdk/globus_compute_sdk/version.py: __version__

# compare against scriv default:
# categories = Removed, Added, Changed, Deprecated, Fixed, Security
Expand Down
File renamed without changes.
4 changes: 4 additions & 0 deletions compute_endpoint/globus_compute_endpoint/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
from globus_compute_endpoint.version import __version__ as _version

__author__ = "The Globus Compute Team"
__version__ = _version
Loading

0 comments on commit 9706b5f

Please sign in to comment.