-
Notifications
You must be signed in to change notification settings - Fork 44
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Separate contribution guide and catalog dev guide from README
- Loading branch information
Showing
3 changed files
with
94 additions
and
48 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,51 +1,9 @@ | ||
| # appyter-catalog: A catalog of [appyters](https://github.com/maayanLab/appyter/) | ||
| # appyter-catalog: A catalog of appyters | ||
|
|
||
| Pull requests encouraged, please refer to the [example](./appyters/example/) for registering your own appyter. | ||
| <https://github.com/maayanLab/appyter/> | ||
|
|
||
| ## Deployment | ||
|
|
||
| Currently, because this application deals with several independent appyters, we construct Dockerfiles independently for each and facilitate deployment with docker-compose.yml. In the future this can be extended to automatically generating a kubernetes deployment or simply using docker-in-docker, but for now a simple Makefile will do the trick of hosting the docker-compose on a single system. | ||
|
|
||
| ```bash | ||
| # Download the catalog locally | ||
| git clone [email protected]:MaayanLab/appyter-catalog.git | ||
|
|
||
| # Build+run everything | ||
| make deploy | ||
|
|
||
| # Build+run specific components | ||
| make appyters/example/.deploy app/.deploy | ||
|
|
||
| # Publish specific component | ||
| make appyters/example/.publish | ||
| ``` | ||
|
|
||
| ## Details | ||
|
|
||
| The appter-catalog does several things to permit integration of several independent appyters with their own dependencies while permitting various modifications performed at the entire application level. | ||
|
|
||
| 1. Submit pull request with new appyter added to `appyters` directory. | ||
| 2. `.github/workflows/validate_merge.yml` instructs github to execute `validate/validate_merge.py` | ||
| 3. `validate/validate_merge.py` executes, validating the structure of the directory including | ||
| 1. Asserting that `appyter.json` is formatted according to the `schema/appyter-validator.json` json-schema validator | ||
| 2. Asserting that other relevant files are present | ||
| 3. Uses `compose/build_dockerfile.py` to construct and build a Dockerfile the same way it would be done in production | ||
| 4. PR is accepted if and only if the validation and manual review is passed | ||
| 5. `Makefile` can be used to facilitate the remaining steps | ||
| 6. Run `compose/build_dockerfile.py` for each appyter to inject `override`s, `catalog_helper`, and construct a Dockerfile for the `appyter` | ||
| 1. When built, the files in `deploy/override` will be merged (using `compose/catalog_helper.py`) with the appyter's own `appyter` overrides | ||
| 7. Run `compose/build_appyters.py` to build a unified `appyters.json` file, containing information about each appyter for the `app` | ||
| 8. Run `cd app && npm i && npm run build` to build the `app` (written in nodejs) with the most recently rendered `appyters.json` | ||
| 9. Run `compose/build_compose.py` to build a application wide `docker-compose.yml` which includes a unified proxy for serving all apps on one endpoint | ||
| 10. Run `docker-compose build` to build all Dockerfiles for the `appyters` and the `app` | ||
| 1. Variables in `.env` are automatically loaded by `docker-compose` | ||
| 2. `appyter_version` in `.env` is used as a `Dockerfile arg` permitting easy updates to the version used by all `appyters` | ||
| 3. A `postgres` database is used through `postgrest` for `app` state. | ||
| 1. `postgres/migrations` contains the `postgres` schema of that database, which are applied at database initialization in `postgres/Dockerfile` | ||
| 11. Run `docker-compose up -d` to start all docker containers in the application. | ||
| 1. Variables in `.env` are automatically loaded by `docker-compose` | ||
| 2. `maayanlab/proxy` is used to proxy different paths to the respective containers and set up `https` with `letsencrypt` | ||
| 3. `postgrest` exposes `postgres` tables, views, and functions on the `api` schema with the `guest` role over HTTP at `/postgrest` | ||
| 4. `app` facilitates showing all `appyters` and navigating users to the mount location of the actual `appyter` container. | ||
| 5. `data/<container_name>` contains all application data split up by container mounted from the host. | ||
| This repository contains source code for the catalog and the appyters in it and is used to assemble and run them all. | ||
|
|
||
| Quick Links: | ||
| - [Appyter Contribution Guide](./docs/contributions.md) | ||
| - [Catalog Deployment Guide](./docs/catalog-deployment.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,55 @@ | ||
| # Appyter Catalog Deployment Guide | ||
|
|
||
| ## Who is this for? | ||
| This is meant for deploying the whole catalog (all appyters), not for any single appyter. | ||
|
|
||
| For deploying or creating a single appyter, | ||
| - If the appyter is in the catalog you can use the [Run Locally](https://appyters.maayanlab.cloud/#/running-appyters/) instructions. | ||
| - Otherwise, see the appyter repo [appyter](https://github.com/maayanLab/appyter/) for more information. | ||
|
|
||
| ## Quick Start | ||
|
|
||
| Currently, because this application deals with several independent appyters, we construct Dockerfiles independently for each and facilitate deployment with docker-compose.yml. In the future this can be extended to automatically generating a kubernetes deployment or simply using docker-in-docker, but for now a simple Makefile will do the trick of hosting the docker-compose on a single system. | ||
|
|
||
| ```bash | ||
| # Download the catalog locally | ||
| git clone [email protected]:MaayanLab/appyter-catalog.git | ||
|
|
||
| # Build+run everything | ||
| make deploy | ||
|
|
||
| # Build+run specific components | ||
| make appyters/example/.deploy app/.deploy | ||
|
|
||
| # Publish specific component | ||
| make appyters/example/.publish | ||
| ``` | ||
|
|
||
| ## System Details | ||
|
|
||
| The appter-catalog does several things to permit integration of several independent appyters with their own dependencies while permitting various modifications performed at the entire application level. | ||
|
|
||
| 1. Submit pull request with new appyter added to `appyters` directory. | ||
| 2. `.github/workflows/validate_merge.yml` instructs github to execute `validate/validate_merge.py` | ||
| 3. `validate/validate_merge.py` executes, validating the structure of the directory including | ||
| 1. Asserting that `appyter.json` is formatted according to the `schema/appyter-validator.json` json-schema validator | ||
| 2. Asserting that other relevant files are present | ||
| 3. Uses `compose/build_dockerfile.py` to construct and build a Dockerfile the same way it would be done in production | ||
| 4. PR is accepted if and only if the validation and manual review is passed | ||
| 5. `Makefile` can be used to facilitate the remaining steps | ||
| 6. Run `compose/build_dockerfile.py` for each appyter to inject `override`s, `catalog_helper`, and construct a Dockerfile for the `appyter` | ||
| 1. When built, the files in `deploy/override` will be merged (using `compose/catalog_helper.py`) with the appyter's own `appyter` overrides | ||
| 7. Run `compose/build_appyters.py` to build a unified `appyters.json` file, containing information about each appyter for the `app` | ||
| 8. Run `cd app && npm i && npm run build` to build the `app` (written in nodejs) with the most recently rendered `appyters.json` | ||
| 9. Run `compose/build_compose.py` to build a application wide `docker-compose.yml` which includes a unified proxy for serving all apps on one endpoint | ||
| 10. Run `docker-compose build` to build all Dockerfiles for the `appyters` and the `app` | ||
| 1. Variables in `.env` are automatically loaded by `docker-compose` | ||
| 2. `appyter_version` in `.env` is used as a `Dockerfile arg` permitting easy updates to the version used by all `appyters` | ||
| 3. A `postgres` database is used through `postgrest` for `app` state. | ||
| 1. `postgres/migrations` contains the `postgres` schema of that database, which are applied at database initialization in `postgres/Dockerfile` | ||
| 11. Run `docker-compose up -d` to start all docker containers in the application. | ||
| 1. Variables in `.env` are automatically loaded by `docker-compose` | ||
| 2. `maayanlab/proxy` is used to proxy different paths to the respective containers and set up `https` with `letsencrypt` | ||
| 3. `postgrest` exposes `postgres` tables, views, and functions on the `api` schema with the `guest` role over HTTP at `/postgrest` | ||
| 4. `app` facilitates showing all `appyters` and navigating users to the mount location of the actual `appyter` container. | ||
| 5. `data/<container_name>` contains all application data split up by container mounted from the host. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # Appyter Catalog Contribution Guide | ||
|
|
||
| ## Who should contribute? | ||
| Anyone who wants to! Please create a pull request to add new content. | ||
|
|
||
| ## Quick Start | ||
| 1. Install the appyter cli and its optional dependency cookiecutter. | ||
| ```bash | ||
| pip install appyter cookiecutter | ||
| ``` | ||
| 2. Create an initial appyter in the `appyters` directory using `appyter init` | ||
| ```bash | ||
| cd appyters | ||
| appyter init | ||
| # repond to prompts | ||
| ``` | ||
| 3. Review and update all generated files in the directory. | ||
| - `*.ipynb`: The jupyter notebook is the source code of your appyter, for more information about developing appyters, refer to the appyter docs: <https://github.com/maayanLab/appyter/>. It might also be helpful to review other appyters on the catalog. | ||
| - `README.md`: is used on the appyter landing page of the catalog, rather than setup instructions it should have a longer, more elaborate, description of your appyter for potential users of it. | ||
| - `appyter.json`: has basic metadata about your appyter used for filtering and searching it in the catalog. You should update it with pertinent tags (preferrably, overlapping tags to those seen on the current catalog), and you should also add an image. | ||
| - image: An image for your appyter should be located in a directory `static` under your appyter directory, it should be `png` and precisely `1280x720px`. The filename should be used for the `appyter.json` `image` key. | ||
| - `requirements.txt`: has the python packages one needs to install to use all features of the appyter | ||
| 4. Validate your appyter. Note that this is run when you make a Pull Request as well. | ||
| ```bash | ||
| # install dependencies to run the tests | ||
| pip install -r compose/requirements.txt -r validate/requirements.txt | ||
| # run the tests on your changes to the repo | ||
| python validate/validate_merge.py -vvv | ||
| ``` | ||
| 5. If all goes well, submit a pull request and it should validate there too, be merged, and deployed in production for the wider community of users to access! Thank you! | ||
|
|
||
| ## Example | ||
| An example appyter exists featuring a complete appyter using most appyter features for your reference, it's under [appyters/example/](../appyters/example). |