feat: add initial development guidelines to OS

docs/development/domino-py.md

@@ -0,0 +1,87 @@
+# Domino-py
+
+:::warning
+Domino package can **not** be tested in docker compose with hot reload.
+:::
+
+## Kubernetes
+
+When running the application in development mode using the key `DOMINO_DEPLOY_MODE=local-k8s-dev` in your `config.toml` file, the application will use the Kubernetes API to create some volumes mappings, allowing some development features:
+1. Hot reloading for local domino package.
+2. Use of local helm charts when creating the cluster.
+3. Use of local frontend image in cluster.
+4. Use of local rest image in cluster.
+5. Hot reloading for local pieces repositories code.
+
+
+### 1. Activating hot reloading for local domino package
+If you want to run your cluster with hot reloading for your local domino package you should follow the next steps:
+1. Go to your `config.toml` file in the `dev` section
+2. Add the key `DOMINO_LOCAL_DOMINO_PACKAGE` with the value to the path of your `src/domino` folder of your local domino package. Example: `DOMINO_LOCAL_DOMINO_PACKAGE=path/to/your/domino/src/domino`
+
+### 2. Using local helm charts when creating the cluster
+If you need to test run your cluster with local helm charts you should follow the same steps as the previous section.
+The applicaton will get the `helm` folder from the path `path/to/your/domino/helm/domino` based on the path you defined in the `DOMINO_LOCAL_DOMINO_PACKAGE` key.
+So, edit your helm charts in the `path/to/your/domino/helm/domino` folder and the application will use them when creating the cluster.
+
+### 3. Using local frontend image in cluster
+If you want to run your cluster with a local frontend image you should follow the next steps:
+1. Build your local frontend image. You can use the following command from the root of the project: `DOCKER_BUILDKIT=1 docker build -f ./frontend/Dockerfile.dev -t domino-frontend ./frontend`
+2. Add the key `DOMINO_FRONTEND_IMAGE` with the value `domino-frontend` (or other tag you choose) to your `config.toml` file in the `dev` section.
+**Important:** This will not activate hot reloading for your local frontend code. If you need to update your frontend code in the cluster you should build the image again.
+
+### 4. Using local rest image in cluster
+As the frontend image, if you want to run your cluster with a local rest image you should follow the next steps:
+1. Build your local rest image. You can use the following command from the root of the project: `DOCKER_BUILDKIT=1 docker build -f ./rest/Dockerfile -t domino-rest ./rest`
+2. Add the key `DOMINO_REST_IMAGE` with the value `domino-rest` (or other tag you choose) to your `config.toml` file in the `dev` section.
+
+This will load your local rest image in the kind cluster.
+**Important:** This will not activate hot reloading for your local rest code. If you need to update your rest code in the cluster you should build the image again.
+
+### 5. Using local airflow base image
+If you need to update the base airflow image used in worker and scheduler containers you can do it by following the next steps:
+1. Build your local airflow base image. You can use the following command from the root of the project: `DOCKER_BUILDKIT=1 docker build -f Dockerfile-airflow-domino.dev -t domino-airflow .`
+2. Addthe key `DOMINO_AIRFLOW_IMAGE` with the value `domino-airflow` (or other tag you choose) to your `config.toml` file in the `dev` section.
+
+This will load your local airflow base image in the kind cluster.
+**Important:** This is useful if you want to update airflow image dependencies.
+To update only the domino package code in the airflow containers you don't need to build the image again, you can use the hot reloading feature for local domino package describe in the section 1.
+
+
+### 5. Activating hot reloading for local pieces repositories code
+You can activate hot reloading for the local code of your pieces repositories. To do so, you should follow the next steps:
+1. Go to your `config.toml` file in the `dev` section
+2. Add the key `<piece_repository_name>` with the value to the path of your `<piece_repository_name>` folder of your local pieces repository. Example: `default_domino_pieces=path/to/your/default_domino_pieces`
+The `<piece_repository_name>` must be the same name in the piece repository `config.toml` file in `REPOSITORY_NAME` key.
+
+The application will create a volume mapping between the path you specified and the path in the cluster where the pieces repository is mounted. This will allow you to update your pieces repository code and see the changes in the cluster without the need to build the image again.
+
+## Docker Compose
+You can run your application in development mode using docker compose file.
+To do so, you should run using the `docker-compose-dev.yaml` file, as follows:
+```bash
+docker-compose -f docker-compose-dev.yaml up
+```
+This will activate some development features, such as:
+1. Hot reloading for local domino package in `worker` and `scheduler` containers.
+2. Hot reloading for `rest` code.
+3. Hot reloading for `frontend` code.
+
+:::warning
+**Important:** This will **not** activate hot reloading for your pieces repositories code and will **not** activate hot reloading for local domino package in the **pieces containers**. If you are running a piece and you need to have the domino package updated in the piece container you should update the piece image wit the domino development package you want to use.
+:::
+
+:::note
+**Workaround:** As mentioned before, currently we don't have a direct way to allow hot reloading for domino package in pieces containers, however, you can use the following workaround:
+:::
+
+1. Go to `domino/src/domino/custom_operators/docker_operator.py` file.
+2. In the `__init__` method and start the `mounts` variable with the following code:
+```python
+mounts = [
+    Mount(source='/path/to/domino/src/domino', target='/home/domino/domino_py/', type='bind', read_only=True)
+]
+```
+The `source` must be the path to your local domino package. The `target` must be the path where the domino package will be mounted in the pieces container, by default the pieces images are built to use `/home/domino/domino_py/`.
+**DEPRECATED WARNING:** The target path will be deprecated in the next update since the pieces will be built using `/home/domino/domino_py/src/domino` as the default path for the domino package.
+

docs/development/frontend.md

@@ -0,0 +1,103 @@
+---
+sidebar_position: 2
+---
+
+# Domino UI
+
+## ⬇️ Install dependencies
+
+If you choose to use [docker-compose-dev.yml](./run-dev.md) (recommended), you will now be able to make changes and verify the modifications through hot reload. However, for your preferred editor's autocomplete to work, you will need to install the dependencies using pnpm.
+
+To do this, you will need to install node-js > 16 <= 20 and the `pnpm` package manager itself.
+
+As a recommendation, use the [`n`](https://github.com/tj/n) or [`nvm`](https://github.com/nvm-sh/nvm) package to install the desired node version (the recommendation is NodeJS LTS 18.18.2)
+
+After that, just use `npm` (package manager installed with node) to install `pnpm`
+
+```bash
+npm install -g pnpm
+```
+
+with that, navigate to the frontend root folder with `cd ./frontend` and run the command:
+
+```bash
+pnpm install
+```
+
+This way, all dependencies related to the frontend will be installed in the newly created node_modules folder.
+
+:::tip
+If your modifications include an external lib, after adding it with
+```bash
+pnpm install <lib>
+``` 
+make sure to re-build the frontend image with the command
+```bash
+docker compose -f docker-compose-dev.yaml build domino_frontend
+``` 
+at the root of the project, since the libraries are not mapped into the container due to operating system compatibility and performance.
+:::
+
+## 🗄️ Project Structure
+
+Most of the code lives in the `./frontend/src` folder and looks like this:
+
+```sh
+src
+|
++-- @types            # base types used across the application
+|
++-- components        # shared components used across the entire application
+|
++-- config            # all the global configuration, env variables etc. get exported from here and used in the app
+|
++-- context           # shared context used across the entire application
+|
++-- features          # feature based modules
+|
++-- providers         # all of the application providers
+|
++-- routes            # routes configuration
+|
++-- services          # re-exporting different libraries preconfigured for the application
+|
++-- test              # test utilities and mock server
+|
++-- utils             # shared utility functions
+```
+
+In order to scale the application in the easiest and most maintainable way, keep most of the code inside the `features` folder, which should contain different feature-based things. Every `feature` folder should contain domain specific code for a given feature. This will allow you to keep functionalities scoped to a feature and not mix its declarations with shared things. This is much easier to maintain than a flat folder structure with many files.
+
+A feature could have the following structure:
+
+```sh
+src/features/awesome-feature
+|
++-- api         # exported API request declarations and api hooks related to a specific feature
+|
++-- assets      # assets folder can contain all the static files for a specific feature
+|
++-- components  # components scoped to a specific feature
+|
++-- context     # context scoped to a specific feature
+|
++-- pages       # route components for a specific feature pages
+|
++-- types       # typescript types for TS specific feature domain
+|
++-- utils       # utility functions for a specific feature
+|
++-- index.ts    # entry point for the feature, it should serve as the public API of the given feature and exports everything that should be used outside the feature
+```
+
+Everything from a feature should be exported from the `index.ts` file which behaves as the public API of the feature.
+
+You should import stuff from other features only by using:
+
+`import {AwesomeComponent} from "features/awesome-feature"`
+
+and not
+
+`import {AwesomeComponent} from "features/awesome-feature/components/AwesomeComponent`
+
+This was inspired by how [NX](https://nx.dev/) handles libraries that are isolated but available to be used by the other modules. Think of a feature as a library or a module that is self-contained but can expose different parts to other features via its entry point.

docs/development/index.md

@@ -0,0 +1,20 @@
+---
+sidebar_position: 7
+description: "Local development"
+keywords: ["development", "local", "docker", "docker-compose", "kubernetes", "k8s", "python", "react", "typescript", "fastapi", "docker", "github", "pypi", "pip", "kind", "open source",
+"collaboration"]
+---
+
+# Contributing to Domino
+
+There are a few ways to contribute to Domino:
+
+1. Pieces: [Create open source pieces](../pieces/create_pieces.mdx) and request addition to the gallery through this [link](https://github.com/Tauffer-Consulting/domino_pieces_gallery).
+2. Discussions: Ask questions and/or resolve doubts from other members of the domino community through the [github discussions](https://github.com/Tauffer-Consulting/domino/discussions).
+3. Issues: Alert the Domino team about bugs or request new features to be evaluated [here](https://github.com/Tauffer-Consulting/domino/issues/new/choose).
+4. Pull requests: Implement features and/or fix bugs yourself following the guidelines for each sub-project:
+      * [Frontend](./frontend.md)
+      * [Rest](./rest.md)
+      * [Domino-py](./domino-py.md)
+
+Would you like to get your hands on the code but don't know how? Follow the steps on how to [run Domino on your machine](./run-dev.md).

docs/development/rest.md

@@ -0,0 +1,29 @@
+---
+sidebar_position: 3
+---
+
+# Domino REST
+
+## ⬇️ Install dependencies
+
+Domino REST uses FastAPI to expose routes and communicate with the frontend, if you are running with [docker-compose-dev.yml](./run-dev.md) (recommended), you will now be able to make changes and verify the modifications through hot reload. However, for your preferred editor's autocomplete to work, you will need to install the dependencies using [python 3.9 or above](https://www.python.org/downloads/).
+
+We suggest you create a virtual environment and install the dependencies in there. You can choose the way you want to create the virtual environment. Here is a suggestion:
+
+```bash
+# Create the virtual env with python 3.9
+python3.9 -m venv venv 
+# Activate the virtual env
+source venv venv/bin/activate
+```
+
+After that you can install the dependencies and use your preferred editor to modify the code
+
+```bash
+# Install the dependencies
+pip install -r requirements.txt
+```
+
+## Making changes and testing
+
+The backend app will be running at `http://localhost:8000/` and you can check and test the available endpoints at `http://localhost:8000/docs`.

docs/development/run-dev.md

@@ -0,0 +1,50 @@
+---
+sidebar_position: 1
+---
+
+# Development with compose
+
+:::caution
+
+You should never use this mode in production. It is intended for development only. To run Domino locally in an environment closer to a production environment, see [Run locally with Kind](../run_domino/run_locally_kind.md) for test proposes see [Run locally with compose](../run_domino/run_locally_compose.md) 
+
+:::
+
+This method is the simplest way to test Domino locally, it uses Docker Compose to run all the necessary services:
+
+
+## Dependencies
+
+In order to run Domino locally for development, you need to have these dependencies installed:
+
+- **Docker engine** 20.0 or greater. You can install it by following the instructions [here](https://docs.docker.com/engine/install/).
+- **Docker Compose** V2. You can install it by following the instructions [here](https://docs.docker.com/compose/install/).
+
+
+This command might take up to a few minutes to execute, since it will download and run all the necessary docker images.
+If everything worked as expected, after all processes started successfully you should be able to navigate to `localhost:3000` to access the Domino frontend service.
+
+
+```bash
+docker compose -f docker-compose-dev.yaml up
+```
+
+:::tip
+Domino automatically creates an admin user with the following credentials you can use to login:
+
+- **Email**: [email protected]
+- **Password**: admin
+:::
+
+## Change code
+
+For now on, every modification you made in `frontend` and `rest` folders will take effect immediately, but for each sub-project you may need some configurations on your development environment to use autocomplete and references.
+
+* [Frontend](./frontend.md)
+* [Rest](./rest.md)
+
+:::note
+Domino package does not support hot reload in compose mode, but you can follow the [Kubernetes guide](../run_domino/run_locally_kind.md#local-deployment-for-development) to be able to test modifications there.
+:::
+
+* [Domino-py](./domino-py.md)