Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev #216

Merged
merged 74 commits into from
Feb 19, 2024
Merged

Dev #216

Changes from 8 commits
Commits
Show all changes
74 commits
Select commit Hold shift + click to select a range
1b797e2
ref: update utils functions
nathan-vm Jan 3, 2024
5db59e4
refactor: piece and schemas types
nathan-vm Jan 5, 2024
99dd5b3
feat: improve schemas types as real json schema
nathan-vm Jan 14, 2024
d209e75
add k8s to task deploy mode
vinicvaz Jan 18, 2024
41f0c50
ref: improve upstream options object
nathan-vm Jan 17, 2024
0c48b20
remove result print
vinicvaz Jan 19, 2024
f23fd8b
feat: add debug option for frontend
nathan-vm Jan 18, 2024
204fa4d
fix: textarea optional input
nathan-vm Jan 22, 2024
ad0e674
fix: make debug work
nathan-vm Jan 23, 2024
f7677eb
quickfix
vinicvaz Jan 29, 2024
afb10f6
add warning when no releases returned
vinicvaz Jan 29, 2024
8d617ba
Merge pull request #222 from Tauffer-Consulting/fix/warning-releases
vinicvaz Jan 29, 2024
9b89045
add workflow_run with path dependency to action
vinicvaz Jan 29, 2024
5a3f4a2
Merge branch 'main' of https://github.com/Tauffer-Consulting/domino i…
vinicvaz Feb 1, 2024
06c1f58
refactor: update workspace card view
tdari Feb 2, 2024
abc2a41
Merge pull request #228 from tdari/refactor/improve-workspace-card
vinicvaz Feb 5, 2024
3bbd2bc
refactor: use pnpm
nathan-vm Feb 6, 2024
0a66d44
refactor: improve workflow editor page
nathan-vm Feb 6, 2024
ec4bfee
refactor: improve workflow editor context
nathan-vm Feb 7, 2024
c265bd8
fix: dockerfile and pnpm
nathan-vm Feb 7, 2024
cbdbcf6
fix: open settings
nathan-vm Feb 7, 2024
0edf4da
add graph utils function to check if is dag using DFS
vinicvaz Feb 2, 2024
b764f33
using acyclic utils onConnect
vinicvaz Feb 2, 2024
741d2b6
add example
vinicvaz Feb 2, 2024
0efe9aa
Merge pull request #227 from Tauffer-Consulting/fix/circular-workflows
nathan-vm Feb 8, 2024
52f0a0f
fix: delete workspace from local storage on leave
tdari Feb 8, 2024
855cf1e
Update workspaces.tsx
tdari Feb 9, 2024
a2a4f0a
Merge pull request #232 from tdari/fix/error-message-on-leave
vinicvaz Feb 9, 2024
4f79e8c
chore: remove console.log
nathan-vm Feb 9, 2024
8f15e52
storage instance var on base piece
vinicvaz Feb 12, 2024
6221388
add storage var
vinicvaz Feb 12, 2024
71fe3b6
publishing dev image on branch
vinicvaz Feb 12, 2024
25b28c1
add dev option to build images cli
vinicvaz Feb 12, 2024
f0df067
remove prompt
vinicvaz Feb 12, 2024
8300323
fix build
vinicvaz Feb 12, 2024
2a0b6cf
Merge pull request #234 from Tauffer-Consulting/update-base-piece
vinicvaz Feb 12, 2024
af1e116
remove branch from workflow
vinicvaz Feb 12, 2024
e69dd77
xcom obj
vinicvaz Feb 12, 2024
ae5c66e
fix path
vinicvaz Feb 12, 2024
3b58349
mend
vinicvaz Feb 12, 2024
cd01fe0
update var name and add return storage usage to ops
vinicvaz Feb 12, 2024
deb36da
fix base piece shared storage usage
vinicvaz Feb 12, 2024
0f28ab5
update logo in readme
vinicvaz Feb 14, 2024
5258e85
feat: improve form inputs
nathan-vm Feb 13, 2024
70c5eae
add print to test
vinicvaz Feb 15, 2024
2c104e3
trigger
vinicvaz Feb 15, 2024
d00a5ba
increase timeout
vinicvaz Feb 15, 2024
0e1847a
test
vinicvaz Feb 15, 2024
736ceef
secret msg
vinicvaz Feb 15, 2024
d06784d
update piece repo test version
vinicvaz Feb 15, 2024
c1c50cc
LOGS
vinicvaz Feb 15, 2024
358ebe6
Merge pull request #236 from Tauffer-Consulting/test
vinicvaz Feb 15, 2024
d57effc
tests
vinicvaz Feb 15, 2024
699ac18
update logos
vinicvaz Feb 15, 2024
7164b62
add fonts
vinicvaz Feb 15, 2024
49ef892
colors and letter
vinicvaz Feb 15, 2024
01f366c
fix: settings storage
nathan-vm Feb 15, 2024
7aa995b
Merge pull request #209 from Tauffer-Consulting/refactor/parse-schema…
nathan-vm Feb 15, 2024
f95251a
favicon
vinicvaz Feb 15, 2024
e50bb50
Merge pull request #235 from Tauffer-Consulting/visual-identity
vinicvaz Feb 15, 2024
d35b113
fix types erros
nathan-vm Feb 15, 2024
78f99af
Merge pull request #237 from Tauffer-Consulting:fix/frontend-image-build
nathan-vm Feb 15, 2024
f69fc5c
using default none value to avoid breaking old pieces
vinicvaz Feb 16, 2024
e2cc6d6
using main colors in runs states panel and table
vinicvaz Feb 16, 2024
81cd9ec
workspace cards using palette
vinicvaz Feb 16, 2024
64c36f3
Merge branch 'main' of https://github.com/Tauffer-Consulting/domino i…
vinicvaz Feb 16, 2024
4cad784
fix: dockerfile frontend
nathan-vm Feb 19, 2024
9548977
Merge pull request #241 from Tauffer-Consulting/fix/frontend-image-build
vinicvaz Feb 19, 2024
f5af14b
Merge pull request #239 from Tauffer-Consulting/visual-identity
vinicvaz Feb 19, 2024
fe60912
fix: input array itemKey
nathan-vm Feb 19, 2024
8e4b08a
gitignore
vinicvaz Feb 19, 2024
d18494c
Merge branch 'dev' of https://github.com/Tauffer-Consulting/domino in…
vinicvaz Feb 19, 2024
50fab7f
Delete .vscode directory
vinicvaz Feb 19, 2024
a357276
minor version
vinicvaz Feb 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/integration-tests-api.yml
Original file line number Diff line number Diff line change
@@ -5,7 +5,7 @@ on:
branches:
- main
- dev
- test/*
- test

jobs:
api-tests:
@@ -55,7 +55,7 @@ jobs:
DOMINO_GITHUB_ACCESS_TOKEN_WORKFLOWS: ${{ secrets.TESTS_GITHUB_WORKFLOWS_TOKEN }}
DOMINO_GITHUB_WORKFLOWS_REPOSITORY: ${{secrets.TESTS_WORKFLOWS_REPOSITORY}}
run: pytest --cov=rest --cov-report=xml --cov-report=term-missing rest/tests/ -v -s

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v3
env:
2 changes: 1 addition & 1 deletion rest/routers/workflow_router.py
Original file line number Diff line number Diff line change
@@ -216,7 +216,7 @@ def list_run_tasks(
)
except (BaseException, ForbiddenException, ResourceNotFoundException) as e:
raise HTTPException(status_code=e.status_code, detail=e.message)

@router.get(
"/{workflow_id}/runs/{workflow_run_id}/tasks/report",
status_code=200,
24 changes: 12 additions & 12 deletions rest/services/auth_service.py
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@
from passlib.context import CryptContext
import jwt
from schemas.errors.base import ForbiddenError, ResourceNotFoundError
from core.settings import settings
from core.settings import settings
from schemas.context.auth_context import AuthorizationContextData, WorkspaceAuthorizerData
from repository.user_repository import UserRepository
from repository.workspace_repository import WorkspaceRepository
@@ -102,7 +102,7 @@ def wrapper(*args, **kwargs):
workspace_associative_data = cls.workspace_repository.find_by_id_and_user_id(id=repository.workspace_id, user_id=auth_context.user_id)
if not workspace_associative_data:
raise HTTPException(status_code=ResourceNotFoundError().status_code, detail=ResourceNotFoundError().message)

if workspace_associative_data and not workspace_associative_data.permission:
raise HTTPException(status_code=ForbiddenError().status_code, detail=ForbiddenError().message)

@@ -118,20 +118,20 @@ def wrapper(*args, **kwargs):

@classmethod
def workspace_access_authorizer(
cls,
cls,
workspace_id: Optional[int],
auth: HTTPAuthorizationCredentials = Security(security),
):
auth_context = cls.auth_wrapper(auth)
if not workspace_id:
raise HTTPException(status_code=ForbiddenError().status_code, detail=ForbiddenError().message)
workspace_associative_data = cls.workspace_repository.find_by_id_and_user_id(
id=workspace_id,
id=workspace_id,
user_id=auth_context.user_id
)
if not workspace_associative_data:
raise HTTPException(
status_code=ResourceNotFoundError().status_code,
status_code=ResourceNotFoundError().status_code,
detail=ResourceNotFoundError(message='Workspace not found').message
)
if workspace_associative_data and not workspace_associative_data.permission:
@@ -148,7 +148,7 @@ def workspace_access_authorizer(
user_permission=workspace_associative_data.permission
)
return auth_context

@classmethod
def workspace_owner_access_authorizer_body(
cls,
@@ -157,7 +157,7 @@ def workspace_owner_access_authorizer_body(
):
workspace_id = body.get('workspace_id')
return cls.workspace_owner_access_authorizer(workspace_id=workspace_id, auth=auth)


@classmethod
def workspace_owner_access_authorizer(
@@ -169,12 +169,12 @@ def workspace_owner_access_authorizer(
if not workspace_id:
raise HTTPException(status_code=ForbiddenError().status_code, detail=ForbiddenError().message)
workspace_associative_data = cls.workspace_repository.find_by_id_and_user_id(
id=workspace_id,
id=workspace_id,
user_id=auth_context.user_id
)
if not workspace_associative_data:
raise HTTPException(status_code=ResourceNotFoundError().status_code, detail=ResourceNotFoundError().message)

if workspace_associative_data and not workspace_associative_data.permission:
raise HTTPException(status_code=ForbiddenError().status_code, detail=ForbiddenError().message)

@@ -213,7 +213,7 @@ def piece_repository_workspace_owner_access_authorizer(

if workspace_associative_data and not workspace_associative_data.permission:
raise HTTPException(status_code=ForbiddenError().status_code, detail=ForbiddenError().message)

if workspace_associative_data and workspace_associative_data.status != UserWorkspaceStatus.accepted.value:
raise HTTPException(status_code=ForbiddenError().status_code, detail=ForbiddenError().message)

@@ -233,7 +233,7 @@ def wrapper(*args, **kwargs):
workspace_associative_data = cls.workspace_repository.find_by_id_and_user_id(id=workspace_id, user_id=auth_context.user_id)
if not workspace_associative_data:
raise HTTPException(status_code=ResourceNotFoundError().status_code, detail=ResourceNotFoundError().message)

if workspace_associative_data and not workspace_associative_data.permission:
raise HTTPException(status_code=ForbiddenError().status_code, detail=ForbiddenError().message)

@@ -262,7 +262,7 @@ def wrapper(*args, **kwargs):

if workspace_associative_data.permission != Permission.owner.value:
raise HTTPException(status_code=ForbiddenError().status_code, detail=ForbiddenError().message)

if workspace_associative_data and workspace_associative_data.status != UserWorkspaceStatus.accepted.value:
raise HTTPException(status_code=ForbiddenError().status_code, detail=ForbiddenError().message)
return func(*args, **kwargs)
2 changes: 1 addition & 1 deletion rest/services/secret_service.py
Original file line number Diff line number Diff line change
@@ -66,7 +66,7 @@ def update_repository_secret(
self.logger.info(f"Updating secret {secret_id} for repository {piece_repository_id}")
secret = self.secret_repository.find_by_id(id=secret_id)
if not secret:
raise ResourceNotFoundException()
raise ResourceNotFoundException("Secret not found.")

if not body.value:
secret.value = None
2 changes: 1 addition & 1 deletion rest/tests/conftest.py
Original file line number Diff line number Diff line change
@@ -55,7 +55,7 @@ def piece_repository():
id=1,
source="github",
path="Tauffer-Consulting/default_domino_pieces_tests",
version="0.0.3",
version="0.0.4",
workspace_id=1,
url='https://github.com/Tauffer-Consulting/default_domino_pieces_tests'
)
2 changes: 1 addition & 1 deletion rest/tests/piece/fixtures.py
Original file line number Diff line number Diff line change
@@ -47,7 +47,7 @@ def get_pieces_mock_response(piece_repository: PieceRepository):
},
repository_url='https://github.com/Tauffer-Consulting/default_domino_pieces_tests',
source_url='https://github.com/Tauffer-Consulting/default_domino_pieces_tests/tree/main/pieces/SimpleLogPiece',
source_image = "ghcr.io/tauffer-consulting/default_domino_pieces_tests:0.0.3-group0",
source_image = "ghcr.io/tauffer-consulting/default_domino_pieces_tests:0.0.4-group0",
input_schema = {
"title": "SimpleLogPiece",
"description": "Example Operator",
2 changes: 1 addition & 1 deletion rest/tests/workflow/create_workflow_request_model.py
Original file line number Diff line number Diff line change
@@ -393,7 +393,7 @@
"task_id": "SimpleLogP_0298c1669d404e08b631ebe1490e1c45",
"piece": {
"name": "SimpleLogPiece",
"source_image": "ghcr.io/tauffer-consulting/default_domino_pieces_tests:0.0.3-group0"
"source_image": "ghcr.io/tauffer-consulting/default_domino_pieces_tests:0.0.4-group0"
},
"workflow_shared_storage": {
"source": "None",
6 changes: 3 additions & 3 deletions rest/tests/workflow/test_workflow_router.py
Original file line number Diff line number Diff line change
@@ -21,7 +21,7 @@
"tests.workflow.fixtures"
]
@pytest.mark.usefixtures("register", "login", "add_piece_repository", "teardown_piece_repository")
class TestWorkflowRouter:
class TestWorkflowRouter:
@staticmethod
def test_create_workflow(patch_piece_secret: Response, create_workflow: Response, piece_repository: PieceRepository, user: User):
mock_response = CreateWorkflowResponse(
@@ -39,7 +39,7 @@ def test_create_workflow(patch_piece_secret: Response, create_workflow: Response

assert response.status_code == 201
assert content.keys() == mock_response_content.keys()

assert content.get("name") == mock_response_content.get("name")

assert content.get("created_by") == mock_response_content.get("created_by")
@@ -82,4 +82,4 @@ def test_delete_workflow(patch_piece_secret: Response, create_workflow: Response
assert response.status_code == 204
#TODO assert not found with get workflow


8 changes: 4 additions & 4 deletions rest/tests/workspace/fixtures.py
Original file line number Diff line number Diff line change
@@ -90,9 +90,9 @@ def invite_user(client: ApiTestClient, authorization_token: Dict, workspace: Wor

@pytest.fixture(scope='function')
def accept_invite(
client: ApiTestClient,
client: ApiTestClient,
login_user_extra: Response,
authorization_token_user_extra: Dict,
authorization_token_user_extra: Dict,
workspace: Workspace,
):
return client.post(
@@ -103,9 +103,9 @@ def accept_invite(

@pytest.fixture(scope='function')
def reject_invite(
client: ApiTestClient,
client: ApiTestClient,
login_user_extra: Response,
authorization_token_user_extra: Dict,
authorization_token_user_extra: Dict,
workspace: Workspace
):
return client.post(
2 changes: 1 addition & 1 deletion src/domino/cli/utils/platform.py
Original file line number Diff line number Diff line change
@@ -225,7 +225,7 @@ def create_platform(install_airflow: bool = True, use_gpu: bool = False) -> None
console.print("")
console.print("Installing NGINX controller...")
subprocess.run(["kubectl", "apply", "-f", "https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml"], stdout=subprocess.DEVNULL)
result = subprocess.run(["kubectl", "wait", "--namespace", "ingress-nginx", "--for", "condition=ready", "pod", "--selector=app.kubernetes.io/component=controller", "--timeout=180s"])
result = subprocess.run(["kubectl", "wait", "--namespace", "ingress-nginx", "--for", "condition=ready", "pod", "--selector=app.kubernetes.io/component=controller", "--timeout=660s"])
if result.returncode != 0:
error_message = result.stderr.strip() if result.stderr else result.stdout.strip()
raise Exception("An error occurred while installing NGINX controller: {error_message}")