Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[flyte deck] Streaming Decks #2779

Open
wants to merge 68 commits into
base: master
Choose a base branch
from
Open

Conversation

Future-Outlier
Copy link
Member

@Future-Outlier Future-Outlier commented Oct 1, 2024

Tracking issue

flyteorg/flyte#5574

Why are the changes needed?

we want to support streaming decks.

What changes were proposed in this pull request?

add a function publish for the Flyte Deck.

How was this patch tested?

from flytekit import ImageSpec, task, workflow
from flytekit.deck import Deck

flytekit_hash = "6b55930d0a77efc3594ebaac056f2c75024e61b5"
flytekit = f"git+https://github.com/flyteorg/flytekit.git@{flytekit_hash}"

# Define custom image for the task
custom_image = ImageSpec(packages=[flytekit],
                            apt_packages=["git"],
                            registry="localhost:30000",
                            env={"FLYTE_SDK_LOGGING_LEVEL": 10},
                         )

@task(enable_deck=False, container_image=custom_image)
def t_no_deck():
    # Deck.publish()
    print("No Deck")

@task(enable_deck=True, container_image=custom_image)
def t_deck():
    import time
    """
    1st deck only show timeline deck
    2nd will show
    """
    for i in range(3):
        Deck.publish()
        time.sleep(1)

@task(enable_deck=True, container_image=custom_image)
def t_fail_deck():
    import time

    for i in range(3):
        Deck.publish()
        time.sleep(3)
    time.sleep(10)
    raise ValueError("Failed Deck")

@workflow
def wf():
    t_no_deck()
    t_deck()
    t_fail_deck()

if __name__ == "__main__":
    from flytekit.clis.sdk_in_container import pyflyte
    from click.testing import CliRunner
    import os

    runner = CliRunner()
    path = os.path.realpath(__file__)

    result = runner.invoke(pyflyte.main,
                           ["run", path, "t_no_deck"])
    print("Local Execution: ", result.output)

    result = runner.invoke(pyflyte.main,
                           ["run", "--remote", path,"wf"])
    print("Remote Execution: ", result.output)

Setup process

single binary.

flyte: flyteorg/flyte#6053
flytekit: #2779
flyteconsole: flyteorg/flyteconsole#890

Screenshots

flytekit: this branch

NEW FLYTEKIT, NO DECK, RUNNING With Deck, SUCCEED, and FAILED

OSS-STREAMING-DECK-small.mov

OLD FLYTEKIT, NO DECK, RUNNING With Deck, SUCCEED, and FAILED

OSS-STREAMING-DECK-OLD-FLYTEKIT-small.mov

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Summary by Bito

Implementation of streaming deck functionality in Flytekit, featuring Deck.publish() method and infrastructure updates. Enhanced deck generation in base_task.py with proper protobuf boolean wrappers and parameter documentation. Refactored deck enabling mechanism by removing legacy flags and implementing builder pattern. Updated from positive to negative logic check and aligned test assertions with new implementation.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 2

Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Copy link

codecov bot commented Oct 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 82.79%. Comparing base (f99d50e) to head (dc6d203).
Report is 30 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master    #2779       +/-   ##
===========================================
+ Coverage   51.08%   82.79%   +31.71%     
===========================================
  Files         201        3      -198     
  Lines       21231      186    -21045     
  Branches     2731        0     -2731     
===========================================
- Hits        10846      154    -10692     
+ Misses       9787       32     -9755     
+ Partials      598        0      -598     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
@Future-Outlier Future-Outlier changed the title [Flyte Deck] Streaming Decks [wip][Flyte Deck] Streaming Decks Oct 2, 2024
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
@kumare3
Copy link
Contributor

kumare3 commented Oct 9, 2024

I would love to see an example of tqdm

@kumare3
Copy link
Contributor

kumare3 commented Oct 9, 2024

replaces #1704

@Future-Outlier
Copy link
Member Author

I would love to see an example of tqdm

will try it, just let me ship the MSGPACK IDL first.

<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will make it automatically refresh the page.. probably need that on the empty deck but not sure how to do the realtime behavior once it has content (you don't want to refresh while the user is browsing around or clicking through various tabs)

Suggested change
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="refresh" content="5" >

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COOL, will try it, thank you

Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
@Future-Outlier Future-Outlier changed the title [wip][Flyte Deck] Streaming Decks [flytepropeller][flyteadmin] Streaming Decks Nov 27, 2024
@Future-Outlier Future-Outlier changed the title [flytepropeller][flyteadmin] Streaming Decks [flyte deck] Streaming Decks Nov 27, 2024
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 2, 2025

Code Review Agent Run #24744d

Actionable Suggestions - 2
  • flytekit/deck/deck.py - 1
    • Consider using more specific exception type · Line 96-97
  • flytekit/core/base_task.py - 1
Additional Suggestions - 1
  • flytekit/core/context_manager.py - 1
Review Details
  • Files reviewed - 7 · Commit Range: 01182b4..7bcf15e
    • flytekit/bin/entrypoint.py
    • flytekit/core/base_task.py
    • flytekit/core/constants.py
    • flytekit/core/context_manager.py
    • flytekit/deck/deck.py
    • flytekit/models/task.py
    • flytekit/tools/translator.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

Signed-off-by: Future-Outlier <[email protected]>
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 2, 2025

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Feature Improvement - Streaming Deck Implementation

Dockerfile.dev - Updated flyteidl dependency to streaming-deck-v2 branch

entrypoint.py - Updated _output_deck function parameters for better clarity

base_task.py - Added generates_deck flag and related functionality

deck.py - Implemented Deck.publish() method and improved deck generation logic

task.py - Added generates_deck property and related metadata handling

translator.py - Enhanced task serialization to support deck generation

Testing - Deck Feature Testing

test_translator.py - Added comprehensive tests for deck functionality

Comment on lines 96 to 97
# todo: change to a more proper error
raise ValueError("Deck is disabled for this task, please don't call Deck.publish()")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using more specific exception type

Consider using a more specific exception type like DeckDisabledException instead of ValueError to better communicate the error condition. The TODO comment also suggests this needs improvement.

Code suggestion
Check the AI-generated fix before applying
Suggested change
# todo: change to a more proper error
raise ValueError("Deck is disabled for this task, please don't call Deck.publish()")
class DeckDisabledException(Exception):
pass
raise DeckDisabledException("Deck is disabled for this task, please don't call Deck.publish()")

Code Review Run #24744d


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Comment on lines 726 to 728
if not self.disable_deck:
ctx.user_space_params._enable_deck = True # type: ignore
if DeckField.TIMELINE.value in self.deck_fields and ctx.user_space_params is not None and not self.disable_deck:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider combining deck disable checks

Consider combining the two conditional checks into a single condition to avoid redundant disable_deck checks. The current implementation checks disable_deck twice which introduces redundancy.

Code suggestion
Check the AI-generated fix before applying
Suggested change
if not self.disable_deck:
ctx.user_space_params._enable_deck = True # type: ignore
if DeckField.TIMELINE.value in self.deck_fields and ctx.user_space_params is not None and not self.disable_deck:
if not self.disable_deck and ctx.user_space_params is not None:
ctx.user_space_params._enable_deck = True
if DeckField.TIMELINE.value in self.deck_fields:

Code Review Run #24744d


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah we can do this, small change.

Signed-off-by: Future-Outlier <[email protected]>
@Future-Outlier
Copy link
Member Author

FYI @fg91 since you also want to use it. would love to get your feedback

I'm a big fan of such a feature - being able to publish realtime updates from a task to flyte console was the topic of various discussions in the past (e.g. this or this):

    # A very similar API was discussed back then:
    deck = flytekit.Deck("Summary", ...)
    deck.append(MarkdownRenderer.to_html(f"... {run.link} ..."))
    deck.flush()

I have one question about the implementation. Does flytepropeller need to detect whenever the deck html in the bucket changed for the change to be reflected in the UI or does propeller only indicate to the task where to upload the deck to but a request from flyteconsole directly "goes to the bucket" without flytepropeller having to do further work?

flyteconsole will send a request to flyteadmin, and flyteadmin will give it the deck html

@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 2, 2025

Code Review Agent Run #75fbe2

Actionable Suggestions - 0
Review Details
  • Files reviewed - 3 · Commit Range: 7bcf15e..be02f9f
    • flytekit/core/base_task.py
    • flytekit/deck/deck.py
    • flytekit/tools/translator.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

if isinstance(entity.task_function, ClassDecorator):
extra_config = entity.task_function.get_extra_config()
if not entity.disable_deck:
entity.metadata.generates_deck = True
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we add this to a unit test?

_output_deck(task_name=task_name, new_user_params=params)
else:
# todo: change to a more proper error
raise ValueError("Deck is disabled for this task, please don't call Deck.publish()")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just log a warning instead of raising an error? the warning can say that the call to publish is being ignored. that way the task can continue to function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also let's remove the warning on line 46: "This feature is in beta."

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok I'm going to call warning

flytekit/core/constants.py Outdated Show resolved Hide resolved
Comment on lines 726 to 728
if not self.disable_deck:
ctx.user_space_params._enable_deck = True # type: ignore
if DeckField.TIMELINE.value in self.deck_fields and ctx.user_space_params is not None and not self.disable_deck:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah we can do this, small change.

@@ -720,7 +723,9 @@ def dispatch_execute(
may be none
* ``DynamicJobSpec`` is returned when a dynamic workflow is executed
"""
if DeckField.TIMELINE.value in self.deck_fields and ctx.user_space_params is not None:
if not self.disable_deck:
ctx.user_space_params._enable_deck = True # type: ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since the ExecutionParams object has a builder, maybe we should use it here instead of accessing a private field?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok I'll move it to the builder, thank you

@@ -86,6 +86,16 @@ def name(self) -> str:
def html(self) -> str:
return self._html

@classmethod
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically this can be static right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes static is better I think

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since static method is bound to a class rather than the objects for that class.

reference: https://www.digitalocean.com/community/tutorials/python-static-method

@@ -86,6 +86,16 @@ def name(self) -> str:
def html(self) -> str:
return self._html

@classmethod
def publish(cls):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deck.publish, pulls the ExecutionParams from the current flytekit context, and calls _output_deck, which again calls the current context to get at the file_access code, to do an upload. what do you think about moving this error check inside _output_deck? making this function just a wrapper around that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes we can do this, this can make the code more efficient

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now it changed to warning, and move the logic to _output_deck

Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 9, 2025

Code Review Agent Run #6220fd

Actionable Suggestions - 2
  • flytekit/deck/deck.py - 2
Review Details
  • Files reviewed - 3 · Commit Range: be02f9f..dc6d203
    • flytekit/core/base_task.py
    • flytekit/core/constants.py
    • flytekit/deck/deck.py
  • Files skipped - 1
    • .github/workflows/monodocs_build.yml - Reason: Filter setting
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

Comment on lines +88 to +89
task_name = params.task_id.name
_output_deck(task_name=task_name, new_user_params=params)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding task_id null check

Consider checking if params.task_id is not None before accessing the name attribute to avoid potential NoneType errors.

Code suggestion
Check the AI-generated fix before applying
 @@ -87,3 +87,6 @@ def publish():
          params = FlyteContextManager.current_context().user_space_params
 -        task_name = params.task_id.name
 -        _output_deck(task_name=task_name, new_user_params=params)
 +        if params.task_id is None:
 +            raise ValueError("Task ID is not available in the current context")
 +        task_name = params.task_id.name
 +        _output_deck(task_name=task_name, new_user_params=params)

Code Review Run #6220fd


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Comment on lines 184 to 185
if not params.enable_deck:
logger.warning("Deck is disabled for this task, please don't call Deck.publish()")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding None check for params

Consider checking params for None before accessing enable_deck attribute to avoid potential NoneType attribute errors.

Code suggestion
Check the AI-generated fix before applying
Suggested change
if not params.enable_deck:
logger.warning("Deck is disabled for this task, please don't call Deck.publish()")
if params is None or not params.enable_deck:
logger.warning("Deck is disabled for this task or context params are not available, please don't call Deck.publish()")
return

Code Review Run #6220fd


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Signed-off-by: Future-Outlier <[email protected]>
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 9, 2025

Code Review Agent Run #3bf552

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: dc6d203..41d8760
    • flytekit/core/base_task.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 9, 2025

Code Review Agent Run #b6d959

Actionable Suggestions - 0
Additional Suggestions - 1
  • tests/flytekit/unit/test_translator.py - 1
    • Consider more descriptive variable name · Line 96-101
Review Details
  • Files reviewed - 4 · Commit Range: 41d8760..b5976fe
    • flytekit/core/base_task.py
    • flytekit/core/context_manager.py
    • flytekit/deck/deck.py
    • tests/flytekit/unit/test_translator.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

Signed-off-by: Future-Outlier <[email protected]>
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 13, 2025

Code Review Agent Run #f6709f

Actionable Suggestions - 0
Review Details
  • Files reviewed - 1 · Commit Range: b5976fe..0c1a5a3
    • flytekit/deck/deck.py
  • Files skipped - 0
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
Signed-off-by: Future-Outlier <[email protected]>
@flyte-bot
Copy link
Contributor

flyte-bot commented Jan 13, 2025

Code Review Agent Run #7a0ea5

Actionable Suggestions - 0
Review Details
  • Files reviewed - 3 · Commit Range: 0c1a5a3..2764ed4
    • Dockerfile.dev
    • flytekit/core/base_task.py
    • tests/flytekit/unit/test_translator.py
  • Files skipped - 1
    • .github/workflows/monodocs_build.yml - Reason: Filter setting
  • Tools
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful
    • MyPy (Static Code Analysis) - ✔︎ Successful
    • Astral Ruff (Static Code Analysis) - ✔︎ Successful

AI Code Review powered by Bito Logo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants