Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EDR CLI send-report filters doesn't work as expected #1777

Open
LeoLuo123 opened this issue Jan 9, 2025 · 0 comments
Open

EDR CLI send-report filters doesn't work as expected #1777

LeoLuo123 opened this issue Jan 9, 2025 · 0 comments
Labels
Bug Something isn't working Triage 👀

Comments

@LeoLuo123
Copy link

LeoLuo123 commented Jan 9, 2025

Describe the bug

  • EDR CLI send-report --select invocation_id:xxxx doesn't work
  • EDR CLI send-report --select tags:elementary partially works

To Reproduce
Steps to reproduce the behavior:

  1. Run dbt test -s tag:elementary to run elementary tagged anomaly detection test ONLY
  2. Go to BQ dbt_invocations table to get the invocation_id by run_job_id. SELECT invocation_id FROM maven-clinic-bi.dbt_metadata.dbt_invocations where job_run_id = "358771750"
  3. Run edr send-report --select invocation_id:65d29673-08a7-43c9-9e6b-0e3817592e63 --slack-token <SLACK_TOKEN> --slack-channel-name <CHANNEL_NAME>

Here is log:


    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - https://bit.ly/slack-elementary

2025-01-09 08:29:14 — INFO — Running with edr=0.16.1
2025-01-09 08:29:18 — WARNING — You are using incompatible versions between edr (0.16.1) and Elementary's dbt package (0.15.2).
 To fix please update your packages.yml, and run:
dbt deps && dbt run --select elementary

2025-01-09 08:29:19 — INFO — Elementary's database and schema: '"maven-clinic-bi.dbt_metadata"'
2025-01-09 08:29:19 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:33 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_source_freshness_results", "macro_args": {"days_back": 7, "invocations_per_test": 720}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:37 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models", "macro_args": {"exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:41 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_sources", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:43 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_exposures", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:46 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_singular_tests", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:30:48 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_runs", "macro_args": {"days_back": 7, "exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:31:28 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_dbt_models_test_coverage", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:31:30 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_last_invocation", "macro_args": {"invocation_id": "65d29673-08a7-43c9-9e6b-0e3817592e63"}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:32:00 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_nodes_depends_on_nodes", "macro_args": {"exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:32:03 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocation", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:33:00 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocations_data", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:33:57 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:34:18 — INFO — Sent test results summary to Slack
2025-01-09 08:39:27 — INFO — Sent report to Slack.

Issue 2

  1. I also tried to use --select tag:elementary, it partially work, it can send a summary report to Slack, but threw out exception when generating html file

    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - https://bit.ly/slack-elementary

2025-01-09 08:46:53 — INFO — Running with edr=0.16.1
2025-01-09 08:46:57 — WARNING — You are using incompatible versions between edr (0.16.1) and Elementary's dbt package (0.15.2).
 To fix please update your packages.yml, and run:
dbt deps && dbt run --select elementary

2025-01-09 08:46:57 — INFO — Elementary's database and schema: '"maven-clinic-bi.dbt_metadata"'
2025-01-09 08:46:57 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:18 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_source_freshness_results", "macro_args": {"days_back": 7, "invocations_per_test": 720}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:20 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models", "macro_args": {"exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:24 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_sources", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:26 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_exposures", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:28 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_singular_tests", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:30 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_runs", "macro_args": {"days_back": 7, "exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:45 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_dbt_models_test_coverage", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:47 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_nodes_depends_on_nodes", "macro_args": {"exclude_elementary": true}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:50 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocation", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:52 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_models_latest_invocations_data", "macro_args": {}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:47:59 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_test_results", "macro_args": {"days_back": 7, "invocations_per_test": 720, "disable_passed_test_metrics": false}} --project-dir /Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/dbt_project
2025-01-09 08:48:54 — INFO — Sent test results summary to Slack
Traceback (most recent call last):
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/bin/edr", line 8, in <module>
    sys.exit(cli())
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/cli/cli.py", line 67, in invoke
    return super().invoke(ctx)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/cli.py", line 713, in send_report
    sent_report_successfully = data_monitoring.send_report(
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/data_monitoring/report/data_monitoring_report.py", line 227, in send_report
    self.validate_report_selector()
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/data_monitoring/report/data_monitoring_report.py", line 136, in validate_report_selector
    self.selector_filter.validate_report_selector()
  File "/Users/leoluo/.virtualenvs/dbt_3.9.18/lib/python3.9/site-packages/elementary/monitor/data_monitoring/schema.py", line 102, in validate_report_selector
    raise InvalidSelectorError(
elementary.monitor.data_monitoring.schema.InvalidSelectorError: ('Selector is invalid for report: ', 'tag:elementary')

Expected behavior

  • When option --select invocation_id:xxx is passed, cli will only generate and send report for this specific dbt invocation
  • When option --select tags:elementary, cli will send message to Slack channel and also includes html report as attachment

Screenshots
Issue 1:
I only have one anomaly detection test tagged with elementary, but the summary is for all dbt tests
image

Issue2:
In the screenshot, you can see filter tag:elementary is applied
image

But there is no attachment since exception is thrown.

Environment (please complete the following information):

  • Elementary CLI (edr) version: [0.16.1], can be found by running pip show elementary-data
  • Elementary dbt package version: [0.15.2], can be found in packages.yml file
  • dbt version you're using [1.8.7]
  • Data warehouse [BigQuery]
  • Infrastructure details (dev)

Additional context
Add any other context about the problem here.

Would you be willing to contribute a fix for this issue?
I'd love to give a shot if guideline and code reference is provided.

@LeoLuo123 LeoLuo123 added Bug Something isn't working Triage 👀 labels Jan 9, 2025
@LeoLuo123 LeoLuo123 changed the title EDR CLI EDR CLI Bugs Jan 9, 2025
@LeoLuo123 LeoLuo123 changed the title EDR CLI Bugs EDR CLI send-report filters doesn't work as expected Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Triage 👀
Projects
None yet
Development

No branches or pull requests

1 participant