Create the initial version of the OpenCTI charm #2

weiiwang01 · 2024-12-09T04:24:46Z

Overview

Create the initial version of the OpenCTI charm, designed to run the OpenCTI platform and its workers.

Screenshot of the OpenCTI dashboard:

Rationale

Juju Events Changes

Module Changes

Library Changes

Checklist

The charm style guide was applied
The contributing guide was applied
The changes are compliant with ISD054 - Managing Charm Complexity
The documentation is generated using src-docs
The documentation for charmhub is updated
The PR is tagged with appropriate label (urgent, trivial, complex)

github-actions

license-eye has checked 53 files.

Valid	Invalid	Ignored	Fixed
13	9	31	0

Click to see the invalid file list

lib/charms/data_platform_libs/v0/data_interfaces.py
lib/charms/data_platform_libs/v0/s3.py
lib/charms/grafana_k8s/v0/grafana_dashboard.py
lib/charms/loki_k8s/v1/loki_push_api.py
lib/charms/observability_libs/v0/juju_topology.py
lib/charms/prometheus_k8s/v0/prometheus_scrape.py
lib/charms/rabbitmq_k8s/v0/rabbitmq.py
lib/charms/redis_k8s/v0/redis.py
opencti_rock/rockcraft.yaml

Use this command to fix any missing license headers

```bash

docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header fix

</details>

lib/charms/data_platform_libs/v0/data_interfaces.py

lib/charms/data_platform_libs/v0/s3.py

lib/charms/grafana_k8s/v0/grafana_dashboard.py

lib/charms/loki_k8s/v1/loki_push_api.py

lib/charms/observability_libs/v0/juju_topology.py

lib/charms/prometheus_k8s/v0/prometheus_scrape.py

lib/charms/rabbitmq_k8s/v0/rabbitmq.py

lib/charms/redis_k8s/v0/redis.py

opencti_rock/rockcraft.yaml

charmcraft.yaml

opencti_rock/rockcraft.yaml

src/charm.py

jdkandersson · 2024-12-12T03:41:25Z

src/charm.py


-        Learn more about config at https://juju.is/docs/sdk/config
+    def _install_callback_script(self, health_check_url: str) -> pathlib.Path:


I think it might be more efficient if we have a review call for this, could you organise something with me and @gregory-schiano please

Yes, no problem, created.

opencti_rock/rockcraft.yaml

src/charm.py

yhaliaw

Reviewed up to src/charm.py.

yhaliaw · 2024-12-19T02:33:14Z

.github/workflows/test.yaml

@@ -8,5 +8,14 @@ jobs:
    uses: canonical/operator-workflows/.github/workflows/test.yaml@main
    secrets: inherit
    with:
-      self-hosted-runner: true
-      self-hosted-runner-label: "edge"
+      self-hosted-runner: false


For our team, I think we should generally use self-hosted runners on "edge".

On the other hand, I think "edge" does have more wait time than GitHub runner under Canonical org.

Yes, the wait time for self-hosted runners is too long. I am currently rapidly developing the charm and will switch to the self-hosted runner once the charm is more stable. Thanks!

yhaliaw · 2024-12-19T02:33:25Z

.github/workflows/test.yaml

+      juju-channel: 3.6/stable
+      microk8s-addons: "dns ingress rbac storage"
+      pre-run-script: tests/integration/prepare.sh
+      self-hosted-runner: false


See the self-hosted runner comment above.

yhaliaw · 2024-12-19T02:34:27Z

.github/workflows/test.yaml

-      self-hosted-runner: true
-      self-hosted-runner-label: "edge"
+      self-hosted-runner: false
+  integration-tests:


Why is the test.yaml and integration_test.yaml combined?

It saves one workflow file and makes it easier to check CI failures since unit tests and integration tests are in the same job tab.

yhaliaw · 2024-12-19T02:36:18Z

.trivyignore

A lot of linting of "not present anymore, can be safely removed.".
I think we should check to see if these are needed.

Yes, I think most of them are false alarms. Maybe we should upgrade the workflow. Thanks!

yhaliaw · 2024-12-19T02:38:34Z

charmcraft.yaml

+title: OpenCTI Charm
+summary: OpenCTI charm.
+links:
+  documentation: https://discourse.charmhub.io


I think this should be updated to a valid discourse topic.

On the content-cache-operator, not updating this line to a valid topic in the initial commit has caused the issues with upload-docs workflow in future PRs.

Yes, you are right, changed to point to the README for now until we have the documentation page

yhaliaw · 2024-12-19T02:46:48Z

src/charm.py

-
-https://discourse.charmhub.io/t/4208
-"""
+"""OpenCTI charm the service."""


I think the wording here is a bit weird.

Changed to OpenCTI charm thanks!

src/charm.py

.trivyignore

opencti_rock/rockcraft.yaml

src/charm.py

jdkandersson · 2025-01-07T12:58:01Z

src/charm.py

+        redis_url = self._redis.url
+        # bug in the Redis library produces an ill-formed redis_url
+        # when the integration is not ready
+        if not redis_url or redis_url == "redis://None:None":


When does redis://None:None happen? Perhaps this is a bug in the charm? It is weird that it would be passing a URL that doesn't work

This is an issue in the Redis charm library. Perhaps we could contribute to the library and fix it? It should probably just return None in that situation.

Would you like to open an issue on the redis charm?

src/charm.py

charmcraft.yaml

github-actions

license-eye has checked 55 files.

Valid	Invalid	Ignored	Fixed
0	14	41	0

Click to see the invalid file list

charmcraft.yaml
generate-src-docs.sh
opencti_rock/rockcraft.yaml
src/charm.py
tests/conftest.py
tests/integration/init.py
tests/integration/conftest.py
tests/integration/prepare.sh
tests/integration/test_charm.py
tests/unit/init.py
tests/unit/conftest.py
tests/unit/state.py
tests/unit/test_charm.py
tox.ini

Use this command to fix any missing license headers

```bash

docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header fix

</details>

github-actions

license-eye has checked 55 files.

Valid	Invalid	Ignored	Fixed
0	14	41	0

Click to see the invalid file list

charmcraft.yaml
generate-src-docs.sh
opencti_rock/rockcraft.yaml
src/charm.py
tests/conftest.py
tests/integration/init.py
tests/integration/conftest.py
tests/integration/prepare.sh
tests/integration/test_charm.py
tests/unit/init.py
tests/unit/conftest.py
tests/unit/state.py
tests/unit/test_charm.py
tox.ini

Use this command to fix any missing license headers

```bash

docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header fix

</details>

This reverts commit 3de4d69.

github-actions

license-eye has checked 55 files.

Valid	Invalid	Ignored	Fixed
0	14	41	0

Click to see the invalid file list

charmcraft.yaml
generate-src-docs.sh
opencti_rock/rockcraft.yaml
src/charm.py
tests/conftest.py
tests/integration/init.py
tests/integration/conftest.py
tests/integration/prepare.sh
tests/integration/test_charm.py
tests/unit/init.py
tests/unit/conftest.py
tests/unit/state.py
tests/unit/test_charm.py
tox.ini

Use this command to fix any missing license headers

```bash

docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header fix

</details>

github-actions · 2025-01-07T19:14:35Z

Test coverage for b4ab9cd

Name           Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------
src/charm.py     260     24     56      4    89%   127, 202-214, 232-233, 449-450, 537-543, 661, 665, 670
----------------------------------------------------------
TOTAL            260     24     56      4    89%

Static code analysis report

Run started:2025-01-07 18:40:48.964701

Test results:
  No issues identified.

Code scanned:
  Total lines of code: 567
  Total lines skipped (#nosec): 3
  Total potential issues skipped due to specifically being disabled (e.g., #nosec BXXX): 0

Run metrics:
  Total issues (by severity):
  	Undefined: 0
  	Low: 0
  	Medium: 0
  	High: 0
  Total issues (by confidence):
  	Undefined: 0
  	Low: 0
  	Medium: 0
  	High: 0
Files skipped (0):

weiiwang01 added 4 commits December 2, 2024 14:12

add opencti rock

07170cf

Add basic charm

11dbfc8

Fix linting issues

4f210cf

Update config description

1dbcebf

github-actions bot added the Libraries: OK label Dec 9, 2024

github-actions bot reviewed Dec 9, 2024

View reviewed changes

weiiwang01 added 19 commits December 9, 2024 13:08

Fix linting issues

a948dcb

Fix linting issues

e990cd3

Fix linting issues

3632067

Fix linting issues

22077d2

Fix tests

676b9a0

Add license headers

dc9ce01

Update runner

d1c0c05

Improved start-up callback

38db050

Use github hosted runners

8a1264b

Update rockcraft.yaml

0caebdc

Debug

020665a

Remove unnecessary files

336b84c

Remove unnecessary files

702add8

Fix charm.py

1f2b5c5

Remove debug info

403aa75

Stop callback in callback itself

4dbc03a

clean up requirements.txt

e494868

Update documents

875e116

Update unit tests

9387460

weiiwang01 requested review from jdkandersson and yhaliaw December 11, 2024 16:35

jdkandersson reviewed Dec 12, 2024

View reviewed changes

Apply suggestions from review comments

61e7daa

weiiwang01 requested a review from a team as a code owner December 12, 2024 08:32

jdkandersson reviewed Dec 17, 2024

View reviewed changes

weiiwang01 added 3 commits December 18, 2024 13:57

Apply suggestions from review comments

ab0eb50

Merge platform/graphql and platform/graphql-deps in rockcraft.yaml

88ae9c1

Add missing stage-snaps and stage-packages

a781a43

yhaliaw reviewed Dec 19, 2024

View reviewed changes

weiiwang01 added 4 commits December 20, 2024 10:41

Apply suggestions from review comments

758231a

Cleanup secrets

06bce03

Fix linting issues

f99ae13

Update .trivyignore

f5e2459

jdkandersson reviewed Jan 7, 2025

View reviewed changes

weiiwang01 added 2 commits January 8, 2025 02:07

Update linting

5a6e508

Update .trivyignore

3de4d69

github-actions bot reviewed Jan 7, 2025

View reviewed changes

Revert "Update .trivyignore"

5eb4d6c

This reverts commit 3de4d69.

github-actions bot reviewed Jan 7, 2025

View reviewed changes

Update copyright years

b4ab9cd

jdkandersson approved these changes Jan 10, 2025

View reviewed changes

weiiwang01 requested a review from yhaliaw January 13, 2025 04:55

yhaliaw approved these changes Jan 13, 2025

View reviewed changes

weiiwang01 merged commit 6720473 into main Jan 13, 2025
18 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create the initial version of the OpenCTI charm #2

Create the initial version of the OpenCTI charm #2

weiiwang01 commented Dec 9, 2024

github-actions bot left a comment

jdkandersson Dec 12, 2024

weiiwang01 Dec 12, 2024

yhaliaw left a comment

yhaliaw Dec 19, 2024

weiiwang01 Dec 20, 2024

yhaliaw Dec 19, 2024

yhaliaw Dec 19, 2024

weiiwang01 Dec 20, 2024

yhaliaw Dec 19, 2024

weiiwang01 Dec 20, 2024

yhaliaw Dec 19, 2024

weiiwang01 Dec 20, 2024

yhaliaw Dec 19, 2024

weiiwang01 Dec 20, 2024

jdkandersson Jan 7, 2025

weiiwang01 Jan 7, 2025

jdkandersson Jan 10, 2025

github-actions bot left a comment

github-actions bot left a comment

github-actions bot left a comment

github-actions bot commented Jan 7, 2025


		Learn more about config at https://juju.is/docs/sdk/config
		def _install_callback_script(self, health_check_url: str) -> pathlib.Path:

Create the initial version of the OpenCTI charm #2

Create the initial version of the OpenCTI charm #2

Conversation

weiiwang01 commented Dec 9, 2024

Overview

Rationale

Juju Events Changes

Module Changes

Library Changes

Checklist

github-actions bot left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yhaliaw left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot left a comment

Choose a reason for hiding this comment

github-actions bot commented Jan 7, 2025