Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Update aws to v23.3.1 #264

Merged
merged 55 commits into from
Dec 28, 2023
Merged
Changes from 1 commit
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
221f04d
feat: Update `aws_cloudtrail_trail_event_selectors` schema
candiduslynx Nov 21, 2023
7ad758f
Merge branch 'main' into fix/aws_cloudtrail_trail_event_selectors/v23…
candiduslynx Dec 19, 2023
eac98bf
bump aws in data-resilience
candiduslynx Dec 21, 2023
e1018d9
Merge branch 'main' into fix/aws_cloudtrail_trail_event_selectors/v23…
candiduslynx Dec 21, 2023
482310a
bump aws in asset-free
candiduslynx Dec 21, 2023
b0aa0ce
bump aws-pg in compliance free
candiduslynx Dec 21, 2023
303dd10
bumop aws in compliance-free-bq
candiduslynx Dec 21, 2023
46914ba
bump aws in compliance-free snowflake
candiduslynx Dec 21, 2023
33b2ea2
Update bigquery.yml
candiduslynx Dec 21, 2023
c434921
Update postgres.yml
candiduslynx Dec 21, 2023
f560430
Update snowflake.yml
candiduslynx Dec 21, 2023
fb81760
Update postgres.yml
candiduslynx Dec 21, 2023
88d0ad5
Update postgres.yml
candiduslynx Dec 21, 2023
d5bfa30
Update bucket_access_logging.sql
candiduslynx Dec 21, 2023
e6d3f6e
Update detector_enabled.sql
candiduslynx Dec 21, 2023
7e86bb9
Update detector_enabled.sql
candiduslynx Dec 21, 2023
521a791
Update detector_enabled.sql
candiduslynx Dec 21, 2023
938b427
Update unused_directconntect_connections.sql
candiduslynx Dec 21, 2023
df1b25d
Update snowflake.yml
candiduslynx Dec 21, 2023
14a92a5
Update snowflake.yml
candiduslynx Dec 21, 2023
a8b7dc5
Update snowflake.yml
candiduslynx Dec 21, 2023
441e013
Update transformations/aws/compliance-free/tests/snowflake.yml
candiduslynx Dec 21, 2023
add1149
force migration inf compliance-free/bq
candiduslynx Dec 21, 2023
c7cc14b
upd sf for compliance-premium
candiduslynx Dec 21, 2023
a9df666
use request_ cols
candiduslynx Dec 21, 2023
6cd4b81
use aws_iam_policy_versions
candiduslynx Dec 21, 2023
75a11a7
use aws_s3_bucket_policies
candiduslynx Dec 21, 2023
1c49bc9
Merge branch 'main' into fix/aws_cloudtrail_trail_event_selectors/v23…
candiduslynx Dec 21, 2023
b7afc23
aws_s3_buckets relations
candiduslynx Dec 22, 2023
ab8b274
aws_s3_bucket_replications
candiduslynx Dec 22, 2023
935a032
aws_s3_bucket_policies
candiduslynx Dec 22, 2023
b705ea1
rm extra forced mode
candiduslynx Dec 22, 2023
80855d1
cloudtrail/bucket_access_logging:snowflake
candiduslynx Dec 23, 2023
b387f58
cloudtrail/enabled_in_all_regions:snowflake
candiduslynx Dec 23, 2023
cce6d8b
Updated log_metric and enabled_in_all_regions
ronsh12 Dec 26, 2023
b6fb421
change ref to aws@v23.2.0
candiduslynx Dec 27, 2023
fb9320e
Updated queries - no_star, policies_have_wildcard_actions, policies_w…
ronsh12 Dec 27, 2023
e26d2a5
Merge branch 'main' into fix/aws_cloudtrail_trail_event_selectors/v23…
candiduslynx Dec 27, 2023
005aab9
force migrate for tests
candiduslynx Dec 27, 2023
794808f
use v23.3.0
candiduslynx Dec 27, 2023
355ed43
use v23.3.0
candiduslynx Dec 27, 2023
46441bc
Revert "force migrate for tests"
candiduslynx Dec 27, 2023
a0c64e2
Updated queries cloudtrail_enabled_all_regions, bucket_access_logging
ronsh12 Dec 27, 2023
9c52378
Updated manifest compliances
ronsh12 Dec 27, 2023
191fdd2
Updated queries elastic_beanstalk_stream_logs_to_cloudwatch, s3_bucke…
ronsh12 Dec 28, 2023
4e5cc69
check query
ronsh12 Dec 28, 2023
35c0fd0
Update aws to `v23.3.1`
candiduslynx Dec 28, 2023
bdfb813
Updated query s3_bucket_logging_enabled
ronsh12 Dec 28, 2023
4bdcce2
Update transformations/aws/compliance-premium/tests/snowflake.yml
candiduslynx Dec 28, 2023
6b2f7cc
Updated query elastic_beanstalk_stream_logs_to_cloudwatch
ronsh12 Dec 28, 2023
3c80494
Update transformations/aws/compliance-premium/tests/snowflake.yml
candiduslynx Dec 28, 2023
2f9cf11
Merge branch 'main' into fix/aws_cloudtrail_trail_event_selectors/v23…
candiduslynx Dec 28, 2023
946f63f
tmp force migration
candiduslynx Dec 28, 2023
5be873d
no forced migration
candiduslynx Dec 28, 2023
a90cbd7
Merge branch 'main' into fix/aws_cloudtrail_trail_event_selectors/v23…
kodiakhq[bot] Dec 28, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Update detector_enabled.sql
candiduslynx authored Dec 21, 2023
commit e6d3f6e05cb5c5f001cbe2270deb46846256309e
6 changes: 3 additions & 3 deletions transformations/aws/macros/guardduty/detector_enabled.sql
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@

{% macro snowflake__detector_enabled(framework, check_id) %}
with enabled_detector_regions as (
select account_id, region
select request_account_id as account_id, request_region as region
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we using the request_* values here? For services that can aggregate across region and account this can flag wouldn't this flag detectors in other accounts and regions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Previously we used account_id & region columns that were propagated with the same data.
We could, however, parse account_id & region values from the arn column, but IDK if that's in the scope of this upgrade or not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be mistaken, but I believe we used account_id and region I believe at the time of our plugin release Guard Duty didn't support cross region/ cross account aggregation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the PK in cloudquery/cloudquery#15468.
I might've misinterpreted cloudquery/cloudquery#15468 (comment) as a suggestion to include request_ columns to the PK, as we already had ARN there...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ARN should contain the region/account of the resource that actually owns the resource, while the request_* should contain information about where the request was made... If we didn't add the request_* fields to the PK it would be non-deterministic about which requests actually made it to the DB

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Judging by the docs we could parse ARN instead. Do we want this?
cc: @jsonpr

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, it doesn't seem that the request_ prefix is required here: https://docs.aws.amazon.com/guardduty/latest/ug/guardduty_concepts.html
It seems to me that each acc/region will have its own detector ID

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So maybe there should be a fix moving back to acocunt_id & region + actually including them into ARN: https://github.com/cloudquery/cloudquery/blob/main/plugins/source/aws/resources/services/guardduty/detectors.go#L97-L101

from aws_guardduty_detectors
where status = 'ENABLED'
)
@@ -19,7 +19,7 @@ select
enabled = TRUE and e.region is null
then 'fail' else 'pass' end AS status
from aws_regions r
left join enabled_detector_regions e on e.region = r.region AND e.account_id = r.account_id
left join enabled_detector_regions e on e.region = r.region AND e.request_account_id = r.account_id
union
-- Add any detector that is enabled but all data sources are disabled
select
@@ -77,4 +77,4 @@ where
{% endmacro %}

{% macro default__detector_enabled(framework, check_id) %}{% endmacro %}