Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Index more sqlite cache fields #271

Merged

Conversation

ericpromislow
Copy link
Contributor

Related to #46642

I don't see a straightforward way to test this

@ericpromislow ericpromislow requested a review from a team as a code owner August 30, 2024 22:58
pkg/stores/sqlproxy/proxy_store.go Outdated Show resolved Hide resolved
pkg/stores/sqlproxy/proxy_store.go Outdated Show resolved Hide resolved
@ericpromislow ericpromislow force-pushed the 46642-index-more-sqlite-cache-fields branch from 5ca4081 to 67c463f Compare September 4, 2024 20:21
@ericpromislow
Copy link
Contributor Author

I figured out the other fields, except I'm not sure if the events should go in the empty-string group or in group "events.k8s.io"

pkg/stores/sqlproxy/proxy_store.go Outdated Show resolved Hide resolved
pkg/stores/sqlproxy/proxy_store.go Show resolved Hide resolved
@ericpromislow ericpromislow force-pushed the 46642-index-more-sqlite-cache-fields branch 2 times, most recently from cf9dd7e to b97d575 Compare September 11, 2024 19:09
@ericpromislow ericpromislow force-pushed the 46642-index-more-sqlite-cache-fields branch from 494b7ea to 5e3de28 Compare September 11, 2024 23:30
@ericpromislow ericpromislow requested a review from a team September 13, 2024 00:03
@ericpromislow ericpromislow force-pushed the 46642-index-more-sqlite-cache-fields branch from 5e3de28 to 8eb3632 Compare September 13, 2024 19:17
pkg/resources/virtual/common/common.go Outdated Show resolved Hide resolved
pkg/resources/virtual/virtual.go Outdated Show resolved Hide resolved
pkg/resources/virtual/common/common_test.go Outdated Show resolved Hide resolved
pkg/resources/virtual/common/event_fields.go Outdated Show resolved Hide resolved
pkg/resources/virtual/common/common.go Outdated Show resolved Hide resolved
@ericpromislow ericpromislow force-pushed the 46642-index-more-sqlite-cache-fields branch from 88f64b9 to eb55ceb Compare September 26, 2024 00:29
@ericpromislow ericpromislow dismissed MbolotSuse’s stale review September 26, 2024 17:38

I implemented the requested changes

@ericpromislow ericpromislow force-pushed the 46642-index-more-sqlite-cache-fields branch from eb55ceb to c9e8789 Compare October 7, 2024 21:30
@ericpromislow ericpromislow requested a review from moio October 9, 2024 00:35
Copy link
Contributor

@moio moio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks solid to me, the only real small problem being a missed error handling if.

Other than that it's totally optional nitpicks.

Thanks for all the work! Looking forward to see this in.

pkg/resources/virtual/virtual.go Outdated Show resolved Hide resolved
pkg/resources/virtual/virtual.go Show resolved Hide resolved
pkg/resources/virtual/common/common.go Outdated Show resolved Hide resolved
pkg/resources/virtual/common/common.go Outdated Show resolved Hide resolved
@ericpromislow ericpromislow force-pushed the 46642-index-more-sqlite-cache-fields branch from 99fb430 to f226a31 Compare October 10, 2024 19:10
Misc changes:
- Use the builtin Event class, not events.k8s.io (by looking at the dashboard client code)
- Specify full path to the management.cattle.io fields.
- Map `Event.type` to `Event._type` for indexing.

Use a compound transform-func to first check for a "signal",
and then to run all the relevant transformers until either
one fails or the list is exhausted.

- Includes moving the fakeSummaryCache type into a common area.

Use a simpler way of running transforms.
@ericpromislow ericpromislow force-pushed the 46642-index-more-sqlite-cache-fields branch from c3ec254 to 44c7f7a Compare October 10, 2024 22:58
Copy link
Contributor

@moio moio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@MbolotSuse MbolotSuse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've got a few concerns on implementation, but the functionality looks good. If we can't address the issues here due to timeline, let's get a separate issue up to track that as tech-debt.

pkg/resources/virtual/common/common_test.go Show resolved Hide resolved
pkg/resources/virtual/common/testutil.go Show resolved Hide resolved
pkg/resources/virtual/events/events.go Show resolved Hide resolved
)

func TestTransformEvents(t *testing.T) {
tests := []struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a test case where there's no "type" and so we don't add "_type".

pkg/resources/virtual/events/events_test.go Show resolved Hide resolved
Comment on lines +31 to +35
converters := make([]func(*unstructured.Unstructured) (*unstructured.Unstructured, error), 0)
if gvk.Kind == "Event" && gvk.Group == "" && gvk.Version == "v1" {
converters = append(converters, events.TransformEventObject)
}
converters = append(converters, t.defaultFields.TransformCommon)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: You could initialize this with the common converter:

Suggested change
converters := make([]func(*unstructured.Unstructured) (*unstructured.Unstructured, error), 0)
if gvk.Kind == "Event" && gvk.Group == "" && gvk.Version == "v1" {
converters = append(converters, events.TransformEventObject)
}
converters = append(converters, t.defaultFields.TransformCommon)
converters := []func(*unstructured.Unstructured) (*unstructured.Unstructured, error){t.defaultFields.TransformCommon}
if gvk.Kind == "Event" && gvk.Group == "" && gvk.Version == "v1" {
converters = append(converters, events.TransformEventObject)
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather leave the transformers running from specific to general, until we find they can be run in any order.

Comment on lines +187 to +189
raw, isSignal, err := common.GetUnstructured(test.input)
require.False(t, isSignal)
require.Nil(t, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I'm not sure that this should be part of this set of tests. The GetUnstructured call is part of the implementation detail of the Transform function - not something that is directly part of this package.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants