Add support for kind, status, instrumentation library, links, events and state data for traces

[benceszikora]

This PR adds support for the kind, status, instrumentation library, links, events and state fields in the trace panel in Grafana. This will bring the plugin to feature parity with what Tempo can offer today.

[CLAassistant]

All committers have signed the CLA.

[SpencerTorres]

Thank you for the contribution! Glad to see expanded functionality for the trace columns. I've added a note regarding the intended table schema

[SpencerTorres]

I noticed theres a few columns defined in otel.ts that don't exist on the ClickHouse OTel exporter default schema. The intent of these definitions is to match as closely as possible to this table schema for a better out of the box experience.

Should we look into adding these columns to the otel exporter?

Also note that State is defined too, but on the table it is called TraceState.

[Sup3rGeo]

Also AFAIU In the default schema Links is a Nested table and you need to select the fields individually:

 arrayMap(
  (traceid, spanid, attributes) -> tuple(
                     traceid,
                     spanid,
                     arrayMap(
                       key -> map('key', key, 'value', attributes[key]),
                       mapKeys(attributes)
                     )
                   )::Tuple(traceID String, spanID String, tags Array(Map(String, String)))
                   , Links.TraceId, Links.SpanId, Links.Attributes
  ) AS references

[jwhitaker-gridcog]

Also AFAIU In the default schema Links is a Nested table and you need to select the fields individually

@Sup3rGeo not disputing the table schema, but arrayMap(l -> ..., Links) does work here - is it possible this is a newer clickhouse feature?

@SpencerTorres @benceszikora is it worth splitting the bits of this PR that can work with the current schema into something that's mergeable now? In my case I'm itching for span events to be supported, and this achieves that out of the box which is really nice.

[SpencerTorres]

is it worth splitting the bits of this PR that can work with the current schema into something that's mergeable now?

@jwhitaker-gridcog Yes, there's some parts of this that need changing. I don't mind picking apart the code myself for it, but I want to make sure the author is okay with that (@benceszikora)

[harryjph]

Hey everyone! I've updated the code to be based on the OTel Collector Schema as requested, and rebased the PR onto main to incorporate #1128, which was merged last week. I've also updated the test cases to cover code merged in that PR.

I've left a couple of comments explaining the changes which aren't simple column addition.

[harryjph]

There are two ways in ClickHouse to define a Nested Table, controlled by the flatten_nested setting at column creation time.

The query as previously written does not work when flatten_nested=0, as ClickHouse does not recognise the sub-columns such as Events.Names in this mode.

The rewritten query should work with either mode, as ClickHouse will recognise Events as one column even when it has been flattened.

[harryjph]

This change is necessary since at the moment, you cannot type into the "Events Prefix" field in the column selection form.

This function generates a SelectedColumn struct for the arbitrary input, which is necessary since when flatten_nested=1, the ColumnSelect component recognises the Events column as multiple sub-columns.