Feature request: keep all InProc dependencies via sampling overrides

[trask]

For example, to support keeping all Inproc dependencies (INTERNAL spans), but dropping all other dependencies (CLIENT spans).

An option to support this would be to introduce a new telemetryType of dependency/inproc, e.g.

{
  "sampling": {
    "percentage": 100,
    "overrides": [
      {
        "telemetryType": "dependency/inproc",
        "percentage": 100
      },
      {
        "telemetryType": "dependency",
        "percentage": 0
      }
    ]
  }
}

[progxaker]

Background

Application Insights allows to blacklist dependencies - spans with the INTERNAL, PRODUCER, CLIENT and so on types, i.e. exclude the unnecessary ones and keep the necessary ones. There is no problem with this.

{
  "sampling": {
    "percentage": 100,
    "overrides": [
      {
        "telemetryType": "dependency",
        "attributes": [
          {
            "key": "db.system",
            "value": "redis",
            "matchType": "strict"
          }
        ],
        "percentage": 0
      }
    ]
  }
}

If the client wants to maintain a whitelist of dependencies, there may be a problem with span with the INTERNAL kind (InProc type). They can be created using the Span class or the @WithSpan annotation and initially have no attributes that can be used to select and preserve them.

For example, when using the following configuration:

{
  "preview": {
    "captureControllerSpans": true // https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-config#autocollect-inproc-dependencies-preview
  },
  "sampling": {
    "percentage": 0,
    "overrides": [
      {
        "telemetryType": "dependency",
        "attributes": [
          {
            "key": "db.system",
            "value": "redis",
            "matchType": "strict"
          }
        ],
        "percentage": 100
      }
    ]
  }
}

Redis dependencies will be preserved, all others including custom/user (InProc) will be captured (captureControllerSpans) but filtered by sampling. To preserve them, it's necessary to apply an attribute to each custom spans, which can be overhead in a complex product.

P.S. The following configuration will save all spans, not just InProc, which also does not satisfy the whitelist definition.

{
  "preview": {
    "captureControllerSpans": true // https://learn.microsoft.com/en-us/azure/azure-monitor/app/java-standalone-config#autocollect-inproc-dependencies-preview
  },
  "sampling": {
    "percentage": 0,
    "overrides": [
      {
        "telemetryType": "dependency",
        "percentage": 100
      }
    ]
  }
}

[progxaker]

I think sampling on a percentage basis is not a good solution, as you also pointed out in #3654 (comment).

if you sample out the InProc SecretClient.getSecretSync, it will cascade down and sample out the underlying HTTP request as well, which I don't believe is what you want.

I think it's better to provide a switch, like "sampling.keepInProcDependencies" or similar, that allows to toggle keep or exclude InProc dependencies. But in this case, it's necessary to make sure that if the user sets keepInProcDependecies to false and sets the override with the product=test attribute, INTERNAL spans with the specified attribute are preserved.

{
  "sampling": {
    "percentage": 0,
    "keepInProcDependencies": false,
    "overrides": [
      {
        "telemetryType": "dependency",
        "attributes": [
          {
            "key": "product",
            "value": "test",
            "matchType": "strict"
          }
        ],
        "percentage": 100
      }
    ]
  }
}

[trask]

I think it's better to provide a switch, like "sampling.keepInProcDependencies" or similar, that allows to toggle keep or exclude InProc dependencies.

I didn't follow how this is different from providing a new telemetryType of dependencies/inproc that can be configured with either percentage 0 or 100 depending on your need? thanks