DETR XAI

[eugene123tw]

Summary

Enhance XAI capabilities for RTDETR and D-Fine models by implementing support for saliency map generation and feature vector outputs, compatibility with both PyTorch and OpenVINO frameworks.

Person Saliency Map:

Car Saliency Map:

How to test

otx explain --work_dir otx-workspace --dump True --explain_config.postprocess True

Checklist

• I have added unit tests to cover my changes.​
• I have added integration tests to cover my changes.​
• I have ran e2e tests and there is no issues.
• I have added the description of my changes into CHANGELOG in my target branch (e.g., CHANGELOG in develop).​
• I have updated the documentation in my target branch accordingly (e.g., documentation in develop).
• I have linked related issues.

License

• I submit my code changes under the same Apache License that covers the project.
  Feel free to contact the maintainers if that's a concern.
• I have updated the license header for each file (see an example below).

# Copyright (C) 2025 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

[sovrasov]

@eugene123tw thanks for the PR! Could you attach some explain results here for a reference?

[negvet]

As @sovrasov noted, DetClassProbabilityMap has some normalization issues.
I mean it is quite sensitive to the particular model and particular dataset.
Galina tried to come up with some uniformly good solution, but this was not that successful.
The result seems not perfect, but probably acceptable. Assuming that the model is well trained.

For the future, I would propose to revisit DetClassProbabilityMap. Basically, it is just visualization of the classification head output. Probably some modification or other methods might work better to explain object detection models.