-
Notifications
You must be signed in to change notification settings - Fork 424
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: integration with dd-trace-api #12057
Open
emmettbutler
wants to merge
36
commits into
main
Choose a base branch
from
emmett.butler/dd-trace-api-integration
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+411
−12
Open
Changes from all commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
eb66781
stub integration with dd-trace-api
emmettbutler 198aaa9
a basic but valid failing test
emmettbutler d60b73c
fleshing out the test setup
emmettbutler 4e1004d
working on the dummy tracer...
emmettbutler dae3efd
store tracer globally
emmettbutler 3e40d48
pull from github
emmettbutler 181b1c1
add dd_trace_api integration to CI
emmettbutler ffc79e5
linting
emmettbutler 89e04a1
enable snapshots
emmettbutler 9bc3423
doesnt work on 3.7
emmettbutler 60ca763
stub verification
emmettbutler 4ffec90
stub verification
emmettbutler b1db4dc
only hook once
emmettbutler d1a7353
more generic code
emmettbutler 1b3cec2
methods on real spans to avoid bookkeeping
emmettbutler d3a6196
current_span APIs
emmettbutler 026c06d
use singleton tracer
emmettbutler e1f1a68
convert stubs to real spans when passed to API functions
emmettbutler 36cdb43
slight refactors
emmettbutler 9bba449
remove some duplication from handlers list
emmettbutler 2448b3e
remove some duplication from handlers list
emmettbutler 35d8a49
reno
emmettbutler 66bdb34
Merge branch 'main' into emmett.butler/dd-trace-api-integration
emmettbutler b787857
simplify
emmettbutler 86d69dd
simplify
emmettbutler bae4cac
unused import
emmettbutler ee4492c
some more tests
emmettbutler 50ce3bd
Merge branch 'main' into emmett.butler/dd-trace-api-integration
emmettbutler abca08f
Merge branch 'main' into emmett.butler/dd-trace-api-integration
emmettbutler 17552de
add patching for tracer.wrap, which requires API accessing the return…
emmettbutler d147067
remove tests for removed functionality
emmettbutler db87343
remove tests
emmettbutler e043f22
code organization
emmettbutler 5540bd1
no shared state
emmettbutler acac914
update requirements
emmettbutler 0e679f8
one hook name
emmettbutler File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# | ||
# This file is autogenerated by pip-compile with Python 3.10 | ||
# by the following command: | ||
# | ||
# pip-compile --no-annotate .riot/requirements/10e65d1.in | ||
# | ||
attrs==24.3.0 | ||
certifi==2024.12.14 | ||
charset-normalizer==3.4.1 | ||
coverage[toml]==7.6.10 | ||
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py | ||
exceptiongroup==1.2.2 | ||
hypothesis==6.45.0 | ||
idna==3.10 | ||
iniconfig==2.0.0 | ||
mock==5.1.0 | ||
opentracing==2.4.0 | ||
packaging==24.2 | ||
pluggy==1.5.0 | ||
pytest==8.3.4 | ||
pytest-cov==6.0.0 | ||
pytest-mock==3.14.0 | ||
pyyaml==6.0.2 | ||
requests==2.32.3 | ||
sortedcontainers==2.4.0 | ||
tomli==2.2.1 | ||
urllib3==2.3.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# | ||
# This file is autogenerated by pip-compile with Python 3.12 | ||
# by the following command: | ||
# | ||
# pip-compile --no-annotate .riot/requirements/1261872.in | ||
# | ||
attrs==24.3.0 | ||
certifi==2024.12.14 | ||
charset-normalizer==3.4.1 | ||
coverage[toml]==7.6.10 | ||
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py | ||
hypothesis==6.45.0 | ||
idna==3.10 | ||
iniconfig==2.0.0 | ||
mock==5.1.0 | ||
opentracing==2.4.0 | ||
packaging==24.2 | ||
pluggy==1.5.0 | ||
pytest==8.3.4 | ||
pytest-cov==6.0.0 | ||
pytest-mock==3.14.0 | ||
requests==2.32.3 | ||
sortedcontainers==2.4.0 | ||
urllib3==2.3.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# | ||
# This file is autogenerated by pip-compile with Python 3.8 | ||
# by the following command: | ||
# | ||
# pip-compile --no-annotate .riot/requirements/14d1688.in | ||
# | ||
attrs==24.3.0 | ||
certifi==2024.12.14 | ||
charset-normalizer==3.4.1 | ||
coverage[toml]==7.6.1 | ||
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py | ||
exceptiongroup==1.2.2 | ||
hypothesis==6.45.0 | ||
idna==3.10 | ||
iniconfig==2.0.0 | ||
mock==5.1.0 | ||
opentracing==2.4.0 | ||
packaging==24.2 | ||
pluggy==1.5.0 | ||
pytest==8.3.4 | ||
pytest-cov==5.0.0 | ||
pytest-mock==3.14.0 | ||
pyyaml==6.0.2 | ||
requests==2.32.3 | ||
sortedcontainers==2.4.0 | ||
tomli==2.2.1 | ||
urllib3==2.2.3 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
# | ||
# This file is autogenerated by pip-compile with Python 3.9 | ||
# by the following command: | ||
# | ||
# pip-compile --no-annotate .riot/requirements/668f2f5.in | ||
# | ||
attrs==24.3.0 | ||
certifi==2024.12.14 | ||
charset-normalizer==3.4.1 | ||
coverage[toml]==7.6.10 | ||
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py | ||
exceptiongroup==1.2.2 | ||
hypothesis==6.45.0 | ||
idna==3.10 | ||
iniconfig==2.0.0 | ||
mock==5.1.0 | ||
opentracing==2.4.0 | ||
packaging==24.2 | ||
pluggy==1.5.0 | ||
pytest==8.3.4 | ||
pytest-cov==6.0.0 | ||
pytest-mock==3.14.0 | ||
pyyaml==6.0.2 | ||
requests==2.32.3 | ||
sortedcontainers==2.4.0 | ||
tomli==2.2.1 | ||
urllib3==2.3.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
# | ||
# This file is autogenerated by pip-compile with Python 3.13 | ||
# by the following command: | ||
# | ||
# pip-compile --no-annotate .riot/requirements/d5f777e.in | ||
# | ||
attrs==24.3.0 | ||
certifi==2024.12.14 | ||
charset-normalizer==3.4.1 | ||
coverage[toml]==7.6.10 | ||
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py | ||
hypothesis==6.45.0 | ||
idna==3.10 | ||
iniconfig==2.0.0 | ||
mock==5.1.0 | ||
opentracing==2.4.0 | ||
packaging==24.2 | ||
pluggy==1.5.0 | ||
pytest==8.3.4 | ||
pytest-cov==6.0.0 | ||
pytest-mock==3.14.0 | ||
pyyaml==6.0.2 | ||
requests==2.32.3 | ||
sortedcontainers==2.4.0 | ||
urllib3==2.3.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
# | ||
# This file is autogenerated by pip-compile with Python 3.11 | ||
# by the following command: | ||
# | ||
# pip-compile --no-annotate .riot/requirements/e49670c.in | ||
# | ||
attrs==24.3.0 | ||
certifi==2024.12.14 | ||
charset-normalizer==3.4.1 | ||
coverage[toml]==7.6.10 | ||
dd-trace-api @ git+https://github.com/DataDog/dd-trace-api-py | ||
hypothesis==6.45.0 | ||
idna==3.10 | ||
iniconfig==2.0.0 | ||
mock==5.1.0 | ||
opentracing==2.4.0 | ||
packaging==24.2 | ||
pluggy==1.5.0 | ||
pytest==8.3.4 | ||
pytest-cov==6.0.0 | ||
pytest-mock==3.14.0 | ||
pyyaml==6.0.2 | ||
requests==2.32.3 | ||
sortedcontainers==2.4.0 | ||
urllib3==2.3.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
from sys import addaudithook | ||
from typing import Any | ||
from typing import Dict | ||
from typing import List | ||
from typing import Optional | ||
from typing import Tuple | ||
import weakref | ||
|
||
import dd_trace_api | ||
|
||
import ddtrace | ||
|
||
|
||
_DD_HOOK_NAME = "dd.hook" | ||
_TRACER_KEY = "Tracer" | ||
_STUB_TO_REAL = weakref.WeakKeyDictionary() | ||
_STUB_TO_REAL[dd_trace_api.tracer] = ddtrace.tracer | ||
|
||
|
||
def _proxy_span_arguments(args: List, kwargs: Dict) -> Tuple[List, Dict]: | ||
"""Convert all dd_trace_api.Span objects in the args/kwargs collections to their held ddtrace.Span objects""" | ||
proxied_args = [] | ||
for arg in args: | ||
if isinstance(arg, dd_trace_api.Span): | ||
proxied_args.append(_STUB_TO_REAL[arg]) | ||
else: | ||
proxied_args.append(arg) | ||
proxied_kwargs = {} | ||
for name, kwarg in kwargs.items(): | ||
if isinstance(kwarg, dd_trace_api.Span): | ||
proxied_kwargs[name] = _STUB_TO_REAL[kwarg] | ||
else: | ||
proxied_kwargs[name] = kwarg | ||
return proxied_args, proxied_kwargs | ||
|
||
|
||
def _call_on_real_instance( | ||
operand_stub: dd_trace_api._Stub, method_name: str, retval_from_api: Optional[Any], *args: List, **kwargs: Dict | ||
) -> None: | ||
""" | ||
Call `method_name` on the real object corresponding to `operand_stub` with `args` and `kwargs` as arguments. | ||
|
||
Store the value that will be returned from the API call we're in the middle of, for the purpose | ||
of mapping from those Stub objects to their real counterparts. | ||
""" | ||
args, kwargs = _proxy_span_arguments(args, kwargs) | ||
retval_from_impl = getattr(_STUB_TO_REAL[operand_stub], method_name)(*args, **kwargs) | ||
if retval_from_api is not None: | ||
_STUB_TO_REAL[retval_from_api] = retval_from_impl | ||
|
||
|
||
def _hook(name, hook_args): | ||
"""Called in response to `sys.audit` events""" | ||
if not dd_trace_api.__datadog_patch or name != _DD_HOOK_NAME: | ||
return | ||
args = hook_args[0][0] | ||
api_return_value, stub_self, event_name = args[0:3] | ||
_call_on_real_instance(stub_self, event_name, api_return_value, *args[3:], **hook_args[0][1]) | ||
|
||
|
||
def get_version() -> str: | ||
return getattr(dd_trace_api, "__version__", "") | ||
|
||
|
||
def patch(tracer=None): | ||
if getattr(dd_trace_api, "__datadog_patch", False): | ||
return | ||
dd_trace_api.__datadog_patch = True | ||
_STUB_TO_REAL[dd_trace_api.tracer] = tracer | ||
if not getattr(dd_trace_api, "__dd_has_audit_hook", False): | ||
addaudithook(_hook) | ||
emmettbutler marked this conversation as resolved.
Show resolved
Hide resolved
|
||
dd_trace_api.__dd_has_audit_hook = True | ||
|
||
|
||
def unpatch(): | ||
if not getattr(dd_trace_api, "__datadog_patch", False): | ||
return | ||
dd_trace_api.__datadog_patch = False | ||
# NB sys.addaudithook's cannot be removed |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 changes: 4 additions & 0 deletions
4
releasenotes/notes/dd-trace-api-integration-0fa7ea051a4d6ce1.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--- | ||
features: | ||
- | | ||
dd-trace-api: adds a simple and minimal instrumentation for the dd-trace-api-py package |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure the integration patch/unpatch approach makes sense here, should we always register an audit hook that listens for these events?
are there ever any cases when we wouldn't want to listen to the events? it would probably be better to have
dd_trace_api
not emit events if you want to "disable" vs having that handled here.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
patch
,unpatch
andget_version
are here just to make this code work with the existing contrib structure. The contrib is enabled by default, and it can be disabled by the same method as any other default-on contrib. As far as I can tell, this code makes sense as a contrib because that provides lazy loading.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, the lazy loading makes sense, but I am not sure we should provide all the same functionality, like
DD_DD_TRACE_API_ENABLED=false
orDD_PATCH_MODULES=dd_trace_api:false
orpatch_all(dd_trace_api=False)
, etc (or w/e all those options are).lazy loading, reporting usage and the version to telemetry both definitely make sense.