Repository navigation
OpenAI and Anthropic integrations record no output or usage for with_raw_response and with_streaming_response calls #7948
Description
How do you use Sentry?
Sentry Saas (sentry.io)
Version
2.71.0 (master 7cb8094)
Steps to Reproduce
client.messages.with_raw_response.create(...) and client.messages.with_streaming_response.create(...) get a gen_ai.chat span with no response id, finish reason, token usage or response text. Both return an APIResponse and not a Message, so the hasattr(result, "content") check in anthropic.py#L794 fails and the span gets unknown_response. The OpenAI integration loses the same data on both calls.
import httpx2, anthropic, sentry_sdk
from sentry_sdk.integrations.anthropic import AnthropicIntegration
BODY = {"id": "msg_01", "type": "message", "role": "assistant", "model": "claude-sonnet-4-5",
"content": [{"type": "text", "text": "hi there"}], "stop_reason": "end_turn",
"stop_sequence": None, "usage": {"input_tokens": 7, "output_tokens": 3}}
events = []
sentry_sdk.init(dsn="https://k@o0.ingest.sentry.io/0", traces_sample_rate=1.0, send_default_pii=True,
integrations=[AnthropicIntegration(include_prompts=True)],
before_send_transaction=lambda e, h: (events.append(e), None)[1])
client = anthropic.Anthropic(api_key="x", http_client=httpx2.Client(transport=httpx2.MockTransport(lambda r: httpx2.Response(200, json=BODY))))
kw = dict(model="claude-sonnet-4-5", max_tokens=10, messages=[{"role": "user", "content": "hello"}])
for label in ("create", "with_raw_response", "with_streaming_response"):
with sentry_sdk.start_transaction(name=label):
if label == "create":
client.messages.create(**kw)
elif label == "with_raw_response":
client.messages.with_raw_response.create(**kw).parse()
else:
with client.messages.with_streaming_response.create(**kw) as r:
r.parse()
KEYS = ("gen_ai.response.id", "gen_ai.response.finish_reasons", "gen_ai.usage.input_tokens", "gen_ai.usage.output_tokens", "gen_ai.response.text")
for e in events:
for s in e["spans"]:
if s["op"] == "gen_ai.chat":
d = s.get("data", {})
print(f"{e['transaction']:24}", {k: k in d for k in KEYS}, "unknown_response" if d.get("unknown_response") else "")Expected Result
The raw and streaming calls carry the same gen_ai.* data as create.
Actual Result
$ python -W ignore repro.py # master 7cb8094, anthropic 1.13.0
create {'gen_ai.response.id': True, 'gen_ai.response.finish_reasons': True, 'gen_ai.usage.input_tokens': True, 'gen_ai.usage.output_tokens': True, 'gen_ai.response.text': True}
with_raw_response {'gen_ai.response.id': False, 'gen_ai.response.finish_reasons': False, 'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False} unknown_response
with_streaming_response {'gen_ai.response.id': False, 'gen_ai.response.finish_reasons': False, 'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False} unknown_response
# the same loop on client.chat.completions with OpenAIIntegration and three keys, openai 3.29.0
create {'gen_ai.usage.input_tokens': True, 'gen_ai.usage.output_tokens': True, 'gen_ai.response.text': True}
with_raw_response {'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False}
with_streaming_response {'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False}
# not run: the async clients, or a real endpoint
For with_raw_response the body is already read when create returns, so I'd parse it there and end the span the way create does. parse() caches its result, so the caller pays nothing for it. Only with_streaming_response needs the Stream treatment from _sentry_patched_create_sync: wrap parse() to set the output and close() to end the span, since parsing at return would buffer a body the caller asked to stream.
Related: #5939 would still route these to unknown_response, because an APIResponse has no type, and #7891 is the parse() gap next to it.
@alexander-alderman-webb this is the span-lifetime question from #7847 again, for APIResponse. I'd fix both in one change, since both touch openai.py. Is that already in the work you have planned for #7891?
Metadata
Metadata
Assignees
Projects
- StatusShow more project fieldsNo status
Activity
linear-code commentedon Oct 12, 2026
PY-2950
alexander-alderman-webb commentedon Oct 12, 2026
Thanks for reporting!
We'll be tackling these gaps in the coming months.