镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

OpenAI and Anthropic integrations record no output or usage for with_raw_response and with_streaming_response calls #7948

Open

Description

@chrikrah

How do you use Sentry?

Sentry Saas (sentry.io)

Version

2.71.0 (master 7cb8094)

Steps to Reproduce

client.messages.with_raw_response.create(...) and client.messages.with_streaming_response.create(...) get a gen_ai.chat span with no response id, finish reason, token usage or response text. Both return an APIResponse and not a Message, so the hasattr(result, "content") check in anthropic.py#L794 fails and the span gets unknown_response. The OpenAI integration loses the same data on both calls.

import httpx2, anthropic, sentry_sdk
from sentry_sdk.integrations.anthropic import AnthropicIntegration

BODY = {"id": "msg_01", "type": "message", "role": "assistant", "model": "claude-sonnet-4-5",
        "content": [{"type": "text", "text": "hi there"}], "stop_reason": "end_turn",
        "stop_sequence": None, "usage": {"input_tokens": 7, "output_tokens": 3}}
events = []
sentry_sdk.init(dsn="https://k@o0.ingest.sentry.io/0", traces_sample_rate=1.0, send_default_pii=True,
                integrations=[AnthropicIntegration(include_prompts=True)],
                before_send_transaction=lambda e, h: (events.append(e), None)[1])
client = anthropic.Anthropic(api_key="x", http_client=httpx2.Client(transport=httpx2.MockTransport(lambda r: httpx2.Response(200, json=BODY))))
kw = dict(model="claude-sonnet-4-5", max_tokens=10, messages=[{"role": "user", "content": "hello"}])

for label in ("create", "with_raw_response", "with_streaming_response"):
    with sentry_sdk.start_transaction(name=label):
        if label == "create":
            client.messages.create(**kw)
        elif label == "with_raw_response":
            client.messages.with_raw_response.create(**kw).parse()
        else:
            with client.messages.with_streaming_response.create(**kw) as r:
                r.parse()

KEYS = ("gen_ai.response.id", "gen_ai.response.finish_reasons", "gen_ai.usage.input_tokens", "gen_ai.usage.output_tokens", "gen_ai.response.text")
for e in events:
    for s in e["spans"]:
        if s["op"] == "gen_ai.chat":
            d = s.get("data", {})
            print(f"{e['transaction']:24}", {k: k in d for k in KEYS}, "unknown_response" if d.get("unknown_response") else "")

Expected Result

The raw and streaming calls carry the same gen_ai.* data as create.

Actual Result

$ python -W ignore repro.py    # master 7cb8094, anthropic 1.13.0
create                   {'gen_ai.response.id': True, 'gen_ai.response.finish_reasons': True, 'gen_ai.usage.input_tokens': True, 'gen_ai.usage.output_tokens': True, 'gen_ai.response.text': True}
with_raw_response        {'gen_ai.response.id': False, 'gen_ai.response.finish_reasons': False, 'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False} unknown_response
with_streaming_response  {'gen_ai.response.id': False, 'gen_ai.response.finish_reasons': False, 'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False} unknown_response

# the same loop on client.chat.completions with OpenAIIntegration and three keys, openai 3.29.0
create                   {'gen_ai.usage.input_tokens': True, 'gen_ai.usage.output_tokens': True, 'gen_ai.response.text': True}
with_raw_response        {'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False}
with_streaming_response  {'gen_ai.usage.input_tokens': False, 'gen_ai.usage.output_tokens': False, 'gen_ai.response.text': False}
# not run: the async clients, or a real endpoint

For with_raw_response the body is already read when create returns, so I'd parse it there and end the span the way create does. parse() caches its result, so the caller pays nothing for it. Only with_streaming_response needs the Stream treatment from _sentry_patched_create_sync: wrap parse() to set the output and close() to end the span, since parsing at return would buffer a body the caller asked to stream.

Related: #5939 would still route these to unknown_response, because an APIResponse has no type, and #7891 is the parse() gap next to it.

@alexander-alderman-webb this is the span-lifetime question from #7847 again, for APIResponse. I'd fix both in one change, since both touch openai.py. Is that already in the work you have planned for #7891?

Activity

  1. linear-code commented on Oct 12, 2026

    @linear-code
  2. alexander-alderman-webb commented on Oct 12, 2026

    @alexander-alderman-webb
    Collaborator

    Thanks for reporting!
    We'll be tackling these gaps in the coming months.

  3. moved this from Waiting for: Product Owner to No status in GitHub Issues with 👀 3on Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions