镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Add opt-in correlation fields to Logback and Log4j2 text patterns - #13277

Open
lrwh wants to merge 1 commit into
DataDog:masterfrom
lrwh:lrwh/log-pattern-injection
Open

lrwh wants to merge 1 commit into
DataDog:masterfrom
lrwh:lrwh/log-pattern-injection

Conversation

@lrwh

@lrwh lrwh commented Nov 6, 2026

Copy link
Copy Markdown

What Does This Do

Add DD_LOGS_PATTERN_INJECT=true (-Ddd.logs.pattern.inject=true) to make service, trace ID, and span ID visible in existing Logback classic and Log4j2 plain-text logs. The option is disabled by default. It adds only missing MDC converters after the first complete field, preserving timestamp options and the rest of the application's pattern.

Motivation

MDC injection alone does not display correlation fields when a text layout does not reference their keys. Applications currently have to update each logging configuration or replace the complete pattern, which can disrupt the timestamp prefix expected by log collectors.

Solves #13276.

Additional Notes

Pattern parsing runs during layout initialization. Logback uses its native parser to recognize existing converters; Log4j2 handles both legacy single-key MDC options and newer key lists. Layout restarts do not duplicate fields. Existing application MDC values are preserved, and DD_LOGS_INJECTION continues to control correlation values.

On Log4j2 versions before 2.7, adding service requires enriching the event's context map; a nonempty application map is copied to preserve its contents. This path is enabled only with the new option. Newer versions reuse the existing context-data injector.

JSON layouts, Logback access layouts, and custom Log4j2 PatternSelector branches are outside this change. Usage and configuration details are documented in docs/log-pattern-injection.md.

Validation on JDK 25:

  • Logback 1.2.13: 27 tests passed, including asynchronous formatting, existing converters, layout restarts, and disabled MDC injection.
  • Log4j2 2.0: all 20 module tests passed.
  • Logback 1.5.25 and Log4j2 2.26.1: the same pattern tests passed with dependency overrides (27 and 14 tests respectively).
  • Configuration defaults, environment variables, and system-property precedence: 4 tests passed.
  • Spotless checks passed for all affected modules.

Contributor Checklist

Requested labels: type: feature, comp: logs, tag: ai generated.

@lrwh
lrwh marked this pull request as ready for review November 6, 2026 09:40
@lrwh
lrwh requested review from a team as code owners November 6, 2026 09:40
@lrwh
lrwh requested review from jordan-wong and sarahchen6 and removed request for a team November 6, 2026 09:40
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-06T09:47:21.344115Z 74d4ab9 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@datadog-prod-us1-3 datadog-prod-us1-3 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: FAIL

Valid Log4j2 patterns can separate correlation fields from their message or duplicate existing MDC values. Legacy asynchronous Log4j2 events also bypass service enrichment, leaving the newly inserted service field empty.

Open Bits AI session

🤖 Bits Code Review · Commit 74d4ab9

if (c == quote) {
quote = 0;
}
} else if (c == '\'' || c == '"') {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Use framework-specific quoting rules for the insertion point

With injection enabled, the valid Log4j2 pattern %replace{%m}{'}{}%n acquires correlation fields after its newline. This scanner incorrectly treats the apostrophe as opening quoted text, although Log4j2 treats it as a literal regex character. The message and correlation fields end up on separate lines, with the fields running into the next event. Restrict this quote handling to Logback.

Suggested change
} else if (c == '\'' || c == '"') {
} else if (logback && (c == '\'' || c == '"')) {

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means
  • 👍 Useful: correct, and you're fixing it or will.
  • 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
  • 😕 Unclear: you can't tell what it means or why it matters.
  • 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

🤖 Bits Code Review · Open Bits AI session

}

private static boolean isComposite(String name) {
return "style".equals(name)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Recognize MDC converters inside built-in Log4j2 wrappers

The composite list omits Log4j2's built-in color wrappers and maxLength alias, so existing MDC converters inside them are missed. For example, %cyan{%X{dd.trace_id}} %m%n gains a duplicate trace ID, while %blue{%X} %m%n gains all three fields despite already printing the entire MDC. Recognize these wrappers to preserve the promise to add only missing fields.

Suggested change
return "style".equals(name)
return "black".equals(name)
|| "blue".equals(name)
|| "cyan".equals(name)
|| "green".equals(name)
|| "magenta".equals(name)
|| "red".equals(name)
|| "white".equals(name)
|| "yellow".equals(name)
|| "maxLength".equals(name)
|| "style".equals(name)

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means
  • 👍 Useful: correct, and you're fixing it or will.
  • 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
  • 😕 Unclear: you can't tell what it means or why it matters.
  • 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

🤖 Bits Code Review · Open Bits AI session


@Override
public String instrumentedType() {
return "org.apache.logging.log4j.core.impl.Log4jLogEvent";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Enrich legacy asynchronous Log4j2 events with service

On Log4j2 versions before 2.7, AsyncLogger delivers RingBufferLogEvent instances directly to appenders, bypassing the Log4jLogEvent constructors instrumented here. The legacy ThreadContext updater supplies only trace/span IDs, and the newer context-data injector is unavailable. Consequently, enabling pattern injection leaves the added service field empty despite a configured service name. Service enrichment must also cover asynchronous event population.

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means
  • 👍 Useful: correct, and you're fixing it or will.
  • 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
  • 😕 Unclear: you can't tell what it means or why it matters.
  • 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

🤖 Bits Code Review · Open Bits AI session

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 74d4ab92a0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".


@Override
public String instrumentedType() {
return "org.apache.logging.log4j.core.impl.Log4jLogEvent";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Enrich legacy async Log4j events with service

When Log4j 2.0–2.6 is configured with AsyncLogger, events are populated as RingBufferLogEvent rather than constructed as Log4jLogEvent, so this advice is never invoked (and no other instrumentation in the Log4j modules targets that event type). The pre-2.7 scope listener only places trace/span IDs in ThreadContext, so the newly inserted %X{dd.service} renders empty even with logs injection enabled; cover the async event path as well.

Useful? React with 👍 / 👎.

Comment on lines +98 to +100
private static boolean isComposite(String name) {
return "style".equals(name)
|| "highlight".equals(name)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Recognize all Log4j composite converter aliases

When a supported Log4j pattern wraps an existing correlation converter in %upperCase{...}, %lowerCase{...}, or the maxLength alias, fields() only descends into names accepted by this incomplete allowlist. It therefore fails to mark the nested key and injects a second converter (for %upperCase{%X{dd.trace_id}}, the ID is printed twice), so include all Log4j composite names and aliases.

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant