镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Add batch mode for JSON Lines inputs - #620

Open
dnagoda wants to merge 5 commits into
dc.compile-oncefrom
dc.batch-jsonl
Open

dnagoda wants to merge 5 commits into
dc.compile-oncefrom
dc.batch-jsonl

Conversation

@dnagoda

@dnagoda dnagoda commented Oct 1, 2026 •

Copy link
Copy Markdown

Stacked on #619. Review that PR first.

Why

To test a Function against many inputs today (a regression suite, recorded inputs, or fuzzing), you start function-runner once per input. Each process loads and compiles the Function and its provider again before it runs one input. Startup costs much more than the run itself.

1,000 inputs, release build, Apple M4 Pro, Wasmtime compilation cache warm:

Fixture One process per input (main, v9.2.2) --batch, one process
exit_code.wasm (Rust, WASI) 5.46 s 0.05 s
js_function_javy_plugin_v3.wasm (Javy plugin) 17.54 s 0.07 s

That is about 110x faster for the Rust fixture and about 250x faster for the Javy fixture.

What

--batch reads JSON Lines from --input or stdin and runs the Function once for each non-blank line. The Function module is compiled once, and #619's provider cache compiles the provider once. Each input gets a new store and instance, so state does not carry over between inputs.

For each input, the runner writes one JSON record on one line to stdout:

{"line":1,"success":true,"instructions":5069,"memory_usage":1088,"logs":"","output":{"exit":0}}
{"line":2,"success":false,"instructions":5069,"memory_usage":1088,"logs":"module exited with code: 1","output":{"exit":1}}
{"line":3,"success":false,"error":"Invalid input JSON: EOF while parsing a value at line 2 column 0"}
  • line is the 1-based input line number. Blank lines are skipped, and you can still match each record to its input.
  • All records go through serde. An error message with quotes or newlines stays one valid JSON line. If a result cannot be serialized, the runner writes an error record for that input, never partial output.
  • If the Function output is not valid JSON, output is null and output_error gives the reason. This matches single-run mode, where that run still counts as successful.
  • If the Function itself cannot run, for example because it imports both WASI and a provider that does not allow WASI, each input gets an error record with that reason.
  • The default record has only the fields needed to check an outcome. Inputs can be large, and echoing each one back would roughly double the output size. --batch-full-output writes the same fields as --json, plus line.

Failure handling:

  • By default, the batch stops after the first failed input. The record for that input is still written.
  • --batch-continue-on-error runs all inputs.
  • The exit code is 0 only if every input succeeds. A failed input means the Function failed (success: false) or the input could not run.
  • A summary goes to stderr, for example Batch complete: 4 inputs processed, 2 successful, 2 failed.

CLI rules:

  • --batch-continue-on-error and --batch-full-output require --batch.
  • --batch cannot be used with the profiling flags.
  • --schema-path and --query-path work with --batch. The new BluejaySchemaAnalyzer::with_analyzer parses and validates the schema and query once, then computes the scale factor for each input. With a 4,451-line Payment Customization Function API schema, 1,000 inputs to noop.wasm take 0.06 s. Parsing again for each input takes 0.23 s.

The README documents the input format, the record format, exit codes, and with_analyzer.

Single-run mode does not change.

Testing

  • cargo test --locked: 35 unit tests and 36 integration tests pass; one existing test remains ignored.
  • 13 new integration tests cover the record format, stdin and blank lines, fail-fast and continue-on-error exit codes, error records with quotes and newlines, invalid output, an invalid Function, full output, a Javy plugin Function, schema and query, an invalid schema, and flag rules.
  • A new unit test covers with_analyzer with many inputs.
  • cargo clippy --locked -- -D warnings and cargo fmt --all -- --check pass.

`--batch` runs one Function against many inputs in one process. It reads
JSON Lines from `--input` or stdin and writes one JSON record per input
line to stdout. The Function module, its provider, and the optional
schema and query are loaded once, so each input pays only for the run.

- Each record has the 1-based input `line`, so records match inputs even
  when blank lines are skipped.
- Records are written with serde, so errors with quotes or newlines stay
  valid JSON Lines.
- By default the batch stops at the first failed input.
  `--batch-continue-on-error` runs all inputs. The exit code is non-zero
  if any input failed.
- `--batch-full-output` writes the full run result for each input.
- A summary goes to stderr.

`BluejaySchemaAnalyzer::with_analyzer` parses the schema and query once
and computes the scale factor for each input.
Batch mode moved the input read below schema loading and Function
compilation for every run, so a run with both a bad input and a bad
Function reported the Function error instead of the input error.

Read the whole input first again when the run is not a batch. Only batch
mode defers the read, because it streams the input line by line.
Full batch records serialized FunctionRunResult, whose BytesContainer
flattens its JSON value into the parent. That serializer rejects arrays,
strings, and booleans, changes null to {}, and writes numbers as
{"$serde_json::private::Number":"123"}. Output that is not valid JSON
became {} instead of null.

Write the full record fields directly, with input and output taken from
their JSON values, the same as minimal records. Single runs keep the
existing serializer.
--batch accepted --json and ignored it, so a caller who asked for the
full result silently got minimal records. Fail with a usage error that
points to --batch-full-output instead.
with_analyzer parses the schema and query; it does not check the query
against the schema. Say so, and note that analyze can also fail when it
cannot select an operation in the query.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant