镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

[POC][multi-vector] Add MinMax MaxSim benchmark - #1509

Draft
juchen-ms (partychen) wants to merge 3 commits into
microsoft:mainfrom
partychen:minmax-benchmark
Draft

juchen-ms (partychen) wants to merge 3 commits into
microsoft:mainfrom
partychen:minmax-benchmark

Conversation

@partychen

@partychen juchen-ms (partychen) commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor
  • Does this PR have a descriptive title that could go in our release notes?
  • Does this PR add any new dependencies?
  • Does this PR modify any existing APIs?
  • Is the change to the API backwards compatible?
  • Should this result in any changes to our documentation, either updating existing docs or adding new ones?

Reference Issues/PRs

What does this implement/fix? Briefly explain your changes.

Any other comments?

MinMax MaxSim benchmark results

Timings are microseconds per MaxSim call (lower is better). The comparison workload uses 32 query vectors, 128 document vectors, and 128 dimensions, with independent query/document quantization widths of 1, 2, 4, and 8 bits.

  • Pairwise / V3 / V4: measured natively on an Intel Xeon Platinum 8370C. Each configuration uses 100 measurements of 200 calls; the reported time is the minimum batch latency divided by 200. Data generation, quantization, query packing, and score-buffer allocation are outside the timed region. Repeated calls reuse the same data; document decoding and scratch allocation inside the matrix kernel remain included.
  • Pairwise is the existing MinMaxIP query/document double loop, compiled for x86-64-v3—not the scalar/reference factory backend. A dash means the existing pairwise API does not support that ordered bit-width combination; no substitute implementation was measured.
  • NEON: timings supplied by the author. The ARM CPU model and measurement setup have not been recorded here. Pairwise / NEON is a cross-platform timing ratio against the x86 pairwise baseline, not a same-machine SIMD speedup. V3/V4 ratios use the same-machine pairwise baseline.
Query bit Document bit Pairwise V3 V4 NEON Pairwise / V3 Pairwise / V4 Pairwise / NEON
1 1 15.560 8.065 5.445 5.775 1.93x 2.86x 2.69x
1 2 — 7.650 5.355 4.976 — — —
1 4 — 8.330 5.425 5.034 — — —
1 8 — 9.715 9.380 4.878 — — —
2 1 — 8.040 5.430 5.775 — — —
2 2 46.255 7.660 5.340 4.989 6.04x 8.66x 9.27x
2 4 — 8.350 5.470 5.036 — — —
2 8 — 9.625 9.375 4.878 — — —
4 1 — 8.095 5.430 5.784 — — —
4 2 — 7.640 5.340 4.972 — — —
4 4 47.215 8.260 5.135 5.042 5.72x 9.19x 9.36x
4 8 — 9.785 9.505 4.876 — — —
8 1 31.860 8.115 5.260 5.776 3.93x 6.06x 5.52x
8 2 53.925 7.660 5.270 4.977 7.04x 10.23x 10.83x
8 4 39.695 8.340 5.430 5.043 4.76x 7.31x 7.87x
8 8 48.460 9.680 8.905 4.881 5.01x 5.44x 9.93x

These results characterize repeated scoring of warm data at the stated fixed workload; they do not measure end-to-end search, cold-cache behavior, quantization accuracy, or query preparation cost.

Integrate the merged MinMax matrix kernel with an owned-query factory, runtime ISA selection, an independent scalar reference, and shared construction errors. Preserve QueryMatRef inputs and add focused factory and query-cloning coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 94d30f43-fadc-4a6b-889b-406de424a1ab
@partychen juchen-ms (partychen) changed the title Minmax benchmark [multi-vector] Add MinMax MaxSim benchmark Oct 23, 2026
@partychen juchen-ms (partychen) changed the title [multi-vector] Add MinMax MaxSim benchmark [POC][multi-vector] Add MinMax MaxSim benchmark Oct 23, 2026
@codecov-commenter

Codecov Comments Bot (codecov-commenter) commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.94521% with 9 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.08%. Comparing base (8a9ff2d) to head (f484895).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
diskann-quantization/src/minmax/multi/factory.rs 98.00% 6 Missing ⚠️
diskann-quantization/src/minmax/multi/reference.rs 92.30% 3 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1509      +/-   ##
==========================================
- Coverage   92.16%   91.08%   -1.09%     
==========================================
  Files         592      596       +4     
  Lines      123582   124095     +513     
==========================================
- Hits       113901   113031     -870     
- Misses       9681    11064    +1383     
Flag Coverage Δ
miri 91.08% <97.94%> (-1.09%) ⬇️
unittests 90.79% <97.92%> (-1.33%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
..._kernels/maxsim/packed_minmax_x_unpacked_minmax.rs 94.63% <100.00%> (-3.56%) ⬇️
diskann-quantization/src/minmax/multi/kernel.rs 100.00% <100.00%> (ø)
diskann-quantization/src/minmax/multi/meta.rs 100.00% <100.00%> (ø)
diskann-quantization/src/minmax/multi/prepared.rs 100.00% <100.00%> (ø)
...-quantization/src/multi_vector/distance/factory.rs 77.51% <ø> (ø)
diskann-quantization/src/minmax/multi/reference.rs 92.30% <92.30%> (ø)
diskann-quantization/src/minmax/multi/factory.rs 98.00% <98.00%> (ø)

... and 64 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Revert cb89267 while the MinMax factory API is redesigned.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants