[pfoxARM]

What does this implement/fix?

Adds prefetching into diskann-vector and diskann-inmem for aarch64 using prfm pldl1keep, [{ptr}]

Any other comments?

I did test both prfm pldl1keep, [{ptr}] and prfm pldl1strm, [{ptr}] (Streaming or non-temporal L1 prefetch) on C100 VM and saw l1keep was marginally better. Although it'd be nice to compare on C200
But l1keep is also closer to the other architecture paths that already exist.

More info on prfm here https://support.arm.com/documentation/ddi0602/2024-06/Base-Instructions/PRFM--immediate---Prefetch-memory--immediate--?lang=en

[Copilot]

🟢 Approval recommended

No unresolved review issues were identified.

Pull request overview

Adds AArch64 L1 prefetch support to diskann-vector and diskann-inmem using prfm pldl1keep.

Changes:

• Adds AArch64 vector prefetch helpers.
• Adds AArch64 raw-memory prefetching with fallback behavior.

File summaries

File	Description
diskann-vector/src/lib.rs	Adds AArch64 vector prefetch helpers.
diskann-inmem/src/prefetch.rs	Adds AArch64 cache-line prefetching.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Mark Hildebrand (hildebrandmw)]

Thanks - one small maintenance ask on my side.

Did you observe much of a performance difference when prefetching was enabled?

[pfoxARM]

Thanks - one small maintenance ask on my side.

Did you observe much of a performance difference when prefetching was enabled?

Using C100 and C200 16-thread VMs and looking at 8-thread results:

Comparing QPS diff against main = cdaf3b4 (16th September)

• C100, exhaustive u8x2:
  -- l1keep -0.31%, l2keep -0.25%.

• C100, exhaustive u8x4:
  -- l1keep +0.67% , l2keep -0.77% .

• C100, spherical u2:
  -- l1keep +60.21%, l2keep +72.88%.

• C100, spherical u4:
  -- l1keep +23.34%, l2keep +20.69%.

• C200, exhaustive u8x2:
  -- l1keep -7.02%, l2keep +1.92%.

• C200, exhaustive u8x4:
  -- l1keep +6.93%, l2keep +5.13%.

• C200, graph u2:
  -- l1keep +45.62%, l2keep +50.98%.

• C200, graph u4:
  -- l1keep +32.91%, l2keep +9.29%.

I also looked at Keep vs Streamed (non-temporal) prefetch for L1 and L2 on int8 * int8, but saw mixed marginal gains/loss on C100 and C200.

I'll keep looking into this for now, check out cache PMU metrics to see how aggressive it's being.

[pfoxARM]

I think we should go with L2keep for aarch64, we see improved search performance on C100 and C200 for this across many kernels.

Kernel	Variant	Backend stall	IPC	L2 miss
f32xf32 inner prod	main / L1 / L2	72.5% / 72.0% / 71.5%	0.88 / 0.93 / 0.95	45.7% / 38.5% / 32.2%
int8xint8 inner prod	main / L1 / L2	78.0% / 73.3% / 63.3%	1.08 / 1.42 / 2.02	74.7% / 57.4% / 10.6%
u2xu2 inner prod (scalar quant)	main / L1 / L2	58.8% / 43.0% / 39.9%	1.89 / 2.63 / 2.76	49.4% / 30.4% / 12.4%

L2KEEP gives the strongest PMU improvement, especially for int8/u2: much lower L2 miss rates, fewer backend stalls, and higher IPC.

[pfoxARM]

Refactored prefetching now in the latest commit, sorry for the force push.

[Mark Hildebrand (hildebrandmw)]

Thanks Patrick! One more small ask - otherwise looks good to me!

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

❌ Patch coverage is 38.70968% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 91.08%. Comparing base (177c03b) to head (48c987d).
⚠️ Report is 17 commits behind head on main.

Files with missing lines	Patch %	Lines
diskann-vector/src/lib.rs	34.48%	19 Missing ⚠️

❌ Your patch status has failed because the patch coverage (38.70%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1408      +/-   ##
==========================================
+ Coverage   90.95%   91.08%   +0.13%     
==========================================
  Files         586      590       +4     
  Lines      118173   121495    +3322     
==========================================
+ Hits       107483   110667    +3184     
- Misses      10690    10828     +138     

Flag	Coverage Δ
miri	91.08% <38.70%> (+0.13%)	⬆️
unittests	90.79% <38.70%> (+0.14%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
diskann-inmem/src/prefetch.rs	95.78% <100.00%> (ø)
diskann-vector/src/lib.rs	38.70% <34.48%> (-5.74%)	⬇️

... and 46 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

🟡 Changes recommended

diskann-vector/src/lib.rs emits pldl2keep instead of the specified pldl1keep instruction.

1 open issue

• Use the specified L1 prefetch instruction instead of L2 · New

_{Review effort: Lite}