[Mark Hildebrand (hildebrandmw)]

It has been pointed out that locating start points at the beginning of the internal ID space has some really nice properties. In particular, it means that a serialized index can be reloaded at a potentially larger size without needing to renumber any internal edges.

Since diskann-inmem decouples the external vs internal ID, making this change is relatively straight-forward outside of updating a few tests that relied pretty closely on this behavior. There are several categories of changes:

• full.rs/quantization/mod.rs: These mechanically move the start point range to the beginning of the ID space and update a few tests as mentioned above.
• ids.rs: Id translation had a small optimization where it didn't allocate translation space for ids beyond the Capacity. With this change, the upper-bound is changed to IdLimit` and while this means we allocate a bit of unused space for translating start point ID slots, these translation slots are never populated so we still cannot accidentally return start points.
• Docs: This PR also remove mention and guarantee of the location of the starting point in the internal ID space. Some situations in the disk index will embed start points within the ID space, so we might as well make no guarantees.

This does technically break Full::ConfigError's API, but this is likely not impactful.

[Copilot]

🟡 Changes recommended

Initialization leaves the freelist on frozen start-point slots in both representations, causing the first insertion to fail when at least 20 start points are present despite writable capacity.

1 open issue

• Freelist initialization exhausts retries on frozen slots · New

What changed in this PR

This PR moves diskann-inmem start points to the beginning of the internal ID space, allowing serialized indexes to grow on reload without relocating start-point edges.

Changes:

• Updates full-precision and quantized representations and their tests for the new layout.
• Sizes ID translation tables using IdLimit rather than writable capacity.
• Removes public guarantees about start-point placement and wraps full-precision configuration errors in an opaque type.

File	Description
diskann-inmem/​src/​store/​mod.rs	Removes documentation assuming start points occupy trailing slots.
diskann-inmem/​src/​repr/​quantized/​mod.rs	Moves quantized start points to leading slots and adjusts tests.
diskann-inmem/​src/​repr/​mod.rs	Explains serialization benefits and relaxes slot-layout guarantees.
diskann-inmem/​src/​repr/​full.rs	Moves full-precision start points, updates tests, and changes configuration errors.
diskann-inmem/​src/​provider.rs	Uses the representation’s ID limit for mapping allocation and updates tests.
diskann-inmem/​src/​ids.rs	Updates mapping allocation, bounds checks, and tests to use IdLimit.

_{Review effort: Balanced}

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

❌ Patch coverage is 83.33333% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.06%. Comparing base (9a01c65) to head (1c6e44c).

Files with missing lines	Patch %	Lines
diskann-inmem/src/repr/full.rs	51.72%	14 Missing ⚠️

❌ Your patch status has failed because the patch coverage (83.33%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1528      +/-   ##
==========================================
+ Coverage   91.02%   92.06%   +1.03%     
==========================================
  Files         602      602              
  Lines      124081   124109      +28     
==========================================
+ Hits       112948   114257    +1309     
+ Misses      11133     9852    -1281     

Flag	Coverage Δ
miri	92.06% <83.33%> (+1.03%)	⬆️
unittests	92.00% <83.33%> (+1.27%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
diskann-inmem/src/ids.rs	100.00% <100.00%> (ø)
diskann-inmem/src/provider.rs	96.21% <100.00%> (+0.01%)	⬆️
diskann-inmem/src/repr/mod.rs	93.75% <ø> (ø)
diskann-inmem/src/repr/quantized/mod.rs	98.48% <100.00%> (+0.02%)	⬆️
diskann-inmem/src/store/mod.rs	91.88% <ø> (ø)
diskann-inmem/src/repr/full.rs	95.29% <51.72%> (-1.84%)	⬇️

... and 50 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Jack Moffitt (metajack)]

I don't quite understand why you need renumbering if the start point is u32::MAX. You can just store them at the beginning when serialized as metadata.

[Mark Hildebrand (hildebrandmw)]

The start point was not u32::MAX. It was located just past the end of the writable ID space.