Repository navigation
Conversation
distance_cosine_float(), cosine_float_neon() and vec_normalize() summed squared elements in float32. The square of an element below about 3e-23 underflows to zero and the square of one above about 2e19 overflows to Inf, so a nonzero finite vector got a cosine distance of +-Inf, NaN or 1.0, ranked ahead of or behind its true neighbours in a cosine vec0 table, and vec_normalize() returned [Inf, NaN] or a vector that was not of unit length (asg017#324). The scalar loops now sum in double, where every product of two float32 values is representable, as vlasky/sqlite-vec@d75c245 does in that fork; its tests come with it. cosine_float_neon(), which that fork lacks and which fails the same way above 16 dimensions, keeps its 16-element blocks and four accumulators per sum but widens each float32x4 load to two float64x2 halves before it is multiplied and added, and finishes as the scalar loop does. A zero vector gets the same results as before: NULL from vec_distance_cosine() and NaN elements from vec_normalize(). On Linux arm64 (Clang 19, -O3; medians of 20 queries on a 100,000-row float[768] cosine table, each build in its own process), a KNN query takes 77.5 ms instead of 72.0 ms for k = 10 and 84.5 ms instead of 78.1 ms for k = 100 on a scalar build, and 25.3 ms instead of 18.3 ms and 32.0 ms instead of 26.8 ms on a NEON build, whose float64x2 lanes hold half as many elements as float32x4. The queries return the same rows at every rank. Their distances change in the last digits, and the NEON build's now equal the scalar build's on every row, where on main they differed; test_vec_distance_cosine expects the new value of one. The tests check the cases from asg017#324 on the scalar path and, padded to 40 dimensions, on the NEON path: vec_distance_cosine() under positive scaling from 1e-45 to 3e38, vec_normalize() of tiny and huge vectors, and the KNN order of a cosine table holding them, along with lengths of 17 to 768 against a float64 reference. All four tests fail on main with a scalar and a NEON build. The suite passes on GCC and Clang scalar builds and a Clang NEON build (482 passed, 132 skipped); GCC builds the NEON code once asg017#322 is in.
This was referenced Oct 23, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
distance_cosine_float(),cosine_float_neon()andvec_normalize()sum squared elements infloat32. The square of an element below about 3e-23 underflows to zero and the square of one above about 2e19 overflows to Inf, so a nonzero finite vector gets a cosine distance of ±Inf, NaN or 1.0, ranks ahead of or behind its true neighbours in a cosinevec0table, andvec_normalize()returns[Inf, NaN]or a vector that is not of unit length. Fixes #324.The scalar loops now sum in
double, where every product of twofloat32values is representable, as vlasky/sqlite-vec@d75c245 does in that fork (vlasky's comment on #324 describes it), and its tests come with it.cosine_float_neon(), which that fork lacks and which fails the same way above 16 dimensions, as that comment notes, keeps its 16-element blocks and four accumulators per sum but widens eachfloat32x4load to twofloat64x2halves before it is multiplied and added, and finishes as the scalar loop does. A zero vector gets the same results as before: NULL fromvec_distance_cosine()and NaN elements fromvec_normalize().The cost, on Linux arm64 (Clang 19,
-O3; medians of 20 queries on a 100,000-rowfloat[768]cosine table, each build in its own process):The NEON kernel's
float64x2lanes hold half as many elements asfloat32x4. The queries return the same rows at every rank. Their distances change in the last digits, and the NEON build's now equal the scalar build's on every row, where on main they differ;test_vec_distance_cosineexpects the new value ofvec_distance_cosine('[1.1, 1.0]', '[1.2, 1.2]').The tests check the cases from #324 on the scalar path and, padded to 40 dimensions, on the NEON path:
vec_distance_cosine()under positive scaling from 1e-45 to 3e38,vec_normalize()of tiny and huge vectors, and the KNN order of a cosine table holding them, along with lengths 17 to 768 against a float64 reference. All four tests fail on main with a scalar and a NEON build. The suite passes on GCC and Clang scalar builds and on a Clang NEON build (482 passed, 132 skipped, 204 snapshots); GCC builds the NEON code once #322 is in.It merges cleanly with #322, #327 through #331, #336, #337 and #339, which turns NEON on for Linux arm64.
Origin. The scalar loops' sums and their tests are vlasky/sqlite-vec@d75c245's, which mceachen's photostructure/sqlite-vec@bb967e1 also follows. The NEON kernel's double sums are photostructure/sqlite-vec@050bb59 (photostructure#5), and vlasky#20 brings them to vlasky's fork. No other open or closed PR here addresses #324.