镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Fix MinBy/MaxBy null-key handling with index-friendly selection - #39653

Open
emremenekse wants to merge 4 commits into
dotnet:mainfrom
emremenekse:fix/minmaxby-null-keys
Open

emremenekse wants to merge 4 commits into
dotnet:mainfrom
emremenekse:fix/minmaxby-null-keys

Conversation

@emremenekse

Copy link
Copy Markdown
Contributor

MinBy/MaxBy currently lower to ordering followed by First/FirstOrDefault. Database null ordering can select a null-key row even when non-null keys exist, unlike LINQ-to-Objects. For example, MinBy over keys [null, 1, 2] can select the null-key row instead of the row with key 1.

Prefer non-null keys while retaining the first source row for an all-null source and preserving empty-source behavior. For eligible plain entity queries, find the extremal key separately from source ordering, then select its first occurrence. The general lowering remains available for unsupported candidate shapes, including owned/JSON entities, scalar sources and computed selectors.

Add a SQLite-specific translation for the recognized two-candidate lowering. Match the source, key and partition predicates, then replace the candidate union with a null-safe SQLite IS comparison against the extremal-key subquery. This removes the fallback candidate and final sort for the optimized shape. Decline the rewrite for different sources, unrecognized method calls, explicit volatile expressions and view/function/raw-SQL roots. SQL Server continues to use the common candidate lowering; the IS optimization is confined to SQLite. The new SQL expression supports quoting for precompiled queries and is non-nullable even when either operand is nullable.

Fixes #39276.

Validation

Targeted local tests on Windows:

  • Nullable-key regression matrix: 56 passed each on SQLite, SQL Server LocalDB and InMemory.
  • Additional SQLite guard and relational-null tests: 10 passed.
  • Existing NorthwindAggregateOperatorsQuery tests selected with --filter-method 'By': 104 passed each on SQLite and SQL Server.
  • Precompiled-query regression: 1 passed each on SQLite and SQL Server, including plain entity MinBy/MaxBy, owned JSON and a scalar source.
  • Total functional tests: 388 passed, 0 failed, 0 skipped.
  • SQLite API baseline test: 1 passed.
  • Standalone owner/note filter and parameterization matrix: 96 passed.
  • git diff --check passed.

Coverage includes integer/string nullable keys, sync/async, mixed-null keys, all-null keys, an empty table, tied keys, reverse ordering, pagination and Id=0. SQLite SQL baselines assert the new translation. Separate tests verify that different candidate/key sources and volatile sources retain the general translation, and that all-null behavior is preserved with UseRelationalNulls.

Precompiled MinByAsync/MaxByAsync recognition remains disabled in the existing query locator/code generator. This revision tests synchronous precompiled calls and normal asynchronous execution.

Local performance checks

Repeated the original 32 Release comparisons: SQLite/SQL Server, 1,000/100,000-row ranges, indexed/unindexed, int/int? keys, MinBy/MaxBy. The short run showed substantial timing noise even for identical-SQL controls, so its raw alarms were retained and the critical indexed cases were rechecked with longer batches.

A second run used 100 warmup calls and seven alternating batches of 1,000 EF operations for each of four indexed 100,000-row comparisons. None crossed the unchanged local >=2x diagnostic threshold. This is a diagnostic threshold, not evidence of zero overhead:

EF operation Original simple ordering reference Corrected translation
SQLite MinBy 133.72 us 156.94 us
SQLite MaxBy 55.08 us 61.18 us
SQL Server MinBy 179.13 us 224.04 us
SQL Server MaxBy 160.53 us 201.68 us

The simple reference returns the wrong row for mixed-null MinBy. Both MaxBy versions return the same row in these cases. The measurements show remaining overhead compared with that simple reference, not an end-to-end speedup.

The previous correct candidate SQL and the new SQLite SQL were also executed on the same database with fresh ADO.NET commands and identical parameters. Both were required to return the same expected record. With 100 warmup calls and seven alternating batches of 500 operations:

SQLite operation Previous correct candidate SQL Native IS SQL
MinBy 80.80 us 24.15 us
MaxBy 70.68 us 17.72 us

This approximately 70-75% reduction covers SQL command creation/preparation/execution and reading the selected ID, not the full EF pipeline or application throughput. These are local medians, not BenchmarkDotNet confidence intervals or production guarantees.

The measured SQLite indexed plans use two covering-index searches without UNION ALL or a temporary sorting B-tree. All 32 captured SQL Server SQL files were byte-identical before and after adding the SQLite-specific optimization; no SQL Server performance improvement is claimed.

Limits

  • Optimization is deliberately conservative. Owned/JSON, scalar/computed-selector and other fallback shapes can retain the general discriminator's cost.
  • The SQLite matcher adds compilation-time work; cold compilation was not measured.
  • PostgreSQL, Cosmos, full test suites, production load and all include/projection/concurrency combinations were not tested.
  • Source expressions are referenced more than once; the rewrite requires matching sources and rejects known unrecognized/volatile shapes.
  • No production dependency files changed.
  • Targeted owner/filter checks do not establish general application authorization safety.

Contribution checklist

  • I've read the guidelines for contributing and seen the walkthrough
  • I've posted a comment on an issue with a detailed description of how I am planning to contribute and got approval from a member of the team
  • The code builds and tests pass locally (also verified by our automated build checks)
  • Commit messages follow the repository's summary, details, and Fixes #bugnumber format
  • Tests for the changes have been added (for bug fixes / features)
  • Code follows the same patterns and style as existing code in this repo

Local targeted validation passed as described above. CI, the walkthrough and team approval have not been verified.

- Order non-null keys first and retain explicit source ordering for ties.
- Preserve empty and all-null source behavior.
- Add shared regression coverage for SQLite, SQL Server, and InMemory.

Fixes dotnet#39276
- Find extremal keys separately while preserving source order and all-null fallback.
- Use guarded SQLite IS translation to avoid the candidate union and final sort.
- Cover ties, pagination, empty sources, relational nulls and precompiled queries.

Fixes dotnet#39276
emremenekse and others added 2 commits November 14, 2026 14:47
- Override the shared nullable-key test and assert all five SQL commands.
- Satisfy the existing check that requires provider-specific test overrides.

Fixes dotnet#39276
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MinBy/MaxBy lowering does not skip null keys as LINQ-to-Objects does

1 participant