Repository navigation
fix: Keep entity rows apart in the ibis point-in-time join - #6884
Merged
ntkathole merged 1 commit intoSep 29, 2026
Merged
Conversation
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6884 +/- ##
=======================================
Coverage 47.83% 47.84%
=======================================
Files 422 422
Lines 52727 52727
Branches 7663 7662 -1
=======================================
+ Hits 25223 25225 +2
+ Misses 25683 25680 -3
- Partials 1821 1822 +1
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
ntkathole
approved these changes
Sep 29, 2026
The ibis point-in-time join (used by the DuckDB, MSSQL and Oracle offline stores) identifies each entity row by concatenating the string values of its join keys and event timestamp. Different rows could produce the same id, e.g. driver_id="1", customer_id="11" and driver_id="11", customer_id="1", so one row got the other's feature values. A null key also made the whole id null, so the row lost the features of every other feature view, not just the one keyed on the null column. Length-prefix each value and give nulls their own marker so the id is only shared by rows with equal values. Signed-off-by: Mohammad Hijjawi <mohammad.hijjawi1997@gmail.com>
ntkathole
force-pushed
the
fix/ibis-entity-row-id
branch
from
September 29, 2026 05:56
412adf3 to
881cc39
Compare
5 of 7 tasks
ntkathole
pushed a commit
to sohailshk/feast
that referenced
this pull request
Nov 13, 2026
The point-in-time join templates of the Redshift, Snowflake, BigQuery, Postgres, Athena, Spark, Trino, ClickHouse and Couchbase offline stores identify each entity row by concatenating the string values of its join keys and event timestamp. Different rows could produce the same id, e.g. driver_id=1, customer_id=12 and driver_id=11, customer_id=2 at the same timestamp. Such rows got the other row's feature value, or, when both feature rows had the same event timestamp, each came back twice, once with each value. Length-prefix each join key value in the id, as the ibis join does since feast-dev#6884, so the id is only shared by rows with equal values. Signed-off-by: sohailshk <122166523+sohailshk@users.noreply.github.com>
ntkathole
pushed a commit
that referenced
this pull request
Nov 13, 2026
* fix: Keep entity rows apart in the SQL point-in-time joins The point-in-time join templates of the Redshift, Snowflake, BigQuery, Postgres, Athena, Spark, Trino, ClickHouse and Couchbase offline stores identify each entity row by concatenating the string values of its join keys and event timestamp. Different rows could produce the same id, e.g. driver_id=1, customer_id=12 and driver_id=11, customer_id=2 at the same timestamp. Such rows got the other row's feature value, or, when both feature rows had the same event timestamp, each came back twice, once with each value. Length-prefix each join key value in the id, as the ibis join does since #6884, so the id is only shared by rows with equal values. Signed-off-by: sohailshk <122166523+sohailshk@users.noreply.github.com> * test: pass a subquery as the feature source so DuckDB 1.5 parses the Trino template Signed-off-by: sohailshk <122166523+sohailshk@users.noreply.github.com> --------- Signed-off-by: sohailshk <122166523+sohailshk@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
The ibis point-in-time join (used by the DuckDB, MSSQL and Oracle offline stores) identifies each entity row by concatenating the string values of its join keys and event timestamp into
entity_row_id, then deduplicates and joins feature rows back on that id. Two problems with plain concatenation:driver_id="1", customer_id="11"anddriver_id="11", customer_id="1"at the same timestamp both become"_111...", so one row silently receives the other row's feature values (whether it happens depends on the column order, which comes from aset).Repro with
point_in_time_join(drivers FV keyed ondriver_id, customers FV keyed oncustomer_id):This length-prefixes each value and gives nulls their own marker, so the id is only shared by rows with equal values. Both places that build the id now use the same helper.
Which issue(s) this PR fixes:
No existing issue.
Checks
git commit -s)Testing Strategy
Misc
Added
test_point_in_time_join_keeps_entity_rows_aparttosdk/python/tests/unit/infra/offline_stores/test_ibis.py; it fails on master (deterministically, through the null-key case) and passes with this change.tests/unit/infra/offline_stores/test_filter_by_created_timestamp.py(the other caller ofpoint_in_time_join) gives the same results as on master.ruffandmypyonibis.pyare clean. The id format is internal and dropped before results are returned.