Repository navigation
[R] "Error: Filter expression not supported for Arrow Datasets" using "date" expression rigth hand side of a filter #39688
Description
Describe the bug, including details regarding any error messages, version, and platform.
Hello,
Here is a repex of a bug occuring when I try to filter a dataset like "filter(Date == date).
Maybe the expression "date" has a special interpretation ?
Thx,
library(arrow)
library(dplyr)
library(lubridate)
# Create a simple df
set.seed(123)
Date <- rep(c("2022-01-01", "2022-01-02"), each = 5)
Volume <- sample(100:200, 10, replace = TRUE)
df <- tibble(
Date = Date,
Volume = Volume
)
# save as partitioned parquet
temp_dir <- tempdir()
write_dataset(df, temp_dir, partitioning = "Date")
# Lazy import
reimported_data <- open_dataset(temp_dir)
# Bug
date = "2022-01-02"
reimported_data |> filter(Date == date)
# Possible workaround
my_date = "2022-01-02"
reimported_data |> filter(Date == my_date)
Component(s)
Parquet, R
Activity
[-]"Error: Filter expression not supported for Arrow Datasets" using "date" expression rigth hand side of a filter[/-][+][R] "Error: Filter expression not supported for Arrow Datasets" using "date" expression rigth hand side of a filter[/+]thisisnic commentedon Jan 30, 2024
Thanks for reporting this @jpramil! I can confirm that I can replicate it and it is a bug. I think we'll need to update something around the environment in which we look for the value we are comparing against.
mohdbilal87 commentedon Mar 5, 2024
Confirming the above behavior. For me it was
This kept segfaulting. I suspect, as noted above, this is due to
day == day.day == dworks as expected.GH-39688: [R] "Error: Filter expression not supported for Arrow Datas…
thisisnic commentedon Sep 23, 2026
Issue resolved by pull request 51291
#51291