You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
ShreddedVariantArrayBuilder throws InvalidCastException for Decimal16 typed values #498
Describe the bug, including details regarding any error messages, version, and platform.
ShreddedVariantArrayBuilder.Build throws InvalidCastException whenever a shred schema has a Decimal16 typed value and a row shreds into it. Shredding a variant into a 16-byte decimal column therefore fails for every input; the size of the value doesn't matter.
Repro, against the Apache.Arrow.Operations 23.0.0 package on .NET 10 (Windows):
usingSystem.Data.SqlTypes;usingApache.Arrow.Operations.Shredding;usingApache.Arrow.Scalars.Variant;varschema=ShredSchema.Primitive(ShredType.Decimal16);// Fits System.Decimal; the failure does not depend on the magnitude.varvalues=newList<VariantValue>{VariantValue.FromDecimal16(SqlDecimal.Parse("1.50"))};var(metadata,rows)=VariantShredder.Shred(values,schema);vararray=ShreddedVariantArrayBuilder.Build(schema,metadata,rows);// throws
System.InvalidCastException: Unable to cast object of type 'System.Data.SqlTypes.SqlDecimal' to type 'System.Decimal'.
at Apache.Arrow.Operations.Shredding.ShreddedVariantArrayBuilder.BuildDecimalArray(ShredType shredType, IList`1 typedValues, MemoryAllocator allocator)
at Apache.Arrow.Operations.Shredding.ShreddedVariantArrayBuilder.BuildPrimitiveTyped(...)
at Apache.Arrow.Operations.Shredding.ShreddedVariantArrayBuilder.BuildTypedValueArray(...)
at Apache.Arrow.Operations.Shredding.ShreddedVariantArrayBuilder.Build(...)
Cause (as of main at bb2f7ed). For a Decimal16 slot, VariantShredder extracts the typed value as a SqlDecimal (VariantShredder.cs line 320: case ShredType.Decimal16: return value.AsSqlDecimal();). BuildDecimalArray in ShreddedVariantArrayBuilder.cs assumes every value is a System.Decimal in two places:
it unboxes each value with b.Append((decimal)v), which throws for a SqlDecimal;
it takes the column's scale from v is decimal d values only, so a Decimal16 column's scale would come out as 0 even without the cast.
Expected. The rows build a Decimal128 typed column holding the values at their scale, the same way Decimal4 and Decimal8 already work. Values outside System.Decimal's range (the reason Decimal16 goes through SqlDecimal) should keep their exact digits. Decimal128Array.Builder has an Append(SqlDecimal) overload, which would avoid narrowing through System.Decimal.
As far as I can see there's no test that builds a shredded array with a Decimal16 typed value. The existing Decimal16 tests in ShreddedVariantReaderTests read arrays that were constructed directly.
Activity