[C#] Expose method to compare 2 Tables

Describe the enhancement requested

Motivation

I am working on integration tests where I am testing that a service returns an arrow table that equals to an expected arrow table.

Feature

Expose a method that compares 2 Tables

Considerations

I see that there's an ArrowReaderVerifier class that does have some similar functionality here to compare record batches -

arrow/csharp/test/Apache.Arrow.Tests/ArrowReaderVerifier.cs

Lines 52 to 65 in 1521148

    
           public static void CompareBatches(RecordBatch expectedBatch, RecordBatch actualBatch, bool strictCompare = true) 
        
           { 
        
               SchemaComparer.Compare(expectedBatch.Schema, actualBatch.Schema); 
        
               Assert.Equal(expectedBatch.Length, actualBatch.Length); 
        
               Assert.Equal(expectedBatch.ColumnCount, actualBatch.ColumnCount); 
        
               for (int i = 0; i < expectedBatch.ColumnCount; i++) 
        
               { 
        
                   IArrowArray expectedArray = expectedBatch.Arrays.ElementAt(i); 
        
                   IArrowArray actualArray = actualBatch.Arrays.ElementAt(i); 
        
                   CompareArrays(expectedArray, actualArray, strictCompare); 
        
               } 
        
           }

But as far as I understand Apache.Arrow.Tests namespace is not publicly available. Is that correct? Is it possible to expose some of the methods from it to be public?

Component(s)

C#

@andrewshandy

Hi @andrewshandy, I think a PR adding this to the public API would be welcome.

From a code search and seeing its presence on https://github.com/apache/arrow/blob/main/csharp/README.md#not-implemented, it looks like comparison isn't (publicly) supported for Table as well as other types. If you do take this on, it might be good to (1) file a tracking issue for the overall feature and use sub-issues/PRs for the pieces and (2) review the semantics of similar operations in C++, Python, etc. and figure out if C# can match or if it needs to deviate for some reason.

cc @CurtHagenlocher @adamreeve in case they want to add any context

But as far as I understand Apache.Arrow.Tests namespace is not publicly available. Is that correct?

Yes that's correct, to expose this functionality publicly it would need to be moved into the Apache.Arrow project or a new project.

I don't imagine we'd want to expose the ArrowReaderVerifier or ArrayComparer classes directly, as once something is in the public API it's hard to change later, but adding new methods for comparing arrays, record batches and tables to the public API would be useful.

Comparing tables is a little bit more complicated than the existing functionality for comparing record batches, because two tables that contain the same data but are chunked differently should be considered equal. If you expect chunking to be the same, you might want to start with adding a way to compare record batches and compare tables by comparing all of their record batches.

Great! Thank you both for the info! I will start working on this then, I will probably start from comparing record batches and see what else would be needed to implement it 🤗

Superseded by apache/arrow-dotnet#183.

Navigation Menu

Repository navigation

[C#] Expose method to compare 2 Tables #46464

Describe the enhancement requested

Motivation

Feature

Considerations

Component(s)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	public static void CompareBatches(RecordBatch expectedBatch, RecordBatch actualBatch, bool strictCompare = true)
	{
	SchemaComparer.Compare(expectedBatch.Schema, actualBatch.Schema);
	Assert.Equal(expectedBatch.Length, actualBatch.Length);
	Assert.Equal(expectedBatch.ColumnCount, actualBatch.ColumnCount);

	for (int i = 0; i < expectedBatch.ColumnCount; i++)
	{
	IArrowArray expectedArray = expectedBatch.Arrays.ElementAt(i);
	IArrowArray actualArray = actualBatch.Arrays.ElementAt(i);

	CompareArrays(expectedArray, actualArray, strictCompare);
	}
	}

[C#] Expose method to compare 2 Tables #46464

Description

Describe the enhancement requested

Motivation

Feature

Considerations

Component(s)

Activity

amoeba commented on May 20, 2025

adamreeve commented on May 20, 2025

andrewshandy commented on May 20, 2025

CurtHagenlocher commented on Nov 18, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions