[C++][Parquet] Partially encrypted parquet files with plaintext footers should allow reading plaintext columns

Describe the bug, including details regarding any error messages, version, and platform.

Encrypted parquet files with plaintext footers conceptually allow reading plaintext columns without any decryption configuration. However, this does not work:

OSError: Cannot decrypt ColumnMetadata. FileDecryption is not setup correctly

There is even a disabled test for this:

arrow/python/pyarrow/tests/parquet/test_encryption.py

Lines 504 to 542 in 07be48c

    
           @pytest.mark.xfail(reason="Plaintext footer - reading plaintext column subset" 
        
                              " reads encrypted columns too") 
        
           def test_encrypted_parquet_write_read_plain_footer_single_wrapping( 
        
                   tempdir, data_table): 
        
               """Write an encrypted parquet, with plaintext footer 
        
               and with single wrapping, 
        
               verify it's encrypted, and then read plaintext columns.""" 
        
               path = tempdir / PARQUET_NAME 
        
               # Encrypt the footer with the footer key, 
        
               # encrypt column `a` and column `b` with another key, 
        
               # keep `c` plaintext 
        
               encryption_config = pe.EncryptionConfiguration( 
        
                   footer_key=FOOTER_KEY_NAME, 
        
                   column_keys={ 
        
                       COL_KEY_NAME: ["a", "b"], 
        
                   }, 
        
                   plaintext_footer=True, 
        
                   double_wrapping=False) 
        
               kms_connection_config = pe.KmsConnectionConfig( 
        
                   custom_kms_conf={ 
        
                       FOOTER_KEY_NAME: FOOTER_KEY.decode("UTF-8"), 
        
                       COL_KEY_NAME: COL_KEY.decode("UTF-8"), 
        
                   } 
        
               ) 
        
               def kms_factory(kms_connection_configuration): 
        
                   return InMemoryKmsClient(kms_connection_configuration) 
        
               crypto_factory = pe.CryptoFactory(kms_factory) 
        
               # Write with encryption properties 
        
               write_encrypted_parquet(path, data_table, encryption_config, 
        
                                       kms_connection_config, crypto_factory) 
        
               # # Read without decryption properties only the plaintext column 
        
               # result = pq.ParquetFile(path) 
        
               # result_table = result.read(columns='c', use_threads=False) 
        
               # assert table.num_rows == result_table.num_rows

Turns out, this is affected by a bug unrelated to encryption, but only encryption is sensitive to the bug.

Component(s)

C++

apacheGH-51361: [C++][Parquet] Derive records_to_read in FileReaderIm…

Navigation Menu

Repository navigation

[C++][Parquet] Partially encrypted parquet files with plaintext footers should allow reading plaintext columns #51361

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	@pytest.mark.xfail(reason="Plaintext footer - reading plaintext column subset"
	" reads encrypted columns too")
	def test_encrypted_parquet_write_read_plain_footer_single_wrapping(
	tempdir, data_table):
	"""Write an encrypted parquet, with plaintext footer
	and with single wrapping,
	verify it's encrypted, and then read plaintext columns."""
	path = tempdir / PARQUET_NAME

	# Encrypt the footer with the footer key,
	# encrypt column `a` and column `b` with another key,
	# keep `c` plaintext
	encryption_config = pe.EncryptionConfiguration(
	footer_key=FOOTER_KEY_NAME,
	column_keys={
	COL_KEY_NAME: ["a", "b"],
	},
	plaintext_footer=True,
	double_wrapping=False)

	kms_connection_config = pe.KmsConnectionConfig(
	custom_kms_conf={
	FOOTER_KEY_NAME: FOOTER_KEY.decode("UTF-8"),
	COL_KEY_NAME: COL_KEY.decode("UTF-8"),
	}
	)

	def kms_factory(kms_connection_configuration):
	return InMemoryKmsClient(kms_connection_configuration)

	crypto_factory = pe.CryptoFactory(kms_factory)
	# Write with encryption properties
	write_encrypted_parquet(path, data_table, encryption_config,
	kms_connection_config, crypto_factory)

	# # Read without decryption properties only the plaintext column
	# result = pq.ParquetFile(path)
	# result_table = result.read(columns='c', use_threads=False)
	# assert table.num_rows == result_table.num_rows

[C++][Parquet] Partially encrypted parquet files with plaintext footers should allow reading plaintext columns #51361

Description

Describe the bug, including details regarding any error messages, version, and platform.

Component(s)

Activity

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions