Prove that an exported archive matches a network server #386

Overview

Integration tests prove that a full export produces a complete archive with the same data as a network server that receives the same push. This issue is part of milestone M7 (verification).

Background & Motivation

The parity rule is the central promise of the design. An archive holds the same morango state as a server that receives the same push, plus a manifest. Future import (Phase 2) and incremental update (Phase 4) depend on this promise.

These tests also cover the full export flow through the public API, chunked transfers, and cleanup after a failure.

Design: spec. Plan: implementation plan, Task 25.

Description & Expected Outcomes

A parity test pushes the same facility data twice with the same filter. One push goes to the second-server test harness, and one push goes to a disk connection. The test compares the Store, RecordMaxCounter and DatabaseMaxCounter rows of the two receivers. The test ignores rows that use the instance ID of each receiver.

An end-to-end test runs a full export through the public API and reopens the archive. The archive must match every row of the "Archive contents at completion" table in spec §3.4.

A chunking test uses a small chunk size, so that the transfer takes more than one cycle. A failure test forces an error during the transfer. After the host closes the sync session and the connection, no archive file, partial file, lock file or archive alias remains.

Deliverables & Contracts

The feature delivers these capabilities:

• A parity test against the second-server harness.
• An end-to-end export test through create_disk_connection and the push client.
• A chunking test and a failure-injection test.

Acceptance Criteria

• The parity test finds equal Store, RecordMaxCounter and DatabaseMaxCounter rows in default2 and in the reopened archive.
• The end-to-end test finds every row of the spec §3.4 contents table in the archive, including ScopeDefinition.
• With chunk_size=2 and 5 records, the push transfer runs 3 times, with results PENDING, PENDING, COMPLETED.
• If DiskSyncConnection.push_record_chunk raises OSError, run() raises MorangoError.
• After the failure and a call to close_sync_session() and close(), no path, .partial or .lock file exists.
• After the failure, no key that starts with ARCHIVE_ALIAS_PREFIX is in connections.databases.
• All tests pass on SQLite and in the postgres tox environment.

Technical Pointers & Architecture

• Target Components / Context: tests/testapp/tests/integration/test_disk_export.py.
• Related Patterns: PushPullClientTestCase and LiveServer in tests/testapp/tests/integration/test_syncsession.py give the seed data and the second server.
• Data Model & Schema Considerations: None.
• Resilience & Failure Modes: The failure test covers the discard path of the archive.

Notes & Tradeoffs

• Dependencies: Blocked by: Keep old network operation names and allow connection-specific overrides #375 (deprecated network operation names and override order), Receive pushed data into an archive through the Peer protocol #384 (disk Peer protocol). Blocks: Document disk export and deprecations in the changelog #387.
• Kolibri companion changes: See spec §3.9. Host routers must allow morango migrations on aliases that start with morango_archive_, and must return None from db_for_read/db_for_write for morango models. Kolibri's routers already do both (Spike: prove that a Django-managed archive database works #362), so Kolibri needs no router change. Kolibri settings overrides must list the Peer* operations.
• Testing Requirements: Run python -O -m pytest tests/testapp/tests/integration/test_disk_export.py -v and tox -e postgres -- tests/testapp/tests/integration. With a PostgreSQL host, the archive stays SQLite.
• Tradeoffs & Alternatives Considered: The tests read the server data from the default2 alias of the test settings.

Metadata

• Complexity: Medium
• Target Branch: release-v0.9.x

AI Usage

Drafted with Claude (Claude Code) from the approved design spec and implementation plan. The author reviewed the requirements, and the code references were checked against the release-v0.9.x codebase.