Repository navigation
Conversation
Drive sorts names as strings, so a message at 0.95 can list after one at 0.9. The listing stopped at the older one and skipped the newer message.
A failed read of a version file returned None, the same as a missing file. At startup, a Drive outage made every peer look unversioned, so sync skipped them all. Login skipped the version check the same way. Both reads now raise TransportUnavailableError, and sync aborts.
The search for orphaned files matched every syft file the account owns. A DS still owns the proposals that the DO moved into its archive, so deleting the DS's SyftBox also deleted them. An orphan is now a match with no parent.
The orphan search still deleted the DS proposals in the DO archive. Drive hides a parent folder that the caller cannot see, so these proposals looked parentless. A file that loses its folder moves to the root of its owner, so the search never found a real orphan. Syft now tags every file and folder that it creates. An orphan is a tagged file in My Drive root, other than /SyftBox.
The DS read each job file as text. A binary file failed the submit, and CRLF line endings reached the DO as LF. The DS now reads bytes and sends text only when the bytes are valid UTF-8. The job hashes no longer copy the old text read, so the DS and the DO hash the same bytes.
Three uploads created files without the syft tag, so the orphan search could not find them. They now use the tagging helper, which also retries them.
The version reads turned every HttpError into TransportUnavailableError, so a 403 looked like an outage. Offline errors, such as an unresolved host or a failed token refresh, escaped unwrapped. A corrupt version file read as a missing one. Now only errors that mean Drive cannot be reached become TransportUnavailableError, other errors pass through, and a corrupt file raises ValueError.
The queue carries bytes since job files go byte for byte, but on_file_change still typed its content as text only.
The mock deleted only the folder itself, so its subfolders stayed alive. Drive also deletes every item below the folder that the user owns, and moves items of other accounts to the root of their owners.
A failed or corrupt read of SYFT_peers.json returned an empty map. The next peer state update wrote it back with only one peer, which wiped every other peer's state and encryption bundle on Drive. The read now raises instead.
Rename find_orphaned_message_files to find_orphaned_syft_files, since it returns every tagged file in root. Type the peers map with Any values, pass fields by name, and list ValueError where the version reads raise it.
read_own_peer_info_file and read_peer_info_file repeated the same lookup, download and parse. Both now call _read_peer_info_file with their own lookup and label.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the four Drive transport bugs left open in connsplit v1.1, plus the same "failure looks like a missing file" pattern in the peers file, which could wipe every peer's state.
Changes
Message listing: a newer message from the same second is no longer skipped. Drive sorts names as strings, so the listing now stops only at an earlier second.
Outages: version and peers file reads raise
TransportUnavailableErrorwhen Drive can't be reached, andValueErrorwhen the file is corrupt, instead of returningNoneor{}. Other errors, such as a 403, pass through unchanged.Orphan search: syft tags every file it creates.
find_orphaned_syft_filesdeletes only tagged files in My Drive root, soundelivered messages and archived proposals in other accounts' folders are kept.
Job files: sent byte for byte. Binary files work, and CRLF line endings survive. syft-job's
as_senthash is removed.Mock Drive: behaves like real Drive for hidden parents, orphans and folder deletes.
Testing
Asana task