Skip to content

Error Handling

aiogzip normalizes errors into a small, stable taxonomy. The engine backing decompression (stdlib zlib or zlib-ng from the [fast] extra) never changes which exception type you see: engine-specific errors are caught internally and re-raised uniformly, so except clauses written against stdlib behavior keep working when the fast engine is installed.

The exception types and the explicitly documented decompression-limit message prefix are part of the 2.0 stability contract. Complete dynamic messages are not: offsets, sizes, member numbers, engine names, and operating system text may vary.

Taxonomy

Situation Raises Notes
Corrupt or malformed gzip data gzip.BadGzipFile Bad magic/header, CRC mismatch, wrong ISIZE, truncated or garbled deflate stream. Identical across engines.
Underlying I/O failure OSError Missing file, permission denied, disk errors — whatever the OS raised.
max_decompressed_size exceeded OSError (not BadGzipFile) Message starts with "decompressed output exceeded max_decompressed_size".
Wrong argument types / values TypeError / ValueError Raised eagerly at the call with a corrective message.
Overlapping operations on one async file aiogzip.ConcurrentOperationError An OSError subtype reporting a shared-handle ownership bug before state corruption.
Codec operation still active RuntimeError Exhaust the returned iterator before starting another operation.
Codec abandoned or partially closed OSError on later use The instance is unusable; call discard() and create a new codec.
Codec used after successful decoder finalization ValueError feed() and repeated finish() are terminal-state misuse.
Operations on a closed file ValueError Matches io module conventions, except that async for over a closed handle ends without raising (see below).
Reading a write-mode file (or vice versa) OSError e.g. "File not open for reading".

gzip.BadGzipFile subclasses OSError, so order your handlers from specific to general:

import gzip

import aiogzip

try:
    data = await aiogzip.read("untrusted.gz", max_decompressed_size=100 * 2**20)
except gzip.BadGzipFile:
    ...  # corrupt or hostile input: reject the file
except OSError as exc:
    ...  # I/O failure, or the decompression cap tripped (see below)

Closed and unopened handles

Reads, writes and seeks on a closed handle raise ValueError. Iteration is the exception: async for over a closed binary or text handle yields nothing and ends, as if at end of file, instead of raising. Check closed first if a loop must distinguish a closed handle from an empty stream.

flush() on a write handle that has not been opened yet (constructed but not entered with async with) returns without error: no gzip member has started, so there is nothing to flush.

Custom sink errors in write() and flush()

When a custom fileobj's write() raises, the two calls report it differently:

  • write() lets the sink's exception propagate unchanged, whatever its type.
  • flush() wraps any Exception that is not an OSError as OSError("Unexpected error during flush: ..."), with the original as __cause__. A sink's OSError normally propagates as raised (aiogzip rewrites only one whose message begins with its own internal Unexpected error during compression flush: prefix). Cancellation and interrupts such as KeyboardInterrupt propagate unwrapped.

Either way the writer is broken afterwards: discard the incomplete output and create a new writer. Code that catches a sink's own exception type around flush() should inspect __cause__. Unifying the two is a candidate for a later release because it changes an exception type.

Same-handle concurrency

One AsyncGzipBinaryFile or AsyncGzipTextFile handle is intentionally owned by one task at a time. If a read, seek, write, flush, or close overlaps an operation already in flight on that handle, the rejected call raises the public ConcurrentOperationError. The active call is left intact. Treat the exception as a programming error: it is not a lock, and catching it in a retry loop is not supported synchronization. Separate handles may progress concurrently because they do not share gzip or buffer state.

Handles are asyncio objects bound to the event loop that uses them; they are not thread-safe. Do not call one handle from several threads or event loops.

When several tasks intentionally share a handle, use an application lock that covers the complete logical operation, not only the first low-level call:

import asyncio

lock = asyncio.Lock()

async with lock:
    batch = await stream.readlines(1024 * 1024)

The reservation covers text already decoded into the wrapper's buffer as well as binary codec work. Composite readlines(), writelines(), and write-mode seek() calls retain ownership across all of their internal reads or writes; another task cannot consume a buffered prefix or splice data between batches. If a text refill fails before the call can publish a result, characters decoded before that failure remain buffered for the documented retry.

On a clean async with exit, aiogzip waits until all calls from a looping producer finish and then closes normally. Concurrent close calls wait for the same binary or text finalization. If the context body is already raising, exit preserves that primary exception and attempts abortive owned-resource cleanup. An active read, write, or flush that resumes after this abort raises OSError; it never returns a truncated prefix or reports success for a gzip member that can no longer receive its trailer. Cancellation delivered during cleanup still propagates, and a failed abortive underlying close leaves the handle reportably open so close() can be retried.

Once a handle has submitted the native close of an aiofiles file it owns (a path, or a file object passed with closefd=True), cancellation cannot abandon that close: it runs to completion before the cancellation propagates, and a close failure becomes the cancellation's __cause__. During an abortive exit, the handle is closed only if that close returned normally; otherwise it stays reportably open and close() retries. A cancellation that arrives while the exit is still waiting for an active call to settle no longer skips the close: once that call has settled, the exit closes the file in the same way and then propagates the cancellation. When the context body raised, that cancellation keeps the body's exception in its chain: as its __context__, or, when the close failed, as the context of the close failure that is the cancellation's __cause__. Custom file objects' async close() methods are cancelled cooperatively, as before.

Cancellation while a clean context exit is waiting for an active call also attempts abortive owned-resource cleanup before the cancellation propagates.

If that abortive cleanup fails, for example because a custom source's close() raises, the failure is recorded as a note beginning Context-exit cleanup also failed: instead of being dropped. Exception types and precedence are unchanged:

  • When the context body raised, an ordinary exception from the cleanup is noted on the body's exception. A cancellation or interrupt delivered during the cleanup propagates instead, as before.
  • When a cancellation interrupted a clean exit, the cancellation propagates and any cleanup failure is noted on it, including a failure carried as the cause of a repeated cancellation. A repeated cancellation on its own adds no note.

To abort an active read or write on a custom file object, the exit cancels the task running that call once, and consumes that request however the call ends, even if the custom object swallows the cancellation. An enclosing asyncio.timeout() in that task therefore still reports its own expiry as TimeoutError, and a separate outside cancellation of the task is kept.

The maintained examples avoid same-handle overlap entirely: each shard task owns a separate file handle, and each transport codec operation is exhausted before the next begins.

Telling the decompression cap apart from corruption

A stream that exceeds max_decompressed_size (a decompression bomb, or just a bigger file than expected) raises a plain OSError, not BadGzipFile — the data may be perfectly valid gzip; it is the output budget that was exceeded. Because the cap error is not a BadGzipFile, the handler split above distinguishes the two cases by exception type alone. If you need to single the cap out from real I/O errors within the OSError branch, match on its stable message prefix:

try:
    data = await aiogzip.read("untrusted.gz", max_decompressed_size=100 * 2**20)
except gzip.BadGzipFile:
    raise  # corrupt input: not the cap
except OSError as exc:
    if str(exc).startswith("decompressed output exceeded max_decompressed_size"):
        ...  # over budget: maybe retry with a larger cap
    else:
        raise

Where errors surface

Decompressed bytes reach the caller in one of three states:

  • Provisional output: bytes from a member whose trailer has not been checked yet. Normal reads deliver these as they decode; a later CRC, ISIZE, truncation or trailing-data error can still invalidate them.
  • Validated output: every member has passed its trailer check once a file read reaches the end of the stream (read() or read(n) with n > 0 returns empty without an error), once the operation returned by GzipDecoder.finish() is exhausted without an error, or once decompress_chunks() is exhausted without an error. An empty read(0) and an exhausted feed() operation validate nothing.
  • Recovery data: bytes still readable after an integrity failure has been raised (see below). They are not validated and must not be trusted as that member's content.

Warning — successful output is not yet proof of integrity. GzipDecoder.feed() and decompress_chunks() can produce payload before the member trailer arrives. CRC, ISIZE, truncation, and trailing-data errors may surface later. Exhaust GzipDecoder.finish() or the complete async iterator before treating the stream as valid.

Streaming helpers validate lazily where the work happens: the iterator returned by decompress_chunks() raises BadGzipFile only as the corrupt region is consumed, and complete trailer validation happens when the iterator is exhausted. verify() and inspect() scan the whole stream eagerly and raise before returning a result. See Processing untrusted gzip input for a full defensive-reading recipe.

Recovery data after an integrity failure

High-level binary and text readers distinguish clean EOF from a poisoned terminal state. Their integrity-failure lifecycle is:

healthy
  -> integrity failure raised
  -> previously decoded recovery data may remain readable
  -> recovery data is drained
  -> stable terminal OSError

rewindable source:
  absolute seek(0) may construct a fresh decoder and recover

non-rewindable source:
  close and reopen/recreate the source

Recovery data is not proof that its gzip member is valid. Deflate output can be emitted before a later CRC-32 or ISIZE trailer check fails, so applications must not publish or trust that member's bytes merely because they were readable. When codec metadata collection is enabled, trailer-validated prior members remain in GzipDecoder.members; the failing member is absent. Those records validate only the completed prefix, not the full concatenated stream.

After the recovery buffer is empty, later reads raise the same terminal OSError instead of returning clean EOF. An absolute seek(0) can rebuild the decoder only when the source can rewind (directly or through its replay cache). Close and recreate a non-rewindable source. Decompression-limit failures, cancellation, and unexpected internal failures poison the reader without automatically enabling the integrity-failure salvage path.

For text readers, the only guaranteed recovery is calling seek(0) directly. A cookie saved with tell() before the failure is not a recovery point: seek(cookie) may raise the same terminal OSError instead of rewinding. Call seek(0) first; once the reader has recovered, a saved cookie is an ordinary position again.

An absolute text seek(), to a position or a tell() cookie, that fails or is cancelled after it has moved the underlying binary reader makes the reader terminal: later reads raise the same terminal OSError until seek(0) recovers it. A seek that fails before moving the binary reader leaves the text position where it was. If the binary reader is still healthy, the text reader is unchanged and the seek can be retried. Otherwise the binary reader's failure rules above apply: a gzip integrity failure leaves the recovery state, so later reads return the recovered text from the original position, and any other failure makes the reader terminal. Either way, a failed absolute seek is never followed by text from the wrong position. seek(0, SEEK_END) reads to the end, so its failures follow the read() rules above, including the UnicodeDecodeError caution below.

Do not retry a text read that raised UnicodeDecodeError. The reader does not become terminal, but the failed read has already consumed the whole chunk of decompressed bytes it was decoding, so the next read resumes after that chunk and silently skips text, possibly far more than the undecodable bytes themselves. Call seek(0) to start again from the beginning, or reopen the file with a different encoding or an errors handler such as "replace".

Codec finalization and operation abandonment

Every state-changing GzipEncoder or GzipDecoder call reserves the codec and returns a lazy CodecOperation. Engine errors, integrity failures, or closing an operation before it is exhausted make that codec unusable. This is intentional: the engine may already have consumed input, so silently reusing the instance could skip bytes or emit a valid-looking trailer for incomplete output.

Dropping an operation does not invoke state-changing finalizer behavior. The codec remains reserved and another operation raises RuntimeError regardless of garbage-collector timing. If the operation is reachable, exhaust it to continue or call its idempotent close() method to abandon the stream. If the caller no longer retains it, call the codec's idempotent discard() method; this invalidates the operation and promptly releases its captured input and codec state. Neither method resets the instance. Start again with a new codec.

GzipDecoder.finish() checks that the current deflate stream and trailer are complete, that no partial next header remains, and that all CRC and ISIZE values match. An incomplete or corrupt stream raises gzip.BadGzipFile and poisons the decoder. After finish() succeeds, both another feed() and another finish() raise ValueError.

The async wrappers follow the same rule internally. If cancellation occurs while a worker thread is advancing an operation, they wait until that worker has stopped before discarding state and re-raising cancellation. Source and cleanup failures do not replace the primary operation error.