Error Handling
aiogzip normalizes errors into a small, stable taxonomy. The engine backing
decompression (stdlib zlib or zlib-ng from the [fast] extra) never changes
which exception type you see: engine-specific errors are caught internally and
re-raised uniformly, so except clauses written against stdlib behavior keep
working when the fast engine is installed.
The exception types and the explicitly documented decompression-limit message prefix are part of the 2.0 stability contract. Complete dynamic messages are not: offsets, sizes, member numbers, engine names, and operating system text may vary.
Taxonomy
| Situation | Raises | Notes |
|---|---|---|
| Corrupt or malformed gzip data | gzip.BadGzipFile |
Bad magic/header, CRC mismatch, wrong ISIZE, truncated or garbled deflate stream. Identical across engines. |
| Underlying I/O failure | OSError |
Missing file, permission denied, disk errors — whatever the OS raised. |
max_decompressed_size exceeded |
OSError (not BadGzipFile) |
Message starts with "decompressed output exceeded max_decompressed_size". |
| Wrong argument types / values | TypeError / ValueError |
Raised eagerly at the call with a corrective message. |
| Overlapping operations on one async file | aiogzip.ConcurrentOperationError |
An OSError subtype reporting a shared-handle ownership bug before state corruption. |
| Codec operation still active | RuntimeError |
Exhaust the returned iterator before starting another operation. |
| Codec abandoned or partially closed | OSError on later use |
The instance is unusable; call discard() and create a new codec. |
| Codec used after successful decoder finalization | ValueError |
feed() and repeated finish() are terminal-state misuse. |
| Operations on a closed file | ValueError |
Matches io module conventions, except that async for over a closed handle ends without raising (see below). |
| Reading a write-mode file (or vice versa) | OSError |
e.g. "File not open for reading". |
gzip.BadGzipFile subclasses OSError, so order your handlers from specific
to general:
import gzip
import aiogzip
try:
data = await aiogzip.read("untrusted.gz", max_decompressed_size=100 * 2**20)
except gzip.BadGzipFile:
... # corrupt or hostile input: reject the file
except OSError as exc:
... # I/O failure, or the decompression cap tripped (see below)
Closed and unopened handles
Reads, writes and seeks on a closed handle raise ValueError. Iteration is
the exception: async for over a closed binary or text handle yields nothing
and ends, as if at end of file, instead of raising. Check closed first if a
loop must distinguish a closed handle from an empty stream.
flush() on a write handle that has not been opened yet (constructed but not
entered with async with) returns without error: no gzip member has started,
so there is nothing to flush.
Custom sink errors in write() and flush()
When a custom fileobj's write() raises, the two calls report it
differently:
write()lets the sink's exception propagate unchanged, whatever its type.flush()wraps anyExceptionthat is not anOSErrorasOSError("Unexpected error during flush: ..."), with the original as__cause__. A sink'sOSErrornormally propagates as raised (aiogzip rewrites only one whose message begins with its own internalUnexpected error during compression flush:prefix). Cancellation and interrupts such asKeyboardInterruptpropagate unwrapped.
Either way the writer is broken afterwards: discard the incomplete output and
create a new writer. Code that catches a sink's own exception type around
flush() should inspect __cause__. Unifying the two is a candidate for a
later release because it changes an exception type.
Same-handle concurrency
One AsyncGzipBinaryFile or AsyncGzipTextFile handle is intentionally owned
by one task at a time. If a read, seek, write, flush, or close overlaps an
operation already in flight on that handle, the rejected call raises the
public ConcurrentOperationError. The active call is left intact. Treat the
exception as a programming error: it is not a lock, and catching it in a retry
loop is not supported synchronization. Separate handles may progress
concurrently because they do not share gzip or buffer state.
Handles are asyncio objects bound to the event loop that uses them; they are not thread-safe. Do not call one handle from several threads or event loops.
When several tasks intentionally share a handle, use an application lock that covers the complete logical operation, not only the first low-level call:
The reservation covers text already decoded into the wrapper's buffer as well
as binary codec work. Composite readlines(), writelines(), and write-mode
seek() calls retain ownership across all of their internal reads or writes;
another task cannot consume a buffered prefix or splice data between batches.
If a text refill fails before the call can publish a result, characters decoded
before that failure remain buffered for the documented retry.
On a clean async with exit, aiogzip waits until all calls from a looping
producer finish and then closes normally. Concurrent close calls wait for the
same binary or text finalization. If the context body is already raising, exit
preserves that primary exception and attempts abortive owned-resource cleanup.
An active read, write, or flush that resumes after this abort raises OSError;
it never returns a truncated prefix or reports success for a gzip member that
can no longer receive its trailer. Cancellation delivered during cleanup still
propagates, and a failed abortive underlying close leaves the handle reportably
open so close() can be retried.
Once a handle has submitted the native close of an aiofiles file it owns (a
path, or a file object passed with closefd=True), cancellation cannot
abandon that close: it runs to completion before the cancellation propagates,
and a close failure becomes the cancellation's __cause__. During an
abortive exit, the handle is closed only if that close returned normally;
otherwise it stays reportably open and close() retries. A cancellation that
arrives while the exit is still waiting for an active call to settle no longer
skips the close: once that call has settled, the exit closes the file in the
same way and then propagates the cancellation. When the context body raised,
that cancellation keeps the body's exception in its chain: as its
__context__, or, when the close failed, as the context of the close failure
that is the cancellation's __cause__. Custom file objects' async close()
methods are cancelled cooperatively, as before.
Cancellation while a clean context exit is waiting for an active call also attempts abortive owned-resource cleanup before the cancellation propagates.
If that abortive cleanup fails, for example because a custom source's
close() raises, the failure is recorded as a note beginning
Context-exit cleanup also failed: instead of being dropped. Exception types
and precedence are unchanged:
- When the context body raised, an ordinary exception from the cleanup is noted on the body's exception. A cancellation or interrupt delivered during the cleanup propagates instead, as before.
- When a cancellation interrupted a clean exit, the cancellation propagates and any cleanup failure is noted on it, including a failure carried as the cause of a repeated cancellation. A repeated cancellation on its own adds no note.
To abort an active read or write on a custom file object, the exit cancels
the task running that call once, and consumes that request however the call
ends, even if the custom object swallows the cancellation. An enclosing
asyncio.timeout() in that task therefore still reports its own expiry as
TimeoutError, and a separate outside cancellation of the task is kept.
The maintained examples avoid same-handle overlap entirely: each shard task owns a separate file handle, and each transport codec operation is exhausted before the next begins.
Telling the decompression cap apart from corruption
A stream that exceeds max_decompressed_size (a decompression bomb, or just a
bigger file than expected) raises a plain OSError, not BadGzipFile —
the data may be perfectly valid gzip; it is the output budget that was
exceeded. Because the cap error is not a BadGzipFile, the handler split
above distinguishes the two cases by exception type alone. If you need to
single the cap out from real I/O errors within the OSError branch, match on
its stable message prefix:
try:
data = await aiogzip.read("untrusted.gz", max_decompressed_size=100 * 2**20)
except gzip.BadGzipFile:
raise # corrupt input: not the cap
except OSError as exc:
if str(exc).startswith("decompressed output exceeded max_decompressed_size"):
... # over budget: maybe retry with a larger cap
else:
raise
Where errors surface
Decompressed bytes reach the caller in one of three states:
- Provisional output: bytes from a member whose trailer has not been
checked yet. Normal reads deliver these as they decode; a later CRC,
ISIZE, truncation or trailing-data error can still invalidate them. - Validated output: every member has passed its trailer check once a
file read reaches the end of the stream (
read()orread(n)withn > 0returns empty without an error), once the operation returned byGzipDecoder.finish()is exhausted without an error, or oncedecompress_chunks()is exhausted without an error. An emptyread(0)and an exhaustedfeed()operation validate nothing. - Recovery data: bytes still readable after an integrity failure has been raised (see below). They are not validated and must not be trusted as that member's content.
Warning — successful output is not yet proof of integrity.
GzipDecoder.feed()anddecompress_chunks()can produce payload before the member trailer arrives. CRC,ISIZE, truncation, and trailing-data errors may surface later. ExhaustGzipDecoder.finish()or the complete async iterator before treating the stream as valid.
Streaming helpers validate lazily where the work happens: the iterator
returned by decompress_chunks() raises BadGzipFile only as the corrupt
region is consumed, and complete trailer validation happens when the iterator
is exhausted. verify() and inspect() scan the whole stream eagerly and
raise before returning a result. See
Processing untrusted gzip input
for a full defensive-reading recipe.
Recovery data after an integrity failure
High-level binary and text readers distinguish clean EOF from a poisoned terminal state. Their integrity-failure lifecycle is:
healthy
-> integrity failure raised
-> previously decoded recovery data may remain readable
-> recovery data is drained
-> stable terminal OSError
rewindable source:
absolute seek(0) may construct a fresh decoder and recover
non-rewindable source:
close and reopen/recreate the source
Recovery data is not proof that its gzip member is valid. Deflate output can
be emitted before a later CRC-32 or ISIZE trailer check fails, so applications
must not publish or trust that member's bytes merely because they were
readable. When codec metadata collection is enabled, trailer-validated prior
members remain in GzipDecoder.members; the failing member is absent. Those
records validate only the completed prefix, not the full concatenated stream.
After the recovery buffer is empty, later reads raise the same terminal
OSError instead of returning clean EOF. An absolute seek(0) can rebuild the
decoder only when the source can rewind (directly or through its replay cache).
Close and recreate a non-rewindable source. Decompression-limit failures,
cancellation, and unexpected internal failures poison the reader without
automatically enabling the integrity-failure salvage path.
For text readers, the only guaranteed recovery is calling seek(0) directly.
A cookie saved with tell() before the failure is not a recovery point:
seek(cookie) may raise the same terminal OSError instead of rewinding. Call
seek(0) first; once the reader has recovered, a saved cookie is an ordinary
position again.
An absolute text seek(), to a position or a tell() cookie, that fails or
is cancelled after it has moved the underlying binary reader makes the
reader terminal: later reads raise the same terminal OSError until
seek(0) recovers it. A seek that fails before moving the binary reader
leaves the text position where it was. If the binary reader is
still healthy, the text reader is unchanged and the seek can be retried.
Otherwise the binary reader's failure rules above apply: a gzip integrity
failure leaves the recovery state, so later reads return the recovered text
from the original position, and any other failure makes the reader terminal.
Either way, a failed absolute seek is never followed by text from the wrong
position. seek(0, SEEK_END) reads to the end, so its failures follow the
read() rules above, including the UnicodeDecodeError caution below.
Do not retry a text read that raised UnicodeDecodeError. The reader does
not become terminal, but the failed read has already consumed the whole chunk
of decompressed bytes it was decoding, so the next read resumes after that
chunk and silently skips text, possibly far more than the undecodable bytes
themselves. Call seek(0) to
start again from the beginning, or reopen the file with a different encoding
or an errors handler such as "replace".
Codec finalization and operation abandonment
Every state-changing GzipEncoder or GzipDecoder call reserves the codec and
returns a lazy CodecOperation. Engine errors, integrity failures, or closing
an operation before it is exhausted make that codec unusable. This is
intentional: the engine may already have consumed input, so silently reusing
the instance could skip bytes or emit a valid-looking trailer for incomplete
output.
Dropping an operation does not invoke state-changing finalizer behavior. The
codec remains reserved and another operation raises RuntimeError regardless
of garbage-collector timing. If the operation is reachable, exhaust it to
continue or call its idempotent close() method to abandon the stream. If the
caller no longer retains it, call the codec's idempotent discard() method;
this invalidates the operation and promptly releases its captured input and
codec state. Neither method resets the instance. Start again with a new codec.
GzipDecoder.finish() checks that the current deflate stream and trailer are
complete, that no partial next header remains, and that all CRC and ISIZE
values match. An incomplete or corrupt stream raises gzip.BadGzipFile and
poisons the decoder. After finish() succeeds, both another feed() and
another finish() raise ValueError.
The async wrappers follow the same rule internally. If cancellation occurs while a worker thread is advancing an operation, they wait until that worker has stopped before discarding state and re-raising cancellation. Source and cleanup failures do not replace the primary operation error.