gh-155649: Bound LZMA dictionary size to prevent memory DoS - #155650
Open
DeseretSaint wants to merge 1 commit into
Open
gh-155649: Bound LZMA dictionary size to prevent memory DoS#155650DeseretSaint wants to merge 1 commit into
DeseretSaint wants to merge 1 commit into
Conversation
liblzma allocates the full LZMA dictionary up front when a decompressor is created, and the dictionary size comes from the stream header. An untrusted stream could declare a multi-gigabyte dictionary (up to 4 GiB) and force that allocation from a handful of bytes, exhausting memory. Two gaps are closed: - parse_filter_spec_lzma() now rejects dict_size above liblzma's own maximum (LZMA_DICT_SIZE_MAX, 1.5 GiB) with LZMAError. This covers raw LZMA1/LZMA2 filter chains, including zipfile's LZMA members. - LZMADecompressor now defaults memlimit to LZMA_DICT_SIZE_MAX instead of UINT64_MAX, bounding the FORMAT_AUTO/FORMAT_ALONE/FORMAT_XZ paths (e.g. tarfile's compression detection and lzma.decompress()). An explicit memlimit can still be passed to adjust the bound. Fixes OSS-Fuzz reports (python3-libraries): - fuzzer-zipfile OOM: https://issues.oss-fuzz.com/issues/495472861 - fuzzer-tarfile OOM: https://issues.oss-fuzz.com/issues/482161128
|
The following commit authors need to sign the Contributor License Agreement: |
Documentation build overview
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
liblzma allocates the full LZMA dictionary up front when a decompressor is created, and the dictionary size comes from the stream header. An untrusted stream can declare a multi-gigabyte dictionary (up to 4 GiB) and force that allocation from a handful of bytes — a memory denial of service.
Both crash paths were reported by OSS-Fuzz (python3-libraries, public):
fuzzer-zipfileOOM: a 1088-byte zip whose LZMA member declares a ~2 GiB dictionary → https://issues.oss-fuzz.com/issues/495472861fuzzer-tarfileOOM: a 13-byte input whose first byte is a valid LZMA-alone properties byte with a ~4 GiB dictionary → https://issues.oss-fuzz.com/issues/482161128Directly reachable too:
lzma.decompress()/LZMADecompressor(FORMAT_AUTO)on untrusted bytes, andLZMADecompressor(FORMAT_RAW, filters=[{"dict_size": ...}]).Changes
parse_filter_spec_lzma()rejectsdict_sizeabove liblzma's own maximum (LZMA_DICT_SIZE_MAX, 1.5 GiB) withLZMAError— covers raw LZMA1/LZMA2 filter chains, includingzipfile's LZMA members (the dictionary is parsed from the 5-byte properties header before any data is decompressed).LZMADecompressordefaultsmemlimittoLZMA_DICT_SIZE_MAXinstead ofUINT64_MAX— bounds theFORMAT_AUTO/FORMAT_ALONE/FORMAT_XZpaths where the header is parsed inside liblzma (tarfile compression detection,lzma.decompress(),LZMAFile).An explicit
memlimitcan still be passed to raise or lower the bound.Compatibility: liblzma presets cap at 64 MiB dictionaries; 1.5 GiB is liblzma's own maximum supported dictionary size, so no legitimate stream is affected.
Tests: new regression tests in
test_lzma.py,test_zipfile(crafted bomb zip →LZMAError), andtest_tarfile(13-byte fake-LZMA input →ReadError).test_lzma test_zipfile test_tarfileall pass (1,487 tests).Companion fuzzer-harness change: https://github.com/python/library-fuzzers/pull/… (zipfile fuzzer now tolerates
lzma.LZMAError).