Thank you for a really great project!
Unfortunately I have observed that when validating invalid events the rapidjson.Validator leaks a little memory. Here is some sample code to reproduce it (tested on Python 3.7.3, MacOS Mojave, with python-rapidjson 0.8.0).
import rapidjson
import gc
import tracemalloc
schema = """{
"$schema": "http://json-schema.org/draft-04/schema#",
"required": ["id", "name"],
"type": "object",
"properties": {
"id": {"type": "integer"},
"name": {"type": "string"}
}
}""".encode("utf-8")
obj = """{
"id": 50
}""".encode("utf-8")
validate = rapidjson.Validator(schema)
snapshots = []
# start the test
tracemalloc.start()
for _ in range(2):
for _ in range(500000):
try:
validate(obj)
except rapidjson.ValidationError:
pass
# garbage collect before taking snapshot to exclude memory that is
gc.collect()
snapshots.append(tracemalloc.take_snapshot())
# report
for stat in snapshots[1].compare_to(snapshots[0], "traceback")[:10]:
print(stat)
This returns something like:
minimal_memory_test.py:27: size=123 MiB (+61.5 MiB), count=1999982 (+1000000), average=64 B
/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tracemalloc.py:397: size=600 B (+600 B), count=4 (+4), average=150 B
/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tracemalloc.py:534: size=488 B (+488 B), count=2 (+2), average=244 B
/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tracemalloc.py:291: size=112 B (+112 B), count=2 (+2), average=56 B
/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tracemalloc.py:532: size=128 B (+64 B), count=2 (+1), average=64 B
minimal_memory_test.py:32: size=464 B (+32 B), count=2 (+1), average=232 B
minimal_memory_test.py:31: size=59 B (+0 B), count=1 (+0), average=59 B
minimal_memory_test.py:24: size=48 B (+0 B), count=1 (+0), average=48 B
minimal_memory_test.py:25: size=28 B (+0 B), count=1 (+0), average=28 B
Note the ~64B leak for the line validate(obj).
Thank you for a really great project!
Unfortunately I have observed that when validating invalid events the
rapidjson.Validatorleaks a little memory. Here is some sample code to reproduce it (tested on Python 3.7.3, MacOS Mojave, with python-rapidjson 0.8.0).This returns something like:
Note the ~64B leak for the line
validate(obj).