OS: MacOS High Sierra
Python version: 3.7.1
Rapidjson version: 0.6.3
It seems I am facing a memory leak problem when using rapidjson.loads and utilizing an object hook. Here is an example code:
import rapidjson
from datetime import timedelta
import tracemalloc
tracemalloc.start()
def object_hook(td):
if '__td__' in td:
return timedelta(td['__td__'])
else:
return td
def parse_json():
data = []
for i in range(1, 100):
data.append({"name": f"a{i}", "timestamp": timedelta(seconds=i)})
a = rapidjson.dumps(
data,
datetime_mode=rapidjson.DM_ISO8601,
default=lambda td: {"__td__": td.total_seconds()} if isinstance(td, timedelta) else td,
)
data = rapidjson.loads(
a,
datetime_mode=rapidjson.DM_ISO8601,
object_hook=object_hook
)
return data
print("before")
if tracemalloc.is_tracing():
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
print([str(x) for x in top_stats[:10]])
for _ in range(1000):
parse_json()
print("after")
if tracemalloc.is_tracing():
snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')
print([str(x) for x in top_stats[:10]])
After running this I get: 'memleaktest.py:9: size=3867 KiB, count=99000, average=40 B' as the first line of the tracemalloc output. I tried using objgraph as well, but it did not show any Python objects, so it seems like a C++ malloc issue here.
I am using rapidjson in production and my, fairly simple, API is slowly eating memory and this is the prime suspect (getting a tracemalloc pointer to the line basically the same as the one here).
Is this working as intended? Do you need any more information to pinpoint the problem?
OS: MacOS High Sierra
Python version: 3.7.1
Rapidjson version: 0.6.3
It seems I am facing a memory leak problem when using rapidjson.loads and utilizing an object hook. Here is an example code:
After running this I get:
'memleaktest.py:9: size=3867 KiB, count=99000, average=40 B'as the first line of the tracemalloc output. I tried usingobjgraphas well, but it did not show any Python objects, so it seems like a C++ malloc issue here.I am using rapidjson in production and my, fairly simple, API is slowly eating memory and this is the prime suspect (getting a tracemalloc pointer to the line basically the same as the one here).
Is this working as intended? Do you need any more information to pinpoint the problem?