You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support for stream (equivalent to json.load/json.dump) #80
That seems to be what the stdlib json and ujson do -- just pass f.read() into the loads().
We could probably take advantage of RapidJSON's file stream class to operate on a file descriptor. Or maybe implement a custom RapidJSON stream object that would go between a Python file like object, but it seems like that could spend a lot of time in Python.
@xavierhardy What is your use case? Is it actual files or other things?
Just for fun^H^H^Hlearning some more C++, I tried a few approaches, and this is currently the best outcome, that implements the custom stream (only for load() right now): it has tricky bugs and I didn't benchmark it.
Anyway, I'm not sure I will be able to spend more time on that in the near future, and it surely needs some love before being considered for merge in master.
A slightly better and more complete implementation here. All tests passes, but there are still some shortcomings: I'm not sure I will ever be able to fix them though.
I was able to implement the text-based streaming and figured out how to configure the writer to support ensure_ascii=True. So the remaining issue seems to be the handling of possible errors while reading/writing the underlying stream.
I think I solved that last issue and everything seems going well, so barring possible objections I'm going to merge PR #92 in the first week of the new year, or so.
Given that it changes the signature of existing loads() and dumps() functions (following stdlib json module the flags are now keyword only arguments), it will eventually land in version 0.4.
Activity
lelit commentedon Sep 22, 2017
Yes.
Ideally it should use the stream interface of Rapidjson, not sure if my C++ skill is up-to-the-task...
As a fallback, I can easily add a
load()function that accepts a file-like object and reads its content behind the scene...kenrobbins commentedon Sep 29, 2017
That seems to be what the stdlib json and ujson do -- just pass
f.read()into theloads().We could probably take advantage of RapidJSON's file stream class to operate on a file descriptor. Or maybe implement a custom RapidJSON stream object that would go between a Python file like object, but it seems like that could spend a lot of time in Python.
@xavierhardy What is your use case? Is it actual files or other things?
lelit commentedon Sep 30, 2017
Just for fun^H^H^Hlearning some more C++, I tried a few approaches, and this is currently the best outcome, that implements the custom stream (only for load() right now): it has tricky bugs and I didn't benchmark it.
Anyway, I'm not sure I will be able to spend more time on that in the near future, and it surely needs some love before being considered for merge in master.
Implement stream-based load()
lelit commentedon Dec 22, 2017
A slightly better and more complete implementation here. All tests passes, but there are still some shortcomings: I'm not sure I will ever be able to fix them though.
lelit commentedon Dec 22, 2017
See also Tencent/rapidjson#1149
lelit commentedon Dec 24, 2017
I was able to implement the text-based streaming and figured out how to configure the writer to support
ensure_ascii=True. So the remaining issue seems to be the handling of possible errors while reading/writing the underlying stream.lelit commentedon Dec 27, 2017
I think I solved that last issue and everything seems going well, so barring possible objections I'm going to merge PR #92 in the first week of the new year, or so.
Given that it changes the signature of existing
loads()anddumps()functions (following stdlibjsonmodule the flags are now keyword only arguments), it will eventually land in version 0.4.lelit commentedon Jan 5, 2018
I just released v0.4 that implements this feature.
Thank you!
xavierhardy commentedon Feb 10, 2018
Great! Thanks a lot!