In the standard json library, bytes (builtin binary type) objects are passed to JSONEncoder.default when they are encountered. Rapidjson does not pass them into Encoder.default, instead it always tries to encode them as utf-8.
I have quite a few places where I am storing file data as a binary blob in a mongodb database. Services can then serialize that to json by converting it to a hex string. However, because I cannot catch bytes values in a custom implementation of default, I get an error.
Example:
from json import JSONEncoder
from rapidjson import Encoder as RAPIDEncoder
class CustomJSONEncoder(JSONEncoder):
def default(self, o):
print("OBJECT PASSED REFERENCE")
if isinstance(o, bytes):
return o.hex()
else:
return o
class CustomRAPIDEncoder(RAPIDEncoder):
def default(self, o):
print("OBJECT PASSED RAPID")
if isinstance(o, bytes):
return o.hex()
else:
return o
json_reference = CustomJSONEncoder()
rapid_implementation = CustomRAPIDEncoder()
if __name__ == '__main__':
with open("./automation.png", mode="rb") as f:
bin_data = f.read()
to_encode = {"data": bin_data}
encoded_reference = json_reference.encode(to_encode)
print("ENCODED BY JSON:", encoded_reference)
encoded_rapid = rapid_implementation(to_encode)
print("ENCODED BY RAPID:", encoded_rapid)
Output:
OBJECT PASSED REFERENCE
ENCODED BY JSON: {"data": "89504e470d0a1a0a000....
Traceback (most recent call last):
File "...", line 38, in <module>
encoded_rapid = rapid_implementation(to_encode)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
Here is the png used in this example for binary data:
In the standard json library, bytes (builtin binary type) objects are passed to JSONEncoder.default when they are encountered. Rapidjson does not pass them into Encoder.default, instead it always tries to encode them as utf-8.
I have quite a few places where I am storing file data as a binary blob in a mongodb database. Services can then serialize that to json by converting it to a hex string. However, because I cannot catch bytes values in a custom implementation of default, I get an error.
Example:
Output:
Here is the png used in this example for binary data: