This repository was archived by the owner on Sep 9, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Sep 9, 2026. It is now read-only.
_pickle.PicklingError encountered when attempting to serialize BaseDoc type #1760
Description
Initial Checks
- I have read and followed the docs and still think this is a bug
Description
After using docarray.documents.helper.create_doc to create a BaseDoc type, this type cannot be serialized by Pickle:
from docarray.documents.helper import create_doc
MyDoc = create_doc(
'MyDoc',
title=(str, ...),
title_tensor= (Optional[NdArray[128]], ...),
)
buffer = pickle.dumps(MyDoc)Out:
_pickle.PicklingError: Can't pickle <class 'docarray.documents.helper.MyDoc'>: attribute lookup MyDoc on docarray.documents.helper failedExample Code
No response
Python, Pydantic & OS Version
docarray: 0.37.1
python: 3.10.12
Activity
JoanFM commentedon Aug 23, 2023
Hello @beidongjiedeguang,
May I ask what are you trying to do here? Here you are pickling a class, is it the intended behavior?
KenyonY commentedon Aug 23, 2023
Hi @JoanFM ,
Yes, because this type is dynamically created. If it is not saved, it will need to be recreated the next time
DocIndex(such as HnswDocumentIndex) is launched. Therefore, I must save the metadata for creating this class. So if possible, I would like to directly save this type.JoanFM commentedon Aug 24, 2023
This is the same problem as seen in Pydantic here pydantic/pydantic#4178
JoanFM commentedon Aug 24, 2023
So, a solution is to declare it with module parameter:
This works for me