Skip to content

Fixed an issue decoding Dataclass and TypedDict instances - #1126

Open
mishmish66 wants to merge 1 commit into
msgspec:mainfrom
mishmish66:main
Open

Fixed an issue decoding Dataclass and TypedDict instances#1126
mishmish66 wants to merge 1 commit into
msgspec:mainfrom
mishmish66:main

Conversation

@mishmish66

Copy link
Copy Markdown

To reproduce try this snippet (in a fresh process):

from dataclasses import dataclass
import msgspec

@dataclass
class Foo:
    π: float
    μ: int

foo = msgspec.json.decode(b'{"\xcf\x80":3.14,"\xce\xbc":0}', type=Foo)
print(foo)

Can't decode π because it is unicode and hasn't been loaded into the unicode cache yet

But this:

from dataclasses import dataclass
import msgspec

@dataclass
class Foo:
    π: float
    μ: int

print(msgspec.json.encode(Foo(3.1416, 1)))
foo = msgspec.json.decode(b'{"\xcf\x80":3.14,"\xce\xbc":0}', type=Foo)
print(foo)

works fine since the cache is populated during encoding.
This PR should have a fix for this and also a test that fails the old unfixed version.

… unicode fields wouldn't decode properly since the nocheck path was run without cached unicode values.

@provinzkraut provinzkraut left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhm. I'm not sure if this is the best way to fix the issue. Invoking PyUnicode_AsUTF8 purely for its side effect doesn't seem to great. We're not doing this elsewhere for UTF8 either, so I would suggest not introducing this pattern for this particular path only.

Have you checked if other types (Struct, NamedTuple, attrs classes) are also affected? If not, dataclasses / TypedDicts should handle this the same way these do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants