Implement TclObject support in tkinter varname_converter - #8465

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

rg -n -C 6 \
  'struct TclObject|TclObject\s*\{|impl Drop for TclObject|Tcl_IncrRefCount|Tcl_DecrRefCount|Tcl_GetVar2Ex' \
  crates/stdlib/src/tkinter.rs

Repository: RustPython/RustPython

Length of output: 4453

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo "== tkinter.rs relevant section =="
sed -n '55,95p;145,175p;300,335p' crates/stdlib/src/tkinter.rs

echo
echo "== pyclass macro impl for TclObject =="
rg -n -C 4 'impl TclObject|TclObject;' crates/stdlib/src/tkinter.rs

echo
echo "== pyobject refcount methods in RustPython source =="
rg -n 'pub fn.*dec[_ ]?ref|pub fn.*inc[_ ]?ref|fn.*decref|fn.*incref|PyPayload|PyRef' crates -g '*.rs' | head -n 80

Repository: RustPython/RustPython

Length of output: 11508

🌐 Web query:

Tcl_GetVar2Ex Tcl_Obj reference count caller owns returned Obj Tcl_DecrRefCount Tcl_IncrRefCount documentation

💡 Result:

When you call Tcl_GetVar2Ex, it returns a pointer to a Tcl_Obj that represents the current value of the variable [1][2]. Crucially, the reference count of the returned Tcl_Obj is not incremented by the function to account for this returned pointer; the caller does not own the object, and Tcl retains control over its lifetime [3][4]. The object returned by Tcl_GetVar2Ex is held by the variable itself [5][2]. Because the caller does not own this reference, the object is subject to being freed or modified by subsequent Tcl commands or variable operations [3][4]. To safely retain or use the returned Tcl_Obj beyond the immediate scope of the call: 1. You must explicitly call Tcl_IncrRefCount on the returned pointer to increment its reference count [6][4]. This marks the reference as owned by your code, ensuring Tcl does not free it prematurely [6]. 2. Once you are finished using the object, you must call Tcl_DecrRefCount to decrement the reference count [6]. If the reference count drops to zero, the storage for the object will be automatically reclaimed [6]. In summary, Tcl_GetVar2Ex returns a "borrowed" reference [3]. If you intend to keep a reference to the object, you are responsible for managing its reference count manually using Tcl_IncrRefCount and Tcl_DecrRefCount [6][3][4].

Citations:

• 1: https://man.archlinux.org/man/extra/tcl/Tcl_GetVar2Ex.3.en
• 2: https://www.tcl-lang.org/man/tcl9.0/TclLib/SetVar.html
• 3: http://svn.code.sf.net/p/brlcad/code/brlcad/tags/rel-6-0/libtcl8.3/generic/tclVar.c
• 4: https://stackoverflow.com/questions/47780726/using-c-and-tcl-error-when-getting-data-from-file-with-tcl-getvar2
• 5: https://www.tcl-lang.org/man/tcl8.7/TclLib/SetVar.html
• 6: https://www.tcl-lang.org/man/tcl8.6.13/TclLib/Object.htm

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

rg -n -C 6 \
  'varname_converter|Tcl_GetString|Tcl_GetUnicodeFromObj|to_str\(\)|CString::new' \
  crates/stdlib/src/tkinter.rs

fd -e rs . | xargs -r rg -n \
  'TclObject|varname_converter|U\+0000|C0 80|embedded null|NUL' || true

Repository: RustPython/RustPython

Length of output: 43692

🌐 Web query:

Tcl_GetString returns Tcl internal string representation NULL encoded as C0 80 Tcl object documentation

💡 Result:

Yes, it is accurate that Tcl represents the NUL character (\u0000) using the two-byte sequence C0 80 (hex) within its internal string representation [1][2][3]. In Tcl, strings are stored internally using a modified UTF-8 encoding [4]. In standard UTF-8, the NUL character is a single byte (0x00), which would terminate C strings and interfere with Tcl's string management routines. To avoid this, Tcl encodes the NUL character as the two-byte sequence 0xC0 0x80 [1][2][3]. This sequence is considered an "overlong" or "non-shortest form" encoding, which is technically illegal in standard UTF-8 as defined by the Unicode Consortium, but is required by Tcl to allow NUL bytes to be stored safely within string objects [5][3]. When you use Tcl_GetString or Tcl_GetStringFromObj to retrieve the string representation of a Tcl object, any embedded NUL characters will appear as this C0 80 sequence [1][4]. If your application requires handling raw binary data or truly literal NUL bytes without this conversion, you should avoid using Tcl_Obj string representation functions and instead use binary-safe types such as Tcl byte arrays (Tcl_NewByteArrayObj) or appropriate encoding/translation settings (e.g., fconfigure -translation binary) [1][5][4][6].

Citations:

• 1: https://github.com/tcltk/tcl/blob/ef3ea0ba/doc/StringObj.3
• 2: https://wiki.tcl-lang.org/page/UTF%2D8+bit+by+bit?R=0
• 3: https://wiki.tcl-lang.org/page/Unicode+and+UTF%2D8?R=0
• 4: https://comp.lang.tcl.narkive.com/PLFSwVLt/invalid-utf-handling-behavior-between-8-6-10-and-8-6-12-13
• 5: https://groups.google.com/g/comp.lang.tcl/c/lyElSLp0Bpg
• 6: https://my.f5.com/manage/s/article/K7875

🌐 Web query:

CPython tkinter cTkinter.py TkAppGetVar Tcl_GetStringVar Tcl_GetVar2Ex variable object bytes conversion

💡 Result:

In CPython's tkinter module, the interaction with Tcl variables and the conversion of data between Tcl objects (Tcl_Obj) and Python objects are handled primarily within the _tkinter C extension module [1][2]. The core mechanism for retrieving Tcl variables from C is Tcl_GetVar2Ex [3][4]. Unlike older functions such as Tcl_GetVar (which return a string representation), Tcl_GetVar2Ex is preferred because it returns a pointer to a Tcl_Obj, which is significantly more efficient and preserves the underlying Tcl object type information [3][5][4]. When CPython's tkinter operates in its default object-aware mode (enabled by the wantobjects flag), it attempts to automatically convert Tcl_Obj types to appropriate native Python types (such as str, int, float, bool, and bytes) using internal conversion logic, often centered around a function named FromObj in Modules/_tkinter.c [6][2][7]. Key points regarding variable and object conversion: 1. Efficiency and Type Integrity: Using Tcl_GetVar2Ex allows tkinter to avoid unnecessary string conversions and leverage Tcl's internal type-caching mechanisms [3][5][8]. 2. FromObj Conversion: When tkinter retrieves a Tcl_Obj, it inspects its type. If the type is recognized, it is converted to the corresponding Python equivalent. For example, bytearray objects in Tcl can be converted to Python bytes, and numeric types are mapped to Python int or float [6][7]. 3. Fallback: If a Tcl_Obj type is not explicitly handled by the conversion logic, it is typically wrapped in an opaque _tkinter.Tcl_Obj instance, which may require the user to explicitly call methods like str or get to access the underlying value [6][2][8]. 4. Evolution: The handling of these types is continuously updated to maintain compatibility with newer Tcl/Tk versions (e.g., Tcl 8.7 and 9.0), which may introduce new object types or change how older types are represented [6][7][9]. Python-level wrappers (such as StringVar, IntVar, etc., defined in tkinter.init.py) facilitate this by calling the underlying globalgetvar or globalsetvar methods, which internally interact with these C-level Tcl variable functions [10][11].

Citations:

• 1: https://github.com/python/cpython/blob/main/Modules/_tkinter.c
• 2: https://bugs.python.org/msg279300
• 3: https://man.archlinux.org/man/extra/tcl/Tcl_GetVar.3.en
• 4: https://manpages.org/tcl_getvar2ex/3
• 5: https://github.com/tcltk/tcl/blob/ef3ea0ba/doc/SetVar.3
• 6: tkinter: return str and numbers for more Tcl object types python/cpython#153513
• 7: Tkinter: C API changes are needed for Tcl 8.7 and 9.0 value types python/cpython#103194
• 8: https://bugs.python.org/issue16809
• 9: gh-103194: Fix Tkinter’s Tcl value type handling for Tcl 8.7/9.0 python/cpython#103846
• 10: https://github.com/python/cpython/blob/ad4ed872415d00fcdfaa52a08108ec752b115000/Lib/tkinter/__init__.py
• 11: https://github.com/python/cpython/blob/main/Doc/library/tkinter.rst

🌐 Web query:

site:github.com/python/cpython Modules/_tkinter.c FromObj Tcl_GetString to_pystringbytes

💡 Result:

In the Python CPython source file Modules/_tkinter.c, FromObj is a central function responsible for converting Tcl objects (Tcl_Obj) into native Python objects [1][2][3]. It inspects the typePtr of a Tcl_Obj to determine the appropriate conversion strategy (e.g., boolean, integer, float, list, or string) [1][2][4]. Regarding the specific identifiers and functions mentioned: Tcl_GetString is a standard Tcl C API function used to retrieve the string representation of a Tcl object [5]. In Modules/_tkinter.c, variants like Tcl_GetStringFromObj are frequently used to obtain both the string pointer and its length, which is safer than accessing the internal bytes member directly, especially to avoid issues with embedded null characters or invalid states [6][7]. to_pystringbytes does not exist as a standard function name within Modules/_tkinter.c. It is likely a misremembered or hypothetical name for one of the internal conversion helpers such as: PyBytes_FromStringAndSize: Used within FromObj when encountering a Tcl ByteArrayType to convert the Tcl byte array into a Python bytes object [1][2][3]. unicodeFromTclObj / unicodeFromTclStringAndSize: Internal functions used to convert Tcl string representations into Python unicode/string objects [1][2][4]. The FromObj function is critical to Python's tkinter "wantobjects" mode [8][9]. When wantobjects is enabled (the default), FromObj attempts to map Tcl types to natural Python types (int, float, str, bytes, tuple) [8][9][10]. For types it cannot resolve to a native Python type, it typically wraps the Tcl_Obj in an opaque Python object, preserving the Tcl-side object for later use [11][9]. This design recognizes that while Tcl often treats all values as strings internally, preserving their specific Tcl type can be necessary for performance and correct round-trip behavior in Tkinter applications [5][9].

Citations:

• 1: https://github.com/python/cpython/blob/main/Modules/_tkinter.c
• 2: https://github.com/python/cpython/blob/38df97a03c5102e717a110ab69bff8e5c9ebfd08/Modules/_tkinter.c
• 3: https://github.com/python/cpython/blob/master/Modules/_tkinter.c
• 4: https://github.com/python/cpython/blob/adf24bd835ed8f76dcc51aa98c8c54275e86965b/Modules/_tkinter.c
• 5: comparing versions - one a float python/cpython#38396
• 6: https://github.com/python/cpython/blob/8feb7ab77c80968a6de6079299a39b0494b1701b/Modules/_tkinter.c
• 7: Tkinter binding involving Control-spacebar raises unicode error python/cpython#45369
• 8: gh-153513: Return str and numbers for more Tcl object types in _tkinter python/cpython#153514
• 9: tkinter: return str and numbers for more Tcl object types python/cpython#153513
• 10: Tkinter: Don't stringify callback arguments python/cpython#66410
• 11: Accessing a tkinter object's string representation converts the object to a string on Windows python/cpython#101830

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

sed -n '130,210p' crates/stdlib/src/tkinter.rs
printf '\n=== lines 260-360 ===\n'
sed -n '260,360p' crates/stdlib/src/tkinter.rs
printf '\n=== TkInterpreter/tcl object conversion context ===\n'
sed -n '320,430p' crates/stdlib/src/tkinter.rs

python3 - <<'PY'
bytes_c080 = b'\xC0\x80'
try:
    print("C080 as standard UTF-8:", bytes_c080.decode("utf-8"))
except UnicodeDecodeError as e:
    print("C080 decoding error:", type(e).__name__, e)

bytes_null_utf8 = b"\x00"
print("U+0000 UTF-8 byte length:", len(bytearray(bytes_null_utf8)))
PY

Repository: RustPython/RustPython

Length of output: 11035