Skip to content

Commit f27721f

Browse files
CopilotsyoyoCopilotCopilotclaude
authored
Rewrite OBJ/MTL parser to byte-stream based with StreamReader, add mmap support, and integrate Windows long path/Unicode support (#416)
* Initial plan * Add max_iter guards to parsing loops and mmap support for .obj/.mtl loading Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Rewrite parser from line-based to byte-stream-based with EOF guards Replace safeGetline + raw const char* token pointer scanning with a StreamReader class that wraps (buf, len, idx) with bounds-checked peek/advance/match operations. Every byte access is guarded by an EOF check. Key changes: - Add StreamReader class for bounds-checked text parsing - Add sr_* parse functions that use StreamReader instead of const char** - Add LoadMtlInternal, LoadObjInternal, LoadObjWithCallbackInternal using StreamReader; public APIs wrap these - mmap paths construct StreamReader directly from mapped memory - Remove safeGetline (no longer needed) - Keep old parse functions for backward compat with public ParseTextureNameAndOption API - Keep IS_SPACE, IS_NEW_LINE, IS_DIGIT macros for old parse functions Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Rewrite parser to byte-stream based with StreamReader for bounds-checked parsing - Add StreamReader class wrapping (buf, length) with EOF-guarded peek/advance/match - Add sr_* parse functions using StreamReader instead of const char** pointers - Add LoadMtlInternal, LoadObjInternal, LoadObjWithCallbackInternal taking StreamReader - mmap paths construct StreamReader directly from mapped memory - Remove safeGetline (no longer needed) - Keep old parse functions + macros for ParseTextureNameAndOption backward compat Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Update tiny_obj_loader.h Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Integrate changes from release: Windows long paths, texcoord w-component, BOM in callback, test fixes - Add UTF8ToWchar/LongPathW helpers for Windows long path and Unicode support - Add Windows long path support (#ifdef _WIN32) in non-mmap ifstream fallbacks - Add vt_w vector and optional w component parsing in LoadObjInternal - Fix texcoord_ws.swap(vt) -> texcoord_ws.swap(vt_w) corruption - Add BOM stripping to LoadObjWithCallbackInternal - Fix invalid-relative-texture-vertex-index.obj -> invalid-relative-texture-index.obj - Add test models: texcoord-w.obj, texcoord-w-mixed.obj, utf8-path-test.obj/.mtl - Add tests: texcoord_w, texcoord_w_mixed, utf8_path, long_path, callback_BOM Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Add clang-style parse error reporting with early return on failure - Add column tracking (col_num_) to StreamReader, updated in all position-advancing methods (get, advance, skip_space, etc.) - Add current_line_text(), format_error(filename, msg) for clang-style diagnostics (file:line:col with source line and caret) - Add error stack to StreamReader (push_error, has_errors, get_errors) - Add error-reporting sr_parse overloads that return bool for success/failure: sr_parseInt, sr_parseReal, sr_parseReal2/3, sr_parseVertexWithColor (returns -1 on error) - Wire error-reporting overloads into LoadMtlInternal (Ka, Kd, Ks, Ni, Ns, illum, d, Tr, PBR properties) with early return on parse failure; change LoadMtlInternal from void to bool - Wire error-reporting overloads into LoadObjInternal (v, vn, vt, vw) with early return on parse failure - Upgrade existing error messages (face, line, points, skin weight) to use sr.format_error() for consistent clang-style output - Pass filename through call chain: LoadObj -> LoadObjInternal, MaterialFileReader -> LoadMtlInternal - Add warning_context.filename for fixIndex warnings - Add 6 new tests: column tracking, format_error output, error stack, malformed vertex/mtl errors, backward compatibility Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Polish: hoist warning_context out of loop, fix no-op test - Move warning_context initialization (warn, filename) before the main parse loop in LoadObjInternal; only update line_number per iteration. Avoids a std::string copy of filename on every face/line/points line. - Replace test_parse_error_backward_compat (was just TEST_CHECK(true)) with an actual round-trip test: parse valid OBJ, verify no errors and correct vertex count. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Update tiny_obj_loader.h Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Update tests/tester.cc Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix sr_parseTagTriple slash-delimited count parsing and add regression assertions Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Fix high/medium review findings: complete attrib reset, usemtl delimiter check, remove duplicate windows.h and dead errss Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Address low severity review items: document read_token, unify line_num naming, extract sr_skipTagField helper, refactor mmap to RAII MappedFile Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Harden MappedFile with is_mapped flag to prevent unmapping string literals Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Run GPT-5.3-Codex comprehensive review Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Remove accidental CTest artifact and ignore Testing/Temporary Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Fix stream offset handling and add oversized stream guards/tests Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Final deep-review fixes for mtllib callback failure path Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Stage-2 deep review fixes for mtllib empty tokens and mmap size guard parity Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Stage-3 deep review: preserve mtllib backslashes and harden stream size check Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Stage-4 review: fix callback usemtl parsing and harden int parsing Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Stage-5 review: harden size parsing and Windows mmap path Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> * Final polish: fix overflow guards, zero-length normalize, and minor inconsistencies - Fix potential size_t wraparound in StreamReader::match/char_at/peek_at bounds checks - Tighten exponent overflow guard to account for subsequent digit addition - Guard Normalize() against division by zero on zero-length vectors - Fix callback API usemtl advance(7)->advance(6) to match regular API - Fix test registration typo: "whitespece" -> "whitespace" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: syoyo <18676+syoyo@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Syoyo Fujita <syoyo@lighttransport.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent dda7950 commit f27721f

File tree

4 files changed

+1888
-542
lines changed

4 files changed

+1888
-542
lines changed

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,5 @@ build/
1111
/python/_version.py
1212
/tinyobjloader.egg-info/
1313
**/__pycache__/
14+
15+
/Testing/Temporary/

‎Testing/Temporary/CTestCostData.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
---

0 commit comments

Comments
 (0)