fix: preserve Parquet root field IDs after nested pruning - #7081

Which issue does this PR close?

Closes #7031.

Rationale for this change

On Spark 3.4, nested schema pruning removes root field ID metadata from the relation’s data schema, while the scan’s required schema retains it. Comet serializes the relation schema, causing renamed structs and arrays of structs to be resolved by name instead of field ID. This produces incorrect NULL values, drops matching rows during filtering, and affects counts.

What changes are included in this PR?

• Preserve root field IDs from the required schema when constructing the native data schema.
• Retain table field types, nullability, other metadata, and projection indexes.
• Use an indexed lookup to avoid quadratic searches.
• Preserve Spark 3.5+ behavior when pruning also removes the required schema’s root ID.
• Add regression coverage for renamed structs and arrays of structs, partitioned data, projections, filter-only nested fields, nullable counts, and pruning and field-ID reading settings.

How are these changes tested?

• Focused serde and Parquet field-ID tests passed on Spark 3.4, 3.5, and 4.1: 12 tests per profile, with no failures, cancellations, or ignored tests.
• Strict-warning compilation passed with Spark 3.5 and Delta.
• [Full fork CI passed] at commit 9a21021: 137 successful jobs, 13 skipped under existing workflow conditions, and 0 failures. Strict Scala warnings, Spark 4.1 expressions, and Required Checks all passed.