Package org.apache.spark.sql.types.ops

Interface TypeApiOps

All Superinterfaces:
Serializable
All Known Implementing Classes:
TimeTypeApiOps
public interface TypeApiOps extends Serializable
Client-side (spark-api) type operations for the Types Framework.

This trait consolidates all client-side operations that a data type must implement to be usable in the Spark SQL API layer. All methods are mandatory because a type cannot function correctly without string formatting (needed for CAST to STRING, EXPLAIN, SHOW) or encoding (needed for Dataset[T] operations).

This single-interface design was chosen over separate FormatTypeOps/EncodeTypeOps traits to make it clear what a new type must implement - there is one mandatory interface, and it contains everything required. Optional capabilities (e.g., proto, Arrow, JDBC) are defined as separate traits that can be mixed in incrementally.

RELATIONSHIP TO TypeOps: - TypeOps (catalyst): Server-side operations - physical types, literals, conversions - TypeApiOps (spark-api): Client-side operations - formatting, encoding

The split exists because sql/api cannot depend on sql/catalyst. For TimeType, TimeTypeOps (catalyst) extends TimeTypeApiOps (sql-api) to inherit both sets of operations.

Since:
4.2.0
See Also:

  • Method Summary

    Modifier and Type
    Method
    Description
    DataType
    dataType()
    The DataType this Ops instance handles.
    String
    format(Object v)
    Formats an internal value as a display string.
    org.apache.spark.unsafe.types.UTF8String
    formatUTF8(Object v)
    Formats an internal value as a UTF8String.
    org.apache.spark.sql.catalyst.encoders.AgnosticEncoder<?>
    getEncoder()
    Returns the AgnosticEncoder for this type.
    String
    toSQLValue(Object v)
    Formats an internal value as a SQL literal string.

  • Method Details

    • dataType

      DataType dataType()
      The DataType this Ops instance handles.

    • format

      String format(Object v)
      Formats an internal value as a display string.

      Used by CAST to STRING, EXPLAIN output, SHOW commands.

      Parameters:
      v - the internal value (e.g., Long nanoseconds for TimeType)
      Returns:
      formatted string (e.g., "10:30:45.123456")

    • formatUTF8

      org.apache.spark.unsafe.types.UTF8String formatUTF8(Object v)
      Formats an internal value as a UTF8String.

      Default implementation wraps format(). Override for performance if needed.

      Parameters:
      v - (undocumented)
      Returns:
      (undocumented)

    • toSQLValue

      String toSQLValue(Object v)
      Formats an internal value as a SQL literal string.

      Parameters:
      v - the internal value
      Returns:
      SQL literal string (e.g., "TIME '10:30:00'")

    • getEncoder

      org.apache.spark.sql.catalyst.encoders.AgnosticEncoder<?> getEncoder()
      Returns the AgnosticEncoder for this type.

      Used by RowEncoder for Dataset[T] operations.

      Returns:
      AgnosticEncoder instance (e.g., LocalTimeEncoder for TimeType)