processing.app.syntax

## Class TokenMarker

- Object
- - TokenMarker

- Direct Known Subclasses:
    PdeKeywords

      public abstract class TokenMarker
      extends Object

  A token marker that splits lines of text into tokens. Each token carries a length field and an indentification tag that can be mapped to a color for painting that token.

  For performance reasons, the linked list of tokens is reused after each line is tokenized. Therefore, the return value of `markTokens` should only be used for immediate painting. Notably, it cannot be cached.

- - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `deleteLines(int index, int lines)` Informs the token marker that line have been deleted from the document.
    `int` | `getLineCount()` Returns the number of lines in this token marker.
    `void` | `insertLines(int index, int lines)` Informs the token marker that lines have been inserted into the document.
    `boolean` | `isNextLineRequested()` Returns true if the next line should be repainted.
    `Token` | `markTokens(Segment line, int lineIndex)` A wrapper for the lower-level `markTokensImpl` method that is called to split a line up into tokens.
    `boolean` | `supportsMultilineTokens()` Returns if the token marker supports tokens that span multiple lines.

    - ### Methods inherited from class Object

      `equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Method Detail

    - #### markTokens

          public Token markTokens(Segment line,
                         int lineIndex)

      A wrapper for the lower-level `markTokensImpl` method that is called to split a line up into tokens.
      Parameters:
        `line` - The line
        `lineIndex` - The line number

    - #### supportsMultilineTokens

          public boolean supportsMultilineTokens()

      Returns if the token marker supports tokens that span multiple lines. If this is true, the object using this token marker is required to pass all lines in the document to the `markTokens()` method (in turn).

      The default implementation returns true; it should be overridden to return false on simpler token markers for increased speed.

    - #### insertLines

          public void insertLines(int index,
                         int lines)

      Informs the token marker that lines have been inserted into the document. This inserts a gap in the `lineInfo` array.
      Parameters:
        `index` - The first line number
        `lines` - The number of lines

    - #### deleteLines

          public void deleteLines(int index,
                         int lines)

      Informs the token marker that line have been deleted from the document. This removes the lines in question from the `lineInfo` array.
      Parameters:
        `index` - The first line number
        `lines` - The number of lines

    - #### getLineCount

          public int getLineCount()

      Returns the number of lines in this token marker.

    - #### isNextLineRequested

          public boolean isNextLineRequested()

      Returns true if the next line should be repainted. This will return true after a line has been tokenized that starts a multiline token that continues onto the next line.