Interface TokenProducer.SequenceParser

Enclosing class:
TokenProducer

public static interface TokenProducer.SequenceParser
Basic access to the current sequence in this tokenizer.

It is intended to be used by objects implementing TokenProducer.CharacterCheck.

  • Method Summary

    Modifier and Type
    Method
    Description
    Give the contents of the word currently being formed.
    boolean
    Was the previous codepoint added to the current word sequence as a CharacterCheck-accepted character ?
    void
    Empty the current sequence.
  • Method Details

    • currentSequence

      CharSequence currentSequence()
      Give the contents of the word currently being formed.
      Returns:
      the contents of the word currently being formed.
    • isPreviousCpWCharacter

      boolean isPreviousCpWCharacter()
      Was the previous codepoint added to the current word sequence as a CharacterCheck-accepted character ?

      A CharacterCheck-accepted character is different to normal word characters, and is only added to a word if it is allowed by the CharacterCheck process.

      Returns:
      true if the previous codepoint was added to the current word sequence as an external, CharacterCheck-accepted character.
      See Also:
    • resetCurrentSequence

      void resetCurrentSequence()
      Empty the current sequence.