Module io.sf.carte.tokenproducer

Package io.sf.carte.uparser

package io.sf.carte.uparser

A small event parser (microparser).

This package provides two low-level parsers that work with three different token handlers.

TokenProducer is a low-level token producer (parser) that can send events to a TokenHandler or TokenHandler2 handler. It should be used when the error handling only produces runtime (unchecked) exceptions.

On the other hand, TokenProducer3 deals with checked exceptions and can only work with a TokenHandler3 handler.

Most token handlers will report problems through error handlers and throw no checked exceptions, in which case you should use TokenProducer together with a TokenHandler or TokenHandler2. In other use cases your handler may want to throw checked exceptions, and then you must use TokenProducer3 with a TokenHandler3 instead.

As an example, this package includes the CommentRemovalHandler which is to be used with TokenProducer.

Class	Description
CommentRemovalHandler	A handler that removes comments.
LocatorAccess	Set a line-column location.
TokenControl	Offers methods to control the parsing process by the handler.
TokenHandler	A TokenHandler2 that is backwards-compatible with TokenProducer 1.x.
TokenHandler2	A TokenHandler3 that has no checked exceptions, backwards-compatible with TokenProducer 2.x.
TokenHandler3<E extends Exception>	To be implemented by listeners that handle the different events generated by the TokenProducer3.
TokenProducer	A simple parser that produces tokens from a String or Reader, and processes them through a user-provided handler.
TokenProducer3<E extends Exception>	A simple parser that produces tokens from a String or Reader, and processes them through a user-provided handler that may throw checked exceptions of type E.
TokenProducer3.CharacterCheck	Check whether a character codepoint would be a valid addition to a word.
TokenProducer3.SequenceParser<E extends Exception>	Basic access to the current sequence in this tokenizer.