Options
All
  • Public
  • Public/Protected
  • All
Menu

Interface EncodedTokensProvider

A "manual" provider of tokens, returning tokens in a binary form.

Hierarchy

  • EncodedTokensProvider

Index

Methods

getInitialState

  • The initial state of a language. Will be the state passed in to tokenize the first line.

    Returns IState

Optional tokenize

  • Tokenize a line given the state at the beginning of the line.

    Parameters

    Returns ILineTokens

tokenizeEncoded

  • Tokenize a line given the state at the beginning of the line.

    Parameters

    Returns IEncodedLineTokens

Generated using TypeDoc