Fork me on GitHub

TokenStream

In package com.mitchellbosecke.pebble.lexer

public class TokenStream extends Object

Constructors

public TokenStream(Collection<Token> tokens, String name)

Constructor for a Token Stream

Parameters:
  • Collection<Token> tokens

    A collection of tokens

  • String name

    The filename of the template that these tokens came from

Methods

public Token next()

Consumes and returns the next token in the stream.

Parameters:
  • No parameters
public Token expect(Type type)

Checks the current token to see if it matches the provided type. If it doesn't match this will throw a SyntaxException. This will consume a token.

Parameters:
  • Type type

    The type of token that we expect

public Token expect(Type type, String value)

Checks the current token to see if it matches the provided type. If it doesn't match this will throw a SyntaxException. This will consume a token.

Parameters:
  • Type type

    The type of token that we expect

  • String value

    The expected value of the token

public Token peek()

Returns the next token in the stream without consuming it.

Parameters:
  • No parameters
public Token peek(int number)

Returns a future token in the stream without consuming any.

Parameters:
  • int number

    How many tokens to lookahead

public boolean isEOF()
Parameters:
  • No parameters
public String toString()
Parameters:
  • No parameters
public Token current()

Looks at the current token. Does not consume the token.

Parameters:
  • No parameters
public String getFilename()
Parameters:
  • No parameters
public ArrayList<Token> getTokens()

used for testing purposes

Parameters:
  • No parameters