|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.antlr.runtime.BufferedTokenStream
org.antlr.runtime.CommonTokenStream
public class CommonTokenStream
The most common stream of tokens where every token is buffered up and tokens are filtered for a certain channel (the parser will only see these tokens). Even though it buffers all of the tokens, this token stream pulls tokens from the tokens source on demand. In other words, until you ask for a token using consume(), LT(), etc. the stream does not pull from the lexer. The only difference between this stream and BufferedTokenStream superclass is that this stream knows how to ignore off channel tokens. There may be a performance advantage to using the superclass if you don't pass whitespace and comments etc. to the parser on a hidden channel (i.e., you set $channel instead of calling skip() in lexer rules.)
UnbufferedTokenStream
,
BufferedTokenStream
Field Summary | |
---|---|
protected int |
channel
Skip tokens on any channel but this one; this is how we skip whitespace... |
Fields inherited from class org.antlr.runtime.BufferedTokenStream |
---|
lastMarker, p, range, tokens, tokenSource |
Constructor Summary | |
---|---|
CommonTokenStream()
|
|
CommonTokenStream(TokenSource tokenSource)
|
|
CommonTokenStream(TokenSource tokenSource,
int channel)
|
Method Summary | |
---|---|
void |
consume()
Always leave p on an on-channel token. |
int |
getNumberOfOnChannelTokens()
Count EOF just once. |
protected Token |
LB(int k)
|
Token |
LT(int k)
Get Token at current input pointer + i ahead where i=1 is next Token. |
void |
reset()
|
void |
setTokenSource(TokenSource tokenSource)
Reset this token stream by setting its token source. |
protected void |
setup()
|
protected int |
skipOffTokenChannels(int i)
Given a starting index, return the index of the first on-channel token. |
protected int |
skipOffTokenChannelsReverse(int i)
|
Methods inherited from class org.antlr.runtime.BufferedTokenStream |
---|
fetch, fill, get, get, getSourceName, getTokens, getTokens, getTokens, getTokens, getTokens, getTokenSource, index, LA, mark, range, release, rewind, rewind, seek, size, sync, toString, toString, toString |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Field Detail |
---|
protected int channel
Constructor Detail |
---|
public CommonTokenStream()
public CommonTokenStream(TokenSource tokenSource)
public CommonTokenStream(TokenSource tokenSource, int channel)
Method Detail |
---|
public void consume()
consume
in interface IntStream
consume
in class BufferedTokenStream
protected Token LB(int k)
LB
in class BufferedTokenStream
public Token LT(int k)
TokenStream
LT
in interface TokenStream
LT
in class BufferedTokenStream
protected int skipOffTokenChannels(int i)
protected int skipOffTokenChannelsReverse(int i)
public void reset()
reset
in class BufferedTokenStream
protected void setup()
setup
in class BufferedTokenStream
public int getNumberOfOnChannelTokens()
public void setTokenSource(TokenSource tokenSource)
setTokenSource
in class BufferedTokenStream
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |