CharScanner Class Reference

#include <CharScanner.hpp>

Inheritance diagram for CharScanner:

Collaboration diagram for CharScanner:


Public Member Functions
	CharScanner (InputBuffer &cb, bool case_sensitive)
	CharScanner (InputBuffer *cb, bool case_sensitive)
	CharScanner (const LexerSharedInputState &state, bool case_sensitive)
virtual	~CharScanner ()
virtual int	LA (unsigned int i)
virtual void	append (char c)
virtual void	append (const ANTLR_USE_NAMESPACE(std) string &s)
virtual void	commit ()
virtual void	recover (const RecognitionException &ex, const BitSet &tokenSet)
virtual void	consume ()
virtual void	consumeUntil (int c)
virtual void	consumeUntil (const BitSet &set)
virtual unsigned int	mark ()
	Mark the current position and return a id for it.
virtual void	rewind (unsigned int pos)
	Rewind the scanner to a previously marked position.
virtual void	match (int c)
	See if input contains character 'c' throw MismatchedCharException if not.
virtual void	match (const BitSet &b)
virtual void	match (const char *s)
virtual void	match (const ANTLR_USE_NAMESPACE(std) string &s)
virtual void	matchNot (int c)
virtual void	matchRange (int c1, int c2)
virtual bool	getCaseSensitive () const
virtual void	setCaseSensitive (bool t)
virtual bool	getCaseSensitiveLiterals () const =0
virtual int	getLine () const
	Get the line the scanner currently is in (starts at 1).
virtual void	setLine (int l)
	set the line number
virtual int	getColumn () const
	Get the column the scanner currently is in (starts at 1).
virtual void	setColumn (int c)
	set the column number
virtual const	ANTLR_USE_NAMESPACE (std) string &getFilename() const
	get the filename for the file currently used
virtual void	setFilename (const ANTLR_USE_NAMESPACE(std) string &f)
	Set the filename the scanner is using (used in error messages).
virtual bool	getCommitToPath () const
virtual void	setCommitToPath (bool commit)
virtual const	ANTLR_USE_NAMESPACE (std) string &getText() const
virtual void	setText (const ANTLR_USE_NAMESPACE(std) string &s)
virtual void	resetText ()
virtual RefToken	getTokenObject () const
virtual void	newline ()
virtual void	tab ()
int	setTabsize (int size)
	set the tabsize. Returns the old tabsize
int	getTabSize () const
	Return the tabsize used by the scanner.
virtual void	reportError (const RecognitionException &e)
virtual void	reportError (const ANTLR_USE_NAMESPACE(std) string &s)
virtual void	reportWarning (const ANTLR_USE_NAMESPACE(std) string &s)
virtual InputBuffer &	getInputBuffer ()
virtual LexerSharedInputState	getInputState ()
virtual void	setInputState (LexerSharedInputState state)
virtual void	setTokenObjectFactory (factory_type factory)
	Set the factory for created tokens.
virtual int	testLiteralsTable (int ttype) const
virtual int	testLiteralsTable (const ANTLR_USE_NAMESPACE(std) string &txt, int ttype) const
virtual int	toLower (int c) const
	Override this method to get more specific case handling.
virtual void	uponEOF ()
virtual void	traceIndent ()
	Methods used to change tracing behavior.
virtual void	traceIn (const char *rname)
virtual void	traceOut (const char *rname)
Static Public Attributes
static const int	EOF_CHAR = EOF
Protected Types
typedef RefToken(*	factory_type )()
Protected Member Functions
	ANTLR_USE_NAMESPACE (std) string text
	ANTLR_USE_NAMESPACE (std) map< ANTLR_USE_NAMESPACE(std) string
virtual RefToken	makeToken (int t)
	Create a new RefToken of type t.
Protected Attributes
bool	saveConsumedInput
	flag indicating wether consume saves characters
factory_type	tokenFactory
	Factory for tokens.
bool	caseSensitive
	Is this lexer case sensitive.
	int
CharScannerLiteralsLess	literals
RefToken	_returnToken
	used to return tokens w/o using return val
LexerSharedInputState	inputState
	Input state, gives access to input stream, shared among different lexers.
bool	commitToPath
int	tabsize
	tab size the scanner uses.
int	traceDepth
Private Member Functions
	CharScanner (const CharScanner &other)
CharScanner &	operator= (const CharScanner &other)
Static Private Attributes
static const int	NO_CHAR = 0
Data Structures
class	Tracer

Detailed Description

Superclass of generated lexers

Definition at line 86 of file CharScanner.hpp.

Member Typedef Documentation

typedef RefToken(* CharScanner::factory_type)() [protected]

Constructor & Destructor Documentation

CharScanner::CharScanner	(	InputBuffer &	cb,
		bool	case_sensitive
	)

CharScanner::CharScanner	(	InputBuffer *	cb,
		bool	case_sensitive
	)

CharScanner::CharScanner	(	const LexerSharedInputState &	state,
		bool	case_sensitive
	)

virtual CharScanner::~CharScanner ( ) [inline, virtual]

Definition at line 94 of file CharScanner.hpp.

CharScanner::CharScanner ( const CharScanner & other ) [private]

Member Function Documentation

int CharScanner::LA ( unsigned int i ) [inline, virtual]

Definition at line 546 of file CharScanner.hpp.

References caseSensitive, inputState, and toLower().

virtual void CharScanner::append ( char c ) [inline, virtual]

Definition at line 100 of file CharScanner.hpp.

virtual void CharScanner::append ( const ANTLR_USE_NAMESPACE(std) string & s ) [inline, virtual]

Definition at line 113 of file CharScanner.hpp.

virtual void CharScanner::commit ( ) [inline, virtual]

Definition at line 119 of file CharScanner.hpp.

virtual void CharScanner::recover	(	const RecognitionException &	ex,
		const BitSet &	tokenSet
	)			`[inline, virtual]`

called by the generated lexer to do error recovery, override to customize the behaviour.

Definition at line 127 of file CharScanner.hpp.

virtual void CharScanner::consume ( ) [inline, virtual]

Definition at line 133 of file CharScanner.hpp.

virtual void CharScanner::consumeUntil ( int c ) [inline, virtual]

Consume chars until one matches the given char

Definition at line 159 of file CharScanner.hpp.

virtual void CharScanner::consumeUntil ( const BitSet & set ) [inline, virtual]

Consume chars until one matches the given set

Definition at line 171 of file CharScanner.hpp.

virtual unsigned int CharScanner::mark ( ) [inline, virtual]

Mark the current position and return a id for it.

Definition at line 183 of file CharScanner.hpp.

virtual void CharScanner::rewind ( unsigned int pos ) [inline, virtual]

Rewind the scanner to a previously marked position.

Definition at line 188 of file CharScanner.hpp.

virtual void CharScanner::match ( int c ) [inline, virtual]

See if input contains character 'c' throw MismatchedCharException if not.

Definition at line 194 of file CharScanner.hpp.

virtual void CharScanner::match ( const BitSet & b ) [inline, virtual]

See if input contains element from bitset b throw MismatchedCharException if not

Definition at line 205 of file CharScanner.hpp.

References BitSet::member().

virtual void CharScanner::match ( const char * s ) [inline, virtual]

See if input contains string 's' throw MismatchedCharException if not

Note:: the string cannot match EOF

Definition at line 217 of file CharScanner.hpp.

virtual void CharScanner::match ( const ANTLR_USE_NAMESPACE(std) string & s ) [inline, virtual]

See if input contains string 's' throw MismatchedCharException if not

Note:: the string cannot match EOF

Definition at line 233 of file CharScanner.hpp.

virtual void CharScanner::matchNot ( int c ) [inline, virtual]

See if input does not contain character 'c' throw MismatchedCharException if not

Definition at line 251 of file CharScanner.hpp.

virtual void CharScanner::matchRange	(	int	c1,
		int	c2
	)			`[inline, virtual]`

See if input contains character in range c1-c2 throw MismatchedCharException if not

Definition at line 263 of file CharScanner.hpp.

virtual bool CharScanner::getCaseSensitive ( ) const [inline, virtual]

Definition at line 273 of file CharScanner.hpp.

virtual void CharScanner::setCaseSensitive ( bool t ) [inline, virtual]

Definition at line 278 of file CharScanner.hpp.

virtual bool CharScanner::getCaseSensitiveLiterals ( ) const [pure virtual]

Referenced by CharScannerLiteralsLess::operator()().

virtual int CharScanner::getLine ( ) const [inline, virtual]

Get the line the scanner currently is in (starts at 1).

Definition at line 286 of file CharScanner.hpp.

virtual void CharScanner::setLine ( int l ) [inline, virtual]

set the line number

Definition at line 292 of file CharScanner.hpp.

virtual int CharScanner::getColumn ( ) const [inline, virtual]

Get the column the scanner currently is in (starts at 1).

Definition at line 298 of file CharScanner.hpp.

virtual void CharScanner::setColumn ( int c ) [inline, virtual]

set the column number

Definition at line 303 of file CharScanner.hpp.

virtual const CharScanner::ANTLR_USE_NAMESPACE ( std ) const [inline, virtual]

get the filename for the file currently used

Definition at line 309 of file CharScanner.hpp.

virtual void CharScanner::setFilename ( const ANTLR_USE_NAMESPACE(std) string & f ) [inline, virtual]

Set the filename the scanner is using (used in error messages).

Definition at line 314 of file CharScanner.hpp.

virtual bool CharScanner::getCommitToPath ( ) const [inline, virtual]

Definition at line 319 of file CharScanner.hpp.

virtual void CharScanner::setCommitToPath ( bool commit ) [inline, virtual]

Definition at line 324 of file CharScanner.hpp.

virtual const CharScanner::ANTLR_USE_NAMESPACE ( std ) const [inline, virtual]

return a copy of the current text buffer

Definition at line 330 of file CharScanner.hpp.

virtual void CharScanner::setText ( const ANTLR_USE_NAMESPACE(std) string & s ) [inline, virtual]

Definition at line 335 of file CharScanner.hpp.

virtual void CharScanner::resetText ( ) [inline, virtual]

Definition at line 340 of file CharScanner.hpp.

virtual RefToken CharScanner::getTokenObject ( ) const [inline, virtual]

Definition at line 347 of file CharScanner.hpp.

virtual void CharScanner::newline ( ) [inline, virtual]

Used to keep track of line breaks, needs to be called from within generated lexers when a
is encountered.

Definition at line 355 of file CharScanner.hpp.

virtual void CharScanner::tab ( ) [inline, virtual]

Advance the current column number by an appropriate amount according to the tabsize. This method needs to be explicitly called from the lexer rules encountering tabs.

Definition at line 365 of file CharScanner.hpp.

int CharScanner::setTabsize ( int size ) [inline]

set the tabsize. Returns the old tabsize

Definition at line 372 of file CharScanner.hpp.

int CharScanner::getTabSize ( ) const [inline]

Return the tabsize used by the scanner.

Definition at line 379 of file CharScanner.hpp.

virtual void CharScanner::reportError ( const RecognitionException & e ) [virtual]

Report exception errors caught in nextToken()

virtual void CharScanner::reportError ( const ANTLR_USE_NAMESPACE(std) string & s ) [virtual]

Parser error-reporting function can be overridden in subclass

virtual void CharScanner::reportWarning ( const ANTLR_USE_NAMESPACE(std) string & s ) [virtual]

Parser warning-reporting function can be overridden in subclass

virtual InputBuffer& CharScanner::getInputBuffer ( ) [inline, virtual]

Definition at line 393 of file CharScanner.hpp.

virtual LexerSharedInputState CharScanner::getInputState ( ) [inline, virtual]

Definition at line 398 of file CharScanner.hpp.

virtual void CharScanner::setInputState ( LexerSharedInputState state ) [inline, virtual]

set the input state for the lexer.

Note:: state is a reference counted object, hence no reference

Definition at line 405 of file CharScanner.hpp.

virtual void CharScanner::setTokenObjectFactory ( factory_type factory ) [inline, virtual]

Set the factory for created tokens.

Definition at line 411 of file CharScanner.hpp.

virtual int CharScanner::testLiteralsTable ( int ttype ) const [inline, virtual]

Test the token text against the literals table Override this method to perform a different literals test

Definition at line 419 of file CharScanner.hpp.

References ANTLR_USE_NAMESPACE.

virtual int CharScanner::testLiteralsTable	(	const ANTLR_USE_NAMESPACE(std) string &	txt,
		int	ttype
	)			const `[inline, virtual]`

Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token

Definition at line 432 of file CharScanner.hpp.

References ANTLR_USE_NAMESPACE.

virtual int CharScanner::toLower ( int c ) const [inline, virtual]

Override this method to get more specific case handling.

Definition at line 441 of file CharScanner.hpp.

Referenced by LA().

virtual void CharScanner::uponEOF ( ) [inline, virtual]

This method is called by YourLexer::nextToken() when the lexer has hit EOF condition. EOF is NOT a character. This method is not called if EOF is reached during syntactic predicate evaluation or during evaluation of normal lexical rules, which presumably would be an IOException. This traps the "normal" EOF condition.

uponEOF() is called after the complete evaluation of the previous token and only if your parser asks for another token beyond that last non-EOF token.

You might want to throw token or char stream exceptions like: "Heh, premature eof" or a retry stream exception ("I found the end of this file, go back to referencing file").

Definition at line 464 of file CharScanner.hpp.

virtual void CharScanner::traceIndent ( ) [virtual]

Methods used to change tracing behavior.

virtual void CharScanner::traceIn ( const char * rname ) [virtual]

virtual void CharScanner::traceOut ( const char * rname ) [virtual]

CharScanner::ANTLR_USE_NAMESPACE ( std ) [protected]

Text of current token

CharScanner::ANTLR_USE_NAMESPACE ( std ) [protected]

virtual RefToken CharScanner::makeToken ( int t ) [inline, protected, virtual]

Create a new RefToken of type t.

Definition at line 502 of file CharScanner.hpp.

CharScanner& CharScanner::operator= ( const CharScanner & other ) [private]

Field Documentation

const int CharScanner::EOF_CHAR = EOF [static]

Definition at line 474 of file CharScanner.hpp.

bool CharScanner::saveConsumedInput [protected]

flag indicating wether consume saves characters

Definition at line 483 of file CharScanner.hpp.

factory_type CharScanner::tokenFactory [protected]

Factory for tokens.

Definition at line 484 of file CharScanner.hpp.

bool CharScanner::caseSensitive [protected]

Is this lexer case sensitive.

Definition at line 485 of file CharScanner.hpp.

Referenced by LA().

CharScanner::int [protected]

Definition at line 486 of file CharScanner.hpp.

CharScannerLiteralsLess CharScanner::literals [protected]

Definition at line 486 of file CharScanner.hpp.

RefToken CharScanner::_returnToken [protected]

used to return tokens w/o using return val

Definition at line 488 of file CharScanner.hpp.

LexerSharedInputState CharScanner::inputState [protected]

Input state, gives access to input stream, shared among different lexers.

Definition at line 491 of file CharScanner.hpp.

Referenced by LA().

bool CharScanner::commitToPath [protected]

Used during filter mode to indicate that path is desired. A subsequent scan error will report an error as usual if acceptPath=true;

Definition at line 497 of file CharScanner.hpp.

int CharScanner::tabsize [protected]

tab size the scanner uses.

Definition at line 499 of file CharScanner.hpp.

int CharScanner::traceDepth [protected]

Definition at line 532 of file CharScanner.hpp.

const int CharScanner::NO_CHAR = 0 [static, private]

Definition at line 538 of file CharScanner.hpp.

The documentation for this class was generated from the following file:

antlr/include/antlr/CharScanner.hpp

CharScanner Class Reference

Public Member Functions

Static Public Attributes

Protected Types

Protected Member Functions

Protected Attributes

Private Member Functions

Static Private Attributes

Data Structures

Detailed Description

Member Typedef Documentation

Constructor & Destructor Documentation

Member Function Documentation

Field Documentation