andyr.jtokeniser
Class StringTokeniser

java.lang.Object
  andyr.jtokeniser.Tokeniser
      andyr.jtokeniser.StringTokeniser

public class StringTokeniser
extends Tokeniser

The StringTokeniser class uses a standard StringTokenizer to find each word in the input.

The following is one example of the use of the tokenizer. The code:

     StringTokeniser st = new StringTokeniser("the cat sat on the mat");
     while (st.hasMoreTokens()) {
         System.out.println(st.nextToken());
     }
 

prints the following output:

     the
     sat
     on
     the
     mat
 

Version:

1.2 (01-Aug-2005)

Author:

Andrew Roberts

Field Summary

Fields inherited from class andyr.jtokeniser.Tokeniser

currentTokenPosition, tokens

Constructor Summary

StringTokeniser(java.lang.String input)
Creates a StringTokeniser that tokenises the input.

StringTokeniser(java.lang.String input, java.lang.String delim)
Creates a StringTokeniser that tokenises the input according the specified delimiter(s).

StringTokeniser(java.lang.String input, java.lang.String delim, boolean keepDelim)
Creates a StringTokeniser that tokenises the input according the specified delimiter(s).

Method Summary

Methods inherited from class andyr.jtokeniser.Tokeniser

countTokens, getTokens, hasMoreTokens, nextToken, numberOfTokens

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StringTokeniser

public StringTokeniser(java.lang.String input,
                       java.lang.String delim,
                       boolean keepDelim)

Creates a StringTokeniser that tokenises the input according the specified delimiter(s). Each character in delim is considered a delimiter for separating tokens. If the returnDelims flag is true, then the delimiter characters are also returned as tokens. Each delimiter is returned as a string of length one. If the flag is false, the delimiter characters are skipped and only serve as separators between tokens. Note that if delim is null, this constructor does not throw an exception. However, trying to invoke other methods on the resulting StringTokeniser may result in a NullPointerException.

Parameters:

input - a string from which the tokens will be extracted.

delim - the delimiters

keepDelim - flag indicating whether to return the delimiters as tokens.

StringTokeniser

public StringTokeniser(java.lang.String input,
                       java.lang.String delim)

Creates a StringTokeniser that tokenises the input according the specified delimiter(s). Each character in delim is considered a delimiter for separating tokens. Delimiter characters themselves will not be treated as tokens. Note that if delim is null, this constructor does not throw an exception. However, trying to invoke other methods on the resulting StringTokeniser may result in a NullPointerException.

Parameters:

input - a string from which the tokens will be extracted.

delim - the delimiters

StringTokeniser

public StringTokeniser(java.lang.String input)

Creates a StringTokeniser that tokenises the input. The tokeniser uses the default delimiter set, which is "\t\n\r\f": the space character, the tab character, the newline character, the carriage-return character, and the form-feed character. Delimiter characters themselves will not be treated as tokens.

Parameters:

input - a string from which the tokens will be extracted.