A good answer might be:

java TokenTester "val = 12+8"
val
12
8

You probably regard the "=" and the "+" symbols as important, so they should be regarded as tokens. But "+" also needs to work as a delimiter so that "12+8" is broken into the tokens "12", "+" and "8". This is done by using true as the third parameter in the constructor: Now individual delimiters (as well as tokens) are returned by nextToken().

import java.util.* ;

public class TokenTester
{

  public static void main ( String[] args )
  {
    StringTokenizer tok;
    if ( args.length > 0 )
    {
      tok = new StringTokenizer( args[0], " =+-", true );

      while ( tok.hasMoreTokens() )
        System.out.println( tok.nextToken() );
    }
  }
}

QUESTION 16:

Now what is the output for the following:

java TokenTester "val = 12+8"