functionscanToken

Scan for the next token and return it

Extends from Modelica.Icons.Function (Icon for functions).

Information

Syntax

(token, nextIndex) = Strings.scanToken(string, startIndex, unsigned=false);

Description

Function scanToken scans the string starting at index "startIndex" and returns the next token, as well as the index directly after the token. The returned token is a record that holds the type of the token and the value of the token:

token.tokenType	Type of the token, see below
token.real	Real value if tokenType == TokenType.RealToken
token.integer	Integer value if tokenType == TokenType.IntegerToken
token.boolean	Boolean value if tokenType == TokenType.BooleanToken
token.string	String value if tokenType == TokenType.StringToken/IdentifierToken/DelimiterToken

Variable token.tokenType is an enumeration (emulated as a package with constants) that can have the following values:

import T = Modelica.Utilities.Types.TokenType;

T.RealToken	Modelica Real literal (e.g., 1.23e-4)
T.IntegerToken	Modelica Integer literal (e.g., 123)
T.BooleanToken	Modelica Boolean literal (e.g., false)
T.StringToken	Modelica String literal (e.g., "string 123")
T.IdentifierToken	Modelica identifier (e.g., "force_a")
T.DelimiterToken	any character without white space that does not appear as first character in the tokens above (e.g., "&")
T.NoToken	White space, line comments and no other token until the end of the string

Modelica line comments ("// ... end-of-line/end-of-string") as well as white space is ignored. If "unsigned=true", a Real or Integer literal is not allowed to start with a "+" or "-" sign.

Example

import Modelica.Utilities.Strings;
import T = Modelica.Utilities.Types.TokenType;
(token, index) := Strings.scanToken(string);
if token.tokenType == T.RealToken then
   realValue := token.real;
elseif token.tokenType == T.IntegerToken then
   integerValue := token.integer;
elseif token.tokenType == T.BooleanToken then
   booleanValue := token.boolean;
elseif token.tokenType == T.Identifier then
   name := token.string;
else
   Strings.syntaxError(string,index,"Expected Real, Integer, Boolean or identifier token");
end if;

Inputs

Type	Name	Default	Description
String	string		String to be scanned
Integer	startIndex	1	Start scanning of string at character startIndex
Boolean	unsigned	false	= true, if Real and Integer tokens shall not start with a sign

Outputs

Type	Name	Default	Description
Types.TokenValue	token		Scanned token
Integer	nextIndex		Index of character after the found token; = 0, if NoToken