/*!
* @defgroup FileFormatTokenization File Format Tokenization
*
* @brief Helper code for tokenization when reading from @link RecordReader
* @endlink.
*
* @fn FileFormatTokenization#readUntilOneOf
*
* @headerfile <seqan/stream.h>
*
* @brief Read characters from RecordReader into buffer until one of the given
* character is encountered.
*
* @signature int readUntilOneOf(buffer, reader, c1[, c2[, c3[, c4[, c5]]]]);
*
* @param[in,out] buffer The buffer to write to. Type: @link SequenceConcept
* @endlink.
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] c1 One of the characters to look for. Type: <tt>char</tt>.
* @param[in] c2 One of the characters to look for. Type: <tt>char</tt>.
* @param[in] c3 One of the characters to look for. Type: <tt>char</tt>.
* @param[in] c4 One of the characters to look for. Type: <tt>char</tt>.
* @param[in] c5 One of the characters to look for. Type: <tt>char</tt>.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops <b>on</b> the found character and the character itself is
* not written to the buffer. Note that even when an error occurs,
* <tt>buffer</tt> will contain the characters read until the error (or EOF)
* occured.
*
* @fn FileFormatTokenization#readUntilWhitespace
*
* @headerfile <seqan/stream.h>
*
* @brief Read characters from stream into buffer until a @link isspace
* whitespace @endlink is encountered.
*
* @signature int readUntilBlank(buffer, reader);
*
* @param[in,out] buffer The buffer to write to. Type: @link SequenceConcept
* @endlink.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* Whitespace is more than <tt>' '</tt> and <tt>'\t'</tt>, see @link isspace
* @endlink. Consider using @link FileFormatTokenization#readUntilBlank @endlink
* for reading until space or tab.
*
* This function stops <b>on</b> the found character and the character itself is
* not written to the buffer. Note that even when an error occurs,
* <tt>buffer</tt> will contain the characters read until the error (or EOF)
* occured.
*
* @fn FileFormatTokenization#readUntilBlank
*
* @headerfile <seqan/stream.h>
*
* @brief Read characters from stream into buffer until a @link isblank blank
* @endlink is encountered.
*
* @signature int readUntilBlank(buffer, reader);
*
* @param[in,out] buffer The buffer to write to. Type: @link SequenceConcept
* @endlink.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops <b>on</b> the found character and the character itself is
* not written to the buffer. Note that even when an error occurs,
* <tt>buffer</tt> will contain the characters read until the error (or EOF)
* occured.
*
* @fn FileFormatTokenization#readUntilChar
*
* @headerfile <seqan/stream.h>
*
* @brief Read characters from stream into buffer until a given character is
* read.
*
* @signature int readUntilChar(buffer, reader, c);
*
* @param[in,out] buffer The buffer to write to. Type: @link SequenceConcept
* @endlink.
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] c The <tt>char</tt> to look for.
*
* @return int 0 if there was no error and non-0 if there were errors. A special
* value is @link TokenizeResult EOF_BEFORE_SUCCESS @endlink.
*
* @section Remarks
*
* This function stops <b>on</b> the found character and the character itself is
* not written to the buffer. Note that even when an error occurs,
* <tt>buffer</tt> will contain the characters read until the error (or EOF)
* occured.
*
* @fn FileFormatTokenization#readNChars
*
* @headerfile <seqan/stream.h>
*
* @brief Read fixed number of characters from stream into buffer until a given
* character is read.
*
* @signature int readUntilChar(buffer, reader, n);
*
* @param[in,out] buffer The buffer to write to. Type: @link SequenceConcept
* @endlink.
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] n The number of characters to read. Type: <tt>unsigned</tt>.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops after <tt>n</tt> read characters. Note that even when an
* error occurs, <tt>buffer</tt> will contain the characters read until the
* error (or EOF) occured.
*
* @fn FileFormatTokenization#readNCharsIgnoringWhitespace
*
* @headerfile <seqan/stream.h>
*
* @brief Read fixed number of non-whitespace characters.
*
* @signature int readUntilChar(buffer, reader, n);
*
* @param[in,out] buffer The buffer to write to. Type: @link SequenceConcept
* @endlink.
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] n The number of non-whitespace characters to read. Type:
* <tt>unsigned</tt>.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* Whitespace is more than <tt>' '</tt> and <tt>'\t'</tt>, see @link isspace
* @endlink.
*
* This function stops after <tt>n</tt> read non-whitespace characters. Note
* that even when an error occurs, <tt>buffer</tt> will contain the characters
* read until the error (or EOF) occured.
*
* @fn FileFormatTokenization#skipNChars
*
* @headerfile <seqan/stream.h>
*
* @brief Skip a fixed number of characters.
*
* @signature int skipNChars(reader, n);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] n The number of characters to read. Type: <tt>unsinged</tt>.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @fn FileFormatTokenization#skipNCharsIgnoringWhitespace
*
* @headerfile <seqan/stream.h>
*
* @brief Skip a fixed number of non-whitespace characters.
*
* @signature int skipNChars(reader, n);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] n The number of characters to read. Type: <tt>unsinged</tt>.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* Whitespace is more than <tt>' '</tt> and <tt>'\t'</tt>, see @link isspace
* @endlink.
*
* @fn FileFormatTokenization#skipUntilWhitespace
*
* @headerfile <seqan/stream.h>
*
* @brief Skip until the first whitespace character is encountered.
*
* @signature int skipUntilWhitespace(reader);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* Whitespace is more than <tt>' '</tt> and <tt>'\t'</tt>, see @link isspace
* @endlink. Consider using @link FileFormatTokenization#skipUntilBlank @endlink
* for skipping until space or tab.
*
* The reader will stop <b>on</b> the first whitespace character.
*
* @fn FileFormatTokenization#skipUntilBlank
*
* @headerfile <seqan/stream.h>
*
* @brief Skip until the first blank character is encountered.
*
* @signature int skipUntilBlank(reader);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The reader will stop <b>on</b> the first blank character.
*
* @fn FileFormatTokenization#skipUntilGraph
*
* @headerfile <seqan/stream.h>
*
* @brief Skip until the first printable character is encountered.
*
* @signature int skipUntilGraph(reader);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The reader will stop <b>on</b> the first printable character.
*
* @see isgraph
*
* @fn FileFormatTokenization#skipUntilChar
*
* @headerfile <seqan/stream.h>
*
* @brief Skip until the a given character is found.
*
* @signature int skipUntilChar(reader, c);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] c The <tt>char</tt> to search for.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The reader will stop <b>on</b> the first found location of <tt>c</tt>.
*
* @fn FileFormatTokenization#skipUntilString
*
* @headerfile <seqan/stream.h>
*
* @brief Skip until the a given string is found.
*
* @signature int skipUntilString(reader, str);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] str The @link SequenceConcept @endlink to find.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The reader will stop <b>behind</b> the location of <tt>str</tt>.
*
* @fn FileFormatTokenization#skipTabOrLineBreak
*
* @headerfile <seqan/stream.h>
*
* @brief Skip over tab or line break characters (<tt>'\r' or '\n'</tt>).
*
* @signature int skipTabOrLineBreak(reader);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The reader will stop <b>behind</b> the (possibly empty) sequence of tab or
* line break characters (regex: <tt>[\r\n\t]</tt>).
*
* @fn FileFormatTokenization#readLetters
*
* @headerfile <seqan/tokenize.h>
*
* @brief Read from a RecordReader as long as the characters are letters (in
* @link isalpha alpha @endlink class).
*
* @signature int readLetters(buffer, reader);
*
* @param[in,out] buffer The @link String @endlink to read the data into.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The function will stop <b>after</b> the last read character.
*
* Note that even when an error occurs, <tt>buffer</tt> will contain the
* characters read until the error (or EOF) occured.
*
* @fn FileFormatTokenization#readDigits
*
* @headerfile <seqan/tokenize.h>
*
* @brief Read from a RecordReader as long as the characters are digits (in
* @link isdigit digit @endlink class).
*
* @signature int readDigits(buffer, reader);
*
* @param[in,out] buffer The @link String @endlink to read the data into.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The function will stop <b>after</b> the last read character.
*
* Note that even when an error occurs, <tt>buffer</tt> will contain the
* characters read until the error (or EOF) occured.
*
* @fn FileFormatTokenization#readGraphs
*
* @headerfile <seqan/tokenize.h>
*
* @brief Read from a RecordReader as long as the characters are printable (in
* @link isgraph graph @endlink class).
*
* @signature int readGraphs(buffer, reader);
*
* @param[in,out] buffer The @link String @endlink to read the data into.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The function will stop <b>after</b> the last read character.
*
* Note that even when an error occurs, <tt>buffer</tt> will contain the
* characters read until the error (or EOF) occured.
*
* @fn FileFormatTokenization#readFloat
*
* @headerfile <seqan/tokenize.h>
*
* @brief Read characters from @link RecordReader @endlink as long as the string
* is a valid floating point numbers.
*
* @signature int readFloat(buffer, reader);
*
* @param[in,out] buffer The @link String @endlink to read the data into.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The function will stop <b>after</b> the last read character.
*
* Note that even when an error occurs, <tt>buffer</tt> will contain the
* characters read until the error (or EOF) occured.
*
* @fn FileFormatTokenization#readAlphaNums
*
* @headerfile <seqan/tokenize.h>
*
* @brief Read from a RecordReader as long as the characters are alphanumeric
* (in @link isalnum alnum @endlink class).
*
* @signature int readAlphaNums(buffer, reader);
*
* @param[in,out] buffer The @link String @endlink to read the data into.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The function will stop <b>after</b> the last read character.
*
* Note that even when an error occurs, <tt>buffer</tt> will contain the
* characters read until the error (or EOF) occured.
*
* @fn FileFormatTokenization#readIdentifier
*
* @headerfile <seqan/tokenize.h>
*
* @brief Read from a RecordReader as long as the characters form an identifier
* (alnum, <tt>'-'</tt>, <tt>'_'</tt>).
*
* @signature int readIdentifier(buffer, reader);
*
* @param[in,out] buffer The @link String @endlink to read the data into.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* The function will stop <b>after</b> the last read character.
*
* Note that even when an error occurs, <tt>buffer</tt> will contain the
* characters read until the error (or EOF) occured.
*
* @fn FileFormatTokenization#skipWhitespaces
*
* @headerfile <seqan/stream.h>
*
* @brief Advance RecordReader until a whitespace occurs.
*
* @signature int skipWhitespaces(reader);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* Whitespace is more than <tt>' '</tt> and <tt>'\t'</tt>, see @link isspace
* @endlink. Consider using @link FileFormatTokenization#skipBlanks @endlink for
* skipping blanks.
*
* This function stops <b>after</b> the last skipped.
*
* @fn FileFormatTokenization#skipChar
*
* @headerfile <seqan/stream.h>
*
* @brief Skip one character that must be equal to a given one for this function
* to succeed.
*
* @signature int skipChar(reader, c);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] c The <tt>char</tt> to skip.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @fn FileFormatTokenization#skipBlanks
*
* @headerfile <seqan/stream.h>
*
* @brief Advance RecordReader until a blank occurs.
*
* @signature int skipBlanks(reader);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops <b>after</b> the last skipped.
*
* @fn FileFormatTokenization#readLine
*
* @headerfile <seqan/stream.h>
*
* @brief Read a line from a RecordReader into a buffer.
*
* @signature int readLine(buffer, reader);
*
* @param[in,out] buffer The @link String @endlink to append the read data to.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops on the beginning of the next line (if there is any). End
* of line characters are not written to buffer. Works on ANSI, Mac, and Unix
* EOL.
*
* @fn FileFormatTokenization#readLineStripTrailingBlanks
*
* @headerfile <seqan/stream.h>
*
* @brief Read a line from a RecordReader into a buffer and remove trailing
* blanks.
*
* @signature int readLineStripTrailingBlanks(buffer, reader);
*
* @param[in,out] buffer The @link String @endlink to append the read data to.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops on the beginning of the next line (if there is any). End
* of line characters and all trailing whitespaces are not written to buffer.
* Works on ANSI, Mac, and Unix EOL.
*
* @fn FileFormatTokenization#skipLine
*
* @headerfile <seqan/stream.h>
*
* @brief Skip a line in stream and go to beginning of next
*
* @signature int skipLine(reader);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops on the beginning of the next line, if there is a next
* line
*
* Works on ANSI EOL and on Unix EOL.
*
* @fn FileFormatTokenization#readDna5IgnoringWhitespaces
*
* @headerfile <seqan/stream.h>
*
* @brief Read characters from stream as long as they are DNA5 characters,
* skipping whitespaces.
*
* @signature int readDna5IgnoringWhitespaces(buffer, reader);
*
* @param[in,out] buffer The @link String @endlink to append the read data to.
* @param[in,out] reader The @link RecordReader @endlink to read from.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops <b>on</b> the first non-matching character, whitespaces
* such as newlines are discarded.
*
* @fn FileFormatTokenization#skipUntilLineBeginsWithChar
*
* @headerfile <seqan/stream.h>
*
* @brief Skip input until the first graphical (see @link isgraph @endlink)
* character of a line is equal to c.
*
* @signature int skipUntilLineBeginsWithChar(reader, c);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] c The <tt>char</tt> to look for.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops <b>on</b> the first occurence of <tt>c</tt> at the
* beginning of a line (ignoring leading whitespace).
*
* @fn FileFormatTokenization#skipUntilLineBeginsWithStr
*
* @headerfile <seqan/stream.h>
*
* @brief Skip input until a line begins with str (str itself must begin with a
* printable character).
*
* @signature int skipUntilLineBeginsWithChar(reader, str);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] c The @link SequenceConcept @endlink to look for.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops <b>behind</b> the first occurence of <tt>str</tt> at the
* beginning of a line. The function skips over leading whitespace.
*
* @fn FileFormatTokenization#skipUntilLineBeginsWithOneCharOfStr
*
* @headerfile <seqan/stream.h>
*
* @brief Skip input until a line begins with one of the character in str.
*
* @signature int skipUntilLineBeginsWithOneCharOfStr(reader, str);
*
* @param[in,out] reader The @link RecordReader @endlink to read from.
* @param[in] str The @link SequenceConcept @endlink with the characters to look
* for.
*
* @return int 0 if there was no error on reading or non-0 if there were errors.
* A special value is @link TokenizeResult EOF_BEFORE_SUCCESS
* @endlink.
*
* @section Remarks
*
* This function stops <b>on</b> the first found occurence. All characters in
* <tt>str</tt> must be printable (see @link isgraph @endlink).
*/