org.embulk.util.csv

Class CsvTokenizer

java.lang.Object

org.embulk.util.csv.CsvTokenizer

public class CsvTokenizer
extends Object

Tokenizes iteration of strings as comma-separated values (CSV) into columns in rows.

Note that this tokenizer is implemented to consume Iterator<String> iteratively, and to return columns one-by-one by nextColumn() or nextColumnOrNull().

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CsvTokenizer.Builder A builder of CsvTokenizer.

Method Summary

Modifier and Type	Method and Description
static CsvTokenizer.Builder	builder(String delimiter) Creates a builder instance for CsvTokenizer.
long	getCurrentLineNumber() Returns the line number where the tokenizer is currently on tokenizing.
boolean	hasNextColumn() Checks if the line has a remaining column to read.
String	nextColumn() Reads the next column tokenized, and moves the tokenizing cursor forward.
String	nextColumnOrNull() Reads the next column tokenized, with nullString considered, and moves the tokenizing cursor forward.
boolean	nextRecord() Moves the tokenizing cursor forward to the next record, with skipping empty lines.
boolean	nextRecord(boolean skipEmptyLine) Moves the tokenizing cursor forward to the next record.
String	skipCurrentLine() Skips a line, where the tokenizer is currently on tokenizing at.
boolean	skipHeaderLine() Skips a line considered as a "header line", where the tokenizer is currently on tokenizing at.
boolean	wasQuotedColumn() Checks if the last column was a quoted column.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

builder

public static CsvTokenizer.Builder builder(String delimiter)

Creates a builder instance for CsvTokenizer.

Parameters:

delimiter - a mandatory column delimiter (ex. "," for CSV, "\t" for TSV)

Returns:

a builder instance

Throws:

NullPointerException - if delimiter is null

IllegalArgumentException - if delimiter is empty

getCurrentLineNumber

public long getCurrentLineNumber()

Returns the line number where the tokenizer is currently on tokenizing.

Returns:

the line number where the tokenizer is currently on tokenizing

skipHeaderLine

public boolean skipHeaderLine()

Skips a line considered as a "header line", where the tokenizer is currently on tokenizing at.

Note that this method does not check the line is really a "header line". It just assumes that the line is a header line, and skips a line unconditionally.

Returns:

false if the tokenizer observes no any more lines. true otherwise.

skipCurrentLine

public String skipCurrentLine()

Skips a line, where the tokenizer is currently on tokenizing at.

If the tokenizer is in the "quoted" state with newline(s), it skips only the first line in the quote, and then unreads following lines.

Returns:

the skipped line

nextRecord

public boolean nextRecord()

Moves the tokenizing cursor forward to the next record, with skipping empty lines.

Returns:

true if the tokenizer still has a line to read

Throws:

RecordHasUnexpectedTrailingColumnException - if called while the line has not reached at an end-of-line yet

nextRecord

public boolean nextRecord(boolean skipEmptyLine)

Moves the tokenizing cursor forward to the next record.

Parameters:

skipEmptyLine - true to skip empty lines

Returns:

true if the tokenizer still has a line to read

Throws:

RecordHasUnexpectedTrailingColumnException - if called while the line has not reached at an end-of-line yet

hasNextColumn

public boolean hasNextColumn()

Checks if the line has a remaining column to read.

Returns:

true if the line has a remaining column

nextColumn

public String nextColumn()

Reads the next column tokenized, and moves the tokenizing cursor forward.

Returns:

the tokenized column as a String

Throws:

EndOfFileInQuotedFieldException - if the tokenizer reaches at the end-of-file while in the "quoted" state

InvalidCharacterAfterQuoteException - if the tokenizer encounters an invalid character next to a quoted field

QuotedFieldLengthLimitExceededException - if the tokenizer encounters too long a quoted field

RecordDoesNotHaveExpectedColumnException - if called while the line has already reached at an end-of-line

nextColumnOrNull

public String nextColumnOrNull()

Reads the next column tokenized, with nullString considered, and moves the tokenizing cursor forward.

If the tokenizer is configured with nullString, it returns null for a column that matches nullString.

If the tokenizer is not configured with nullString, it returns an empty string "" for a quoted field, or null otherwise.

Returns:

the tokenized column as a String

Throws:

EndOfFileInQuotedFieldException - if the tokenizer reaches at the end-of-file while in the "quoted" state

InvalidCharacterAfterQuoteException - if the tokenizer encounters an invalid character next to a quoted field

QuotedFieldLengthLimitExceededException - if the tokenizer encounters too long a quoted field

RecordDoesNotHaveExpectedColumnException - if called while the line has already reached at an end-of-line

wasQuotedColumn

public boolean wasQuotedColumn()

Checks if the last column was a quoted column.

Returns:

true if the last column was a quoted column