@InterfaceAudience.Public @InterfaceStability.Stable public class KeyValueTextInputFormat extends FileInputFormat<Text,Text>
InputFormat
for plain text files. Files are broken into lines.
Either line feed or carriage-return are used to signal end of line.
Each line is divided into key and value parts by a separator byte. If no
such a byte exists, the key will be the entire line and value will be empty.
The separator byte can be specified in config file under the attribute name
mapreduce.input.keyvaluelinerecordreader.key.value.separator. The default
is the tab character ('\t').DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_NONRECURSIVE_IGNORE_SUBDIRS, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE
Constructor and Description |
---|
KeyValueTextInputFormat() |
Modifier and Type | Method and Description |
---|---|
RecordReader<Text,Text> |
createRecordReader(InputSplit genericSplit,
TaskAttemptContext context)
Create a record reader for a given split.
|
protected boolean |
isSplitable(JobContext context,
Path file)
Is the given filename splittable? Usually, true, but if the file is
stream compressed, it will not be.
|
addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getFormatMinSplitSize, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, getSplits, listStatus, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize, shrinkStatus
protected boolean isSplitable(JobContext context, Path file)
FileInputFormat
FileInputFormat
always returns
true. Implementations that may deal with non-splittable files must
override this method.
FileInputFormat
implementations can override this and return
false
to ensure that individual input files are never split-up
so that Mapper
s process entire files.isSplitable
in class FileInputFormat<Text,Text>
context
- the job contextfile
- the file name to checkpublic RecordReader<Text,Text> createRecordReader(InputSplit genericSplit, TaskAttemptContext context) throws IOException
InputFormat
RecordReader.initialize(InputSplit, TaskAttemptContext)
before
the split is used.createRecordReader
in class InputFormat<Text,Text>
genericSplit
- the split to be readcontext
- the information about the taskIOException
Copyright © 2023 Apache Software Foundation. All rights reserved.