Package org.apache.nutch.segment

Class SegmentMerger.ObjectInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<K,V>

org.apache.hadoop.mapreduce.lib.input.FileInputFormat<K,V>

org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat<Text,MetaWrapper>

org.apache.nutch.segment.SegmentMerger.ObjectInputFormat

Enclosing class:

SegmentMerger

public static class SegmentMerger.ObjectInputFormat
extends SequenceFileInputFormat<Text,MetaWrapper>

Wraps inputs in an MetaWrapper, to permit merging different types in reduce and use additional metadata.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

FileInputFormat.Counter

Field Summary

Fields inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

DEFAULT_LIST_STATUS_NUM_THREADS, INPUT_DIR, INPUT_DIR_NONRECURSIVE_IGNORE_SUBDIRS, INPUT_DIR_RECURSIVE, LIST_STATUS_NUM_THREADS, NUM_INPUT_FILES, PATHFILTER_CLASS, SPLIT_MAXSIZE, SPLIT_MINSIZE

Constructor Summary

Constructors

Constructor	Description
ObjectInputFormat()

Method Summary

Modifier and Type	Method	Description
RecordReader<Text,MetaWrapper>	createRecordReader(InputSplit split, TaskAttemptContext context)

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.SequenceFileInputFormat

getFormatMinSplitSize, listStatus

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputDirRecursive, getInputPathFilter, getInputPaths, getMaxSplitSize, getMinSplitSize, getSplits, isSplitable, makeSplit, makeSplit, setInputDirRecursive, setInputPathFilter, setInputPaths, setInputPaths, setMaxInputSplitSize, setMinInputSplitSize, shrinkStatus

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ObjectInputFormat

public ObjectInputFormat()

Method Details

createRecordReader

public RecordReader<Text,MetaWrapper> createRecordReader(InputSplit split,
 TaskAttemptContext context)
                                                  throws IOException

Overrides:

createRecordReader in class SequenceFileInputFormat<Text,MetaWrapper>

Throws:

IOException