org.apache.hadoop.hbase.mapreduce

Class TableOutputFormat<KEY>

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>

org.apache.hadoop.hbase.mapreduce.TableOutputFormat<KEY>

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

@InterfaceAudience.Public
public class TableOutputFormat<KEY>
extends org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>
implements org.apache.hadoop.conf.Configurable

Convert Map/Reduce output and write it to an HBase table. The KEY is ignored while the output value must be either a Put or a Delete instance.

Field Summary

Fields

Modifier and Type	Field and Description
static String	OUTPUT_COMMITTER_CLASS The configuration key for specifying a custom OutputCommitter implementation to be used by TableOutputFormat.
static String	OUTPUT_CONF_PREFIX Prefix for configuration property overrides to apply in setConf(Configuration).
static String	OUTPUT_TABLE Job parameter that specifies the output table.
static String	QUORUM_ADDRESS Optional job parameter to specify a peer cluster.
static String	QUORUM_PORT Optional job parameter to specify peer cluster's ZK client port
static String	REGION_SERVER_CLASS Deprecated. Since 2.5.9, 2.6.1 and 2.7.0, will be removed in 4.0.0. Does not take effect from long ago, see HBASE-6044.
static String	REGION_SERVER_IMPL Deprecated. Since 2.5.9, 2.6.1 and 2.7.0, will be removed in 4.0.0. Does not take effect from long ago, see HBASE-6044.
static boolean	WAL_OFF Property value to disable write-ahead logging
static boolean	WAL_ON Property value to use write-ahead logging
static String	WAL_PROPERTY Set this to WAL_OFF to turn off write-ahead logging (WAL)

Constructor Summary

Constructors

Constructor and Description
TableOutputFormat()

Method Summary

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context) Checks if the output table exists and is enabled.
org.apache.hadoop.conf.Configuration	getConf()
org.apache.hadoop.mapreduce.OutputCommitter	getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Returns the output committer.
org.apache.hadoop.mapreduce.RecordWriter<KEY,Mutation>	getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) Creates a new record writer.
void	setConf(org.apache.hadoop.conf.Configuration otherConf)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OUTPUT_TABLE

public static final String OUTPUT_TABLE

Job parameter that specifies the output table.

See Also:

Constant Field Values

WAL_ON

public static final boolean WAL_ON

Property value to use write-ahead logging

See Also:

Constant Field Values

WAL_OFF

public static final boolean WAL_OFF

Property value to disable write-ahead logging

See Also:

Constant Field Values

WAL_PROPERTY

public static final String WAL_PROPERTY

Set this to WAL_OFF to turn off write-ahead logging (WAL)

See Also:

Constant Field Values

OUTPUT_CONF_PREFIX

public static final String OUTPUT_CONF_PREFIX

Prefix for configuration property overrides to apply in setConf(Configuration). For keys matching this prefix, the prefix is stripped, and the value is set in the configuration with the resulting key, ie. the entry "hbase.mapred.output.key1 = value1" would be set in the configuration as "key1 = value1". Use this to set properties which should only be applied to the TableOutputFormat configuration and not the input configuration.

See Also:

Constant Field Values

OUTPUT_COMMITTER_CLASS

public static final String OUTPUT_COMMITTER_CLASS

The configuration key for specifying a custom OutputCommitter implementation to be used by TableOutputFormat. The value for this property should be the fully qualified class name of the custom committer. If this property is not set, TableOutputCommitter will be used by default.

See Also:

Constant Field Values

QUORUM_ADDRESS

public static final String QUORUM_ADDRESS

Optional job parameter to specify a peer cluster. Used specifying remote cluster when copying between hbase clusters (the source is picked up from hbase-site.xml).

See Also:

TableMapReduceUtil.initTableReducerJob(String, Class, org.apache.hadoop.mapreduce.Job, Class, String), Constant Field Values

QUORUM_PORT

public static final String QUORUM_PORT

Optional job parameter to specify peer cluster's ZK client port

See Also:

Constant Field Values

REGION_SERVER_CLASS

@Deprecated
public static final String REGION_SERVER_CLASS

Deprecated. Since 2.5.9, 2.6.1 and 2.7.0, will be removed in 4.0.0. Does not take effect from long ago, see HBASE-6044.

Optional specification of the rs class name of the peer cluster.

See Also:

Constant Field Values

REGION_SERVER_IMPL

@Deprecated
public static final String REGION_SERVER_IMPL

Deprecated. Since 2.5.9, 2.6.1 and 2.7.0, will be removed in 4.0.0. Does not take effect from long ago, see HBASE-6044.

Optional specification of the rs impl name of the peer cluster

See Also:

Constant Field Values

Constructor Detail

TableOutputFormat

public TableOutputFormat()

Method Detail

getRecordWriter

public org.apache.hadoop.mapreduce.RecordWriter<KEY,Mutation> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                       throws IOException,
                                                                              InterruptedException

Creates a new record writer. Be aware that the baseline javadoc gives the impression that there is a single RecordWriter per job but in HBase, it is more natural if we give you a new RecordWriter per call of this method. You must close the returned RecordWriter when done. Failure to do so will drop writes.

Specified by:

getRecordWriter in class org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>

Parameters:

context - The current task context.

Returns:

The newly created writer instance.

Throws:

IOException - When creating the writer fails.

InterruptedException - When the job is cancelled.

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
                      throws IOException,
                             InterruptedException

Checks if the output table exists and is enabled.

Specified by:

checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>

Parameters:

context - The current context.

Throws:

IOException - When the check fails.

InterruptedException - When the job is aborted.

See Also:

OutputFormat.checkOutputSpecs(JobContext)

getOutputCommitter

public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                               throws IOException,
                                                                      InterruptedException

Returns the output committer.

Specified by:

getOutputCommitter in class org.apache.hadoop.mapreduce.OutputFormat<KEY,Mutation>

Parameters:

context - The current context.

Returns:

The committer.

Throws:

IOException - When creating the committer fails.

InterruptedException - When the job is aborted.

See Also:

OutputFormat.getOutputCommitter(TaskAttemptContext)

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public void setConf(org.apache.hadoop.conf.Configuration otherConf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable