org.apache.hadoop.hive.ql.io.orc

Class NiFiOrcUtils

java.lang.Object

org.apache.hadoop.hive.ql.io.orc.NiFiOrcUtils

public class NiFiOrcUtils
extends java.lang.Object

Utility methods for ORC support (conversion from Avro, conversion to Hive types, e.g.

Constructor Summary

Constructors

Constructor and Description
NiFiOrcUtils()

Method Summary

Modifier and Type	Method and Description
static java.lang.Object	convertToORCObject(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo typeInfo, java.lang.Object o)
static org.apache.hadoop.hive.ql.io.orc.OrcStruct	createOrcStruct(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo typeInfo, java.lang.Object... objs) Create an object of OrcStruct given a TypeInfo and a list of objects
static OrcFlowFileWriter	createWriter(java.io.OutputStream flowFileOutputStream, org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.hive.serde2.typeinfo.TypeInfo orcSchema, long stripeSize, org.apache.hadoop.hive.ql.io.orc.CompressionKind compress, int bufferSize)
static java.lang.String	generateHiveDDL(org.apache.avro.Schema avroSchema, java.lang.String tableName)
static org.apache.avro.Schema.Type	getAvroSchemaTypeOfObject(java.lang.Object o)
static java.lang.String	getHiveTypeFromAvroType(org.apache.avro.Schema avroSchema)
static org.apache.hadoop.hive.serde2.typeinfo.TypeInfo	getOrcField(org.apache.avro.Schema fieldSchema)
static org.apache.hadoop.hive.serde2.typeinfo.TypeInfo	getPrimitiveOrcTypeFromPrimitiveAvroType(org.apache.avro.Schema.Type avroType)
static java.lang.String	normalizeHiveTableName(java.lang.String name)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NiFiOrcUtils

public NiFiOrcUtils()

Method Detail

convertToORCObject

public static java.lang.Object convertToORCObject(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo typeInfo,
                                                  java.lang.Object o)

createOrcStruct

public static org.apache.hadoop.hive.ql.io.orc.OrcStruct createOrcStruct(org.apache.hadoop.hive.serde2.typeinfo.TypeInfo typeInfo,
                                                                         java.lang.Object... objs)

Create an object of OrcStruct given a TypeInfo and a list of objects

Parameters:

typeInfo - The TypeInfo object representing the ORC record schema

objs - ORC objects/Writables

Returns:

an OrcStruct containing the specified objects for the specified schema

normalizeHiveTableName

public static java.lang.String normalizeHiveTableName(java.lang.String name)

generateHiveDDL

public static java.lang.String generateHiveDDL(org.apache.avro.Schema avroSchema,
                                               java.lang.String tableName)

getOrcField

public static org.apache.hadoop.hive.serde2.typeinfo.TypeInfo getOrcField(org.apache.avro.Schema fieldSchema)
                                                                   throws java.lang.IllegalArgumentException

Throws:

java.lang.IllegalArgumentException

getAvroSchemaTypeOfObject

public static org.apache.avro.Schema.Type getAvroSchemaTypeOfObject(java.lang.Object o)

getPrimitiveOrcTypeFromPrimitiveAvroType

public static org.apache.hadoop.hive.serde2.typeinfo.TypeInfo getPrimitiveOrcTypeFromPrimitiveAvroType(org.apache.avro.Schema.Type avroType)
                                                                                                throws java.lang.IllegalArgumentException

Throws:

java.lang.IllegalArgumentException

getHiveTypeFromAvroType

public static java.lang.String getHiveTypeFromAvroType(org.apache.avro.Schema avroSchema)

createWriter

public static OrcFlowFileWriter createWriter(java.io.OutputStream flowFileOutputStream,
                                             org.apache.hadoop.fs.Path path,
                                             org.apache.hadoop.conf.Configuration conf,
                                             org.apache.hadoop.hive.serde2.typeinfo.TypeInfo orcSchema,
                                             long stripeSize,
                                             org.apache.hadoop.hive.ql.io.orc.CompressionKind compress,
                                             int bufferSize)
                                      throws java.io.IOException

Throws:

java.io.IOException