org.apache.datasketches.pig.cpc

## Class DataToSketch

- java.lang.Object
- - org.apache.pig.EvalFunc<org.apache.pig.data.DataByteArray>
  - - org.apache.datasketches.pig.cpc.DataToSketch

All Implemented Interfaces:
  org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>, org.apache.pig.Algebraic

    public class DataToSketch
    extends org.apache.pig.EvalFunc<org.apache.pig.data.DataByteArray>
    implements org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>, org.apache.pig.Algebraic

This is a Pig UDF that builds sketches from data. This class implements both the Accumulator and Algebraic interfaces.

Author:
  Alexander Saydakov

### Nested Class Summary

### Nested classes/interfaces inherited from class org.apache.pig.EvalFunc

`org.apache.pig.EvalFunc.SchemaType`

### Field Summary

### Fields inherited from class org.apache.pig.EvalFunc

`log, pigLogger, reporter, returnType`

### Constructor Summary

Constructors
Constructor and Description
`DataToSketch()` Constructor with default lgK and seed
`DataToSketch(int lgK, long seed)` Base constructor.
`DataToSketch(String lgK)` Constructor with given lgK as string and default seed
`DataToSketch(String lgK, String seed)` Constructor with given lgK and seed as strings

### Method Summary

All Methods Instance Methods Concrete Methods
`void` `accumulate(org.apache.pig.data.Tuple inputTuple)` An Accumulator version of the standard exec() method.
`void` `cleanup()` Cleans up the UDF state after being called using the `Accumulator` interface.
`org.apache.pig.data.DataByteArray` `exec(org.apache.pig.data.Tuple inputTuple)` Top-level exec function.
`String` `getFinal()`
`String` `getInitial()`
`String` `getIntermed()`
`org.apache.pig.data.DataByteArray` `getValue()` Returns the sketch that has been built up by multiple calls to `accumulate(org.apache.pig.data.Tuple)`.

### Methods inherited from class org.apache.pig.EvalFunc

`allowCompileTimeCalculation, finish, getArgToFuncMapping, getCacheFiles, getInputSchema, getLoadCaster, getLogger, getPigLogger, getReporter, getReturnType, getSchemaName, getSchemaType, getShipFiles, isAsynchronous, needEndOfAllInputProcessing, outputSchema, progress, setEndOfAllInput, setInputSchema, setPigLogger, setReporter, setUDFContextSignature, warn`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### DataToSketch

    public DataToSketch()

Constructor with default lgK and seed

#### DataToSketch

    public DataToSketch(String lgK)

Constructor with given lgK as string and default seed

Parameters:
  `lgK` - in a form of a String

#### DataToSketch

    public DataToSketch(String lgK,
                        String seed)

Constructor with given lgK and seed as strings

Parameters:
  `lgK` - in a form of a String
  `seed` - in a form of a String

#### DataToSketch

    public DataToSketch(int lgK,
                        long seed)

Base constructor.

Parameters:
  `lgK` - parameter controlling the sketch size and accuracy
  `seed` - parameter to use during hashing

### Method Detail

#### exec

    public org.apache.pig.data.DataByteArray exec(org.apache.pig.data.Tuple inputTuple)
                                           throws IOException

Top-level exec function. This method accepts an input Tuple containing a Bag of one or more inner Datum Tuples and returns a single serialized CpcSketch as a DataByteArray. Datum Tuple is a Tuple containing a single field, which can be one of the following (Java type: Pig type):

- Byte: BYTE
- Integer: INTEGER
- Long: LONG
- Float: FLOAT
- Double: DOUBLE
- String: CHARARRAY
- DataByteArray: BYTEARRAY

Note Strings as values are normally typed as DataType.CHARARRAY, which will be encoded as UTF-8 prior to being submitted to the sketch. If the user requires a different encoding for cross-platform compatibility, it is recommended that these values be encoded prior to being submitted in a DataBag and then typed as a DataType.BYTEARRAY.

Specified by:
  `exec` in class `org.apache.pig.EvalFunc<org.apache.pig.data.DataByteArray>`
Parameters:
  `inputTuple` - A tuple containing a single bag, containing Datum Tuples.
Returns:
  serialized CpcSketch
Throws:
  `IOException` - from Pig
See Also:
  "org.apache.pig.EvalFunc.exec(org.apache.pig.data.Tuple)"

#### accumulate

    public void accumulate(org.apache.pig.data.Tuple inputTuple)
                    throws IOException

An Accumulator version of the standard exec() method. Like exec(), accumulator is called with a bag of Datum Tuples. Unlike exec(), it doesn't serialize the result at the end. Instead, it can be called multiple times, each time with another bag of Datum Tuples to be input to the sketch.

Specified by:
  `accumulate` in interface `org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>`
Parameters:
  `inputTuple` - A tuple containing a single bag, containing Datum Tuples.
Throws:
  `IOException` - by Pig
See Also:
  `exec(org.apache.pig.data.Tuple)`, "org.apache.pig.Accumulator.accumulate(org.apache.pig.data.Tuple)"

#### getValue

    public org.apache.pig.data.DataByteArray getValue()

Returns the sketch that has been built up by multiple calls to `accumulate(org.apache.pig.data.Tuple)`.

Specified by:
  `getValue` in interface `org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>`
Returns:
  serialized CpcSketch
See Also:
  "org.apache.pig.Accumulator.getValue()"

#### cleanup

    public void cleanup()

Cleans up the UDF state after being called using the `Accumulator` interface.

Specified by:
  `cleanup` in interface `org.apache.pig.Accumulator<org.apache.pig.data.DataByteArray>`
See Also:
  "org.apache.pig.Accumulator.cleanup()"

#### getInitial

    public String getInitial()

Specified by:
  `getInitial` in interface `org.apache.pig.Algebraic`

#### getIntermed

    public String getIntermed()

Specified by:
  `getIntermed` in interface `org.apache.pig.Algebraic`

#### getFinal

    public String getFinal()

Specified by:
  `getFinal` in interface `org.apache.pig.Algebraic`