org.apache.datasketches.pig.theta

## Class DataToSketch

- java.lang.Object
- - org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>
  - - org.apache.datasketches.pig.theta.DataToSketch

All Implemented Interfaces:
  org.apache.pig.Accumulator<org.apache.pig.data.Tuple>, org.apache.pig.Algebraic

    public class DataToSketch
    extends org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>
    implements org.apache.pig.Accumulator<org.apache.pig.data.Tuple>, org.apache.pig.Algebraic

This is a Pig UDF that builds Sketches from data. To assist Pig, this class implements both the Accumulator and Algebraic interfaces.

### Nested Class Summary

Nested Classes
`static class` `DataToSketch.Initial` Class used to calculate the initial pass of an Algebraic sketch operation.
`static class` `DataToSketch.IntermediateFinal` Class used to calculate the intermediate or final combiner pass of an Algebraic sketch operation.

### Nested classes/interfaces inherited from class org.apache.pig.EvalFunc

`org.apache.pig.EvalFunc.SchemaType`

### Field Summary

### Fields inherited from class org.apache.pig.EvalFunc

`log, pigLogger, reporter, returnType`

### Constructor Summary

Constructors
Constructor and Description
`DataToSketch()` Default constructor.
`DataToSketch(int nomEntries, float p, long seed)` Base constructor.
`DataToSketch(String nomEntriesStr)` String constructor.
`DataToSketch(String nomEntriesStr, String pStr)` String constructor.
`DataToSketch(String nomEntriesStr, String pStr, String seedStr)` Full string constructor.

### Method Summary

All Methods Instance Methods Concrete Methods
`void` `accumulate(org.apache.pig.data.Tuple inputTuple)` An Accumulator version of the standard exec() method.
`void` `cleanup()` Cleans up the UDF state after being called using the `Accumulator` interface.
`org.apache.pig.data.Tuple` `exec(org.apache.pig.data.Tuple inputTuple)` Top-level exec function.
`String` `getFinal()`
`String` `getInitial()`
`String` `getIntermed()`
`org.apache.pig.data.Tuple` `getValue()` Returns the sketch that has been built up by multiple calls to `accumulate(org.apache.pig.data.Tuple)`.
`org.apache.pig.impl.logicalLayer.schema.Schema` `outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input)`

### Methods inherited from class org.apache.pig.EvalFunc

`allowCompileTimeCalculation, finish, getArgToFuncMapping, getCacheFiles, getInputSchema, getLoadCaster, getLogger, getPigLogger, getReporter, getReturnType, getSchemaName, getSchemaType, getShipFiles, isAsynchronous, needEndOfAllInputProcessing, progress, setEndOfAllInput, setInputSchema, setPigLogger, setReporter, setUDFContextSignature, warn`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### DataToSketch

    public DataToSketch()

Default constructor. Assumes:

- See Default Nominal Entries
- p = 1.0. See Sampling Probability, p.
- See Default Update Seed

#### DataToSketch

    public DataToSketch(String nomEntriesStr)

String constructor. Assumes:

- p = 1.0. See Sampling Probability, p
- See Default Update Seed

Parameters:
  `nomEntriesStr` - See Nominal Entries

#### DataToSketch

    public DataToSketch(String nomEntriesStr,
                        String pStr)

String constructor. Assumes:

- See Default Update Seed

Parameters:
  `nomEntriesStr` - See Nominal Entries
  `pStr` - See Sampling Probability, p

#### DataToSketch

    public DataToSketch(String nomEntriesStr,
                        String pStr,
                        String seedStr)

Full string constructor.

Parameters:
  `nomEntriesStr` - See Nominal Entries.
  `pStr` - See Sampling Probability, p.
  `seedStr` - See Update Hash Seed.

#### DataToSketch

    public DataToSketch(int nomEntries,
                        float p,
                        long seed)

Base constructor.

Parameters:
  `nomEntries` - See Nominal Entries.
  `p` - See Sampling Probability, p.
  `seed` - See Update Hash Seed.

### Method Detail

#### exec

    public org.apache.pig.data.Tuple exec(org.apache.pig.data.Tuple inputTuple)
                                   throws IOException

Top-level exec function. This method accepts an input Tuple containing a Bag of one or more inner Datum Tuples and returns a single updated Sketch as a Sketch Tuple.

If a large number of calls is anticipated, leveraging either the Algebraic or Accumulator interfaces is recommended. Pig normally handles this automatically.

Internally, this method presents the inner Datum Tuples to a new Sketch, which is returned as a Sketch Tuple

Input Tuple

- Tuple: TUPLE (Must contain only one field)
  - index 0: DataBag: BAG (May contain 0 or more Inner Tuples)
    - index 0: Tuple: TUPLE Datum Tuple
    - ...
    - index n-1: Tuple: TUPLE Datum Tuple

Datum Tuple

- Tuple: TUPLE (Must contain only one field)
  - index 0: Java data type : Pig DataType: may be any one of:
    - Byte: BYTE
    - Integer: INTEGER
    - Long: LONG
    - Float: FLOAT
    - Double: DOUBLE
    - String: CHARARRAY
    - DataByteArray: BYTEARRAY

Note Strings as values are normally typed as DataType.CHARARRAY, which will be encoded as UTF-8 prior to being submitted to the sketch. If the user requires a different encoding for cross-platform compatibility, it is recommended that these values be encoded prior to being submitted in a DataBag and then typed as a DataType.BYTEARRAY.

Sketch Tuple

- Tuple: TUPLE (Contains exactly 1 field)
  - index 0: DataByteArray: BYTEARRAY = The serialization of a Sketch object.

Specified by:
  `exec` in class `org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>`
Parameters:
  `inputTuple` - A tuple containing a single bag, containing Datum Tuples.
Returns:
  Sketch Tuple. If inputTuple is null or empty, returns empty sketch (8 bytes).
Throws:
  `IOException` - from Pig.
See Also:
  "org.apache.pig.EvalFunc.exec(org.apache.pig.data.Tuple)"

#### outputSchema

    public org.apache.pig.impl.logicalLayer.schema.Schema outputSchema(org.apache.pig.impl.logicalLayer.schema.Schema input)

Overrides:
  `outputSchema` in class `org.apache.pig.EvalFunc<org.apache.pig.data.Tuple>`

#### accumulate

    public void accumulate(org.apache.pig.data.Tuple inputTuple)
                    throws IOException

An Accumulator version of the standard exec() method. Like exec(), accumulator is called with a bag of Datum Tuples. Unlike exec(), it doesn't serialize the sketch at the end. Instead, it can be called multiple times, each time with another bag of Datum Tuples to be input to the sketch.

Specified by:
  `accumulate` in interface `org.apache.pig.Accumulator<org.apache.pig.data.Tuple>`
Parameters:
  `inputTuple` - A tuple containing a single bag, containing Datum Tuples.
Throws:
  `IOException` - by Pig
See Also:
  `exec(org.apache.pig.data.Tuple)`, "org.apache.pig.Accumulator.accumulate(org.apache.pig.data.Tuple)"

#### getValue

    public org.apache.pig.data.Tuple getValue()

Returns the sketch that has been built up by multiple calls to `accumulate(org.apache.pig.data.Tuple)`.

Specified by:
  `getValue` in interface `org.apache.pig.Accumulator<org.apache.pig.data.Tuple>`
Returns:
  Sketch Tuple. (see `exec(org.apache.pig.data.Tuple)` for return tuple format)
See Also:
  "org.apache.pig.Accumulator.getValue()"

#### cleanup

    public void cleanup()

Cleans up the UDF state after being called using the `Accumulator` interface.

Specified by:
  `cleanup` in interface `org.apache.pig.Accumulator<org.apache.pig.data.Tuple>`
See Also:
  "org.apache.pig.Accumulator.cleanup()"

#### getInitial

    public String getInitial()

Specified by:
  `getInitial` in interface `org.apache.pig.Algebraic`

#### getIntermed

    public String getIntermed()

Specified by:
  `getIntermed` in interface `org.apache.pig.Algebraic`

#### getFinal

    public String getFinal()

Specified by:
  `getFinal` in interface `org.apache.pig.Algebraic`