2. Introduction — OpenSplice GPB Tutorial

### Navigation

- index
- next |
- previous |
- OpenSplice GPB Tutorial »

# 2. Introduction

## 2.1. Google Protocol Buffers for DDS

Vortex OpenSplice is capable of using the Google Protocol Buffer (GPB) system for publishing and subscribing GPB messages in a DDS system. This makes it possible to use GPB as an alternative to OMG-IDL for those who prefer to use GPB rather than IDL. With the seamless integration of GPB and DDS technologies there is no need for OMG-IDL knowledge or visibility when working with GPB data models, and no OMG-DDS data-types are needed in the application (no explicit type-mapping between GPB and DDS types is required).

This results in an easy migration of GPB users to DDS(-based data-sharing) with data-centric GPB with support for keys, filters and (future) QoS-annotations (ony a few DDS calls are needed). Also easy migration of DDS applications to GPB(-based data-modeling), only the field accessors change.

This Tutorial will describe how this is done for the language bindings Java5 and ISO-C++ by defining a GPB message layout which is compiled into proper interfaces for the Vortex DDS system.

### 2.1.1. GPB Installation and usage with DDS

Google Protocol Buffers (GPB) can be downloaded from the following locations:

Linux: https://github.com/google/protobuf/releases/download/v2.6.1/protobuf-2.6.1.tar.gz

Windows: https://github.com/google/protobuf/releases/download/v2.6.1/protobuf-2.6.1.zip

After unpacking follow the install instructions located in install.txt in the unpacked directory. For windows there is a visual studio solution file that will build everything that is needed.

  In order for GPB to work with DDS an environment variable PROTOBUF_HOME needs to be
  set that points to the unpacked directory.
  For windows also another environment variable PROTOBUF_LIB_HOME needs to be set that
  points to the directory that contains the generated libprotobuf.lib.

### 2.1.2. IDL usage in a DDS system

In a Data Distributed System (DDS) as a Global DataSpace (GDS) for ubiquitous information-sharing in distributed systems as specified by the Object Management Group (OMG), the data is traditionally captured in the platform- and language-independent OMG-IDL language. The relational model of DDS is supported by the notion of identifying key fields in these data structures where structure/content-awareness by the middleware allows for dynamic querying and filtering of data.

### 2.1.3. Google Protocol Buffers

Google Protocol Buffers (GPB) are a flexible, efficient, automated mechanism for serializing structured data; think XML, but smaller, faster, and simpler. One can define how data needs to be structured once, after which language-specific source code can be generated to easily write and read this structured data to and from a variety of data streams using a variety of languages. The information structure is defined in so-called protocol buffer message types in `.proto` files. Each protocol buffer message is a small logical record of information, containing a series of name-value pairs. This approach is quite similar to using IDL for data modeling in combination with an IDL compiler (as available in OpenSplice and DDS implementations in general).

Additionally, the GPB data structure can be updated without breaking deployed programs that are compiled against the ‘old’ format, similar to the xTypes concept as defined for DDS.

#### 2.1.3.1. Using a GPB data-model instead of an IDL data-model

For an IDL-OMG based application, the IDL file is compiled with the IDL-PP compiler to generate the needed classes.

For Java as an example, `Address.idl` will (among others) be compiled into:

> - `Address.java`
> - `AddressTypesSupport.java`
> - `AddressDataWriter.java`
> - `AddressDataReader.java`
> - ...

Using a GPB data-model, it is not necessary to create IDL files. The `protoc_gen_ddsJava` plug-in in OpenSplice will create them from the given `.proto` data-model.

For the GPB `.proto` based application, the `.proto` file is first compiled by the Google `protoc` compiler. This compiler will call the `protoc_gen_ddsJava` plug-in in OpenSplice with the `.proto` data parsed into a `CodeGeneratorRequest` protocol buffer.

The OpenSplice plug-in will generate an IDL file from this data. Any field member that is marked as key or filterable is explicitly mapped to a member in the IDL type.

The complete serialized `.proto` message is stored in the generic `ospl_protobuf_data` attribute as a sequence of bytes (making it opaque data for DDS). The mapping between data types is given in the table Mapping of GPB types to DDS types.

As the next step the IDL-PP compiler will generate the previously-named files from the idl file needed for the DDS domain. The Google `protoc` compiler will generate the classes needed for the GPB domain.

The dds options for the proto file are given in the omg/dds/descriptor.proto file listed below. This proto file shows how the different dds options on the proto file are interpreted, and gives the unique id `1016` to the dds types.

  Note that the id 1016 has officially been granted to the Vortex product by Google.
  This ensures these options are always unique and won’t clash with any options used by users.

How mapping is done between the different languages is shown below in the table Mapping of GPB types to DDS types.

#### 2.1.3.2. omg/dds/descriptor.proto

    import "google/protobuf/descriptor.proto";

    package omg.dds;

    option java_package = "org.omg.dds.protobuf";
    option java_outer_classname = "DescriptorProtos";

    /* These options are required for any .proto message that needs to be available
     * in DDS.
     *
     * - name: An optional scoped name to allow overriding the name of the type in
     *   DDS. The dot('.') can be used as a scoping separator. In case the name
     *   starts with a dot, the name will be interpreted as an absolute scope name.
     *   If not, the name will be considered relative to the scope of the message
     *   including its 'package'.
     */
    message MessageOptions {
        optional string name = 1 [default = ""];
    }

    extend google.protobuf.MessageOptions {
      optional omg.dds.MessageOptions type  = 1016;
    }

    /* These options are provided to assign specific behaviour to a member of a
     * DDS-enabled .proto message in DDS. These options will only be applied in case
     * the omg.dds.MessageOptions.type has been applied to the message in which the
     * member is modeled.
     *
     * - key: Make the member part of the key of the type in DDS. Each unique
     *   key-value will become a separate instance with its own history in DDS. Only
     *   'required' members can be made part of the key and key-definitions cannot
     *   be modified in future versions of the message. Members that are part of the
     *   key are automatically filterable as well.
     *
     * - filterable: Ensure the member is filterable in DDS using a so-called
     *   ContentFilteredTopic or QueryCondition. Only 'required' members can be made
     *   filterable and filterable definitions cannot be modified in future versions
     *   of the message.
     *
     * - name: Override the name of the member in DDS. This only applies to members
     *   that are marked as key and/or filterable.
     */
    message FieldOptions {
        optional bool key = 1 [default = false];
        optional bool filterable = 2 [default = false];
        optional string name = 3 [default = ""];
    }

    extend google.protobuf.FieldOptions {
      optional omg.dds.FieldOptions member = 1016;
    }

#### 2.1.3.3. Mapping of GPB types to DDS types

.proto Type | Notes | C++ Type | Java Type | DDS IDL Type
--- | --- | --- | --- | ---
double |  | double | double | double
float |  | float | float | float
int32 | Uses variable-length encoding. Inefficient for encoding negative numbers; if your field is likely to have negative values, use sint32 instead | int32 | int | long
int64 | Uses variable-length encoding. Inefficient for encoding negative numbers; if your field is likely to have negative values, use sint64 instead | int64 | long | long long
uint32 | Uses variable-length encoding | uint32 | int | unsigned long
uint64 | Uses variable-length encoding | uint64 | long | unsigned long long
sint32 | Uses variable-length encoding. Signed int value. These encode negative numbers more efficiently than regular int32s. | int32 | int | long
sint64 | Uses variable-length encoding. Signed int value. These encode negative numbers more efficiently than regular int64s. | int64 | long | long long
fixed32 | Always four bytes. More efficient than uint32 if values are often greater than 2^28. | uint32 | int | unsigned long
fixed64 | Always eight bytes. More efficient than uint64 if values are often greater than 2^56. | uint64 | long | unsigned long long
sfixed32 | Always four bytes. | int32 | int | long
sfixed64 | Always eight bytes. | int64 | long | long long
bool |  | bool | boolean | bool
string | A string must always contain UTF-8 encoded or 7-bit ASCII text | string | String | string

### Navigation

- index
- next |
- previous |
- OpenSplice GPB Tutorial »