[jira] [Created] (STORM-2669) Extend the BinaryEventDataScheme in storm-eventhubs to include MessageId in addition to system properties

["Ravi Tandon (JIRA)" <ji...@apache.org>]

Ravi Tandon created STORM-2669:
----------------------------------

             Summary: Extend the BinaryEventDataScheme in storm-eventhubs to include MessageId in addition to system properties
                 Key: STORM-2669
                 URL: https://issues.apache.org/jira/browse/STORM-2669
             Project: Apache Storm
          Issue Type: Improvement
          Components: storm-eventhubs
    Affects Versions: 2.0.0
         Environment: Ubuntu/Azure
            Reporter: Ravi Tandon
            Priority: Minor
             Fix For: 2.0.0


Currently there are two types of EventDataScheme included with the storm-eventhubs spout.

The default is the StringEventDataScheme that emits a single output field, the message itself as a string.

There is an additional BinaryEventDataScheme that passes the message as is, but also has two additional fields: metadata and system_metadata that is passed by eventhubs-client.

The system_metadata only contains the sequence number, offset and enqeued time of an event.

As part of recent requirements by certain applications for tracking an event, they also need the partition id. The partition id is NOT sent by the eventhubs-client, instead the partition manager in the spout already has this information.

The goal of this JIRA is to introduce another output field in BinaryEventDataScheme that contains the MessageId for an event. The messageId will contain: partitionId, sequence number and the offset information for any downstream bolt to be able to locate where the message arrived from.

I will also be fixing any maven checkstyle warnings/errors in the files that I will be committing changes in.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)