You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Daljeet Singh (Jira)" <ji...@apache.org> on 2022/05/17 19:55:00 UTC
[jira] [Updated] (BEAM-14482) Mismatch in event publish_time
[ https://issues.apache.org/jira/browse/BEAM-14482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Daljeet Singh updated BEAM-14482:
---------------------------------
Description:
below is the example code, where we are trying to get the publish_time of the pubsub message in our DoFn(). It seems the type has changed in the version of apache beam starting 2.36.0. However, I was not able to find any release notes that talk about this change. Any reference will be helpful.
{code:java}
class ProtoToDictDoFn(beam.DoFn):
def process(self, element, publish_time=beam.DoFn.TimestampParam):
"""
element is type PubsubMessage(),
"""
print('-------------')
print(type(element.publish_time))
print(element.publish_time)
print('-------------')
Output:
— for version 2.35.0 —
<class 'google.protobuf.timestamp_pb2.Timestamp'>
seconds: 1652814206
nanos: 417000000
— for version >= 2.36.0 —
<class 'proto.datetime_helpers.DatetimeWithNanoseconds'>
2022-05-17 19:02:06.314000+00:00 {code}
Which seems to be an issue. As per the [google pubsub documentation publish time should be type "google.protobuf.timestamp_pb2.Timestamp"|https://github.com/googleapis/python-pubsub/blob/main/google/pubsub_v1/types/pubsub.py#L232]
Any clue, what has changed or caused this issue?
was:
below is the example code, where we are trying to get the publish_time of the pubsub message in our DoFn(). It seems the type has changed in the version of apache beam starting 2.36.0. However, I was not able to find any release notes that talk about this change. Any reference will be helpful.
class ProtoToDictDoFn(beam.DoFn):
def process(self, element, publish_time=beam.DoFn.TimestampParam):
"""
element is type PubsubMessage(),
"""
print('-------------')
print(type(element.publish_time))
print(element.publish_time)
print('-------------')
Output:
--- for version 2.35.0 ---
<class 'google.protobuf.timestamp_pb2.Timestamp'>
seconds: 1652814206
nanos: 417000000
--- for version >= 2.36.0 ---
<class 'proto.datetime_helpers.DatetimeWithNanoseconds'>
2022-05-17 19:02:06.314000+00:00
Which seems to be an issue. As per the [google pubsub documentation publish time should be type "google.protobuf.timestamp_pb2.Timestamp"|https://github.com/googleapis/python-pubsub/blob/main/google/pubsub_v1/types/pubsub.py#L232]
Any clue, what has changed or caused this issue?
> Mismatch in event publish_time
> ------------------------------
>
> Key: BEAM-14482
> URL: https://issues.apache.org/jira/browse/BEAM-14482
> Project: Beam
> Issue Type: Bug
> Components: beam-community
> Affects Versions: 2.36.0
> Reporter: Daljeet Singh
> Priority: P2
> Labels: python
>
> below is the example code, where we are trying to get the publish_time of the pubsub message in our DoFn(). It seems the type has changed in the version of apache beam starting 2.36.0. However, I was not able to find any release notes that talk about this change. Any reference will be helpful.
> {code:java}
> class ProtoToDictDoFn(beam.DoFn):
> def process(self, element, publish_time=beam.DoFn.TimestampParam):
> """
> element is type PubsubMessage(),
> """
> print('-------------')
> print(type(element.publish_time))
> print(element.publish_time)
> print('-------------')
> Output:
> — for version 2.35.0 —
> <class 'google.protobuf.timestamp_pb2.Timestamp'>
> seconds: 1652814206
> nanos: 417000000
> — for version >= 2.36.0 —
> <class 'proto.datetime_helpers.DatetimeWithNanoseconds'>
> 2022-05-17 19:02:06.314000+00:00 {code}
> Which seems to be an issue. As per the [google pubsub documentation publish time should be type "google.protobuf.timestamp_pb2.Timestamp"|https://github.com/googleapis/python-pubsub/blob/main/google/pubsub_v1/types/pubsub.py#L232]
> Any clue, what has changed or caused this issue?
--
This message was sent by Atlassian Jira
(v8.20.7#820007)