You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2017/03/01 06:40:45 UTC
[jira] [Commented] (BEAM-1514) change default timestamp in KafkaIO
[ https://issues.apache.org/jira/browse/BEAM-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889605#comment-15889605 ]
ASF GitHub Bot commented on BEAM-1514:
--------------------------------------
GitHub user XuMingmin opened a pull request:
https://github.com/apache/beam/pull/2135
BEAM-1514: change default timestamp in KafkaIO
I use Java reflection here to check if timestamp is available and get the value, based on the performance test as below. As the function is called for every record, it's worth to choose a faster option.
Direct: 1ns
SpEL: 15ns
Reflect: 5ns
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/XuMingmin/beam master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2135.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2135
----
commit 9b0505ca5f244e28f8c83ca3f1037ae45e516791
Author: mingmxu <mi...@ebay.com>
Date: 2017-02-22T00:40:16Z
junit test for both Kafka 0.9 and 0.10
commit 07e207d845943c485e6280e1705cc7c153a2ef35
Author: mingmxu <mi...@ebay.com>
Date: 2017-02-22T07:10:45Z
add doc for how to test, and how it works
commit c1d74af9f69cca6d5a7a79a67ef184a13ea6b2f1
Author: XuMingmin <mm...@gmail.com>
Date: 2017-02-22T17:58:29Z
Merge pull request #1 from apache/master
pull latest code
commit 71f83b7af6379155e30599f331ac39e68b046783
Author: mingmxu <mi...@ebay.com>
Date: 2017-02-22T18:01:29Z
JavaDoc style
commit 931027519be8f15999fef834609db6e2e63e087d
Author: mingmxu <mi...@ebay.com>
Date: 2017-02-23T01:14:55Z
revise to keep default kafka version as 0.9.0.1
commit d7093563067a31f268305bf001eb578e899745ec
Author: mingmxu <mi...@ebay.com>
Date: 2017-02-24T22:12:21Z
remove changes in TimestampTransform and WindowDoFnOperator
commit 280bc870568aff6825168a3faab3ed62870f6ee7
Author: mingmxu <mi...@ebay.com>
Date: 2017-02-24T23:09:12Z
revert TimestampTransform
commit 50d426be9b5cf7cc9d8c68a0a767a8cdcf8ab888
Author: mingmxu <mi...@ebay.com>
Date: 2017-02-24T23:10:19Z
revert TimestampTransform
commit d6ef256f7fee618512fccf622c0088d795a32bea
Author: XuMingmin <mm...@gmail.com>
Date: 2017-03-01T06:12:28Z
Merge pull request #2 from apache/master
pull latest code from apache/beam
commit ad4c208992f1c8612bd319d82f6a28754032c9ad
Author: mingmxu <mi...@ebay.com>
Date: 2017-03-01T06:14:59Z
use Java reflection for timestamp()
commit a95a9a74e08548977569f9b791b6d255468ef5f3
Author: XuMingmin <mm...@gmail.com>
Date: 2017-03-01T06:23:01Z
Merge pull request #3 from XuMingmin/BEAM-1514
merge Beam 1514 to master
commit b496ffcfe2aac851ed9c163747edad1a1677fed8
Author: XuMingmin <mm...@gmail.com>
Date: 2017-03-01T06:24:58Z
Update pom.xml
commit 220b5af8172bee1319394c2cb4168bc6fbbc059e
Author: XuMingmin <mm...@gmail.com>
Date: 2017-03-01T06:28:43Z
Update ConsumerSpEL.java
----
> change default timestamp in KafkaIO
> -----------------------------------
>
> Key: BEAM-1514
> URL: https://issues.apache.org/jira/browse/BEAM-1514
> Project: Beam
> Issue Type: Improvement
> Components: sdk-java-extensions
> Reporter: Xu Mingmin
> Assignee: Xu Mingmin
>
> When user use Kafka 0.10, the field 'timestamp' from Kafka should be used as the default event timestamp.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)