You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@storm.apache.org by sr...@apache.org on 2019/01/30 11:41:26 UTC
[storm] branch master updated (604ecf7 -> 8f49e06)
This is an automated email from the ASF dual-hosted git repository.
srdo pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/storm.git.
from 604ecf7 Merge branch 'master' of govind-menon's fork into master
add 0c82b8f STORM-3309: Fix flaky tick tuple test
new df9fde4 Merge branch 'STORM-3309' of https://github.com/srdo/storm into asfgit-master
add 9069e30 STORM-3310: Make JCQueueTest wait for consumer to read all queued items before terminating
new f7883f8 Merge branch 'STORM-3310' of https://github.com/srdo/storm into asfgit-master
add b1a8a06 Doc clarification on returning `null` from RecordTranslator.apply()
new 31c7e22 Merge branch 'master' of https://github.com/esamson/storm into asfgit-master
add 111aa8d MINOR: Remove unused parameter from storm-kafka-client-example docs
new 564bf83 Merge branch 'fix-kafka-example-docs' of https://github.com/srdo/storm into asfgit-master
add 444e051 STORM-3318: Complete information in Class NewKafkaSpoutOffsetQuery
new 6175a04 Merge branch 'master' of https://github.com/MichealShin/storm into asfgit-master
new c466f5f MINOR: Remove reference to deleted Subscription interface in storm-kafka-client docs
new 8f49e06 Merge branch 'minor-fix-assignment-docs' of https://github.com/srdo/storm into asfgit-master
The 7 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
docs/storm-kafka-client.md | 8 +-
.../storm-kafka-client-examples/README.markdown | 4 +-
.../apache/storm/kafka/spout/RecordTranslator.java | 3 +-
.../kafka/monitor/NewKafkaSpoutOffsetQuery.java | 33 +++-
.../jvm/org/apache/storm/executor/Executor.java | 6 +-
.../apache/storm/executor/bolt/BoltExecutor.java | 2 +-
.../apache/storm/executor/spout/SpoutExecutor.java | 2 +-
.../org/apache/storm/utils/InprocMessaging.java | 2 +-
.../src/jvm/org/apache/storm/utils/Utils.java | 9 ++
.../apache/storm/zookeeper/ClientZookeeper.java | 2 +-
.../jvm/org/apache/storm/utils/JCQueueTest.java | 167 +++++++++++----------
storm-core/pom.xml | 8 +
.../org/apache/storm/integration/TestingTest.java | 8 +-
.../storm/integration/TopologyIntegrationTest.java | 4 +-
storm-server/pom.xml | 5 +
.../main/java/org/apache/storm/LocalCluster.java | 2 +
.../test/java/org/apache/storm}/AssertLoop.java | 6 +-
.../test/java/org/apache/storm/TickTupleTest.java | 141 ++++++++++-------
storm-server/src/test/resources/log4j2.xml | 22 +--
19 files changed, 265 insertions(+), 169 deletions(-)
rename {storm-core/test/jvm/org/apache/storm/integration => storm-server/src/test/java/org/apache/storm}/AssertLoop.java (93%)
[storm] 01/07: MINOR: Remove reference to deleted Subscription
interface in storm-kafka-client docs
Posted by sr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
srdo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/storm.git
commit c466f5f563cf675c3b3c08701e86e4a317741641
Author: Stig Rohde Døssing <sr...@apache.org>
AuthorDate: Wed Jan 23 19:10:07 2019 +0100
MINOR: Remove reference to deleted Subscription interface in storm-kafka-client docs
---
docs/storm-kafka-client.md | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/docs/storm-kafka-client.md b/docs/storm-kafka-client.md
index 1123bdd..73c88c7 100644
--- a/docs/storm-kafka-client.md
+++ b/docs/storm-kafka-client.md
@@ -245,9 +245,13 @@ streams. If you are doing this for Trident a value must be in the List returned
otherwise trident can throw exceptions.
-### Manual Partition Assigment (ADVANCED)
+### Manual Partition Assignment (ADVANCED)
-By default the KafkaSpout instances will be assigned partitions using a round robin strategy. If you need to customize partition assignment, you must implement the `ManualPartitioner` interface. The implementation can be passed to the `ManualPartitionSubscription` constructor, and the `Subscription` can then be set in the `KafkaSpoutConfig` via the `KafkaSpoutConfig.Builder` constructor. Please take care when supplying a custom implementation, since an incorrect `ManualPartitioner` imple [...]
+By default the KafkaSpout instances will be assigned partitions using a round robin strategy. If you need to customize partition assignment, you must implement the `ManualPartitioner` interface. You can pass your implementation to the `KafkaSpoutConfig.Builder` constructor. Please take care when supplying a custom implementation, since an incorrect `ManualPartitioner` implementation could leave some partitions unread, or concurrently read by multiple spout instances. See the `RoundRobinM [...]
+
+### Manual partition discovery
+
+You can customize how the spout discovers existing partitions, by implementing the `TopicFilter` interface. Storm-kafka-client ships with a few implementations. Like `ManualPartitioner`, you can pass your implementation to the `KafkaSpoutConfig.Builder` constructor. Note that the `TopicFilter` is only responsible for discovering partitions, deciding which of the discovered partitions to subscribe to is the responsibility of `ManualPartitioner`.
## Using storm-kafka-client with different versions of kafka
[storm] 06/07: Merge branch 'master' of
https://github.com/MichealShin/storm into asfgit-master
Posted by sr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
srdo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/storm.git
commit 6175a04b1345c49e0f065f5bc6af85ecf8a0f16d
Merge: 564bf83 444e051
Author: Stig Rohde Døssing <sr...@apache.org>
AuthorDate: Wed Jan 30 12:39:30 2019 +0100
Merge branch 'master' of https://github.com/MichealShin/storm into asfgit-master
.../kafka/monitor/NewKafkaSpoutOffsetQuery.java | 33 ++++++++++++++++++----
1 file changed, 27 insertions(+), 6 deletions(-)
[storm] 02/07: Merge branch 'STORM-3309' of
https://github.com/srdo/storm into asfgit-master
Posted by sr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
srdo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/storm.git
commit df9fde4ae4fcea348c3823a1aa0a974ad18304be
Merge: 604ecf7 0c82b8f
Author: Stig Rohde Døssing <sr...@apache.org>
AuthorDate: Wed Jan 30 12:37:48 2019 +0100
Merge branch 'STORM-3309' of https://github.com/srdo/storm into asfgit-master
.../jvm/org/apache/storm/executor/Executor.java | 6 +-
.../apache/storm/executor/bolt/BoltExecutor.java | 2 +-
.../apache/storm/executor/spout/SpoutExecutor.java | 2 +-
.../org/apache/storm/utils/InprocMessaging.java | 2 +-
.../src/jvm/org/apache/storm/utils/Utils.java | 9 ++
.../apache/storm/zookeeper/ClientZookeeper.java | 2 +-
storm-core/pom.xml | 8 ++
.../org/apache/storm/integration/TestingTest.java | 8 +-
.../storm/integration/TopologyIntegrationTest.java | 4 +-
storm-server/pom.xml | 5 +
.../main/java/org/apache/storm/LocalCluster.java | 2 +
.../test/java/org/apache/storm}/AssertLoop.java | 6 +-
.../test/java/org/apache/storm/TickTupleTest.java | 141 +++++++++++++--------
storm-server/src/test/resources/log4j2.xml | 22 ++--
14 files changed, 139 insertions(+), 80 deletions(-)
[storm] 07/07: Merge branch 'minor-fix-assignment-docs' of
https://github.com/srdo/storm into asfgit-master
Posted by sr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
srdo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/storm.git
commit 8f49e06998abb4dfc50f51d78b6784ebd04844fb
Merge: 6175a04 c466f5f
Author: Stig Rohde Døssing <sr...@apache.org>
AuthorDate: Wed Jan 30 12:40:13 2019 +0100
Merge branch 'minor-fix-assignment-docs' of https://github.com/srdo/storm into asfgit-master
docs/storm-kafka-client.md | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
[storm] 05/07: Merge branch 'fix-kafka-example-docs' of
https://github.com/srdo/storm into asfgit-master
Posted by sr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
srdo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/storm.git
commit 564bf83ecbe59d8b951867247edd2bf276b8495a
Merge: 31c7e22 111aa8d
Author: Stig Rohde Døssing <sr...@apache.org>
AuthorDate: Wed Jan 30 12:38:54 2019 +0100
Merge branch 'fix-kafka-example-docs' of https://github.com/srdo/storm into asfgit-master
examples/storm-kafka-client-examples/README.markdown | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
[storm] 03/07: Merge branch 'STORM-3310' of
https://github.com/srdo/storm into asfgit-master
Posted by sr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
srdo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/storm.git
commit f7883f82945113f4999dc2f23b299234b58142a4
Merge: df9fde4 9069e30
Author: Stig Rohde Døssing <sr...@apache.org>
AuthorDate: Wed Jan 30 12:38:03 2019 +0100
Merge branch 'STORM-3310' of https://github.com/srdo/storm into asfgit-master
.../jvm/org/apache/storm/utils/JCQueueTest.java | 167 +++++++++++----------
1 file changed, 89 insertions(+), 78 deletions(-)
[storm] 04/07: Merge branch 'master' of
https://github.com/esamson/storm into asfgit-master
Posted by sr...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
srdo pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/storm.git
commit 31c7e22f809e4d5de27c06fbfd4ddf380e1059e1
Merge: f7883f8 b1a8a06
Author: Stig Rohde Døssing <sr...@apache.org>
AuthorDate: Wed Jan 30 12:38:36 2019 +0100
Merge branch 'master' of https://github.com/esamson/storm into asfgit-master
.../src/main/java/org/apache/storm/kafka/spout/RecordTranslator.java | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)