You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by GitBox <gi...@apache.org> on 2019/12/19 13:16:55 UTC
[GitHub] [kylin] hit-lacus opened a new pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
hit-lacus opened a new pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
URL: https://github.com/apache/kylin/pull/1009
…ncel long-running query
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] hit-lacus commented on a change in pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
Posted by GitBox <gi...@apache.org>.
hit-lacus commented on a change in pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
URL: https://github.com/apache/kylin/pull/1009#discussion_r368823403
##########
File path: stream-core/src/main/java/org/apache/kylin/stream/core/query/MultiThreadsResultCollector.java
##########
@@ -69,38 +81,41 @@ public MultiThreadsResultCollector(int numOfWorkers, int timeout) {
@Override
public boolean hasNext() {
- boolean exits = (internalIT.hasNext() || queue.size() > 0);
+ boolean exits = (internalIT.hasNext() || !recordCachePool.isEmpty());
if (!exits) {
while (notCompletedWorkers.get() > 0) {
Thread.yield();
- long takeTime = System.currentTimeMillis() - startTime;
- if (takeTime > timeout) {
+ if (System.currentTimeMillis() > deadline) {
+ masterThread.interrupt(); // notify main thread
Review comment:
1. If `WorkSubmitter` is interrupted, `WorkSubmitter` will leave it `run` method. As you can see in finally block of `run` method, it cancel every sub-thread(`ResultIterateWorker` will thrown InterruptedException).
2. In finally block of `ResultIterateWorker` 's `run`, it will release workersSemaphore.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] hit-lacus commented on a change in pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
Posted by GitBox <gi...@apache.org>.
hit-lacus commented on a change in pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
URL: https://github.com/apache/kylin/pull/1009#discussion_r368823403
##########
File path: stream-core/src/main/java/org/apache/kylin/stream/core/query/MultiThreadsResultCollector.java
##########
@@ -69,38 +81,41 @@ public MultiThreadsResultCollector(int numOfWorkers, int timeout) {
@Override
public boolean hasNext() {
- boolean exits = (internalIT.hasNext() || queue.size() > 0);
+ boolean exits = (internalIT.hasNext() || !recordCachePool.isEmpty());
if (!exits) {
while (notCompletedWorkers.get() > 0) {
Thread.yield();
- long takeTime = System.currentTimeMillis() - startTime;
- if (takeTime > timeout) {
+ if (System.currentTimeMillis() > deadline) {
+ masterThread.interrupt(); // notify main thread
Review comment:
1. If `WorkSubmitter` is interrupted, `WorkSubmitter` will leave it `run` method. As you can see in finally block of `run` method, it [cancel](https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html#cancel(boolean)) every sub-thread(`ResultIterateWorker` will thrown InterruptedException).
2. In finally block of `ResultIterateWorker` 's `run`, it will release workersSemaphore.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
Posted by GitBox <gi...@apache.org>.
nichunen commented on a change in pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
URL: https://github.com/apache/kylin/pull/1009#discussion_r368828939
##########
File path: stream-core/src/main/java/org/apache/kylin/stream/core/query/MultiThreadsResultCollector.java
##########
@@ -69,38 +81,41 @@ public MultiThreadsResultCollector(int numOfWorkers, int timeout) {
@Override
public boolean hasNext() {
- boolean exits = (internalIT.hasNext() || queue.size() > 0);
+ boolean exits = (internalIT.hasNext() || !recordCachePool.isEmpty());
if (!exits) {
while (notCompletedWorkers.get() > 0) {
Thread.yield();
- long takeTime = System.currentTimeMillis() - startTime;
- if (takeTime > timeout) {
+ if (System.currentTimeMillis() > deadline) {
+ masterThread.interrupt(); // notify main thread
Review comment:
OK, I see it
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen commented on a change in pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
Posted by GitBox <gi...@apache.org>.
nichunen commented on a change in pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
URL: https://github.com/apache/kylin/pull/1009#discussion_r368802651
##########
File path: stream-core/src/main/java/org/apache/kylin/stream/core/query/MultiThreadsResultCollector.java
##########
@@ -69,38 +81,41 @@ public MultiThreadsResultCollector(int numOfWorkers, int timeout) {
@Override
public boolean hasNext() {
- boolean exits = (internalIT.hasNext() || queue.size() > 0);
+ boolean exits = (internalIT.hasNext() || !recordCachePool.isEmpty());
if (!exits) {
while (notCompletedWorkers.get() > 0) {
Thread.yield();
- long takeTime = System.currentTimeMillis() - startTime;
- if (takeTime > timeout) {
+ if (System.currentTimeMillis() > deadline) {
+ masterThread.interrupt(); // notify main thread
Review comment:
Will the workersSemaphore be released if masterThread is interrupted?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [kylin] nichunen merged pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
Posted by GitBox <gi...@apache.org>.
nichunen merged pull request #1009: KYLIN-4305 Streaming Receiver cannot limit income query request or ca…
URL: https://github.com/apache/kylin/pull/1009
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services