You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@spark.apache.org by frreiss <gi...@git.apache.org> on 2016/09/03 00:00:37 UTC

[GitHub] spark pull request #14945: [SPARK-17386] Set default trigger interval to 1/1...

GitHub user frreiss opened a pull request:

    https://github.com/apache/spark/pull/14945

    [SPARK-17386] Set default trigger interval to 1/10 second

    ## What changes were proposed in this pull request?
    
    This pull request implements the most expedient change to fix SPARK-17386: Use a default trigger interval of 100ms instead of polling continuously for new data by default. I've changed the default value used in both `StreamingQueryManager` and `StreamTest` and created a new constant to facilitate changing both of those defaults simultaneously in the future.
    
    ## How was this patch tested?
    
    All existing regression tests pass, and most of those tests use the default trigger intervals that this PR changes.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/frreiss/spark-fred fred-17386

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/14945.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #14945
    
----
commit 13496416bf2409082e292314444cad504d1b3bf1
Author: frreiss <fr...@us.ibm.com>
Date:   2016-09-02T23:22:59Z

    Change default trigger to 1/10 second

commit b1e7fd55adaa7faf5344ea5ddc135cf9cbaf3507
Author: frreiss <fr...@us.ibm.com>
Date:   2016-09-02T23:23:22Z

    Merge branch 'master' of https://github.com/apache/spark into fred-17386

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14945: [SPARK-17386] Set default trigger interval to 1/10 secon...

Posted by AmplabJenkins <gi...@git.apache.org>.
Github user AmplabJenkins commented on the issue:

    https://github.com/apache/spark/pull/14945
  
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #14945: [SPARK-17386] Set default trigger interval to 1/1...

Posted by frreiss <gi...@git.apache.org>.
Github user frreiss closed the pull request at:

    https://github.com/apache/spark/pull/14945


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark issue #14945: [SPARK-17386] Set default trigger interval to 1/10 secon...

Posted by frreiss <gi...@git.apache.org>.
Github user frreiss commented on the issue:

    https://github.com/apache/spark/pull/14945
  
    On a closer reading of the code, there is a more expedient fix; change the default STREAMING_POLLING_DELAY parameter. Will redo.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org


[GitHub] spark pull request #14945: [SPARK-17386] Set default trigger interval to 1/1...

Posted by wangmiao1981 <gi...@git.apache.org>.
Github user wangmiao1981 commented on a diff in the pull request:

    https://github.com/apache/spark/pull/14945#discussion_r77442963
  
    --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala ---
    @@ -152,8 +152,8 @@ trait StreamTest extends QueryTest with SharedSQLContext with Timeouts {
     
       /** Starts the stream, resuming if data has already been processed. It must not be running. */
       case class StartStream(
    -      trigger: Trigger = ProcessingTime(0),
    -      triggerClock: Clock = new SystemClock)
    +                          trigger: Trigger = ProcessingTime.defaultTriggerInterval,
    --- End diff --
    
    Leading spaces should be removed.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org