You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@carbondata.apache.org by sraghunandan <gi...@git.apache.org> on 2017/08/07 05:32:25 UTC

[GitHub] carbondata pull request #1242: [WIP]Offheap sort size validation

GitHub user sraghunandan opened a pull request:

    https://github.com/apache/carbondata/pull/1242

    [WIP]Offheap sort size validation

    

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/sraghunandan/carbondata-1 offheap_sort_size_validation

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1242.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1242
    
----
commit 69220fce0a4e2196e0a094ac3a0f5c5fb4013bd5
Author: sraghunandan <ca...@gmail.com>
Date:   2017-08-07T01:38:45Z

    Request 1 executor when preferred locations are empty

commit 8b7027ac98090dcd799f2e91feb106ef3a02915a
Author: sraghunandan <ca...@gmail.com>
Date:   2017-08-07T02:03:29Z

    Validate off heap sort size value configured is in valid range

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #1242: [WIP]Offheap sort size validation

Posted by asfgit <gi...@git.apache.org>.

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1242
  
    Can one of the admins verify this patch?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #1242: [CARBONDATA-1389]Offheap sort size validation

Posted by sraghunandan <gi...@git.apache.org>.

Github user sraghunandan closed the pull request at:

    https://github.com/apache/carbondata/pull/1242


---

[GitHub] carbondata issue #1242: [CARBONDATA-1389]Offheap sort size validation

Posted by sraghunandan <gi...@git.apache.org>.

Github user sraghunandan commented on the issue:

    https://github.com/apache/carbondata/pull/1242
  
    retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata pull request #1242: [CARBONDATA-1389]Offheap sort size validation

Posted by sraghunandan <gi...@git.apache.org>.

Github user sraghunandan commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1242#discussion_r135423319
  
    --- Diff: integration/spark-common/src/main/scala/org/apache/spark/sql/hive/DistributionUtil.scala ---
    @@ -115,7 +115,9 @@ object DistributionUtil {
           sparkContext: SparkContext): Seq[String] = {
         val confExecutors: Int = getConfiguredExecutors(sparkContext)
         LOGGER.info(s"Executors configured : $confExecutors")
    -    val requiredExecutors = if (nodesOfData < 1 || nodesOfData > confExecutors) {
    +    val requiredExecutors = if (nodesOfData < 1) {
    +      1
    +    } else if (nodesOfData > confExecutors) {
    --- End diff --
    
    This is worked over 1241.When 1241 is merged, this change will not reflect


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #1242: [WIP]Offheap sort size validation

Posted by ravipesala <gi...@git.apache.org>.

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1242
  
    I think validation already present. `CarbonProperties.getSortMemoryChunkSizeInMB`. It is duplicated


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #1242: [CARBONDATA-1389]Offheap sort size validation

Posted by ravipesala <gi...@git.apache.org>.

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1242
  
    SDV Build Success with Spark 2.1, Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/230/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #1242: [WIP]Offheap sort size validation

Posted by ravipesala <gi...@git.apache.org>.

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1242
  
    Build Failed with Spark 1.6, Please check CI http://144.76.159.231:8080/job/ApacheCarbonPRBuilder/806/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #1242: [WIP]Offheap sort size validation

Posted by ravipesala <gi...@git.apache.org>.

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1242
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/3404/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] carbondata issue #1242: [CARBONDATA-1389]Offheap sort size validation

Posted by sraghunandan <gi...@git.apache.org>.

Github user sraghunandan commented on the issue:

    https://github.com/apache/carbondata/pull/1242
  
    retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---