You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by Sephiroth-Lin <gi...@git.apache.org> on 2017/04/06 13:04:10 UTC
[GitHub] incubator-carbondata pull request #752: [CARBONDATA-879] Clear driver side d...
GitHub user Sephiroth-Lin opened a pull request:
https://github.com/apache/incubator-carbondata/pull/752
[CARBONDATA-879] Clear driver side dict cache for CarbonDictionaryDecoder
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Sephiroth-Lin/incubator-carbondata carbondata-879
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/752.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #752
----
commit 07c4c71bcea737d9db87ac426c1fe077801e3b24
Author: Sephiroth-Lin <li...@gmail.com>
Date: 2017-04-06T13:01:13Z
Clear driver side dict cache for CarbonDictionaryDecoder
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #752: [CARBONDATA-879] Clear driver side dict cac...
Posted by QiangCai <gi...@git.apache.org>.
Github user QiangCai commented on the issue:
https://github.com/apache/incubator-carbondata/pull/752
LGTM
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #752: [CARBONDATA-879] Clear driver side dict cac...
Posted by Sephiroth-Lin <gi...@git.apache.org>.
Github user Sephiroth-Lin commented on the issue:
https://github.com/apache/incubator-carbondata/pull/752
@kumarvishal09 @QiangCai @watermen Please help review, tks
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #752: [CARBONDATA-879] Clear driver side dict cac...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/incubator-carbondata/pull/752
@Sephiroth-Lin Actually clearing the cache at the time of codegen generation does not help as it just returns the code string to spark , Spark compiles it and executes as part of WholeCodeGen. We need to find a way to clear inside the code gen.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #752: [CARBONDATA-879] Clear driver side dict cac...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/752
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #752: [CARBONDATA-879] Clear driver side dict cac...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/incubator-carbondata/pull/752
@Sephiroth-Lin Yes you are right, but I feel it is not required to create dictionary while doing codegen, so better avoid dictionary creation while doing codegen. just create only at executor side.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #752: [CARBONDATA-879] Clear driver side dict cac...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/752
Build Success with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/1493/
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #752: [CARBONDATA-879] Clear driver side dict cac...
Posted by chenliang613 <gi...@git.apache.org>.
Github user chenliang613 commented on the issue:
https://github.com/apache/incubator-carbondata/pull/752
add to whitelist
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata issue #752: [CARBONDATA-879] Clear driver side dict cac...
Posted by Sephiroth-Lin <gi...@git.apache.org>.
Github user Sephiroth-Lin commented on the issue:
https://github.com/apache/incubator-carbondata/pull/752
@ravipesala Yes, inside the code gen, then it will clear for executor side, but for driver side after code gen we also need to clear the access count. From my test, if we don't clear it, then we can't remove it from memory if we don't have enough memory to load another cache.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---