You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@carbondata.apache.org by kumarvishal09 <gi...@git.apache.org> on 2017/09/25 13:06:59 UTC

[GitHub] carbondata pull request #1384: [CARBONDATA-1515]Fixed NPE in data loading in...

GitHub user kumarvishal09 opened a pull request:

    https://github.com/apache/carbondata/pull/1384

    [CARBONDATA-1515]Fixed NPE in data loading in long run

    **Scenario:** 
    Data size: 3.5 billion rows(4.1 tb data)
    3 node cluster
    Number of core while data loading 12.
    No. of loads 100 times
    **Problem:** In DataConverterProcessorStepImpl it is using array list for adding all the local converter, in case of multiple thread scenario it is creating a hole (null value)(as array list if not synchronized). while closing the converter it is it is throwing NPE
    **Solution:** Add local converter in synchronized block

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kumarvishal09/incubator-carbondata NPEInDataLoading

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1384.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1384
    
----
commit 7991ca02205fe37d63d6bfa1307a4ca48dd057e1
Author: kumarvishal <ku...@gmail.com>
Date:   2017-09-25T12:57:03Z

    Fixed NPE in data loading

----


---

[GitHub] carbondata pull request #1384: [CARBONDATA-1515]Fixed NPE in data loading in...

Posted by asfgit <gi...@git.apache.org>.

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1384


---

[GitHub] carbondata issue #1384: [CARBONDATA-1515]Fixed NPE in data loading in long r...

Posted by ravipesala <gi...@git.apache.org>.

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1384
  
    LGTM


---

[GitHub] carbondata issue #1384: [CARBONDATA-1515]Fixed NPE in data loading in long r...

Posted by CarbonDataQA <gi...@git.apache.org>.

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1384
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/291/



---

[GitHub] carbondata issue #1384: [CARBONDATA-1515]Fixed NPE in data loading in long r...

Posted by ravipesala <gi...@git.apache.org>.

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1384
  
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/922/



---

[GitHub] carbondata issue #1384: [CARBONDATA-1515]Fixed NPE in data loading in long r...

Posted by CarbonDataQA <gi...@git.apache.org>.

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1384
  
    Build Success with Spark 1.6, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/166/



---