You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by ajantha-bhat <gi...@git.apache.org> on 2018/07/11 07:11:22 UTC
[GitHub] carbondata pull request #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWrite...
GitHub user ajantha-bhat opened a pull request:
https://github.com/apache/carbondata/pull/2485
[CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue fixes
[CARBONDATA-2722][SDK] [JsonWriter] NPE when schema and data are not of same length or Data is null.
problem: Null data is not handled in the json object to carbon row conversion.
solution: add a null check when object is fetched from json map.
[CARBONDATA-2721][SDK] [JsonWriter] Json writer is writing only first element of an array and discarding the rest of the elements.
problem: converting json object to carbon row array object is based on array children count , instead of array element count.
Hence as array will always have one children. only one element is filled.
solution: use array element count instead of array children count.Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?NA
- [ ] Any backward compatibility impacted?NA
- [ ] Document update required?NA
- [ ] Testing done
updated the UT
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ajantha-bhat/carbondata master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2485.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2485
----
commit 1152176762cba7c6b4686fbc2d7825c82e373e3c
Author: ajantha-bhat <aj...@...>
Date: 2018-07-09T13:15:30Z
[CARBONDATA-2722][SDK] [JsonWriter] NPE when schema and data are not of same length or Data is null.
problem: Null data is not handled in the json object to carbon row
conversion.
solution: add a null check when object is fetched from json map.
[CARBONDATA-2721][SDK] [JsonWriter] Json writer is writing only first
element of an array and discarding the rest of the elements.
problem: converting json object to carbon row array object is based on
array children count , instead of array element count.
Hence as array will always have one children. only one element is
filled.
solution: use array element count instead of array children count.
----
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7066/
---
[GitHub] carbondata pull request #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWrite...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2485#discussion_r202277326
--- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/parser/impl/JsonRowParser.java ---
@@ -56,6 +56,9 @@ public JsonRowParser(DataField[] dataFields) {
Map<String, Object> jsonNodeMap =
objectMapper.readValue(jsonString, new TypeReference<Map<String, Object>>() {
});
+ if (jsonNodeMap == null) {
--- End diff --
yeah, not required. removed it.
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485
Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5828/
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2485
LGTM
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5864/
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5843/
---
[GitHub] carbondata pull request #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWrite...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2485
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7015/
---
[GitHub] carbondata pull request #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWrite...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2485#discussion_r202256075
--- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/parser/impl/JsonRowParser.java ---
@@ -56,6 +56,9 @@ public JsonRowParser(DataField[] dataFields) {
Map<String, Object> jsonNodeMap =
objectMapper.readValue(jsonString, new TypeReference<Map<String, Object>>() {
});
+ if (jsonNodeMap == null) {
--- End diff --
Here `|| jsonNodeMap.size() == 0` check is not required?
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7087/
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2485
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5823/
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7051/
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2485
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5767/
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2485
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5809/
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5791/
---
[GitHub] carbondata issue #2485: [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2485
retest this please
---