You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by "dylan (JIRA)" <ji...@apache.org> on 2017/09/19 09:09:00 UTC
[jira] [Created] (CARBONDATA-1487) problem with the conversion data
, using the carbondata struct type
dylan created CARBONDATA-1487:
---------------------------------
Summary: problem with the conversion data , using the carbondata struct type
Key: CARBONDATA-1487
URL: https://issues.apache.org/jira/browse/CARBONDATA-1487
Project: CarbonData
Issue Type: Bug
Components: data-load, data-query, spark-integration
Affects Versions: 1.1.0
Environment: spark1.6
Reporter: dylan
i'm use spark1.6 and carbondata1.1.0.
i use struct type,and i find the data is wrong when i load data into carbondata.
my test data int csv like this :
device,source
d803c517c971,NULL
d803c517c972,NULL
d803c517c973,"{""lat"":null,""lon"":null,""province"":null}"
d803c517c974,NULL
d803c517c975,NULL
d803c517c976,NULL
d803c517c977,NULL
d803c517c978,"{""lat"":""22.696963"",""lon"":""113.948252"",""province"":""cn2""}"
d803c517c979,NULL
d803c517c980,NULL
d803c517c981,NULL
d803c517c982,NULL
d803c517c983,NULL
d803c517c984,NULL
d803c517c985,NULL
d803c517c986,"{""lat"":""22.542006"",""lon"":""113.954106"",""province"":""cn2""}"
d803c517c987,NULL
d803c517c988,NULL
d803c517c989,NULL
d803c517c990,NULL
d803c517c991,NULL
d803c517c992,"{""lat"":null,""lon"":null,""province"":null}"
d803c517c993,NULL
d803c517c994,NULL
d803c517c995,NULL
d803c517c996,NULL
and i select the result in carbondata use sparksql like this:
+------------+---------------------------------------------------+
|device |source |
+------------+---------------------------------------------------+
|d803c517c971|[NULL,null,null] |
|d803c517c972|[NULL,null,null] |
|d803c517c973|[{"lat":null,"lon":null,"province":null},null,null]|
|d803c517c974|[NULL,null,null] |
|d803c517c975|[NULL,null,null] |
|d803c517c976|[NULL,null,null] |
|d803c517c977|[NULL,null,null] |
|d803c517c978|[{"lat":"22.696963,null,null] |
|d803c517c979|[NULL,null,null] |
|d803c517c980|[NULL,null,null] |
|d803c517c981|[NULL,null,null] |
|d803c517c982|[NULL,null,null] |
|d803c517c983|[NULL,null,null] |
|d803c517c984|[NULL,null,null] |
|d803c517c985|[NULL,null,null] |
|d803c517c986|[{"lat":"22.542006,null,null] |
|d803c517c987|[NULL,null,null] |
|d803c517c988|[NULL,null,null] |
|d803c517c989|[NULL,null,null] |
|d803c517c990|[NULL,null,null] |
|d803c517c991|[NULL,null,null] |
|d803c517c992|[{"lat":null,"lon":null,"province":null},null,null]|
|d803c517c993|[NULL,null,null] |
|d803c517c994|[NULL,null,null] |
|d803c517c995|[NULL,null,null] |
|d803c517c996|[NULL,null,null] |
+------------+---------------------------------------------------+
the field source.lon has lost,and data format are not the same as in my csv file.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)