You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by manishgupta88 <gi...@git.apache.org> on 2018/09/27 12:32:43 UTC

[GitHub] carbondata pull request #2774: [CARBONDATA-2979] select count fails when car...

GitHub user manishgupta88 opened a pull request:

    https://github.com/apache/carbondata/pull/2774

    [CARBONDATA-2979] select count fails when carbondata file is written through SDK and read through sparkfileformat for complex datatype map(struct->array->map)

    **Problem**
    Select query failed issue for map type when data is loaded using avro SDK and external table using carbon file format is used to query the data
    
    **Analysis**
    When data is loaded through Avro SDK which has a schema of type struct<array<map>>, fieldName was hard coded to "val" because of which during query the schema written in the file footer and schema inferred for the external table had a mismatch which lead to failure.
    
    **Solution**
    Instead of hard coding the field value as "val" use the given field name in the schema
    
     - [ ] Any interfaces changed?
    No 
     - [ ] Any backward compatibility impacted?
     No
     - [ ] Document update required?
    No
     - [ ] Testing done
    UT Added
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. 
    NA


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/manishgupta88/carbondata fix_map_type_issues

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/2774.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2774
    
----
commit 1e1eda1dcfcfdf424c9fa7748555d4400c6c14c3
Author: manishgupta88 <to...@...>
Date:   2018-09-27T12:32:34Z

    Fixed query failure issue for map type when data is loaded using avro SDK and external table using carbon file format is used to query the data

----


---

[GitHub] carbondata issue #2774: [CARBONDATA-2979] select count fails when carbondata...

Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2774
  
    retest this please


---

[GitHub] carbondata issue #2774: [CARBONDATA-2979] select count fails when carbondata...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2774
  
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8868/



---

[GitHub] carbondata issue #2774: [CARBONDATA-2979] select count fails when carbondata...

Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2774
  
    LGTM


---

[GitHub] carbondata issue #2774: [CARBONDATA-2979] select count fails when carbondata...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2774
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/674/



---

[GitHub] carbondata issue #2774: [CARBONDATA-2979] select count fails when carbondata...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2774
  
    Build Success with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/8938/



---

[GitHub] carbondata pull request #2774: [CARBONDATA-2979] select count fails when car...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2774


---

[GitHub] carbondata issue #2774: [CARBONDATA-2979] select count fails when carbondata...

Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2774
  
    LGTM


---

[GitHub] carbondata issue #2774: [CARBONDATA-2979] select count fails when carbondata...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2774
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/607/



---

[GitHub] carbondata issue #2774: [CARBONDATA-2979] select count fails when carbondata...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2774
  
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/799/



---

[GitHub] carbondata issue #2774: [CARBONDATA-2979] select count fails when carbondata...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2774
  
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/870/



---