You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by scwf <gi...@git.apache.org> on 2016/11/30 09:33:48 UTC

[GitHub] incubator-carbondata pull request #372: [CARBONDATA-473] integrate with spar...

GitHub user scwf opened a pull request:

    https://github.com/apache/incubator-carbondata/pull/372

    [CARBONDATA-473] integrate with spark 2 stable datasource api

    1.  read path: based on spark2 datasource api
    2.  load path: give a tool to do data load


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/KirinKing/incubator-carbondata spark2

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-carbondata/pull/372.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #372
    
----
commit a78efaea951194b1e9a3273b817d5e1641ed0f2c
Author: jackylk <ja...@huawei.com>
Date:   2016-11-30T08:28:34Z

    add spark2 module

commit 8d536ced3c1c147bb734c769a28cebbf33f2f035
Author: jackylk <ja...@huawei.com>
Date:   2016-11-30T08:49:03Z

    rebase

commit 308fe24500a81373dac7d87d5f2a563a35ff547d
Author: jackylk <ja...@huawei.com>
Date:   2016-11-30T08:49:57Z

    rebase

commit d3416dc128c0d4c37d9f8191796e596d9ca7a77b
Author: jackylk <ja...@huawei.com>
Date:   2016-11-29T12:12:22Z

    rebase

commit 120914467ee93a6613911fcf25171fa952eccd83
Author: jackylk <ja...@huawei.com>
Date:   2016-11-30T08:50:32Z

    rebase

commit d4cee19fbd54aa008662e36479b0ee0dda0ef384
Author: QiangCai <qi...@qq.com>
Date:   2016-11-29T13:25:58Z

    carbon api for spark2

commit bdfa20346d2ce432f925c02c47a9d2dd7dd5d807
Author: QiangCai <qi...@qq.com>
Date:   2016-11-29T16:38:53Z

    fix late decoder

commit 9a96466209a5918b2fc4f55a728055ffffbc0093
Author: jackylk <ja...@huawei.com>
Date:   2016-11-30T09:16:16Z

    fix

commit 68f61c195a90b027e9d0dbc5a87915ca68d90c00
Author: Fei Wang <wa...@126.com>
Date:   2016-11-30T09:31:56Z

    Merge pull request #26 from jackylk/kingmaster
    
    Kingmaster

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #372: [CARBONDATA-473] integrate with spark 2 sta...

Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
  
    CI passed
    http://136.243.101.176:8080/job/ApacheCarbonManualPRBuilder/717/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #372: [CARBONDATA-473] integrate with spark 2 sta...

Posted by scwf <gi...@git.apache.org>.
Github user scwf commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
  
    the result has some issues for the carbon example:
    ```
    +----------+--------+-----------+-----------+-----------+
    |shortField|intField|bigintField|doubleField|stringField|
    +----------+--------+-----------+-----------+-----------+
    |         1|      11|       null|       44.4|      flink|
    |         3|      14|       null|       43.4|       hive|
    |         2|      10|       null|       43.4|     impala|
    |         4|      16|       null|       42.4|     impala|
    |         1|      10|       null|       48.4|      spark|
    |         5|      17|       null|       43.4|      spark|
    |         1|      10|       null|       43.4|      spark|
    |         1|      10|       null|       47.4|      spark|
    |         1|      10|       null|       43.4|      spark|
    |         1|      10|       null|       43.4|      spark|
    +----------+--------+-----------+-----------+-----------+
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #372: [CARBONDATA-473] integrate with spark 2 sta...

Posted by scwf <gi...@git.apache.org>.
Github user scwf commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
  
    @jacklk added test case from empty table


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #372: [CARBONDATA-473] integrate with spark 2 sta...

Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
  
    Thanks for working for this


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #372: [CARBONDATA-473] integrate with spar...

Posted by scwf <gi...@git.apache.org>.
Github user scwf commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/372#discussion_r90207315
  
    --- Diff: integration/spark-common/pom.xml ---
    @@ -84,6 +79,11 @@
           <artifactId>junit</artifactId>
         </dependency>
         <dependency>
    +      <groupId>com.databricks</groupId>
    +      <artifactId>spark-csv_${scala.binary.version}</artifactId>
    +      <version>1.2.0</version>
    --- End diff --
    
    we can extract this as a property


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #372: [CARBONDATA-473] integrate with spark 2 sta...

Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
  
    CI running
    http://136.243.101.176:8080/job/ApacheCarbonManualPRBuilder/716/


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata pull request #372: [CARBONDATA-473] integrate with spar...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-carbondata/pull/372


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #372: [CARBONDATA-473] integrate with spark 2 sta...

Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
  
    CI running
    http://136.243.101.176:8080/job/ApacheCarbonManualPRBuilder/715/


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #372: [CARBONDATA-473] integrate with spark 2 sta...

Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
  
    There are codestyle failing, please correct them


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #372: [CARBONDATA-473] integrate with spark 2 sta...

Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
  
    Is there an example to run carbon with spark2?
    Can these code work?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-carbondata issue #372: [CARBONDATA-473] integrate with spark 2 sta...

Posted by jackylk <gi...@git.apache.org>.
Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
  
    Yes, bigint datatype has some issue, please raise a JIRA for it.
    And can you add some basic testcase for this PR


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---