You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by manishgupta88 <gi...@git.apache.org> on 2018/01/18 08:59:10 UTC

[GitHub] carbondata pull request #1830: [CARBONDATA-2051] Added like query ends with ...

GitHub user manishgupta88 opened a pull request:

    https://github.com/apache/carbondata/pull/1830

    [CARBONDATA-2051] Added like query ends with and contains with filter push down suport to carbondata

    **Problem**
    Current like filter with start with expression is only pushed down to carbondata. In case of ends with and contains like filter all the data is given back to spark and then spark applies the filter on it.
    This behavior is fine for the queries which has lesser number of queried columns. But as the number of columns and data increases there is performance impact because the data being sent to spark becomes more thereby increasing the IO. 
    If like filter is push down then first filter column is read and blocks are pruned. In this cases the data returned to the spark is after applying the filter and only blocklets matching the data are fully read. This reduces IO and increases the query performance.
    
    **Solution**
    Modify code to push down like query with ends and contains with filter
    
     - [ ] Any interfaces changed?
     No
     - [ ] Any backward compatibility impacted?
     No
     - [ ] Document update required?
    No
     - [ ] Testing done
    Added test case to verify push down is happening
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. 
    NA


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/manishgupta88/carbondata like_query_pushdown

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1830.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1830
    
----
commit bff9fbf316941d0c732c04d3c9b7e775285ec893
Author: manishgupta88 <to...@...>
Date:   2018-01-18T08:53:17Z

    Added like query ends with and contains with filter push down suport to carbondata

----


---

[GitHub] carbondata issue #1830: [CARBONDATA-2051] Added like query ends with and con...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1830
  
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1716/



---

[GitHub] carbondata issue #1830: [CARBONDATA-2051] Added like query ends with and con...

Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1830
  
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2940/



---

[GitHub] carbondata pull request #1830: [CARBONDATA-2051] Added like query ends with ...

Posted by kumarvishal09 <gi...@git.apache.org>.
Github user kumarvishal09 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1830#discussion_r162298090
  
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/strategy/CarbonLateDecodeStrategy.scala ---
    @@ -612,6 +612,10 @@ private[sql] class CarbonLateDecodeStrategy extends SparkStrategy {
             CastExpressionOptimization.checkIfCastCanBeRemove(c)
           case StartsWith(a: Attribute, Literal(v, t)) =>
             Some(sources.StringStartsWith(a.name, v.toString))
    +      case c@EndsWith(a: Attribute, Literal(v, t)) =>
    --- End diff --
    
    I think we need to handle Like case also when filter is [column like '%_a'] expression will be like expression so we need to pushdown the same


---

[GitHub] carbondata pull request #1830: [CARBONDATA-2051] Added like query ends with ...

Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/1830


---

[GitHub] carbondata issue #1830: [CARBONDATA-2051] Added like query ends with and con...

Posted by manishgupta88 <gi...@git.apache.org>.
Github user manishgupta88 commented on the issue:

    https://github.com/apache/carbondata/pull/1830
  
    retest sdv please


---

[GitHub] carbondata issue #1830: [CARBONDATA-2051] Added like query ends with and con...

Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1830
  
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2981/



---

[GitHub] carbondata issue #1830: [CARBONDATA-2051] Added like query ends with and con...

Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1830
  
    LGTM


---