You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@impala.apache.org by chen <xi...@gmail.com> on 2017/11/08 08:13:34 UTC

long codegen time while codegen disabled

 I have a query,tooks a long time on codegen:

      CodeGen:(Total: 32m22s, non-child: 32m22s, % non-child: 100.00%)
         - CodegenTime: 0ns
         - CompileTime: 53.143ms
         - LoadTime: 58.680us
         - ModuleFileSize: 1.96 MB (2054956)
         - OptimizationTime: 32m22s
         - PrepareTime: 157.700ms

but from the profile ,we can see that codegen is diabled for this query:

ExecOption: Codegen enabled: 0 out of 1

attached is the complete profile.


can anyone help to firgure out a way to bypass.


Chen

Re: long codegen time while codegen disabled

Posted by Tim Armstrong <ta...@cloudera.com>.

This was cross-posted to a Cloudera forum:
http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/long-codegen-time-while-codegen-disabled/m-p/61635#M3812?eid=1&aid=1,
where I gave the same answer that Mostafa gave.

If you cross-post to multiple related mailing lists and forums, can you
please at least link to the previous post so that people don't waste time
answering the same question twice? We're happy to help but don't appreciate
having our time wasted.

On Wed, Nov 8, 2017 at 8:29 AM, Mostafa Mokhtar <mm...@cloudera.com>
wrote:

> From the profile codegen is disabled for HDFS_SCAN_NODE (id=8) and not the
> entire query.
> If you wish to disable codegen run "set disable_codegen=1;" before
> executing the query from impala-shell or add it to the connection string if
> using JDBC.
>
>        HDFS_SCAN_NODE (id=8):(Total: 2.314ms, non-child: 2.314ms, %
> non-child: 100.00%)
>           Hdfs split stats (<volume id>:<# splits>/<split lengths>):
> 2:1/14.28 KB
>           Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0%
>           File Formats: PARQUET/SNAPPY:3
>           ExecOption: Codegen enabled: 0 out of 1
>
>
> On a side not I recommend trying out a more recent version of Impala
> as a lot has improved since.
>
>
> On Wed, Nov 8, 2017 at 12:13 AM, chen <xi...@gmail.com> wrote:
>
> >  I have a query,tooks a long time on codegen:
> >
> >       CodeGen:(Total: 32m22s, non-child: 32m22s, % non-child: 100.00%)
> >          - CodegenTime: 0ns
> >          - CompileTime: 53.143ms
> >          - LoadTime: 58.680us
> >          - ModuleFileSize: 1.96 MB (2054956)
> >          - OptimizationTime: 32m22s
> >          - PrepareTime: 157.700ms
> >
> > but from the profile ,we can see that codegen is diabled for this query:
> >
> > ExecOption: Codegen enabled: 0 out of 1
> >
> > attached is the complete profile.
> >
> >
> > can anyone help to firgure out a way to bypass.
> >
> >
> > Chen
> >
> >
>

Re: long codegen time while codegen disabled

Posted by Mostafa Mokhtar <mm...@cloudera.com>.

From the profile codegen is disabled for HDFS_SCAN_NODE (id=8) and not the
entire query.
If you wish to disable codegen run "set disable_codegen=1;" before
executing the query from impala-shell or add it to the connection string if
using JDBC.

       HDFS_SCAN_NODE (id=8):(Total: 2.314ms, non-child: 2.314ms, %
non-child: 100.00%)
          Hdfs split stats (<volume id>:<# splits>/<split lengths>):
2:1/14.28 KB
          Hdfs Read Thread Concurrency Bucket: 0:0% 1:0% 2:0% 3:0%
          File Formats: PARQUET/SNAPPY:3
          ExecOption: Codegen enabled: 0 out of 1

On a side not I recommend trying out a more recent version of Impala
as a lot has improved since.

On Wed, Nov 8, 2017 at 12:13 AM, chen <xi...@gmail.com> wrote:

>  I have a query,tooks a long time on codegen:
>
>       CodeGen:(Total: 32m22s, non-child: 32m22s, % non-child: 100.00%)
>          - CodegenTime: 0ns
>          - CompileTime: 53.143ms
>          - LoadTime: 58.680us
>          - ModuleFileSize: 1.96 MB (2054956)
>          - OptimizationTime: 32m22s
>          - PrepareTime: 157.700ms
>
> but from the profile ,we can see that codegen is diabled for this query:
>
> ExecOption: Codegen enabled: 0 out of 1
>
> attached is the complete profile.
>
>
> can anyone help to firgure out a way to bypass.
>
>
> Chen
>
>