You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Nirav Shah <ni...@games24x7.com> on 2015/12/04 12:42:18 UTC
Drill Query Problem
Hello,
I am getting below error while running big query.
=======================================
Error: SYSTEM ERROR: CompileException: File
'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
public void doEval(int inIndex, int outIndex)
^ (compiler.err.limit.code)
Regards,
Nirav
Re: Drill Query Problem
Posted by Jinfeng Ni <ji...@gmail.com>.
Can you please check if you have JDK installed on the drillbit where
the forman node is located for this query? or can you check if the
drillbit.log for that bit contains the following WARN msg. That would
help us understand the cause of the problem. Thanks.
"JDK Java compiler not available - probably you're running Drill with
a JRE and not a JDK"
On Sun, Dec 6, 2015 at 10:53 PM, Boris Chmiel
<bo...@yahoo.com.invalid> wrote:
> Finally it does not work on 1.2 either...
>
>
>
>
> Le Samedi 5 décembre 2015 16h51, Boris Chmiel <bo...@yahoo.com.INVALID> a écrit :
>
>
> Hello
> I am getting the same error on huge queries. My query is about 10K characters. This error is a regression and is directly related to 1.3
> since I have already ran queries more than twice biggers (nearly 30K characters) with 1.1 and 1.2 I'm downgrading to 1.2
>
> Boris
>
>
>
>
> Le Vendredi 4 décembre 2015 12h42, Nirav Shah <ni...@games24x7.com> a écrit :
>
>
> Hello,
>
> I am getting below error while running big query.
>
> =======================================
>
> Error: SYSTEM ERROR: CompileException: File 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]', Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> public void doEval(int inIndex, int outIndex)
> ^ (compiler.err.limit.code)
>
>
> Regards,
> Nirav
>
>
>
>
>
Re: Drill Query Problem
Posted by Boris Chmiel <bo...@yahoo.com.INVALID>.
Finally it does not work on 1.2 either...
Le Samedi 5 décembre 2015 16h51, Boris Chmiel <bo...@yahoo.com.INVALID> a écrit :
Hello
I am getting the same error on huge queries. My query is about 10K characters. This error is a regression and is directly related to 1.3
since I have already ran queries more than twice biggers (nearly 30K characters) with 1.1 and 1.2 I'm downgrading to 1.2
Boris
Le Vendredi 4 décembre 2015 12h42, Nirav Shah <ni...@games24x7.com> a écrit :
Hello,
I am getting below error while running big query.
=======================================
Error: SYSTEM ERROR: CompileException: File 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]', Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
public void doEval(int inIndex, int outIndex)
^ (compiler.err.limit.code)
Regards,
Nirav
Re: Drill Query Problem
Posted by Nirav Shah <ni...@games24x7.com>.
Sure, I will check and revert. Anyway thanks for spending time on this.
Regards,
Nirav
On Thu, Dec 17, 2015 at 10:10 PM, Jinfeng Ni <ji...@gmail.com> wrote:
> @Nirav,
>
> I tried your query on the sample dataset you sent, with the latest
> Drill master branch. The query is successful, either as a standalone
> query, or embedded in a CTAS.
>
> Can you please double check if you have JDK install on "every" node in
> your drill cluster? The run-time code generation and compilation
> happen in execution phase. This means any drillbit assigned to execute
> the Project operator would have to go through code compilation,
> requiring to have JDK to compile the large generated code for your
> query. It's not sufficient to only have JDK installed on the foreman
> node.
>
>
>
> On Wed, Dec 9, 2015 at 2:17 AM, Boris Chmiel
> <bo...@yahoo.com.invalid> wrote:
> > Thanks Nirav,I did the same.
> > I think the limitation, in my case, is due to the number of cast /
> regex_replace / substr etc to handle in the query, not the number of Kline
> of code
> > Boris
> >
> >
> > Le Mercredi 9 décembre 2015 6h41, Nirav Shah <
> nirav.shah@games24x7.com> a écrit :
> >
> >
> > Hi Boris,
> >
> > As a workaround I broke my query and rejoin results. In case need any
> help
> > on this let me know.
> >
> > Best,
> > Nirav
> >
> > On Sat, Dec 5, 2015 at 9:17 PM, Boris Chmiel
> <boris.chmiel@yahoo.com.invalid
> >> wrote:
> >
> >> Hello
> >> I am getting the same error on huge queries. My query is about 10K
> >> characters. This error is a regression and is directly related to 1.3
> >> since I have already ran queries more than twice biggers (nearly 30K
> >> characters) with 1.1 and 1.2 I'm downgrading to 1.2
> >>
> >> Boris
> >>
> >>
> >>
> >>
> >> Le Vendredi 4 décembre 2015 12h42, Nirav Shah <
> >> nirav.shah@games24x7.com> a écrit :
> >>
> >>
> >> Hello,
> >>
> >> I am getting below error while running big query.
> >>
> >> =======================================
> >>
> >> Error: SYSTEM ERROR: CompileException: File
> >>
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> >> Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> >> public void doEval(int inIndex, int outIndex)
> >> ^ (compiler.err.limit.code)
> >>
> >>
> >> Regards,
> >> Nirav
> >>
> >>
> >>
> >>
> >
> >
>
Re: Drill Query Problem
Posted by Jinfeng Ni <ji...@gmail.com>.
@Nirav,
I tried your query on the sample dataset you sent, with the latest
Drill master branch. The query is successful, either as a standalone
query, or embedded in a CTAS.
Can you please double check if you have JDK install on "every" node in
your drill cluster? The run-time code generation and compilation
happen in execution phase. This means any drillbit assigned to execute
the Project operator would have to go through code compilation,
requiring to have JDK to compile the large generated code for your
query. It's not sufficient to only have JDK installed on the foreman
node.
On Wed, Dec 9, 2015 at 2:17 AM, Boris Chmiel
<bo...@yahoo.com.invalid> wrote:
> Thanks Nirav,I did the same.
> I think the limitation, in my case, is due to the number of cast / regex_replace / substr etc to handle in the query, not the number of Kline of code
> Boris
>
>
> Le Mercredi 9 décembre 2015 6h41, Nirav Shah <ni...@games24x7.com> a écrit :
>
>
> Hi Boris,
>
> As a workaround I broke my query and rejoin results. In case need any help
> on this let me know.
>
> Best,
> Nirav
>
> On Sat, Dec 5, 2015 at 9:17 PM, Boris Chmiel <boris.chmiel@yahoo.com.invalid
>> wrote:
>
>> Hello
>> I am getting the same error on huge queries. My query is about 10K
>> characters. This error is a regression and is directly related to 1.3
>> since I have already ran queries more than twice biggers (nearly 30K
>> characters) with 1.1 and 1.2 I'm downgrading to 1.2
>>
>> Boris
>>
>>
>>
>>
>> Le Vendredi 4 décembre 2015 12h42, Nirav Shah <
>> nirav.shah@games24x7.com> a écrit :
>>
>>
>> Hello,
>>
>> I am getting below error while running big query.
>>
>> =======================================
>>
>> Error: SYSTEM ERROR: CompileException: File
>> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
>> Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
>> public void doEval(int inIndex, int outIndex)
>> ^ (compiler.err.limit.code)
>>
>>
>> Regards,
>> Nirav
>>
>>
>>
>>
>
>
Re: Drill Query Problem
Posted by Boris Chmiel <bo...@yahoo.com.INVALID>.
Thanks Nirav,I did the same.
I think the limitation, in my case, is due to the number of cast / regex_replace / substr etc to handle in the query, not the number of Kline of code
Boris
Le Mercredi 9 décembre 2015 6h41, Nirav Shah <ni...@games24x7.com> a écrit :
Hi Boris,
As a workaround I broke my query and rejoin results. In case need any help
on this let me know.
Best,
Nirav
On Sat, Dec 5, 2015 at 9:17 PM, Boris Chmiel <boris.chmiel@yahoo.com.invalid
> wrote:
> Hello
> I am getting the same error on huge queries. My query is about 10K
> characters. This error is a regression and is directly related to 1.3
> since I have already ran queries more than twice biggers (nearly 30K
> characters) with 1.1 and 1.2 I'm downgrading to 1.2
>
> Boris
>
>
>
>
> Le Vendredi 4 décembre 2015 12h42, Nirav Shah <
> nirav.shah@games24x7.com> a écrit :
>
>
> Hello,
>
> I am getting below error while running big query.
>
> =======================================
>
> Error: SYSTEM ERROR: CompileException: File
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> public void doEval(int inIndex, int outIndex)
> ^ (compiler.err.limit.code)
>
>
> Regards,
> Nirav
>
>
>
>
Re: Drill Query Problem
Posted by Jinfeng Ni <ji...@gmail.com>.
@Nirav,
If possible, could you please post a small sample dataset? For the
compilation problem of run-time generated code, the thing matters is
the input column type. It would be nice if you could share some sample
data. Also, which version of Drill are you using?
I tried to use the following data in JSON format, by guessing the
column type from the query. I then run the query against the JSON file
in both 1.2 and 1.4.0. Both are running ok. However, I did notice for
one of the Project operator, Drill uses JDKClassCompiler, and the
total java source file has more than 17k line of codes, totaling 752.4
KB size.
Here is the compiling info for Project's generated code:
"
org.apache.drill.exec.compile.JDKClassCompiler: Compiling (source
size=752.4 KiB):
"
{"userid": "USER1",
"channelid" : 2,
"id" : "/window/rewardtooltip/",
"event" : "EVENT1",
"mprizetype" : 2,
"mgametype": "cfp",
"misfmg" : "MG1",
"sessionid": "SESSION1"
}
On Tue, Dec 8, 2015 at 9:36 PM, Nirav Shah <ni...@games24x7.com> wrote:
> Hi Boris,
>
> As a workaround I broke my query and rejoin results. In case need any help
> on this let me know.
>
> Best,
> Nirav
>
> On Sat, Dec 5, 2015 at 9:17 PM, Boris Chmiel <boris.chmiel@yahoo.com.invalid
>> wrote:
>
>> Hello
>> I am getting the same error on huge queries. My query is about 10K
>> characters. This error is a regression and is directly related to 1.3
>> since I have already ran queries more than twice biggers (nearly 30K
>> characters) with 1.1 and 1.2 I'm downgrading to 1.2
>>
>> Boris
>>
>>
>>
>>
>> Le Vendredi 4 décembre 2015 12h42, Nirav Shah <
>> nirav.shah@games24x7.com> a écrit :
>>
>>
>> Hello,
>>
>> I am getting below error while running big query.
>>
>> =======================================
>>
>> Error: SYSTEM ERROR: CompileException: File
>> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
>> Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
>> public void doEval(int inIndex, int outIndex)
>> ^ (compiler.err.limit.code)
>>
>>
>> Regards,
>> Nirav
>>
>>
>>
>>
Re: Drill Query Problem
Posted by Nirav Shah <ni...@games24x7.com>.
Hi Boris,
As a workaround I broke my query and rejoin results. In case need any help
on this let me know.
Best,
Nirav
On Sat, Dec 5, 2015 at 9:17 PM, Boris Chmiel <boris.chmiel@yahoo.com.invalid
> wrote:
> Hello
> I am getting the same error on huge queries. My query is about 10K
> characters. This error is a regression and is directly related to 1.3
> since I have already ran queries more than twice biggers (nearly 30K
> characters) with 1.1 and 1.2 I'm downgrading to 1.2
>
> Boris
>
>
>
>
> Le Vendredi 4 décembre 2015 12h42, Nirav Shah <
> nirav.shah@games24x7.com> a écrit :
>
>
> Hello,
>
> I am getting below error while running big query.
>
> =======================================
>
> Error: SYSTEM ERROR: CompileException: File
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> public void doEval(int inIndex, int outIndex)
> ^ (compiler.err.limit.code)
>
>
> Regards,
> Nirav
>
>
>
>
Re: Drill Query Problem
Posted by Boris Chmiel <bo...@yahoo.com.INVALID>.
Hello
I am getting the same error on huge queries. My query is about 10K characters. This error is a regression and is directly related to 1.3
since I have already ran queries more than twice biggers (nearly 30K characters) with 1.1 and 1.2 I'm downgrading to 1.2
Boris
Le Vendredi 4 décembre 2015 12h42, Nirav Shah <ni...@games24x7.com> a écrit :
Hello,
I am getting below error while running big query.
=======================================
Error: SYSTEM ERROR: CompileException: File 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]', Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
public void doEval(int inIndex, int outIndex)
^ (compiler.err.limit.code)
Regards,
Nirav
Re: Drill Query Problem
Posted by Sanjeev Verma <sa...@gmail.com>.
it seems that this exception is not related to the JDK compiler, not sure
what is triggering this
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
JaninoRuntimeException: Code attribute in class "org.apache.
drill.exec.test.generated.ProjectorGen189" grows beyond 64 KB
On Tue, Dec 8, 2015 at 10:39 AM, Nirav Shah <ni...@games24x7.com>
wrote:
> Hi Abdel & Jacques
> I have observed that if we put more than 40 columns with count or some
> aggregation function this issue arises. I have attached query in the first
> mail which gives this error, if you need sample data let me know.
> If we break this query it works fine.
>
> Thanks,
> Nirav
> On Dec 4, 2015 11:03 PM, "Abdel Hakim Deneche" <ad...@maprtech.com>
> wrote:
>
> > Hi Nirav,
> >
> > can you give us more information to help reproduce this issue ?
> >
> > thanks
> >
> > On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <ni...@games24x7.com>
> > wrote:
> >
> > > Hello,
> > >
> > > I am getting below error while running big query.
> > >
> > > =======================================
> > >
> > > Error: SYSTEM ERROR: CompileException: File
> > >
> >
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> > > Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> > > public void doEval(int inIndex, int outIndex)
> > > ^ (compiler.err.limit.code)
> > >
> > >
> > > Regards,
> > > Nirav
> > >
> >
> >
> >
> > --
> >
> > Abdelhakim Deneche
> >
> > Software Engineer
> >
> > <http://www.mapr.com/>
> >
> >
> > Now Available - Free Hadoop On-Demand Training
> > <
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > >
> >
>
Re: Drill Query Problem
Posted by Nirav Shah <ni...@games24x7.com>.
Let me know if you don't get attached query.
@Jinfeng,
I am not getting "JDK error".
I have tried "alter session set `exec.java_compiler` = 'JDK';" this doesn't
help.
Best,
Nirav
On Tue, Dec 8, 2015 at 8:20 PM, Ted Dunning <te...@gmail.com> wrote:
> Nirav,
>
> Attachments typically don't work since Apache mailing lists strip them off.
>
> Can you put the query into a gist on github or some such?
>
>
>
> On Tue, Dec 8, 2015 at 1:09 PM, Nirav Shah <ni...@games24x7.com>
> wrote:
>
> > Hi Abdel & Jacques
> > I have observed that if we put more than 40 columns with count or some
> > aggregation function this issue arises. I have attached query in the
> first
> > mail which gives this error, if you need sample data let me know.
> > If we break this query it works fine.
> >
> > Thanks,
> > Nirav
> > On Dec 4, 2015 11:03 PM, "Abdel Hakim Deneche" <ad...@maprtech.com>
> > wrote:
> >
> > > Hi Nirav,
> > >
> > > can you give us more information to help reproduce this issue ?
> > >
> > > thanks
> > >
> > > On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <ni...@games24x7.com>
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I am getting below error while running big query.
> > > >
> > > > =======================================
> > > >
> > > > Error: SYSTEM ERROR: CompileException: File
> > > >
> > >
> >
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> > > > Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too
> large
> > > > public void doEval(int inIndex, int outIndex)
> > > > ^ (compiler.err.limit.code)
> > > >
> > > >
> > > > Regards,
> > > > Nirav
> > > >
> > >
> > >
> > >
> > > --
> > >
> > > Abdelhakim Deneche
> > >
> > > Software Engineer
> > >
> > > <http://www.mapr.com/>
> > >
> > >
> > > Now Available - Free Hadoop On-Demand Training
> > > <
> > >
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > > >
> > >
> >
>
Re: Drill Query Problem
Posted by Ted Dunning <te...@gmail.com>.
Nirav,
Attachments typically don't work since Apache mailing lists strip them off.
Can you put the query into a gist on github or some such?
On Tue, Dec 8, 2015 at 1:09 PM, Nirav Shah <ni...@games24x7.com> wrote:
> Hi Abdel & Jacques
> I have observed that if we put more than 40 columns with count or some
> aggregation function this issue arises. I have attached query in the first
> mail which gives this error, if you need sample data let me know.
> If we break this query it works fine.
>
> Thanks,
> Nirav
> On Dec 4, 2015 11:03 PM, "Abdel Hakim Deneche" <ad...@maprtech.com>
> wrote:
>
> > Hi Nirav,
> >
> > can you give us more information to help reproduce this issue ?
> >
> > thanks
> >
> > On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <ni...@games24x7.com>
> > wrote:
> >
> > > Hello,
> > >
> > > I am getting below error while running big query.
> > >
> > > =======================================
> > >
> > > Error: SYSTEM ERROR: CompileException: File
> > >
> >
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> > > Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> > > public void doEval(int inIndex, int outIndex)
> > > ^ (compiler.err.limit.code)
> > >
> > >
> > > Regards,
> > > Nirav
> > >
> >
> >
> >
> > --
> >
> > Abdelhakim Deneche
> >
> > Software Engineer
> >
> > <http://www.mapr.com/>
> >
> >
> > Now Available - Free Hadoop On-Demand Training
> > <
> >
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> > >
> >
>
Re: Drill Query Problem
Posted by Nirav Shah <ni...@games24x7.com>.
Hi Abdel & Jacques
I have observed that if we put more than 40 columns with count or some
aggregation function this issue arises. I have attached query in the first
mail which gives this error, if you need sample data let me know.
If we break this query it works fine.
Thanks,
Nirav
On Dec 4, 2015 11:03 PM, "Abdel Hakim Deneche" <ad...@maprtech.com>
wrote:
> Hi Nirav,
>
> can you give us more information to help reproduce this issue ?
>
> thanks
>
> On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <ni...@games24x7.com>
> wrote:
>
> > Hello,
> >
> > I am getting below error while running big query.
> >
> > =======================================
> >
> > Error: SYSTEM ERROR: CompileException: File
> >
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> > Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> > public void doEval(int inIndex, int outIndex)
> > ^ (compiler.err.limit.code)
> >
> >
> > Regards,
> > Nirav
> >
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
> <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >
>
Re: Drill Query Problem
Posted by Jason Altekruse <al...@gmail.com>.
I think we might want to do more than just log in this case. Here enabling
the option is a good suggestion for possibly fixing the issue, but if a JDK
is not installed it looks like there will now just be a warning logged and
the same error will be produced.
It would probably be worth it to just throw the Janino compilation call in
a try block and produce a message that recommends turning on the option, as
well as confirming the a JDK is available on all nodes running Drillbits.
On Fri, Dec 4, 2015 at 11:41 AM, Jinfeng Ni <ji...@gmail.com> wrote:
> If JDK is not available, according to [1], there would be a warning in
> drillbit.log.
>
>
> [1]
> https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/compile/JDKClassCompiler.java#L47
>
> On Fri, Dec 4, 2015 at 11:11 AM, Jason Altekruse
> <al...@gmail.com> wrote:
> > Do we have something that checks a JDK is installed when launching Drill?
> > Based on this JIRA it looks like we will fall back to janino if we cannot
> > find a JDK. It is possible that this option does nothing if a JDK is not
> > available, or it may fail with an error. We should test this out in
> various
> > environments and make sure that the action to take is clear in the
> > exception messages.
> >
> > https://issues.apache.org/jira/browse/DRILL-1919
> >
> > On Fri, Dec 4, 2015 at 11:01 AM, Jinfeng Ni <ji...@gmail.com>
> wrote:
> >
> >> You may consider switching the java_compiler from DEFALT to JDK, by :
> >>
> >> alter session set `exec.java_compiler` = 'JDK';
> >>
> >> All Drill will automatically switch fro janino compiler to JDK, when
> >> the source code length is beyond certain limit. If your query happens
> >> to have smaller code than that limit, but still hit the limit for
> >> janino compiler, then it's better to set the option to JDK.
> >>
> >>
> >>
> >> On Fri, Dec 4, 2015 at 10:48 AM, Jacques Nadeau <ja...@dremio.com>
> >> wrote:
> >> > I'm guessing this is due to an extremely large number of columns,
> >> > complicated expressions or extremely long column names. We test up to
> a
> >> > fairly large size here [1] but you may have exceeded what we test. Can
> >> you
> >> > share the query?
> >> >
> >> > [1]
> >> >
> >>
> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/exec/compile/TestLargeFileCompilation.java
> >> >
> >> > --
> >> > Jacques Nadeau
> >> > CTO and Co-Founder, Dremio
> >> >
> >> > On Fri, Dec 4, 2015 at 9:33 AM, Abdel Hakim Deneche <
> >> adeneche@maprtech.com>
> >> > wrote:
> >> >
> >> >> Hi Nirav,
> >> >>
> >> >> can you give us more information to help reproduce this issue ?
> >> >>
> >> >> thanks
> >> >>
> >> >> On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <nirav.shah@games24x7.com
> >
> >> >> wrote:
> >> >>
> >> >> > Hello,
> >> >> >
> >> >> > I am getting below error while running big query.
> >> >> >
> >> >> > =======================================
> >> >> >
> >> >> > Error: SYSTEM ERROR: CompileException: File
> >> >> >
> >> >>
> >>
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> >> >> > Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too
> >> large
> >> >> > public void doEval(int inIndex, int outIndex)
> >> >> > ^ (compiler.err.limit.code)
> >> >> >
> >> >> >
> >> >> > Regards,
> >> >> > Nirav
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >>
> >> >> Abdelhakim Deneche
> >> >>
> >> >> Software Engineer
> >> >>
> >> >> <http://www.mapr.com/>
> >> >>
> >> >>
> >> >> Now Available - Free Hadoop On-Demand Training
> >> >> <
> >> >>
> >>
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >> >> >
> >> >>
> >>
>
Re: Drill Query Problem
Posted by Jinfeng Ni <ji...@gmail.com>.
If JDK is not available, according to [1], there would be a warning in
drillbit.log.
[1] https://github.com/apache/drill/blob/master/exec/java-exec/src/main/java/org/apache/drill/exec/compile/JDKClassCompiler.java#L47
On Fri, Dec 4, 2015 at 11:11 AM, Jason Altekruse
<al...@gmail.com> wrote:
> Do we have something that checks a JDK is installed when launching Drill?
> Based on this JIRA it looks like we will fall back to janino if we cannot
> find a JDK. It is possible that this option does nothing if a JDK is not
> available, or it may fail with an error. We should test this out in various
> environments and make sure that the action to take is clear in the
> exception messages.
>
> https://issues.apache.org/jira/browse/DRILL-1919
>
> On Fri, Dec 4, 2015 at 11:01 AM, Jinfeng Ni <ji...@gmail.com> wrote:
>
>> You may consider switching the java_compiler from DEFALT to JDK, by :
>>
>> alter session set `exec.java_compiler` = 'JDK';
>>
>> All Drill will automatically switch fro janino compiler to JDK, when
>> the source code length is beyond certain limit. If your query happens
>> to have smaller code than that limit, but still hit the limit for
>> janino compiler, then it's better to set the option to JDK.
>>
>>
>>
>> On Fri, Dec 4, 2015 at 10:48 AM, Jacques Nadeau <ja...@dremio.com>
>> wrote:
>> > I'm guessing this is due to an extremely large number of columns,
>> > complicated expressions or extremely long column names. We test up to a
>> > fairly large size here [1] but you may have exceeded what we test. Can
>> you
>> > share the query?
>> >
>> > [1]
>> >
>> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/exec/compile/TestLargeFileCompilation.java
>> >
>> > --
>> > Jacques Nadeau
>> > CTO and Co-Founder, Dremio
>> >
>> > On Fri, Dec 4, 2015 at 9:33 AM, Abdel Hakim Deneche <
>> adeneche@maprtech.com>
>> > wrote:
>> >
>> >> Hi Nirav,
>> >>
>> >> can you give us more information to help reproduce this issue ?
>> >>
>> >> thanks
>> >>
>> >> On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <ni...@games24x7.com>
>> >> wrote:
>> >>
>> >> > Hello,
>> >> >
>> >> > I am getting below error while running big query.
>> >> >
>> >> > =======================================
>> >> >
>> >> > Error: SYSTEM ERROR: CompileException: File
>> >> >
>> >>
>> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
>> >> > Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too
>> large
>> >> > public void doEval(int inIndex, int outIndex)
>> >> > ^ (compiler.err.limit.code)
>> >> >
>> >> >
>> >> > Regards,
>> >> > Nirav
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >>
>> >> Abdelhakim Deneche
>> >>
>> >> Software Engineer
>> >>
>> >> <http://www.mapr.com/>
>> >>
>> >>
>> >> Now Available - Free Hadoop On-Demand Training
>> >> <
>> >>
>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>> >> >
>> >>
>>
Re: Drill Query Problem
Posted by Jason Altekruse <al...@gmail.com>.
Do we have something that checks a JDK is installed when launching Drill?
Based on this JIRA it looks like we will fall back to janino if we cannot
find a JDK. It is possible that this option does nothing if a JDK is not
available, or it may fail with an error. We should test this out in various
environments and make sure that the action to take is clear in the
exception messages.
https://issues.apache.org/jira/browse/DRILL-1919
On Fri, Dec 4, 2015 at 11:01 AM, Jinfeng Ni <ji...@gmail.com> wrote:
> You may consider switching the java_compiler from DEFALT to JDK, by :
>
> alter session set `exec.java_compiler` = 'JDK';
>
> All Drill will automatically switch fro janino compiler to JDK, when
> the source code length is beyond certain limit. If your query happens
> to have smaller code than that limit, but still hit the limit for
> janino compiler, then it's better to set the option to JDK.
>
>
>
> On Fri, Dec 4, 2015 at 10:48 AM, Jacques Nadeau <ja...@dremio.com>
> wrote:
> > I'm guessing this is due to an extremely large number of columns,
> > complicated expressions or extremely long column names. We test up to a
> > fairly large size here [1] but you may have exceeded what we test. Can
> you
> > share the query?
> >
> > [1]
> >
> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/exec/compile/TestLargeFileCompilation.java
> >
> > --
> > Jacques Nadeau
> > CTO and Co-Founder, Dremio
> >
> > On Fri, Dec 4, 2015 at 9:33 AM, Abdel Hakim Deneche <
> adeneche@maprtech.com>
> > wrote:
> >
> >> Hi Nirav,
> >>
> >> can you give us more information to help reproduce this issue ?
> >>
> >> thanks
> >>
> >> On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <ni...@games24x7.com>
> >> wrote:
> >>
> >> > Hello,
> >> >
> >> > I am getting below error while running big query.
> >> >
> >> > =======================================
> >> >
> >> > Error: SYSTEM ERROR: CompileException: File
> >> >
> >>
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> >> > Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too
> large
> >> > public void doEval(int inIndex, int outIndex)
> >> > ^ (compiler.err.limit.code)
> >> >
> >> >
> >> > Regards,
> >> > Nirav
> >> >
> >>
> >>
> >>
> >> --
> >>
> >> Abdelhakim Deneche
> >>
> >> Software Engineer
> >>
> >> <http://www.mapr.com/>
> >>
> >>
> >> Now Available - Free Hadoop On-Demand Training
> >> <
> >>
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >> >
> >>
>
Re: Drill Query Problem
Posted by Jinfeng Ni <ji...@gmail.com>.
You may consider switching the java_compiler from DEFALT to JDK, by :
alter session set `exec.java_compiler` = 'JDK';
All Drill will automatically switch fro janino compiler to JDK, when
the source code length is beyond certain limit. If your query happens
to have smaller code than that limit, but still hit the limit for
janino compiler, then it's better to set the option to JDK.
On Fri, Dec 4, 2015 at 10:48 AM, Jacques Nadeau <ja...@dremio.com> wrote:
> I'm guessing this is due to an extremely large number of columns,
> complicated expressions or extremely long column names. We test up to a
> fairly large size here [1] but you may have exceeded what we test. Can you
> share the query?
>
> [1]
> https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/exec/compile/TestLargeFileCompilation.java
>
> --
> Jacques Nadeau
> CTO and Co-Founder, Dremio
>
> On Fri, Dec 4, 2015 at 9:33 AM, Abdel Hakim Deneche <ad...@maprtech.com>
> wrote:
>
>> Hi Nirav,
>>
>> can you give us more information to help reproduce this issue ?
>>
>> thanks
>>
>> On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <ni...@games24x7.com>
>> wrote:
>>
>> > Hello,
>> >
>> > I am getting below error while running big query.
>> >
>> > =======================================
>> >
>> > Error: SYSTEM ERROR: CompileException: File
>> >
>> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
>> > Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
>> > public void doEval(int inIndex, int outIndex)
>> > ^ (compiler.err.limit.code)
>> >
>> >
>> > Regards,
>> > Nirav
>> >
>>
>>
>>
>> --
>>
>> Abdelhakim Deneche
>>
>> Software Engineer
>>
>> <http://www.mapr.com/>
>>
>>
>> Now Available - Free Hadoop On-Demand Training
>> <
>> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
>> >
>>
Re: Drill Query Problem
Posted by Jacques Nadeau <ja...@dremio.com>.
I'm guessing this is due to an extremely large number of columns,
complicated expressions or extremely long column names. We test up to a
fairly large size here [1] but you may have exceeded what we test. Can you
share the query?
[1]
https://github.com/apache/drill/blob/master/exec/java-exec/src/test/java/org/apache/drill/exec/compile/TestLargeFileCompilation.java
--
Jacques Nadeau
CTO and Co-Founder, Dremio
On Fri, Dec 4, 2015 at 9:33 AM, Abdel Hakim Deneche <ad...@maprtech.com>
wrote:
> Hi Nirav,
>
> can you give us more information to help reproduce this issue ?
>
> thanks
>
> On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <ni...@games24x7.com>
> wrote:
>
> > Hello,
> >
> > I am getting below error while running big query.
> >
> > =======================================
> >
> > Error: SYSTEM ERROR: CompileException: File
> >
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> > Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> > public void doEval(int inIndex, int outIndex)
> > ^ (compiler.err.limit.code)
> >
> >
> > Regards,
> > Nirav
> >
>
>
>
> --
>
> Abdelhakim Deneche
>
> Software Engineer
>
> <http://www.mapr.com/>
>
>
> Now Available - Free Hadoop On-Demand Training
> <
> http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available
> >
>
Re: Drill Query Problem
Posted by Abdel Hakim Deneche <ad...@maprtech.com>.
Hi Nirav,
can you give us more information to help reproduce this issue ?
thanks
On Fri, Dec 4, 2015 at 3:42 AM, Nirav Shah <ni...@games24x7.com> wrote:
> Hello,
>
> I am getting below error while running big query.
>
> =======================================
>
> Error: SYSTEM ERROR: CompileException: File
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> public void doEval(int inIndex, int outIndex)
> ^ (compiler.err.limit.code)
>
>
> Regards,
> Nirav
>
--
Abdelhakim Deneche
Software Engineer
<http://www.mapr.com/>
Now Available - Free Hadoop On-Demand Training
<http://www.mapr.com/training?utm_source=Email&utm_medium=Signature&utm_campaign=Free%20available>
Re: Drill Query Problem
Posted by Sanjeev Verma <sa...@gmail.com>.
I am not sure but seeing a related exception while executing a big query.
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
JaninoRuntimeException: Code attribute in class "org.apache.
drill.exec.test.generated.ProjectorGen189" grows beyond 64 KB
On Fri, Dec 4, 2015 at 5:12 PM, Nirav Shah <ni...@games24x7.com> wrote:
> Hello,
>
> I am getting below error while running big query.
>
> =======================================
>
> Error: SYSTEM ERROR: CompileException: File
> 'org.apache.drill.exec.compile.DrillJavaFileObject[ProjectorGen2825.java]',
> Line 5799, Column 17: ProjectorGen2825.java:5799: error: code too large
> public void doEval(int inIndex, int outIndex)
> ^ (compiler.err.limit.code)
>
>
> Regards,
> Nirav
>