You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by iryoung jeong <ir...@gmail.com> on 2010/12/20 07:16:40 UTC
hive - comment in sql file cause error message?
Hello, there.
I found that running hive with sql file caused error messages because of
comments.
For example, me and data analysts in my company think these sql file is make
sense.
~/Desktop/projecs/hive/hive-trunk/build/dist$ cat q.sql
-- hello
-- hello ;
set;
-- hello again
-- hello again;
but, hive think it doesn't.
When I run the sql file using "hive -f q.sql", then hive complains like
this:
FAILED: Parse Error: line 0:-1 cannot recognize input '<EOF>'
Analyst people confused that this was caused by their SQLs or just comments.
So, I had to apply quick-fix to hive source.
~/Desktop/projecs/hive/hive-trunk$ svn diff
Index: cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
===================================================================
--- cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java (revision
1050978)
+++ cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java (working copy)
@@ -245,11 +245,14 @@
StringBuilder qsb = new StringBuilder();
while ((line = r.readLine()) != null) {
- qsb.append(line + "\n");
+ qsb.append(removeComment(line) + "\n");
}
return (processLine(qsb.toString()));
}
+ private String removeComment(String line) {
+ return line.replaceFirst("--.+$","").trim();
+ }
public int processFile(String fileName) throws IOException {
FileReader fileReader = null;
And, there are no more error messages related to comments.
I know this temporary quick-fix doesn't solve the root of trouble, but I
hope this can be useful to other people until the problem is sovled
correctly.
btw, I just wonder I have to bring up an issue to hive-jira about this
problem or just ask other people in here to make the issue. If someone can
do that, then please make a issue or tell me that this is a too small issue
to raise issue in jira.
Thanks.
iryoung.
Re: hive - comment in sql file cause error message?
Posted by iryoung jeong <ir...@gmail.com>.
oh, that might be a right issue. It's already there long time ago.
I'll submit patch to there.
thanks :)
On Tue, Dec 21, 2010 at 1:27 AM, Edward Capriolo <ed...@gmail.com>wrote:
> On Mon, Dec 20, 2010 at 10:30 AM, Edward Capriolo <ed...@gmail.com>
> wrote:
> > On Mon, Dec 20, 2010 at 1:16 AM, iryoung jeong <ir...@gmail.com>
> wrote:
> >> Hello, there.
> >> I found that running hive with sql file caused error messages because of
> >> comments.
> >> For example, me and data analysts in my company think these sql file is
> make
> >> sense.
> >> ~/Desktop/projecs/hive/hive-trunk/build/dist$ cat q.sql
> >> -- hello
> >> -- hello ;
> >> set;
> >> -- hello again
> >> -- hello again;
> >> but, hive think it doesn't.
> >> When I run the sql file using "hive -f q.sql", then hive complains like
> >> this:
> >> FAILED: Parse Error: line 0:-1 cannot recognize input '<EOF>'
> >> Analyst people confused that this was caused by their SQLs or just
> comments.
> >> So, I had to apply quick-fix to hive source.
> >> ~/Desktop/projecs/hive/hive-trunk$ svn diff
> >> Index: cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
> >> ===================================================================
> >> --- cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java (revision
> >> 1050978)
> >> +++ cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java (working
> copy)
> >> @@ -245,11 +245,14 @@
> >> StringBuilder qsb = new StringBuilder();
> >>
> >> while ((line = r.readLine()) != null) {
> >> - qsb.append(line + "\n");
> >> + qsb.append(removeComment(line) + "\n");
> >> }
> >>
> >> return (processLine(qsb.toString()));
> >> }
> >> + private String removeComment(String line) {
> >> + return line.replaceFirst("--.+$","").trim();
> >> + }
> >>
> >> public int processFile(String fileName) throws IOException {
> >> FileReader fileReader = null;
> >>
> >> And, there are no more error messages related to comments.
> >> I know this temporary quick-fix doesn't solve the root of trouble, but I
> >> hope this can be useful to other people until the problem is sovled
> >> correctly.
> >> btw, I just wonder I have to bring up an issue to hive-jira about this
> >> problem or just ask other people in here to make the issue. If someone
> can
> >> do that, then please make a issue or tell me that this is a too small
> issue
> >> to raise issue in jira.
> >> Thanks.
> >> iryoung.
> >>
> >
> > QL can not handle comments before set commands. Comments are actually
> > handed up to the QueryParser. Your patch is removing them at the CLI
> > level. (which is "ok") we have a few little hacks baked into the CLI
> > and how it "parses" things.
> >
>
> You can upload your patch here, This issues is very close to what you
> are fixing.
>
> https://issues.apache.org/jira/browse/HIVE-1431
>
Re: hive - comment in sql file cause error message?
Posted by Edward Capriolo <ed...@gmail.com>.
On Mon, Dec 20, 2010 at 10:30 AM, Edward Capriolo <ed...@gmail.com> wrote:
> On Mon, Dec 20, 2010 at 1:16 AM, iryoung jeong <ir...@gmail.com> wrote:
>> Hello, there.
>> I found that running hive with sql file caused error messages because of
>> comments.
>> For example, me and data analysts in my company think these sql file is make
>> sense.
>> ~/Desktop/projecs/hive/hive-trunk/build/dist$ cat q.sql
>> -- hello
>> -- hello ;
>> set;
>> -- hello again
>> -- hello again;
>> but, hive think it doesn't.
>> When I run the sql file using "hive -f q.sql", then hive complains like
>> this:
>> FAILED: Parse Error: line 0:-1 cannot recognize input '<EOF>'
>> Analyst people confused that this was caused by their SQLs or just comments.
>> So, I had to apply quick-fix to hive source.
>> ~/Desktop/projecs/hive/hive-trunk$ svn diff
>> Index: cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
>> ===================================================================
>> --- cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java (revision
>> 1050978)
>> +++ cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java (working copy)
>> @@ -245,11 +245,14 @@
>> StringBuilder qsb = new StringBuilder();
>>
>> while ((line = r.readLine()) != null) {
>> - qsb.append(line + "\n");
>> + qsb.append(removeComment(line) + "\n");
>> }
>>
>> return (processLine(qsb.toString()));
>> }
>> + private String removeComment(String line) {
>> + return line.replaceFirst("--.+$","").trim();
>> + }
>>
>> public int processFile(String fileName) throws IOException {
>> FileReader fileReader = null;
>>
>> And, there are no more error messages related to comments.
>> I know this temporary quick-fix doesn't solve the root of trouble, but I
>> hope this can be useful to other people until the problem is sovled
>> correctly.
>> btw, I just wonder I have to bring up an issue to hive-jira about this
>> problem or just ask other people in here to make the issue. If someone can
>> do that, then please make a issue or tell me that this is a too small issue
>> to raise issue in jira.
>> Thanks.
>> iryoung.
>>
>
> QL can not handle comments before set commands. Comments are actually
> handed up to the QueryParser. Your patch is removing them at the CLI
> level. (which is "ok") we have a few little hacks baked into the CLI
> and how it "parses" things.
>
You can upload your patch here, This issues is very close to what you
are fixing.
https://issues.apache.org/jira/browse/HIVE-1431
Re: hive - comment in sql file cause error message?
Posted by Edward Capriolo <ed...@gmail.com>.
On Mon, Dec 20, 2010 at 1:16 AM, iryoung jeong <ir...@gmail.com> wrote:
> Hello, there.
> I found that running hive with sql file caused error messages because of
> comments.
> For example, me and data analysts in my company think these sql file is make
> sense.
> ~/Desktop/projecs/hive/hive-trunk/build/dist$ cat q.sql
> -- hello
> -- hello ;
> set;
> -- hello again
> -- hello again;
> but, hive think it doesn't.
> When I run the sql file using "hive -f q.sql", then hive complains like
> this:
> FAILED: Parse Error: line 0:-1 cannot recognize input '<EOF>'
> Analyst people confused that this was caused by their SQLs or just comments.
> So, I had to apply quick-fix to hive source.
> ~/Desktop/projecs/hive/hive-trunk$ svn diff
> Index: cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
> ===================================================================
> --- cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java (revision
> 1050978)
> +++ cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java (working copy)
> @@ -245,11 +245,14 @@
> StringBuilder qsb = new StringBuilder();
>
> while ((line = r.readLine()) != null) {
> - qsb.append(line + "\n");
> + qsb.append(removeComment(line) + "\n");
> }
>
> return (processLine(qsb.toString()));
> }
> + private String removeComment(String line) {
> + return line.replaceFirst("--.+$","").trim();
> + }
>
> public int processFile(String fileName) throws IOException {
> FileReader fileReader = null;
>
> And, there are no more error messages related to comments.
> I know this temporary quick-fix doesn't solve the root of trouble, but I
> hope this can be useful to other people until the problem is sovled
> correctly.
> btw, I just wonder I have to bring up an issue to hive-jira about this
> problem or just ask other people in here to make the issue. If someone can
> do that, then please make a issue or tell me that this is a too small issue
> to raise issue in jira.
> Thanks.
> iryoung.
>
QL can not handle comments before set commands. Comments are actually
handed up to the QueryParser. Your patch is removing them at the CLI
level. (which is "ok") we have a few little hacks baked into the CLI
and how it "parses" things.