You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by IT CTO <go...@gmail.com> on 2015/10/01 14:17:14 UTC
Hive SerDe regex error
Hi,
I am trying to create a table with Regex SerDe but failing with no good
reason:
CREATE EXTERNAL TABLE syslog (
month STRING,
day STRING,
time STRING,
source STRING,
process STRING,
pid STRING,
uname STRING,
message STRING)
COMMENT 'This is the syslog sample table'
ROW FORMAT SERDE "org.apache.hadoop.hive.serde2.RegexSerDe"
WITH SERDEPROPERTIES (
"input.regex" = "([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ \[]*)\[([^ ]*)\]:
\(([^ ]*)\) ([^]*)"
)
STORED AS TEXTFILE
LOCATION 'dfs://localhost:8020/data/flumeTest/flume-test-spoolDir';
The regex iself works on regex tester so I don't understand why I am
getting:
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask.
java.util.regex.PatternSyntaxException: Unclosed character class near index
66
([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ []*)[([^ ]*)]: (([^ ]*)) ([^]*)
Any help?
--
Eran | "You don't need eyes to see, you need vision" (Faithless)
RE: Hive SerDe regex error
Posted by Ryan Harris <Ry...@zionsbancorp.com>.
depending on how you are submitting the statement to hive, you'll probably need to escape the backslash...
try replacing every \ with \\
From: IT CTO [mailto:goi.cto@gmail.com]
Sent: Thursday, October 01, 2015 6:25 AM
To: user@hive.apache.org
Subject: Re: Hive SerDe regex error
Your Regex gives different results. I want the last part to capture the rest of the string.
I also tried this
CREATE EXTERNAL TABLE syslog (
month STRING,
day STRING,
time STRING,
source STRING,
process STRING,
pid STRING,
uname STRING,
message STRING)
COMMENT 'This is the syslog sample table'
ROW FORMAT SERDE "org.apache.hadoop.hive.serde2.RegexSerDe"
WITH SERDEPROPERTIES (
"input.regex" = "([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ \[]*)\[([^ ]*)\]: \(([^ ]*)\) ([\S|\s]*)"
)
STORED AS TEXTFILE
LOCATION 'dfs://localhost:8020/data/flumeTest/flume-test-spoolDir';
And got the same result
Eran
On Thu, Oct 1, 2015 at 3:20 PM Daniel Haviv <da...@veracity-group.com>> wrote:
Hi,
You didn't escape the ^ character at the end.
Try using this string instead: ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ \[]*)\[([^ ]*)\]: \(([^ ]*)\) ([\^]*)
Daniel
On Thu, Oct 1, 2015 at 3:17 PM, IT CTO <go...@gmail.com>> wrote:
Hi,
I am trying to create a table with Regex SerDe but failing with no good reason:
CREATE EXTERNAL TABLE syslog (
month STRING,
day STRING,
time STRING,
source STRING,
process STRING,
pid STRING,
uname STRING,
message STRING)
COMMENT 'This is the syslog sample table'
ROW FORMAT SERDE "org.apache.hadoop.hive.serde2.RegexSerDe"
WITH SERDEPROPERTIES (
"input.regex" = "([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ \[]*)\[([^ ]*)\]: \(([^ ]*)\) ([^]*)"
)
STORED AS TEXTFILE
LOCATION 'dfs://localhost:8020/data/flumeTest/flume-test-spoolDir';
The regex iself works on regex tester so I don't understand why I am getting:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.util.regex.PatternSyntaxException: Unclosed character class near index 66
([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ []*)[([^ ]*)]: (([^ ]*)) ([^]*)
Any help?
--
Eran | "You don't need eyes to see, you need vision" (Faithless)
--
Eran | "You don't need eyes to see, you need vision" (Faithless)
======================================================================
THIS ELECTRONIC MESSAGE, INCLUDING ANY ACCOMPANYING DOCUMENTS, IS CONFIDENTIAL and may contain information that is privileged and exempt from disclosure under applicable law. If you are neither the intended recipient nor responsible for delivering the message to the intended recipient, please note that any dissemination, distribution, copying or the taking of any action in reliance upon the message is strictly prohibited. If you have received this communication in error, please notify the sender immediately. Thank you.
Re: Hive SerDe regex error
Posted by IT CTO <go...@gmail.com>.
Your Regex gives different results. I want the last part to capture the
rest of the string.
I also tried this
CREATE EXTERNAL TABLE syslog (
month STRING,
day STRING,
time STRING,
source STRING,
process STRING,
pid STRING,
uname STRING,
message STRING)
COMMENT 'This is the syslog sample table'
ROW FORMAT SERDE "org.apache.hadoop.hive.serde2.RegexSerDe"
WITH SERDEPROPERTIES (
"input.regex" = "([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ \[]*)\[([^ ]*)\]:
\(([^ ]*)\) ([\S|\s]*)"
)
STORED AS TEXTFILE
LOCATION 'dfs://localhost:8020/data/flumeTest/flume-test-spoolDir';
And got the same result
Eran
On Thu, Oct 1, 2015 at 3:20 PM Daniel Haviv <da...@veracity-group.com>
wrote:
> Hi,
> You didn't escape the ^ character at the end.
> Try using this string instead: ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^
> \[]*)\[([^ ]*)\]: \(([^ ]*)\) ([\^]*)
>
> Daniel
>
> On Thu, Oct 1, 2015 at 3:17 PM, IT CTO <go...@gmail.com> wrote:
>
>> Hi,
>> I am trying to create a table with Regex SerDe but failing with no good
>> reason:
>> CREATE EXTERNAL TABLE syslog (
>> month STRING,
>> day STRING,
>> time STRING,
>> source STRING,
>> process STRING,
>> pid STRING,
>> uname STRING,
>> message STRING)
>> COMMENT 'This is the syslog sample table'
>> ROW FORMAT SERDE "org.apache.hadoop.hive.serde2.RegexSerDe"
>> WITH SERDEPROPERTIES (
>> "input.regex" = "([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ \[]*)\[([^ ]*)\]:
>> \(([^ ]*)\) ([^]*)"
>> )
>> STORED AS TEXTFILE
>> LOCATION 'dfs://localhost:8020/data/flumeTest/flume-test-spoolDir';
>>
>> The regex iself works on regex tester so I don't understand why I am
>> getting:
>>
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.DDLTask.
>> java.util.regex.PatternSyntaxException: Unclosed character class near index
>> 66
>> ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ []*)[([^ ]*)]: (([^ ]*)) ([^]*)
>>
>> Any help?
>> --
>> Eran | "You don't need eyes to see, you need vision" (Faithless)
>>
>
> --
Eran | "You don't need eyes to see, you need vision" (Faithless)
Re: Hive SerDe regex error
Posted by Daniel Haviv <da...@veracity-group.com>.
Hi,
You didn't escape the ^ character at the end.
Try using this string instead: ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^
\[]*)\[([^ ]*)\]: \(([^ ]*)\) ([\^]*)
Daniel
On Thu, Oct 1, 2015 at 3:17 PM, IT CTO <go...@gmail.com> wrote:
> Hi,
> I am trying to create a table with Regex SerDe but failing with no good
> reason:
> CREATE EXTERNAL TABLE syslog (
> month STRING,
> day STRING,
> time STRING,
> source STRING,
> process STRING,
> pid STRING,
> uname STRING,
> message STRING)
> COMMENT 'This is the syslog sample table'
> ROW FORMAT SERDE "org.apache.hadoop.hive.serde2.RegexSerDe"
> WITH SERDEPROPERTIES (
> "input.regex" = "([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ \[]*)\[([^ ]*)\]:
> \(([^ ]*)\) ([^]*)"
> )
> STORED AS TEXTFILE
> LOCATION 'dfs://localhost:8020/data/flumeTest/flume-test-spoolDir';
>
> The regex iself works on regex tester so I don't understand why I am
> getting:
>
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask.
> java.util.regex.PatternSyntaxException: Unclosed character class near index
> 66
> ([^ ]*) ([^ ]*) ([^ ]*) ([^ ]*) ([^ []*)[([^ ]*)]: (([^ ]*)) ([^]*)
>
> Any help?
> --
> Eran | "You don't need eyes to see, you need vision" (Faithless)
>