You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Terje Marthinussen (JIRA)" <ji...@apache.org> on 2010/12/14 14:19:00 UTC
[jira] Created: (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
--------------------------------------------------------------------------------------------------
Key: HIVE-1850
URL: https://issues.apache.org/jira/browse/HIVE-1850
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.7.0
Environment: Trunk build from a few days ago, but seen once before with older version as well.
Reporter: Terje Marthinussen
create table aa ( test STRING )
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
This will fail. Great!
create table aa ( test STRING )
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
Works, no problem there.
alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
Wups... I can set that without any problems!
alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
[^\](.*)
^)
java.util.regex.PatternSyntaxException: Unclosed character class near index 7
[^\](.*)
^
at java.util.regex.Pattern.error(Pattern.java:1713)
at java.util.regex.Pattern.clazz(Pattern.java:2254)
at java.util.regex.Pattern.sequence(Pattern.java:1818)
at java.util.regex.Pattern.expr(Pattern.java:1752)
at java.util.regex.Pattern.compile(Pattern.java:1460)
at java.util.regex.Pattern.<init>(Pattern.java:1133)
at java.util.regex.Pattern.compile(Pattern.java:847)
at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
After this, all further commands on the table fails, including drop table :)
1. The alter table command should probably check the regexp just like the create table command does
2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HIVE-1850:
------------------------------------------
Attachment: patch-1850-2.txt
This patch fixes protectmode failure.
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.8.0
>
> Attachments: patch-1850-2.txt, patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HIVE-1850:
------------------------------------------
Status: Open (was: Patch Available)
canceling patch as one of the tests is failing.
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.8.0
>
> Attachments: patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HIVE-1850:
------------------------------------------
Status: Patch Available (was: Open)
Fixed the test failure(protectmode.q) with the patch on review board.
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.8.0
>
> Attachments: patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "jiraposter@reviews.apache.org (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073434#comment-13073434 ]
jiraposter@reviews.apache.org commented on HIVE-1850:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1233/
-----------------------------------------------------------
Review request for hive.
Summary
-------
alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
This addresses bug HIVE-1850.
https://issues.apache.org/jira/browse/HIVE-1850
Diffs
-----
trunk/ql/src/test/results/clientnegative/create_table_wrong_regex.q.out PRE-CREATION
trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java 1151047
trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 1151047
trunk/ql/src/test/queries/clientnegative/alter_table_wrong_regex.q PRE-CREATION
trunk/ql/src/test/queries/clientnegative/create_table_wrong_regex.q PRE-CREATION
trunk/ql/src/test/results/clientnegative/alter_table_wrong_regex.q.out PRE-CREATION
Diff: https://reviews.apache.org/r/1233/diff
Testing
-------
All tests pass with the patch.
Thanks,
Amareshwari
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.8.0
>
> Attachments: patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135828#comment-13135828 ]
Hudson commented on HIVE-1850:
------------------------------
Integrated in Hive-trunk-h0.21 #1036 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1036/])
HIVE-1850 alter table set serdeproperties bypasses regexps checks
(leaves table in a non-recoverable state?) (Amareshwari Sriramadasu via namit)
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1189026
Files :
* /hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreUtils.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
* /hive/trunk/ql/src/test/queries/clientnegative/alter_table_wrong_regex.q
* /hive/trunk/ql/src/test/queries/clientnegative/create_table_wrong_regex.q
* /hive/trunk/ql/src/test/results/clientnegative/alter_table_wrong_regex.q.out
* /hive/trunk/ql/src/test/results/clientnegative/create_table_wrong_regex.q.out
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-1850-2.txt, patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1850:
---------------------------------
Fix Version/s: (was: 0.8.0)
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-1850-2.txt, patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HIVE-1850:
------------------------------------------
Fix Version/s: 0.8.0
Status: Patch Available (was: Open)
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.8.0
>
> Attachments: patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HIVE-1850:
------------------------------------------
Attachment: patch-1850.txt
Even though, DDLTask.alterTable() does a checkValidity for the table after all the alterations, this problem is not found. Because getDeserializer() was not getting it from the Metastore with modified properties.
Patch does the required change and add regression test.
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu updated HIVE-1850:
------------------------------------------
Status: Patch Available (was: Open)
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.8.0
>
> Attachments: patch-1850-2.txt, patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Terje Marthinussen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Terje Marthinussen updated HIVE-1850:
-------------------------------------
Description:
{code}
create table aa ( test STRING )
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
{code}
This will fail. Great!
{code}
create table aa ( test STRING )
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
{code}
Works, no problem there.
{code}
alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
{code}
Wups... I can set that without any problems!
{code}
alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
[^\](.*)
^)
java.util.regex.PatternSyntaxException: Unclosed character class near index 7
[^\](.*)
^
at java.util.regex.Pattern.error(Pattern.java:1713)
at java.util.regex.Pattern.clazz(Pattern.java:2254)
at java.util.regex.Pattern.sequence(Pattern.java:1818)
at java.util.regex.Pattern.expr(Pattern.java:1752)
at java.util.regex.Pattern.compile(Pattern.java:1460)
at java.util.regex.Pattern.<init>(Pattern.java:1133)
at java.util.regex.Pattern.compile(Pattern.java:847)
at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
{code}
After this, all further commands on the table fails, including drop table :)
1. The alter table command should probably check the regexp just like the create table command does
2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
was:
create table aa ( test STRING )
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
This will fail. Great!
create table aa ( test STRING )
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
Works, no problem there.
alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
Wups... I can set that without any problems!
alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
[^\](.*)
^)
java.util.regex.PatternSyntaxException: Unclosed character class near index 7
[^\](.*)
^
at java.util.regex.Pattern.error(Pattern.java:1713)
at java.util.regex.Pattern.clazz(Pattern.java:2254)
at java.util.regex.Pattern.sequence(Pattern.java:1818)
at java.util.regex.Pattern.expr(Pattern.java:1752)
at java.util.regex.Pattern.compile(Pattern.java:1460)
at java.util.regex.Pattern.<init>(Pattern.java:1133)
at java.util.regex.Pattern.compile(Pattern.java:847)
at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
After this, all further commands on the table fails, including drop table :)
1. The alter table command should probably check the regexp just like the create table command does
2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] [Updated] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Carl Steinbach (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Carl Steinbach updated HIVE-1850:
---------------------------------
Status: Open (was: Patch Available)
@Amareshwari: I see a diff in TestCliDriver.protectmode. Can you please take a look and see if you get the same result? Thanks.
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.8.0
>
> Attachments: patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Amareshwari Sriramadasu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Amareshwari Sriramadasu reassigned HIVE-1850:
---------------------------------------------
Assignee: Amareshwari Sriramadasu
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Namit Jain (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Namit Jain updated HIVE-1850:
-----------------------------
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed. Thanks Amareshwari
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-1850-2.txt, patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1850) alter table set serdeproperties
bypasses regexps checks (leaves table in a non-recoverable state?)
Posted by "Namit Jain (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HIVE-1850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13135242#comment-13135242 ]
Namit Jain commented on HIVE-1850:
----------------------------------
+1
> alter table set serdeproperties bypasses regexps checks (leaves table in a non-recoverable state?)
> --------------------------------------------------------------------------------------------------
>
> Key: HIVE-1850
> URL: https://issues.apache.org/jira/browse/HIVE-1850
> Project: Hive
> Issue Type: Bug
> Components: Serializers/Deserializers
> Affects Versions: 0.7.0
> Environment: Trunk build from a few days ago, but seen once before with older version as well.
> Reporter: Terje Marthinussen
> Assignee: Amareshwari Sriramadasu
> Attachments: patch-1850-2.txt, patch-1850.txt
>
>
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> This will fail. Great!
> {code}
> create table aa ( test STRING )
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
> WITH SERDEPROPERTIES ("input.regex" = "(.*)", "output.format.string" = "$1s");
> {code}
> Works, no problem there.
> {code}
> alter table aa set serdeproperties ("input.regex" = "[^\\](.*)", "output.format.string" = "$1s");
> {code}
> Wups... I can set that without any problems!
> {code}
> alter table aa set serdeproperties ("input.regex" = "(.*)", "output.format.string" = "$1s");
> FAILED: Hive Internal Error: java.util.regex.PatternSyntaxException(Unclosed character class near index 7
> [^\](.*)
> ^)
> java.util.regex.PatternSyntaxException: Unclosed character class near index 7
> [^\](.*)
> ^
> at java.util.regex.Pattern.error(Pattern.java:1713)
> at java.util.regex.Pattern.clazz(Pattern.java:2254)
> at java.util.regex.Pattern.sequence(Pattern.java:1818)
> at java.util.regex.Pattern.expr(Pattern.java:1752)
> at java.util.regex.Pattern.compile(Pattern.java:1460)
> at java.util.regex.Pattern.<init>(Pattern.java:1133)
> at java.util.regex.Pattern.compile(Pattern.java:847)
> at org.apache.hadoop.hive.contrib.serde2.RegexSerDe.initialize(RegexSerDe.java:101)
> at org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:199)
> at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
> at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:484)
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:161)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:803)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerdeProps(DDLSemanticAnalyzer.java:558)
> at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:232)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:238)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:686)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:142)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:216)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:370)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> {code}
> After this, all further commands on the table fails, including drop table :)
> 1. The alter table command should probably check the regexp just like the create table command does
> 2. Even though the regexp is bad, it should be possible to do things like set the regexp again or drop the table.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira