You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Bill Graham (JIRA)" <ji...@apache.org> on 2011/05/20 23:21:47 UTC
[jira] [Created] (PIG-2085) HBaseStorage fails with multiple STORE
statements
HBaseStorage fails with multiple STORE statements
-------------------------------------------------
Key: PIG-2085
URL: https://issues.apache.org/jira/browse/PIG-2085
Project: Pig
Issue Type: Bug
Reporter: Bill Graham
Assignee: Bill Graham
Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
{code}
raw = LOAD 'hbase_split_load_bug.txt' AS
(f1: chararray, f2:chararray);
SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');
STORE apples INTO 'hbase://test_table'
USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');
STORE oranges INTO 'hbase://test_table'
USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
{code}
The server throws the following exception after {{apples}} is successfully stored:
{code}
Backend error message
---------------------
java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2085) HBaseStorage fails with multiple
STORE statements
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038268#comment-13038268 ]
Bill Graham commented on PIG-2085:
----------------------------------
>From discussions on the HBase list, I think this could be an issue in TableOutputFormat in 0.90, where closing the connection on one table killed the connections for all tables:
http://mail-archives.apache.org/mod_mbox/hbase-user/201105.mbox/%3cBANLkTimCXKvtPAqi-HY2uT-h434xub8SNA@mail.gmail.com%3e
If anyone has an HBase cluster running off the trunk to test this theory on (we're still on 0.90), please do so with the attached scripts and report back. HBASE-3777 is the relevant fix.
> HBaseStorage fails with multiple STORE statements
> -------------------------------------------------
>
> Key: PIG-2085
> URL: https://issues.apache.org/jira/browse/PIG-2085
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2085_example_input.txt, PIG-2085_example_script.pig, PIG-2085_schema.hbase
>
>
> Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
> {code}
> raw = LOAD 'hbase_split_load_bug.txt' AS
> (f1: chararray, f2:chararray);
> SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');
> STORE apples INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');
> STORE oranges INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
> {code}
> The server throws the following exception after {{apples}} is successfully stored:
> {code}
> Backend error message
> ---------------------
> java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2085) HBaseStorage fails with multiple
STORE statements
Posted by "Royston Sellman (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13212546#comment-13212546 ]
Royston Sellman commented on PIG-2085:
--------------------------------------
9 months since last comment but in case it's still relevant: we are running HBase off trunk and this test PASSES using Pig 0.9.2.
> HBaseStorage fails with multiple STORE statements
> -------------------------------------------------
>
> Key: PIG-2085
> URL: https://issues.apache.org/jira/browse/PIG-2085
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2085_example_input.txt, PIG-2085_example_script.pig, PIG-2085_schema.hbase
>
>
> Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
> {code}
> raw = LOAD 'hbase_split_load_bug.txt' AS
> (f1: chararray, f2:chararray);
> SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');
> STORE apples INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');
> STORE oranges INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
> {code}
> The server throws the following exception after {{apples}} is successfully stored:
> {code}
> Backend error message
> ---------------------
> java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2085) HBaseStorage fails with multiple
STORE statements
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260658#comment-13260658 ]
Dmitriy V. Ryaboy commented on PIG-2085:
----------------------------------------
Kevin,
This is an HBase bug as described above. The HBase bug was fixed in 0.92 (not 0.90.2).
> HBaseStorage fails with multiple STORE statements
> -------------------------------------------------
>
> Key: PIG-2085
> URL: https://issues.apache.org/jira/browse/PIG-2085
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2085_example_input.txt, PIG-2085_example_script.pig, PIG-2085_schema.hbase
>
>
> Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
> {code}
> raw = LOAD 'hbase_split_load_bug.txt' AS
> (f1: chararray, f2:chararray);
> SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');
> STORE apples INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');
> STORE oranges INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
> {code}
> The server throws the following exception after {{apples}} is successfully stored:
> {code}
> Backend error message
> ---------------------
> java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-2085) HBaseStorage fails with multiple STORE
statements
Posted by "Bill Graham (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bill Graham updated PIG-2085:
-----------------------------
Attachment: PIG-2085_example_input.txt
PIG-2085_schema.hbase
PIG-2085_example_script.pig
Attaching scripts to create the HBase table and to reproduce, along with sample input data.
> HBaseStorage fails with multiple STORE statements
> -------------------------------------------------
>
> Key: PIG-2085
> URL: https://issues.apache.org/jira/browse/PIG-2085
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2085_example_input.txt, PIG-2085_example_script.pig, PIG-2085_schema.hbase
>
>
> Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
> {code}
> raw = LOAD 'hbase_split_load_bug.txt' AS
> (f1: chararray, f2:chararray);
> SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');
> STORE apples INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');
> STORE oranges INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
> {code}
> The server throws the following exception after {{apples}} is successfully stored:
> {code}
> Backend error message
> ---------------------
> java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2085) HBaseStorage fails with multiple
STORE statements
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038271#comment-13038271 ]
Dmitriy V. Ryaboy commented on PIG-2085:
----------------------------------------
We will likely be upgrading to 0.93 this week, I'll test once we do.
> HBaseStorage fails with multiple STORE statements
> -------------------------------------------------
>
> Key: PIG-2085
> URL: https://issues.apache.org/jira/browse/PIG-2085
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2085_example_input.txt, PIG-2085_example_script.pig, PIG-2085_schema.hbase
>
>
> Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
> {code}
> raw = LOAD 'hbase_split_load_bug.txt' AS
> (f1: chararray, f2:chararray);
> SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');
> STORE apples INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');
> STORE oranges INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
> {code}
> The server throws the following exception after {{apples}} is successfully stored:
> {code}
> Backend error message
> ---------------------
> java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (PIG-2085) HBaseStorage fails with multiple STORE
statements
Posted by "Dmitriy V. Ryaboy (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy V. Ryaboy resolved PIG-2085.
------------------------------------
Resolution: Not A Problem
Just doing some housecleaning.
> HBaseStorage fails with multiple STORE statements
> -------------------------------------------------
>
> Key: PIG-2085
> URL: https://issues.apache.org/jira/browse/PIG-2085
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2085_example_input.txt, PIG-2085_example_script.pig, PIG-2085_schema.hbase
>
>
> Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
> {code}
> raw = LOAD 'hbase_split_load_bug.txt' AS
> (f1: chararray, f2:chararray);
> SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');
> STORE apples INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');
> STORE oranges INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
> {code}
> The server throws the following exception after {{apples}} is successfully stored:
> {code}
> Backend error message
> ---------------------
> java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2085) HBaseStorage fails with multiple
STORE statements
Posted by "Dmitriy V. Ryaboy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038225#comment-13038225 ]
Dmitriy V. Ryaboy commented on PIG-2085:
----------------------------------------
I bet HBaseOutputFormat gets confused when the Pig does its optimizations and tries to do 2 stores in 1 reduce phase.
> HBaseStorage fails with multiple STORE statements
> -------------------------------------------------
>
> Key: PIG-2085
> URL: https://issues.apache.org/jira/browse/PIG-2085
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2085_example_input.txt, PIG-2085_example_script.pig, PIG-2085_schema.hbase
>
>
> Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
> {code}
> raw = LOAD 'hbase_split_load_bug.txt' AS
> (f1: chararray, f2:chararray);
> SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');
> STORE apples INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');
> STORE oranges INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
> {code}
> The server throws the following exception after {{apples}} is successfully stored:
> {code}
> Backend error message
> ---------------------
> java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-2085) HBaseStorage fails with multiple
STORE statements
Posted by "Kevin Lion (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-2085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13260483#comment-13260483 ]
Kevin Lion commented on PIG-2085:
---------------------------------
Using HBase 0.90.3 and Pig 0.9.2 : the bug is still here.
> HBaseStorage fails with multiple STORE statements
> -------------------------------------------------
>
> Key: PIG-2085
> URL: https://issues.apache.org/jira/browse/PIG-2085
> Project: Pig
> Issue Type: Bug
> Reporter: Bill Graham
> Assignee: Bill Graham
> Attachments: PIG-2085_example_input.txt, PIG-2085_example_script.pig, PIG-2085_schema.hbase
>
>
> Scripts with multiple STORE statements using HBaseStorage fail when run against a cluster (they succeed in local mode). Below is an example script:
> {code}
> raw = LOAD 'hbase_split_load_bug.txt' AS
> (f1: chararray, f2:chararray);
> SPLIT raw INTO apples IF (f2 == 'apple'), oranges IF (f2 == 'orange');
> STORE apples INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:apple');
> STORE oranges INTO 'hbase://test_table'
> USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('info:orange');
> {code}
> The server throws the following exception after {{apples}} is successfully stored:
> {code}
> Backend error message
> ---------------------
> java.io.IOException: org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@6273305c closed
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:566)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1113)
> at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchOfPuts(HConnectionManager.java:1233)
> at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:819)
> at org.apache.hadoop.hbase.mapreduce.TableOutputFormat$TableRecordWriter.close(TableOutputFormat.java:106)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl.tearDown(MapReducePOStoreImpl.java:96)
> at org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStore.tearDown(POStore.java:122)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.cleanup(PigMapBase.java:128)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
> at org.apache.hadoop.mapred.Child.main(Child.java:170)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira