[jira] [Created] (PHOENIX-976) bulk load issue with file permissions

["Cristian Armaselu (JIRA)" <ji...@apache.org>]

Cristian Armaselu created PHOENIX-976:
-----------------------------------------

             Summary: bulk load issue with file permissions
                 Key: PHOENIX-976
                 URL: https://issues.apache.org/jira/browse/PHOENIX-976
             Project: Phoenix
          Issue Type: Bug
    Affects Versions: 3.0.0
         Environment: CDH 4.8.0
            Reporter: Cristian Armaselu


Created and copied a file in hdfs in /tmp/phload/customers.dat
/tmp/phload folder permission is 777

Executed:
hadoop --config /etc/hadoop/conf/ jar phoenix-3.0.0-incubating-client.jar org.apache.phoenix.mapreduce.CsvBulkLoadTool -libjars antlr-runtime-3.4.jar --table CUSTOMERS3 --input /tmp/phload/customers.dat --output /tmp/phload/tmp

MR completes the task successfully
In the client loading we can see:
14/05/11 13:39:32 INFO mapreduce.LoadIncrementalHFiles: Trying to load hfile=hdfs://localhost.localdomain:8020/tmp/phload/tmp/default/0f281fbd70e6443e82c1a559441654e3 first=0-customer_id 0_0 last=9-customer_id 9_0

Then nothing moves past that point.
In hbase region server we can see:
2014-05-11 13:39:32,950 INFO org.apache.hadoop.hbase.regionserver.Store: Validating hfile at hdfs://localhost.localdomain:8020/tmp/phload/tmp/default/0f281fbd70e6443e82c1a559441654e3 for inclusion in store default region CUSTOMERS3,,1399840091691.da914ff9abd642725ac5839b8787c0bb.
2014-05-11 13:39:32,962 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 1
2014-05-11 13:39:33,964 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 2
2014-05-11 13:39:35,966 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 3
2014-05-11 13:39:38,969 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 4
2014-05-11 13:39:42,972 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 5
2014-05-11 13:39:47,975 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 6
2014-05-11 13:39:53,977 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 7
2014-05-11 13:40:00,980 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 8
2014-05-11 13:40:08,983 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 9
2014-05-11 13:40:17,987 INFO org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, sleeping 1000 times 10
2014-05-11 13:40:27,989 WARN org.apache.hadoop.hbase.HBaseFileSystem: Rename Directory, retries exhausted
2014-05-11 13:40:27,990 ERROR org.apache.hadoop.hbase.regionserver.HRegion: There was a partial failure due to IO when attempting to load default : hdfs://localhost.localdomain:8020/tmp/phload/tmp/default/0f281fbd70e6443e82c1a559441654e3

The error is caused by hbase trying to read the folder:
/tmp/phload/tmp/default/0f281fbd70e6443e82c1a559441654e3

As soon as the folder permission is changed to 777 the import continues and data is loaded.

I wold have expected that as soon as I provided a 777 folder in the first place (--output) everything to work smooth.





--
This message was sent by Atlassian JIRA
(v6.2#6252)