You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by mo...@open-insights.co.in on 2017/11/10 04:56:20 UTC
PutHiveStreaming Error : java.lang.RuntimeException: java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwx------
Hi all,
I am using PutHiveStreaming processor to ingest the data into a partitioned
table. I have given all the permissions to /tmp/hive folder.
Still, it prompts the error :
2017-11-09 11:48:06,849 ERROR [Timer-Driven Process Thread-8]
o.a.n.processors.hive.PutHiveStreaming
PutHiveStreaming[id=9b7b6af6-015f-1000-8772-6b70eb0a4841]
PutHiveStreaming[id=9b7b6af6-015f-1000-8772-6b70eb0a4841] failed to process
due to java.lang.RuntimeException: java.lang.RuntimeException: The root
scratch dir: /tmp/hive on HDFS should be writable. Current permissions are:
rwx------; rolling back session: {}
java.lang.RuntimeException: java.lang.RuntimeException: The root scratch
dir: /tmp/hive on HDFS should be writable. Current permissions are:
rwx------
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:523)
at
org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.createPartiti
onIfNotExists(HiveEndPoint.java:454)
at
org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEn
dPoint.java:318)
at
org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.<init>(HiveEn
dPoint.java:278)
at
org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnectionImpl(HiveEndPoi
nt.java:215)
at
org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.j
ava:192)
at
org.apache.hive.hcatalog.streaming.HiveEndPoint.newConnection(HiveEndPoint.j
ava:122)
at
org.apache.nifi.util.hive.HiveWriter.lambda$newConnection$6(HiveWriter.java:
237)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11
49)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6
24)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: The root scratch dir: /tmp/hive on
HDFS should be writable. Current permissions are: rwx------
at
org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionStat
e.java:613)
at
org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionStat
e.java:555)
at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:509)
... 11 common frames omitted
I'm sharing the detail error in the screenshot.
Please let me know if anyone is aware about this issue.
Regards,
Mohit Jain
Re: PutHiveStreaming Error : java.lang.RuntimeException:
java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be
writable. Current permissions are: rwx------
Posted by Pierre Villard <pi...@gmail.com>.
Hi Mohit,
As far as I know, the Hive Streaming API is directly writing into HDFS and,
based on the stack trace, is trying to write into /tmp/hive. Right now this
folder is not writable by your user and it is the reason why it's failing.
I would try changing the chmod permissions in HDFS.
Pierre
2017-11-10 5:56 GMT+01:00 <mo...@open-insights.co.in>:
> Hi all,
>
>
>
> I am using PutHiveStreaming processor to ingest the data into a
> partitioned table. I have given all the permissions to /tmp/hive folder.
>
> Still, it prompts the error :
>
>
>
> 2017-11-09 11:48:06,849 ERROR [Timer-Driven Process Thread-8]
> o.a.n.processors.hive.PutHiveStreaming PutHiveStreaming[id=9b7b6af6-015f-1000-8772-6b70eb0a4841]
> PutHiveStreaming[id=9b7b6af6-015f-1000-8772-6b70eb0a4841] failed to
> process due to java.lang.RuntimeException: java.lang.RuntimeException: The
> root scratch dir: /tmp/hive on HDFS should be writable. Current permissions
> are: rwx------; rolling back session: {}
>
> java.lang.RuntimeException: java.lang.RuntimeException: The root scratch
> dir: /tmp/hive on HDFS should be writable. Current permissions are:
> rwx------
>
> at org.apache.hadoop.hive.ql.session.SessionState.start(
> SessionState.java:523)
>
> at org.apache.hive.hcatalog.streaming.HiveEndPoint$ConnectionImpl.
> createPartitionIfNotExists(HiveEndPoint.java:454)
>
> at org.apache.hive.hcatalog.streaming.HiveEndPoint$
> ConnectionImpl.<init>(HiveEndPoint.java:318)
>
> at org.apache.hive.hcatalog.streaming.HiveEndPoint$
> ConnectionImpl.<init>(HiveEndPoint.java:278)
>
> at org.apache.hive.hcatalog.streaming.HiveEndPoint.
> newConnectionImpl(HiveEndPoint.java:215)
>
> at org.apache.hive.hcatalog.streaming.HiveEndPoint.
> newConnection(HiveEndPoint.java:192)
>
> at org.apache.hive.hcatalog.streaming.HiveEndPoint.
> newConnection(HiveEndPoint.java:122)
>
> at org.apache.nifi.util.hive.HiveWriter.lambda$
> newConnection$6(HiveWriter.java:237)
>
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1149)
>
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:624)
>
> at java.lang.Thread.run(Thread.java:748)
>
> Caused by: java.lang.RuntimeException: The root scratch dir: /tmp/hive on
> HDFS should be writable. Current permissions are: rwx------
>
> at org.apache.hadoop.hive.ql.session.SessionState.
> createRootHDFSDir(SessionState.java:613)
>
> at org.apache.hadoop.hive.ql.session.SessionState.
> createSessionDirs(SessionState.java:555)
>
> at org.apache.hadoop.hive.ql.session.SessionState.start(
> SessionState.java:509)
>
> ... 11 common frames omitted
>
>
> I’m sharing the detail error in the screenshot.
>
> Please let me know if anyone is aware about this issue.
>
>
>
> Regards,
>
> Mohit Jain
>