You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by "Venkat, Ankam" <An...@centurylink.com> on 2014/11/19 22:15:08 UTC

Encrypting data in Hive

I need to encrypt few columns (like SSN, Credit Card Number etc) in Hive Table.

Is there a built-in feature available in Hive?   I see a Jira at https://issues.apache.org/jira/browse/HIVE-6329 which is still unresolved.

If there is no built-in feature,  are there any open source UDFs available for this?

Thanks in advance.

Regards,
Venkat

This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.

Hive on Tez Error

Posted by peterm_second <re...@gmail.com>.
Hi Guys,
I am trying to enable tez on my hiveserver2 but everytime I get this error

Exception in thread "main" java.lang.RuntimeException: 
java.io.IOException: Previous writer likely failed to write 
hdfs://hadoop-nn.mo-data.com:9000/tmp/hive/root/_tez_session_dir/a0087fb2-1430-43fa-b3e1-06644ab4961d/*. 
Failing because I am unlikely to write too.
     at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:457)
     at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
     at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.io.IOException: Previous writer likely failed to write 
hdfs://hadoop-nn.mo-data.com:9000/tmp/hive/root/_tez_session_dir/a0087fb2-1430-43fa-b3e1-06644ab4961d/*. 
Failing because I am unlikely to write too.
     at 
org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeResource(DagUtils.java:974)
     at 
org.apache.hadoop.hive.ql.exec.tez.DagUtils.addTempResources(DagUtils.java:855)
     at 
org.apache.hadoop.hive.ql.exec.tez.DagUtils.localizeTempFilesFromConf(DagUtils.java:798)
     at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.refreshLocalResourcesFromConf(TezSessionState.java:228)
     at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:154)
     at 
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:122)
     at 
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:454)
     ... 7 more

I am a noob when it comes to tez and hive, has anyone seen this problem 
? Any ideas what might be causing it.
As far as I understand tez doesn't use the distributed cashe so it tries 
to localize all the resources for each session. I have no idea what 
process is holding those files and causing it to fail with this error.
I am using Hive 0.14.0 and tez 0.6

Regards,
Peter

RE: Encrypting data in Hive

Posted by "Xu, Cheng A" <ch...@intel.com>.
Actually Xiaomeng is working on this jira https://issues.apache.org/jira/browse/HIVE-7934
And we have a plan to move this feature forwards recently.

+ xiaomeng

From: Venkat, Ankam [mailto:Ankam.Venkat@centurylink.com]
Sent: Thursday, November 20, 2014 5:15 AM
To: 'user@hive.apache.org'
Subject: Encrypting data in Hive

I need to encrypt few columns (like SSN, Credit Card Number etc) in Hive Table.

Is there a built-in feature available in Hive?   I see a Jira at https://issues.apache.org/jira/browse/HIVE-6329 which is still unresolved.

If there is no built-in feature,  are there any open source UDFs available for this?

Thanks in advance.

Regards,
Venkat

This communication is the property of CenturyLink and may contain confidential or privileged information. Unauthorized use of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender by reply e-mail and destroy all copies of the communication and any attachments.