You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Mostafa Alaa Mohamed <mo...@etisalat.ae> on 2016/10/04 05:27:22 UTC
RE: Spark Hive Rejection
No, Sometime when you have table with column int and you insert in this column string job will be fail some times
Best Regards,
Mostafa Alaa Mohamed,
Technical Expert Big Data,
M: +971506450787
Email: mohamedamostafa@etisalat.ae<ma...@etisalat.ae>
From: Michael Segel [mailto:msegel_hadoop@hotmail.com]
Sent: Thursday, September 29, 2016 04:26 PM
To: Mostafa Alaa Mohamed
Cc: user@spark.apache.org
Subject: Re: Spark Hive Rejection
Correct me if I’m wrong but isn’t hive schema on read and not on write?
So you shouldn’t fail on write.
On Sep 29, 2016, at 1:25 AM, Mostafa Alaa Mohamed <mo...@etisalat.ae>> wrote:
Dears,
I want to ask
• What will happened if there are rejections rows when inserting dataframe into hive?
o Rejection will be for example table required integer into column and dataframe include string.
o Duplication rejection restriction from the table itself?
• How can we specify the rejection directory?
If not avaiable do you recommend to open Jira issue?
Best Regards,
Mostafa Alaa Mohamed,
Technical Expert Big Data,
M: +971506450787
Email: mohamedamostafa@etisalat.ae<ma...@etisalat.ae>
________________________________
The content of this email together with any attachments, statements and opinions expressed herein contains information that is private and confidential are intended for the named addressee(s) only. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this message in error please notify postmaster@etisalat.ae<ma...@etisalat.ae> by email immediately and delete the message without making any copies.
________________________________
The content of this email together with any attachments, statements and opinions expressed herein contains information that is private and confidential are intended for the named addressee(s) only. If you are not the addressee of this email you may not copy, forward, disclose or otherwise use it or any part of it in any form whatsoever. If you have received this message in error please notify postmaster@etisalat.ae by email immediately and delete the message without making any copies.