You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by Smruti Ranjan <sm...@tcs.com> on 2018/03/28 09:19:30 UTC
Issue in accessing ORC transactional table through apache drill
Dear Team,
Issue while accessing ORC transactional hive table through apache drill
and below are the stack versions.
Apache drill 1.10.0
Hive 1.2.1 [HDP 2.6.0 versions]
Below is the error coming while accessing data from the mentioned table
through apache drill.
Query Failed: An Error Occurred
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
NumberFormatException: For input string: "0000112_0000" [Error Id:
ad9b4243-d48d-43c7-9755-388202d7c54d on inbbrdssvm16.india.tcs.com:31010]
Please help me in resolving the issue.
Thanks & Regards
Smruti Ranjan Jena
Tata Consultancy Services
Mailto: smruti.ranjan@tcs.com
Website: http://www.tcs.com
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you
Re: Issue in accessing ORC transactional table through apache drill
Posted by Vitalii Diravka <vi...@gmail.com>.
Hi Smruti,
I suggest you to move onto latest Drill and Hive versions, it is resolved
there.
Regarding your issue, Drill 1.10 uses Hive 1.2.1 client version,
so it could be compatible with Hive server/metastore 1.2.1 version, but it
wasn't verified.
0000112_0000 is a delta_name for the bucket. Your issue happened in
OrcInputFormat#generateSplitsInfo()
method. Looks like a wrong splitStrategy is selected.
It can be related to enabled custom properties before creating Hive ORC
bucketed transactional table.
So double check all table properties, the properties in your hive-site.xml
and properties, which were set in hive shell.
If these properties are changes, this is a root cause:
https://github.com/apache/hive/blob/branch-1.2/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java#L1023
https://github.com/apache/hive/blob/branch-1.2/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java#L1023
If you want to set some Hive specific option in Drill, add it in Drill Hive
plugin and restart drillbit.
Please ask such kind of questions in user mailing list. Devs are also
there,
but other users can read and also suggest you some decisions and
workarounds.
Kind regards
Vitalii
On Wed, Mar 28, 2018 at 9:05 PM, Prasad Nagaraj Subramanya <
prasadns14@gmail.com> wrote:
> Hi Smruti,
>
> Hive orc transactional support is available from Drill 1.13.0 onwards.
> Please update to the latest release version.
>
> Thanks,
> Prasad
>
> On Wed, Mar 28, 2018 at 10:22 AM, Gautam Parai <gp...@mapr.com> wrote:
>
> > From the error it seems like Drill barfs when trying to convert
> > "0000112_0000" to a number which would be expected? I am not familiar
> with
> > Hive ORC but maybe you could (temporarily) remove the offending row and
> try
> > it?
> >
> >
> > Gautam
> >
> > ________________________________
> > From: Smruti Ranjan <sm...@tcs.com>
> > Sent: Wednesday, March 28, 2018 2:19:30 AM
> > To: dev@drill.apache.org
> > Subject: Issue in accessing ORC transactional table through apache drill
> >
> > Dear Team,
> >
> > Issue while accessing ORC transactional hive table through apache drill
> > and below are the stack versions.
> > Apache drill 1.10.0
> > Hive 1.2.1 [HDP 2.6.0 versions]
> >
> > Below is the error coming while accessing data from the mentioned table
> > through apache drill.
> >
> > Query Failed: An Error Occurred
> > org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> > NumberFormatException: For input string: "0000112_0000" [Error Id:
> > ad9b4243-d48d-43c7-9755-388202d7c54d on inbbrdssvm16.india.tcs.com:31010
> ]
> >
> > Please help me in resolving the issue.
> >
> >
> > Thanks & Regards
> > Smruti Ranjan Jena
> > Tata Consultancy Services
> > Mailto: smruti.ranjan@tcs.com
> > Website: https://urldefense.proofpoint.com/v2/url?u=http-3A__www.tcs.
> > com&d=DwIBAg&c=cskdkSMqhcnjZxdQVpwTXg&r=jGaWXfAULy7L7yLSDM6rFQ&m=
> > kVHrzlhCd8duKi3rDtQYq29gjO6EiXKxCg8iaz-x2h8&s=
> > iOWB2L2JkjCw5RXZ9bkXhYhDDMFA3oOd6vPfvE-Jqnk&e=
> > ____________________________________________
> > Experience certainty. IT Services
> > Business Solutions
> > Consulting
> > ____________________________________________
> > =====-----=====-----=====
> > Notice: The information contained in this e-mail
> > message and/or attachments to it may contain
> > confidential or privileged information. If you are
> > not the intended recipient, any dissemination, use,
> > review, distribution, printing or copying of the
> > information contained in this e-mail message
> > and/or attachments to it are strictly prohibited. If
> > you have received this communication in error,
> > please notify us by reply e-mail or telephone and
> > immediately and permanently delete the message
> > and any attachments. Thank you
> >
> >
> >
>
Re: Issue in accessing ORC transactional table through apache drill
Posted by Prasad Nagaraj Subramanya <pr...@gmail.com>.
Hi Smruti,
Hive orc transactional support is available from Drill 1.13.0 onwards.
Please update to the latest release version.
Thanks,
Prasad
On Wed, Mar 28, 2018 at 10:22 AM, Gautam Parai <gp...@mapr.com> wrote:
> From the error it seems like Drill barfs when trying to convert
> "0000112_0000" to a number which would be expected? I am not familiar with
> Hive ORC but maybe you could (temporarily) remove the offending row and try
> it?
>
>
> Gautam
>
> ________________________________
> From: Smruti Ranjan <sm...@tcs.com>
> Sent: Wednesday, March 28, 2018 2:19:30 AM
> To: dev@drill.apache.org
> Subject: Issue in accessing ORC transactional table through apache drill
>
> Dear Team,
>
> Issue while accessing ORC transactional hive table through apache drill
> and below are the stack versions.
> Apache drill 1.10.0
> Hive 1.2.1 [HDP 2.6.0 versions]
>
> Below is the error coming while accessing data from the mentioned table
> through apache drill.
>
> Query Failed: An Error Occurred
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> NumberFormatException: For input string: "0000112_0000" [Error Id:
> ad9b4243-d48d-43c7-9755-388202d7c54d on inbbrdssvm16.india.tcs.com:31010]
>
> Please help me in resolving the issue.
>
>
> Thanks & Regards
> Smruti Ranjan Jena
> Tata Consultancy Services
> Mailto: smruti.ranjan@tcs.com
> Website: https://urldefense.proofpoint.com/v2/url?u=http-3A__www.tcs.
> com&d=DwIBAg&c=cskdkSMqhcnjZxdQVpwTXg&r=jGaWXfAULy7L7yLSDM6rFQ&m=
> kVHrzlhCd8duKi3rDtQYq29gjO6EiXKxCg8iaz-x2h8&s=
> iOWB2L2JkjCw5RXZ9bkXhYhDDMFA3oOd6vPfvE-Jqnk&e=
> ____________________________________________
> Experience certainty. IT Services
> Business Solutions
> Consulting
> ____________________________________________
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
>
Re: Issue in accessing ORC transactional table through apache drill
Posted by Gautam Parai <gp...@mapr.com>.
From the error it seems like Drill barfs when trying to convert "0000112_0000" to a number which would be expected? I am not familiar with Hive ORC but maybe you could (temporarily) remove the offending row and try it?
Gautam
________________________________
From: Smruti Ranjan <sm...@tcs.com>
Sent: Wednesday, March 28, 2018 2:19:30 AM
To: dev@drill.apache.org
Subject: Issue in accessing ORC transactional table through apache drill
Dear Team,
Issue while accessing ORC transactional hive table through apache drill
and below are the stack versions.
Apache drill 1.10.0
Hive 1.2.1 [HDP 2.6.0 versions]
Below is the error coming while accessing data from the mentioned table
through apache drill.
Query Failed: An Error Occurred
org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
NumberFormatException: For input string: "0000112_0000" [Error Id:
ad9b4243-d48d-43c7-9755-388202d7c54d on inbbrdssvm16.india.tcs.com:31010]
Please help me in resolving the issue.
Thanks & Regards
Smruti Ranjan Jena
Tata Consultancy Services
Mailto: smruti.ranjan@tcs.com
Website: https://urldefense.proofpoint.com/v2/url?u=http-3A__www.tcs.com&d=DwIBAg&c=cskdkSMqhcnjZxdQVpwTXg&r=jGaWXfAULy7L7yLSDM6rFQ&m=kVHrzlhCd8duKi3rDtQYq29gjO6EiXKxCg8iaz-x2h8&s=iOWB2L2JkjCw5RXZ9bkXhYhDDMFA3oOd6vPfvE-Jqnk&e=
____________________________________________
Experience certainty. IT Services
Business Solutions
Consulting
____________________________________________
=====-----=====-----=====
Notice: The information contained in this e-mail
message and/or attachments to it may contain
confidential or privileged information. If you are
not the intended recipient, any dissemination, use,
review, distribution, printing or copying of the
information contained in this e-mail message
and/or attachments to it are strictly prohibited. If
you have received this communication in error,
please notify us by reply e-mail or telephone and
immediately and permanently delete the message
and any attachments. Thank you