You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by hd...@uni.de on 2016/02/03 15:09:39 UTC
Re: StringIndexOutOfBoundsException: String index out of range: -1
Hi,
we found the reason for the empty output files: the Hive table are
bucketed. It looks like Kylin does not support bucketed tables and is
looking in the wrong folder for the necessary files.
Can anyone confirm this?
2016-01-29 7:34 GMT+01:00 <hd...@uni.de>:
> Hi,
>
> the output file is actually empty (that's probably the cause for "out
> of range -1" -> length (0)-1 = -1). There is no output logging which
> could be used to investigate why the file is actually empty. Any hints
> on how we can debug why it is empty?
>
>
> 2016-01-29 2:52 GMT+01:00 hongbin ma <ma...@apache.org>:
>> HiveColumnCardinalityUpdateJob
>> desc in source code:
>>
>> /**
>> * This job will update save the cardinality result into Kylin table
>> metadata store.
>> * @author shaoshi
>> */
>>
>>
>>
>> it does not belong to a cubing job, it's a separate task to help modeling.
>> can you checkout the output in /tmp/kylin/cardinality/KYLIN_DK.DIM_DTM, it
>> seems the content format is not as expected:
>> https://github.com/apache/kylin/blob/kylin-1.2/job/src/main/java/org/apache/kylin/job/hadoop/cardinality/HiveColumnCardinalityUpdateJob.java#L113
>>
>>
>>
>> --
>> Regards,
>>
>> *Bin Mahone | 马洪宾*
>> Apache Kylin: http://kylin.io
>> Github: https://github.com/binmahone
Re: StringIndexOutOfBoundsException: String index out of range: -1
Posted by ShaoFeng Shi <sh...@gmail.com>.
I see, thanks for reporting!
发送自 Outlook Mobile
On Thu, Feb 4, 2016 at 1:36 AM -0800, <hd...@uni.de> wrote:
Done ;-)
2016-02-04 9:15 GMT+01:00 ShaoFeng Shi :
> Hi, two would be better.
>
> 2016-02-04 16:10 GMT+08:00 :
>
>> Hi,
>>
>> should I file one or two different Jira's (IndexOutOfBounds + Bucket
>> tables)?
>>
>>
>> 2016-02-03 15:34 GMT+01:00 ShaoFeng Shi :
>> > Kylin uses HCatalog to read the hive table, ideally HCatalog will
>> > understand the different formats and partitions; I tried to search
>> whether
>> > HCatalog supports bucket tables, but there is no related discussion.
>> Could
>> > you please report a JIRA with your findings? Firstly we can fix the
>> string
>> > index out of bounds error, and then look into the hive source issue.
>> >
>> > 2016-02-03 22:09 GMT+08:00 :
>> >
>> >> Hi,
>> >>
>> >> we found the reason for the empty output files: the Hive table are
>> >> bucketed. It looks like Kylin does not support bucketed tables and is
>> >> looking in the wrong folder for the necessary files.
>> >>
>> >> Can anyone confirm this?
>> >>
>> >>
>> >> 2016-01-29 7:34 GMT+01:00 :
>> >> > Hi,
>> >> >
>> >> > the output file is actually empty (that's probably the cause for "out
>> >> > of range -1" -> length (0)-1 = -1). There is no output logging which
>> >> > could be used to investigate why the file is actually empty. Any hints
>> >> > on how we can debug why it is empty?
>> >> >
>> >> >
>> >> > 2016-01-29 2:52 GMT+01:00 hongbin ma :
>> >> >> HiveColumnCardinalityUpdateJob
>> >> >> desc in source code:
>> >> >>
>> >> >> /**
>> >> >> * This job will update save the cardinality result into Kylin table
>> >> >> metadata store.
>> >> >> * @author shaoshi
>> >> >> */
>> >> >>
>> >> >>
>> >> >>
>> >> >> it does not belong to a cubing job, it's a separate task to help
>> >> modeling.
>> >> >> can you checkout the output in
>> /tmp/kylin/cardinality/KYLIN_DK.DIM_DTM,
>> >> it
>> >> >> seems the content format is not as expected:
>> >> >>
>> >>
>> https://github.com/apache/kylin/blob/kylin-1.2/job/src/main/java/org/apache/kylin/job/hadoop/cardinality/HiveColumnCardinalityUpdateJob.java#L113
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Regards,
>> >> >>
>> >> >> *Bin Mahone | 马洪宾*
>> >> >> Apache Kylin: http://kylin.io
>> >> >> Github: https://github.com/binmahone
>> >>
>> >
>> >
>> >
>> > --
>> > Best regards,
>> >
>> > Shaofeng Shi
>>
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
Re: StringIndexOutOfBoundsException: String index out of range: -1
Posted by hd...@uni.de.
Done ;-)
2016-02-04 9:15 GMT+01:00 ShaoFeng Shi <sh...@apache.org>:
> Hi, two would be better.
>
> 2016-02-04 16:10 GMT+08:00 <hd...@uni.de>:
>
>> Hi,
>>
>> should I file one or two different Jira's (IndexOutOfBounds + Bucket
>> tables)?
>>
>>
>> 2016-02-03 15:34 GMT+01:00 ShaoFeng Shi <sh...@apache.org>:
>> > Kylin uses HCatalog to read the hive table, ideally HCatalog will
>> > understand the different formats and partitions; I tried to search
>> whether
>> > HCatalog supports bucket tables, but there is no related discussion.
>> Could
>> > you please report a JIRA with your findings? Firstly we can fix the
>> string
>> > index out of bounds error, and then look into the hive source issue.
>> >
>> > 2016-02-03 22:09 GMT+08:00 <hd...@uni.de>:
>> >
>> >> Hi,
>> >>
>> >> we found the reason for the empty output files: the Hive table are
>> >> bucketed. It looks like Kylin does not support bucketed tables and is
>> >> looking in the wrong folder for the necessary files.
>> >>
>> >> Can anyone confirm this?
>> >>
>> >>
>> >> 2016-01-29 7:34 GMT+01:00 <hd...@uni.de>:
>> >> > Hi,
>> >> >
>> >> > the output file is actually empty (that's probably the cause for "out
>> >> > of range -1" -> length (0)-1 = -1). There is no output logging which
>> >> > could be used to investigate why the file is actually empty. Any hints
>> >> > on how we can debug why it is empty?
>> >> >
>> >> >
>> >> > 2016-01-29 2:52 GMT+01:00 hongbin ma <ma...@apache.org>:
>> >> >> HiveColumnCardinalityUpdateJob
>> >> >> desc in source code:
>> >> >>
>> >> >> /**
>> >> >> * This job will update save the cardinality result into Kylin table
>> >> >> metadata store.
>> >> >> * @author shaoshi
>> >> >> */
>> >> >>
>> >> >>
>> >> >>
>> >> >> it does not belong to a cubing job, it's a separate task to help
>> >> modeling.
>> >> >> can you checkout the output in
>> /tmp/kylin/cardinality/KYLIN_DK.DIM_DTM,
>> >> it
>> >> >> seems the content format is not as expected:
>> >> >>
>> >>
>> https://github.com/apache/kylin/blob/kylin-1.2/job/src/main/java/org/apache/kylin/job/hadoop/cardinality/HiveColumnCardinalityUpdateJob.java#L113
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Regards,
>> >> >>
>> >> >> *Bin Mahone | 马洪宾*
>> >> >> Apache Kylin: http://kylin.io
>> >> >> Github: https://github.com/binmahone
>> >>
>> >
>> >
>> >
>> > --
>> > Best regards,
>> >
>> > Shaofeng Shi
>>
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
Re: StringIndexOutOfBoundsException: String index out of range: -1
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi, two would be better.
2016-02-04 16:10 GMT+08:00 <hd...@uni.de>:
> Hi,
>
> should I file one or two different Jira's (IndexOutOfBounds + Bucket
> tables)?
>
>
> 2016-02-03 15:34 GMT+01:00 ShaoFeng Shi <sh...@apache.org>:
> > Kylin uses HCatalog to read the hive table, ideally HCatalog will
> > understand the different formats and partitions; I tried to search
> whether
> > HCatalog supports bucket tables, but there is no related discussion.
> Could
> > you please report a JIRA with your findings? Firstly we can fix the
> string
> > index out of bounds error, and then look into the hive source issue.
> >
> > 2016-02-03 22:09 GMT+08:00 <hd...@uni.de>:
> >
> >> Hi,
> >>
> >> we found the reason for the empty output files: the Hive table are
> >> bucketed. It looks like Kylin does not support bucketed tables and is
> >> looking in the wrong folder for the necessary files.
> >>
> >> Can anyone confirm this?
> >>
> >>
> >> 2016-01-29 7:34 GMT+01:00 <hd...@uni.de>:
> >> > Hi,
> >> >
> >> > the output file is actually empty (that's probably the cause for "out
> >> > of range -1" -> length (0)-1 = -1). There is no output logging which
> >> > could be used to investigate why the file is actually empty. Any hints
> >> > on how we can debug why it is empty?
> >> >
> >> >
> >> > 2016-01-29 2:52 GMT+01:00 hongbin ma <ma...@apache.org>:
> >> >> HiveColumnCardinalityUpdateJob
> >> >> desc in source code:
> >> >>
> >> >> /**
> >> >> * This job will update save the cardinality result into Kylin table
> >> >> metadata store.
> >> >> * @author shaoshi
> >> >> */
> >> >>
> >> >>
> >> >>
> >> >> it does not belong to a cubing job, it's a separate task to help
> >> modeling.
> >> >> can you checkout the output in
> /tmp/kylin/cardinality/KYLIN_DK.DIM_DTM,
> >> it
> >> >> seems the content format is not as expected:
> >> >>
> >>
> https://github.com/apache/kylin/blob/kylin-1.2/job/src/main/java/org/apache/kylin/job/hadoop/cardinality/HiveColumnCardinalityUpdateJob.java#L113
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Regards,
> >> >>
> >> >> *Bin Mahone | 马洪宾*
> >> >> Apache Kylin: http://kylin.io
> >> >> Github: https://github.com/binmahone
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> > Shaofeng Shi
>
--
Best regards,
Shaofeng Shi
Re: StringIndexOutOfBoundsException: String index out of range: -1
Posted by hd...@uni.de.
Hi,
should I file one or two different Jira's (IndexOutOfBounds + Bucket tables)?
2016-02-03 15:34 GMT+01:00 ShaoFeng Shi <sh...@apache.org>:
> Kylin uses HCatalog to read the hive table, ideally HCatalog will
> understand the different formats and partitions; I tried to search whether
> HCatalog supports bucket tables, but there is no related discussion. Could
> you please report a JIRA with your findings? Firstly we can fix the string
> index out of bounds error, and then look into the hive source issue.
>
> 2016-02-03 22:09 GMT+08:00 <hd...@uni.de>:
>
>> Hi,
>>
>> we found the reason for the empty output files: the Hive table are
>> bucketed. It looks like Kylin does not support bucketed tables and is
>> looking in the wrong folder for the necessary files.
>>
>> Can anyone confirm this?
>>
>>
>> 2016-01-29 7:34 GMT+01:00 <hd...@uni.de>:
>> > Hi,
>> >
>> > the output file is actually empty (that's probably the cause for "out
>> > of range -1" -> length (0)-1 = -1). There is no output logging which
>> > could be used to investigate why the file is actually empty. Any hints
>> > on how we can debug why it is empty?
>> >
>> >
>> > 2016-01-29 2:52 GMT+01:00 hongbin ma <ma...@apache.org>:
>> >> HiveColumnCardinalityUpdateJob
>> >> desc in source code:
>> >>
>> >> /**
>> >> * This job will update save the cardinality result into Kylin table
>> >> metadata store.
>> >> * @author shaoshi
>> >> */
>> >>
>> >>
>> >>
>> >> it does not belong to a cubing job, it's a separate task to help
>> modeling.
>> >> can you checkout the output in /tmp/kylin/cardinality/KYLIN_DK.DIM_DTM,
>> it
>> >> seems the content format is not as expected:
>> >>
>> https://github.com/apache/kylin/blob/kylin-1.2/job/src/main/java/org/apache/kylin/job/hadoop/cardinality/HiveColumnCardinalityUpdateJob.java#L113
>> >>
>> >>
>> >>
>> >> --
>> >> Regards,
>> >>
>> >> *Bin Mahone | 马洪宾*
>> >> Apache Kylin: http://kylin.io
>> >> Github: https://github.com/binmahone
>>
>
>
>
> --
> Best regards,
>
> Shaofeng Shi
Re: StringIndexOutOfBoundsException: String index out of range: -1
Posted by ShaoFeng Shi <sh...@apache.org>.
Kylin uses HCatalog to read the hive table, ideally HCatalog will
understand the different formats and partitions; I tried to search whether
HCatalog supports bucket tables, but there is no related discussion. Could
you please report a JIRA with your findings? Firstly we can fix the string
index out of bounds error, and then look into the hive source issue.
2016-02-03 22:09 GMT+08:00 <hd...@uni.de>:
> Hi,
>
> we found the reason for the empty output files: the Hive table are
> bucketed. It looks like Kylin does not support bucketed tables and is
> looking in the wrong folder for the necessary files.
>
> Can anyone confirm this?
>
>
> 2016-01-29 7:34 GMT+01:00 <hd...@uni.de>:
> > Hi,
> >
> > the output file is actually empty (that's probably the cause for "out
> > of range -1" -> length (0)-1 = -1). There is no output logging which
> > could be used to investigate why the file is actually empty. Any hints
> > on how we can debug why it is empty?
> >
> >
> > 2016-01-29 2:52 GMT+01:00 hongbin ma <ma...@apache.org>:
> >> HiveColumnCardinalityUpdateJob
> >> desc in source code:
> >>
> >> /**
> >> * This job will update save the cardinality result into Kylin table
> >> metadata store.
> >> * @author shaoshi
> >> */
> >>
> >>
> >>
> >> it does not belong to a cubing job, it's a separate task to help
> modeling.
> >> can you checkout the output in /tmp/kylin/cardinality/KYLIN_DK.DIM_DTM,
> it
> >> seems the content format is not as expected:
> >>
> https://github.com/apache/kylin/blob/kylin-1.2/job/src/main/java/org/apache/kylin/job/hadoop/cardinality/HiveColumnCardinalityUpdateJob.java#L113
> >>
> >>
> >>
> >> --
> >> Regards,
> >>
> >> *Bin Mahone | 马洪宾*
> >> Apache Kylin: http://kylin.io
> >> Github: https://github.com/binmahone
>
--
Best regards,
Shaofeng Shi