You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Jim Huang <ge...@gmail.com> on 2014/04/07 23:37:06 UTC
HBase 0.94 - TableMapReduceUtil class, overloaded method
"initTableMapperJob" behaves differently
I am looking at the 0.94 branch and I have noticed that the overloaded
methods for "initTableMapperJob" behaves differently in that one of them
calls the "initCredentials(job);" while the others don't. For people who
are using Kerberos security on their cluster, the initCredentials(job) sets
up the HBase security tokens for all the mapper tasks only for that
overloaded method. Is there a specific reason why this could be
intentional? Otherwise, I would like to create a new JIRA to see if I can
work on this as a newbie issue.
Thanks for any pointer.
Jim
Re: HBase 0.94 - TableMapReduceUtil class, overloaded method
"initTableMapperJob" behaves differently
Posted by Jim Huang <ge...@gmail.com>.
Yes, that's precisely the method that should get initCredentials(job) as
well.
All the other overloaded methods with table are covered.
On Mon, Apr 7, 2014 at 2:48 PM, Ted Yu <yu...@gmail.com> wrote:
> You are talking about this method, right ?
>
> public static void initTableMapperJob(List<Scan> scans,
> Class<? extends TableMapper> mapper,
> Class<? extends WritableComparable> outputKeyClass,
> Class<? extends Writable> outputValueClass, Job job,
> boolean addDependencyJars) throws IOException {
>
> Looks like it should call initCredentials(job) as well.
>
>
> On Mon, Apr 7, 2014 at 2:37 PM, Jim Huang <ge...@gmail.com> wrote:
>
> > I am looking at the 0.94 branch and I have noticed that the overloaded
> > methods for "initTableMapperJob" behaves differently in that one of them
> > calls the "initCredentials(job);" while the others don't. For people who
> > are using Kerberos security on their cluster, the initCredentials(job)
> sets
> > up the HBase security tokens for all the mapper tasks only for that
> > overloaded method. Is there a specific reason why this could be
> > intentional? Otherwise, I would like to create a new JIRA to see if I
> can
> > work on this as a newbie issue.
> >
> > Thanks for any pointer.
> > Jim
> >
>
Re: HBase 0.94 - TableMapReduceUtil class, overloaded method
"initTableMapperJob" behaves differently
Posted by Nick Dimiduk <nd...@gmail.com>.
I vaguely recall a ticket being raised and committed for this problem, but
the bug is clearly present and I'm not finding the issue. Maybe @larsh
recalls?
On Mon, Apr 7, 2014 at 2:48 PM, Ted Yu <yu...@gmail.com> wrote:
> You are talking about this method, right ?
>
> public static void initTableMapperJob(List<Scan> scans,
> Class<? extends TableMapper> mapper,
> Class<? extends WritableComparable> outputKeyClass,
> Class<? extends Writable> outputValueClass, Job job,
> boolean addDependencyJars) throws IOException {
>
> Looks like it should call initCredentials(job) as well.
>
>
> On Mon, Apr 7, 2014 at 2:37 PM, Jim Huang <ge...@gmail.com> wrote:
>
> > I am looking at the 0.94 branch and I have noticed that the overloaded
> > methods for "initTableMapperJob" behaves differently in that one of them
> > calls the "initCredentials(job);" while the others don't. For people who
> > are using Kerberos security on their cluster, the initCredentials(job)
> sets
> > up the HBase security tokens for all the mapper tasks only for that
> > overloaded method. Is there a specific reason why this could be
> > intentional? Otherwise, I would like to create a new JIRA to see if I
> can
> > work on this as a newbie issue.
> >
> > Thanks for any pointer.
> > Jim
> >
>
Re: HBase 0.94 - TableMapReduceUtil class, overloaded method
"initTableMapperJob" behaves differently
Posted by Ted Yu <yu...@gmail.com>.
Looking at references to this method, it is used by MultiTableInputFormat.
It seems the next step is to enable MultiTableInputFormat in secure cluster.
Cheers
On Mon, Apr 7, 2014 at 2:48 PM, Ted Yu <yu...@gmail.com> wrote:
> You are talking about this method, right ?
>
> public static void initTableMapperJob(List<Scan> scans,
> Class<? extends TableMapper> mapper,
> Class<? extends WritableComparable> outputKeyClass,
> Class<? extends Writable> outputValueClass, Job job,
> boolean addDependencyJars) throws IOException {
>
> Looks like it should call initCredentials(job) as well.
>
>
> On Mon, Apr 7, 2014 at 2:37 PM, Jim Huang <ge...@gmail.com> wrote:
>
>> I am looking at the 0.94 branch and I have noticed that the overloaded
>> methods for "initTableMapperJob" behaves differently in that one of them
>> calls the "initCredentials(job);" while the others don't. For people who
>> are using Kerberos security on their cluster, the initCredentials(job)
>> sets
>> up the HBase security tokens for all the mapper tasks only for that
>> overloaded method. Is there a specific reason why this could be
>> intentional? Otherwise, I would like to create a new JIRA to see if I can
>> work on this as a newbie issue.
>>
>> Thanks for any pointer.
>> Jim
>>
>
>
Re: HBase 0.94 - TableMapReduceUtil class, overloaded method
"initTableMapperJob" behaves differently
Posted by Jim Huang <ge...@gmail.com>.
Good to see this patch is already in action, so we can avoid others
spending time on this once this is committed.
On Mon, Apr 7, 2014 at 7:09 PM, liushaohui <li...@xiaomi.com> wrote:
> Please see https://issues.apache.org/jira/browse/HBASE-10692
>
> This issue is about this problem.
>
> May someone help to push it ? Thanks.
>
> - Shaohui Liu
>
>
>
> On 04/08/2014 05:48 AM, Ted Yu wrote:
>
>> You are talking about this method, right ?
>>
>> public static void initTableMapperJob(List<Scan> scans,
>> Class<? extends TableMapper> mapper,
>> Class<? extends WritableComparable> outputKeyClass,
>> Class<? extends Writable> outputValueClass, Job job,
>> boolean addDependencyJars) throws IOException {
>>
>> Looks like it should call initCredentials(job) as well.
>>
>>
>> On Mon, Apr 7, 2014 at 2:37 PM, Jim Huang <ge...@gmail.com> wrote:
>>
>> I am looking at the 0.94 branch and I have noticed that the overloaded
>>> methods for "initTableMapperJob" behaves differently in that one of them
>>> calls the "initCredentials(job);" while the others don't. For people who
>>> are using Kerberos security on their cluster, the initCredentials(job)
>>> sets
>>> up the HBase security tokens for all the mapper tasks only for that
>>> overloaded method. Is there a specific reason why this could be
>>> intentional? Otherwise, I would like to create a new JIRA to see if I
>>> can
>>> work on this as a newbie issue.
>>>
>>> Thanks for any pointer.
>>> Jim
>>>
>>>
>
Re: HBase 0.94 - TableMapReduceUtil class, overloaded method "initTableMapperJob"
behaves differently
Posted by liushaohui <li...@xiaomi.com>.
Please see https://issues.apache.org/jira/browse/HBASE-10692
This issue is about this problem.
May someone help to push it ? Thanks.
- Shaohui Liu
On 04/08/2014 05:48 AM, Ted Yu wrote:
> You are talking about this method, right ?
>
> public static void initTableMapperJob(List<Scan> scans,
> Class<? extends TableMapper> mapper,
> Class<? extends WritableComparable> outputKeyClass,
> Class<? extends Writable> outputValueClass, Job job,
> boolean addDependencyJars) throws IOException {
>
> Looks like it should call initCredentials(job) as well.
>
>
> On Mon, Apr 7, 2014 at 2:37 PM, Jim Huang <ge...@gmail.com> wrote:
>
>> I am looking at the 0.94 branch and I have noticed that the overloaded
>> methods for "initTableMapperJob" behaves differently in that one of them
>> calls the "initCredentials(job);" while the others don't. For people who
>> are using Kerberos security on their cluster, the initCredentials(job) sets
>> up the HBase security tokens for all the mapper tasks only for that
>> overloaded method. Is there a specific reason why this could be
>> intentional? Otherwise, I would like to create a new JIRA to see if I can
>> work on this as a newbie issue.
>>
>> Thanks for any pointer.
>> Jim
>>
Re: HBase 0.94 - TableMapReduceUtil class, overloaded method
"initTableMapperJob" behaves differently
Posted by Ted Yu <yu...@gmail.com>.
You are talking about this method, right ?
public static void initTableMapperJob(List<Scan> scans,
Class<? extends TableMapper> mapper,
Class<? extends WritableComparable> outputKeyClass,
Class<? extends Writable> outputValueClass, Job job,
boolean addDependencyJars) throws IOException {
Looks like it should call initCredentials(job) as well.
On Mon, Apr 7, 2014 at 2:37 PM, Jim Huang <ge...@gmail.com> wrote:
> I am looking at the 0.94 branch and I have noticed that the overloaded
> methods for "initTableMapperJob" behaves differently in that one of them
> calls the "initCredentials(job);" while the others don't. For people who
> are using Kerberos security on their cluster, the initCredentials(job) sets
> up the HBase security tokens for all the mapper tasks only for that
> overloaded method. Is there a specific reason why this could be
> intentional? Otherwise, I would like to create a new JIRA to see if I can
> work on this as a newbie issue.
>
> Thanks for any pointer.
> Jim
>