You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by Kiriti Sai <ki...@gmail.com> on 2016/01/05 11:27:03 UTC
Java.lang.NoSuchMethodError: org.apache.hadoop.yarn.conf.YarnConfiguration.getServiceAddressConfKeys
(Lorg/apache/hadoop/conf/Configuration; ) Ljava/util/List;
Hi,
I've recently update the binaries in my Kylin setup from v1.1 incubating to
v1.2. The cubes which were building fine till now are throwing the above
error.
This error is occuring in the extract fact table distinct columns step.
(Step 2).
Can you please point out any mistakes with the upgrading procedure or
anything else.
Thank you,
Sai Kiriti B.
Re: Java.lang.NoSuchMethodError:
org.apache.hadoop.yarn.conf.YarnConfiguration.getServiceAddressConfKeys
(Lorg/apache/hadoop/conf/Configuration; ) Ljava/util/List;
Posted by 王晓雨 <wa...@jd.com>.
Hi，
When kylin build cube，it will execute mapreduce job，kylin use ResourceManager address to get the mapreduce job status。kylin read the kylin.properties file to get the RM url first。if not config，kylin will read hadoop config to get the RM address by HAUtils api in hadoop 2.4+。because hadoop support RM HA after hadoop2.4。
but hbase0.98x complie with hadoop 2.2 by default。does not support RM HA。so you got the exception！
the config property is correct，kylin will replace the $｛job_id｝to real jobid，you can only replace the YOUR_RM_AND_PORT to your RM address and port like：192.168.2.2:8088
thanks！
> 在 2016年1月5日，20:29，Kiriti Sai <ki...@gmail.com> 写道：
>
> Hi,
> Can you please explain it in a slightly detailed manner. I understand that
> the url you are referring to is the resource manager url, but it's
> particular to a job right? How can something particular to a job be set as
> a property for Kylin. I'm sorry if I'm mistaken.
> Or are you intending that {job_id} will actually get the id of the MR job
> running? Sorry for these naive questions.
>
> Thank you.
>> On Jan 5, 2016 8:42 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
>>
>> Hi,
>> You can set the property in kylin.properties file
>> kylin.job.yarn.app.rest.check.status.url=
>> https://YOUR_RM_AND_PORT/ws/v1/cluster/apps/${job_id}?anonymous=true
>>
>>> 在 2016年01月05日 19:38, Kiriti Sai 写道:
>>>
>>> Hi Wang,
>>> The version of Hadoop in the cluster is 2.6. I've setup Hbase 0.98.15
>>> using
>>> binaries, in which there are some jar files of the Hadoop version 2.2,
>>> like
>>> hadoop-yarn-client-2.2.jar.
>>> As I've mentioned already, this setup has worked with the previous version
>>> of Kylin 1.1-incubating, but has been throwing this error after updating
>>> to
>>> v1.2. (Dont know if there is anything due to this, but just mentioning
>>> it).
>>> So, is there any other to solve this other than building HBase from source
>>> using the latest Hadoop libraries.
>>>
>>> Thank You.
>>> On Jan 5, 2016 8:26 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
>>>
>>> Hi,
>>>> The api YarnConfiguration.getServiceAddressConfKeys required Hadoop2.4+
>>>> Which version Hadoop do you use ?
>>>> You can recompile the hbase with hadoop 2.4+ version or your hadoop
>>>> cluster version.
>>>>
>>>>
>>>>
>>>>
>>>> 在 2016年01月05日 18:52, Kiriti Sai 写道:
>>>>
>>>> Hi,
>>>>> I have looked at the suggested link before posting the question here. I
>>>>> didn't understand how to resolve this issue.
>>>>> I've tried replacing the 2.2 hadoop yarn libs present in the HBase lib
>>>>> directory but then it throws FileNotFoundException.
>>>>> Can you please explain in a detailed way how to resolve this issue.
>>>>> I'm using Hbase 0.98.15-hadoop2 version.
>>>>>
>>>>> Thank you,
>>>>> Sai Kiriti B
>>>>> On Jan 5, 2016 7:38 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
>>>>>
>>>>> Hi Sai!
>>>>>
>>>>>> You can see the same topic :
>>>>>>
>>>>>>
>>>>>>
>>>>>> http://apache-kylin.74782.x6.nabble.com/NoSuchMethodError-org-apache-hadoop-yarn-conf-YarnConfiguration-getServiceAddressConfKeys-td2937.html#a2943
>>>>>>
>>>>>> 在 2016年01月05日 18:27, Kiriti Sai 写道:
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>> I've recently update the binaries in my Kylin setup from v1.1
>>>>>>> incubating
>>>>>>> to
>>>>>>> v1.2. The cubes which were building fine till now are throwing the
>>>>>>> above
>>>>>>> error.
>>>>>>> This error is occuring in the extract fact table distinct columns
>>>>>>> step.
>>>>>>> (Step 2).
>>>>>>> Can you please point out any mistakes with the upgrading procedure or
>>>>>>> anything else.
>>>>>>>
>>>>>>> Thank you,
>>>>>>> Sai Kiriti B.
>>
Re: Java.lang.NoSuchMethodError: org.apache.hadoop.yarn.conf.YarnConfiguration.getServiceAddressConfKeys
(Lorg/apache/hadoop/conf/Configuration; ) Ljava/util/List;
Posted by Kiriti Sai <ki...@gmail.com>.
Hi,
Can you please explain it in a slightly detailed manner. I understand that
the url you are referring to is the resource manager url, but it's
particular to a job right? How can something particular to a job be set as
a property for Kylin. I'm sorry if I'm mistaken.
Or are you intending that {job_id} will actually get the id of the MR job
running? Sorry for these naive questions.
Thank you.
On Jan 5, 2016 8:42 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
> Hi,
> You can set the property in kylin.properties file
> kylin.job.yarn.app.rest.check.status.url=
> https://YOUR_RM_AND_PORT/ws/v1/cluster/apps/${job_id}?anonymous=true
>
> 在 2016年01月05日 19:38, Kiriti Sai 写道:
>
>> Hi Wang,
>> The version of Hadoop in the cluster is 2.6. I've setup Hbase 0.98.15
>> using
>> binaries, in which there are some jar files of the Hadoop version 2.2,
>> like
>> hadoop-yarn-client-2.2.jar.
>> As I've mentioned already, this setup has worked with the previous version
>> of Kylin 1.1-incubating, but has been throwing this error after updating
>> to
>> v1.2. (Dont know if there is anything due to this, but just mentioning
>> it).
>> So, is there any other to solve this other than building HBase from source
>> using the latest Hadoop libraries.
>>
>> Thank You.
>> On Jan 5, 2016 8:26 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
>>
>> Hi,
>>> The api YarnConfiguration.getServiceAddressConfKeys required Hadoop2.4+
>>> Which version Hadoop do you use ?
>>> You can recompile the hbase with hadoop 2.4+ version or your hadoop
>>> cluster version.
>>>
>>>
>>>
>>>
>>> 在 2016年01月05日 18:52, Kiriti Sai 写道:
>>>
>>> Hi,
>>>> I have looked at the suggested link before posting the question here. I
>>>> didn't understand how to resolve this issue.
>>>> I've tried replacing the 2.2 hadoop yarn libs present in the HBase lib
>>>> directory but then it throws FileNotFoundException.
>>>> Can you please explain in a detailed way how to resolve this issue.
>>>> I'm using Hbase 0.98.15-hadoop2 version.
>>>>
>>>> Thank you,
>>>> Sai Kiriti B
>>>> On Jan 5, 2016 7:38 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
>>>>
>>>> Hi Sai!
>>>>
>>>>> You can see the same topic :
>>>>>
>>>>>
>>>>>
>>>>> http://apache-kylin.74782.x6.nabble.com/NoSuchMethodError-org-apache-hadoop-yarn-conf-YarnConfiguration-getServiceAddressConfKeys-td2937.html#a2943
>>>>>
>>>>> 在 2016年01月05日 18:27, Kiriti Sai 写道:
>>>>>
>>>>> Hi,
>>>>>
>>>>>> I've recently update the binaries in my Kylin setup from v1.1
>>>>>> incubating
>>>>>> to
>>>>>> v1.2. The cubes which were building fine till now are throwing the
>>>>>> above
>>>>>> error.
>>>>>> This error is occuring in the extract fact table distinct columns
>>>>>> step.
>>>>>> (Step 2).
>>>>>> Can you please point out any mistakes with the upgrading procedure or
>>>>>> anything else.
>>>>>>
>>>>>> Thank you,
>>>>>> Sai Kiriti B.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>
Re: Java.lang.NoSuchMethodError:
org.apache.hadoop.yarn.conf.YarnConfiguration.getServiceAddressConfKeys
(Lorg/apache/hadoop/conf/Configuration; ) Ljava/util/List;
Posted by Xiaoyu Wang <wa...@jd.com>.
Hi,
You can set the property in kylin.properties file
kylin.job.yarn.app.rest.check.status.url=https://YOUR_RM_AND_PORT/ws/v1/cluster/apps/${job_id}?anonymous=true
在 2016年01月05日 19:38, Kiriti Sai 写道:
> Hi Wang,
> The version of Hadoop in the cluster is 2.6. I've setup Hbase 0.98.15 using
> binaries, in which there are some jar files of the Hadoop version 2.2, like
> hadoop-yarn-client-2.2.jar.
> As I've mentioned already, this setup has worked with the previous version
> of Kylin 1.1-incubating, but has been throwing this error after updating to
> v1.2. (Dont know if there is anything due to this, but just mentioning it).
> So, is there any other to solve this other than building HBase from source
> using the latest Hadoop libraries.
>
> Thank You.
> On Jan 5, 2016 8:26 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
>
>> Hi,
>> The api YarnConfiguration.getServiceAddressConfKeys required Hadoop2.4+
>> Which version Hadoop do you use ?
>> You can recompile the hbase with hadoop 2.4+ version or your hadoop
>> cluster version.
>>
>>
>>
>>
>> 在 2016年01月05日 18:52, Kiriti Sai 写道:
>>
>>> Hi,
>>> I have looked at the suggested link before posting the question here. I
>>> didn't understand how to resolve this issue.
>>> I've tried replacing the 2.2 hadoop yarn libs present in the HBase lib
>>> directory but then it throws FileNotFoundException.
>>> Can you please explain in a detailed way how to resolve this issue.
>>> I'm using Hbase 0.98.15-hadoop2 version.
>>>
>>> Thank you,
>>> Sai Kiriti B
>>> On Jan 5, 2016 7:38 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
>>>
>>> Hi Sai!
>>>> You can see the same topic :
>>>>
>>>>
>>>> http://apache-kylin.74782.x6.nabble.com/NoSuchMethodError-org-apache-hadoop-yarn-conf-YarnConfiguration-getServiceAddressConfKeys-td2937.html#a2943
>>>>
>>>> 在 2016年01月05日 18:27, Kiriti Sai 写道:
>>>>
>>>> Hi,
>>>>> I've recently update the binaries in my Kylin setup from v1.1 incubating
>>>>> to
>>>>> v1.2. The cubes which were building fine till now are throwing the above
>>>>> error.
>>>>> This error is occuring in the extract fact table distinct columns step.
>>>>> (Step 2).
>>>>> Can you please point out any mistakes with the upgrading procedure or
>>>>> anything else.
>>>>>
>>>>> Thank you,
>>>>> Sai Kiriti B.
>>>>>
>>>>>
>>>>>
Re: Java.lang.NoSuchMethodError: org.apache.hadoop.yarn.conf.YarnConfiguration.getServiceAddressConfKeys
(Lorg/apache/hadoop/conf/Configuration; ) Ljava/util/List;
Posted by Kiriti Sai <ki...@gmail.com>.
Hi Wang,
The version of Hadoop in the cluster is 2.6. I've setup Hbase 0.98.15 using
binaries, in which there are some jar files of the Hadoop version 2.2, like
hadoop-yarn-client-2.2.jar.
As I've mentioned already, this setup has worked with the previous version
of Kylin 1.1-incubating, but has been throwing this error after updating to
v1.2. (Dont know if there is anything due to this, but just mentioning it).
So, is there any other to solve this other than building HBase from source
using the latest Hadoop libraries.
Thank You.
On Jan 5, 2016 8:26 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
> Hi,
> The api YarnConfiguration.getServiceAddressConfKeys required Hadoop2.4+
> Which version Hadoop do you use ?
> You can recompile the hbase with hadoop 2.4+ version or your hadoop
> cluster version.
>
>
>
>
> 在 2016年01月05日 18:52, Kiriti Sai 写道:
>
>> Hi,
>> I have looked at the suggested link before posting the question here. I
>> didn't understand how to resolve this issue.
>> I've tried replacing the 2.2 hadoop yarn libs present in the HBase lib
>> directory but then it throws FileNotFoundException.
>> Can you please explain in a detailed way how to resolve this issue.
>> I'm using Hbase 0.98.15-hadoop2 version.
>>
>> Thank you,
>> Sai Kiriti B
>> On Jan 5, 2016 7:38 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
>>
>> Hi Sai!
>>> You can see the same topic :
>>>
>>>
>>> http://apache-kylin.74782.x6.nabble.com/NoSuchMethodError-org-apache-hadoop-yarn-conf-YarnConfiguration-getServiceAddressConfKeys-td2937.html#a2943
>>>
>>> 在 2016年01月05日 18:27, Kiriti Sai 写道:
>>>
>>> Hi,
>>>> I've recently update the binaries in my Kylin setup from v1.1 incubating
>>>> to
>>>> v1.2. The cubes which were building fine till now are throwing the above
>>>> error.
>>>> This error is occuring in the extract fact table distinct columns step.
>>>> (Step 2).
>>>> Can you please point out any mistakes with the upgrading procedure or
>>>> anything else.
>>>>
>>>> Thank you,
>>>> Sai Kiriti B.
>>>>
>>>>
>>>>
>
Re: Java.lang.NoSuchMethodError:
org.apache.hadoop.yarn.conf.YarnConfiguration.getServiceAddressConfKeys
(Lorg/apache/hadoop/conf/Configuration; ) Ljava/util/List;
Posted by Xiaoyu Wang <wa...@jd.com>.
Hi,
The api YarnConfiguration.getServiceAddressConfKeys required Hadoop2.4+
Which version Hadoop do you use ?
You can recompile the hbase with hadoop 2.4+ version or your hadoop
cluster version.
在 2016年01月05日 18:52, Kiriti Sai 写道:
> Hi,
> I have looked at the suggested link before posting the question here. I
> didn't understand how to resolve this issue.
> I've tried replacing the 2.2 hadoop yarn libs present in the HBase lib
> directory but then it throws FileNotFoundException.
> Can you please explain in a detailed way how to resolve this issue.
> I'm using Hbase 0.98.15-hadoop2 version.
>
> Thank you,
> Sai Kiriti B
> On Jan 5, 2016 7:38 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
>
>> Hi Sai!
>> You can see the same topic :
>>
>> http://apache-kylin.74782.x6.nabble.com/NoSuchMethodError-org-apache-hadoop-yarn-conf-YarnConfiguration-getServiceAddressConfKeys-td2937.html#a2943
>>
>> 在 2016年01月05日 18:27, Kiriti Sai 写道:
>>
>>> Hi,
>>> I've recently update the binaries in my Kylin setup from v1.1 incubating
>>> to
>>> v1.2. The cubes which were building fine till now are throwing the above
>>> error.
>>> This error is occuring in the extract fact table distinct columns step.
>>> (Step 2).
>>> Can you please point out any mistakes with the upgrading procedure or
>>> anything else.
>>>
>>> Thank you,
>>> Sai Kiriti B.
>>>
>>>
Re: Java.lang.NoSuchMethodError: org.apache.hadoop.yarn.conf.YarnConfiguration.getServiceAddressConfKeys
(Lorg/apache/hadoop/conf/Configuration; ) Ljava/util/List;
Posted by Kiriti Sai <ki...@gmail.com>.
Hi,
I have looked at the suggested link before posting the question here. I
didn't understand how to resolve this issue.
I've tried replacing the 2.2 hadoop yarn libs present in the HBase lib
directory but then it throws FileNotFoundException.
Can you please explain in a detailed way how to resolve this issue.
I'm using Hbase 0.98.15-hadoop2 version.
Thank you,
Sai Kiriti B
On Jan 5, 2016 7:38 PM, "Xiaoyu Wang" <wa...@jd.com> wrote:
> Hi Sai!
> You can see the same topic :
>
> http://apache-kylin.74782.x6.nabble.com/NoSuchMethodError-org-apache-hadoop-yarn-conf-YarnConfiguration-getServiceAddressConfKeys-td2937.html#a2943
>
> 在 2016年01月05日 18:27, Kiriti Sai 写道:
>
>> Hi,
>> I've recently update the binaries in my Kylin setup from v1.1 incubating
>> to
>> v1.2. The cubes which were building fine till now are throwing the above
>> error.
>> This error is occuring in the extract fact table distinct columns step.
>> (Step 2).
>> Can you please point out any mistakes with the upgrading procedure or
>> anything else.
>>
>> Thank you,
>> Sai Kiriti B.
>>
>>
Re: Java.lang.NoSuchMethodError:
org.apache.hadoop.yarn.conf.YarnConfiguration.getServiceAddressConfKeys
(Lorg/apache/hadoop/conf/Configuration; ) Ljava/util/List;
Posted by Xiaoyu Wang <wa...@jd.com>.
Hi Sai!
You can see the same topic :
http://apache-kylin.74782.x6.nabble.com/NoSuchMethodError-org-apache-hadoop-yarn-conf-YarnConfiguration-getServiceAddressConfKeys-td2937.html#a2943
在 2016年01月05日 18:27, Kiriti Sai 写道:
> Hi,
> I've recently update the binaries in my Kylin setup from v1.1 incubating to
> v1.2. The cubes which were building fine till now are throwing the above
> error.
> This error is occuring in the extract fact table distinct columns step.
> (Step 2).
> Can you please point out any mistakes with the upgrading procedure or
> anything else.
>
> Thank you,
> Sai Kiriti B.
>