You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Kang Minwoo <mi...@outlook.com> on 2019/05/14 02:36:09 UTC
How to reload dynamic Coprocessors
Hello Users,
When I load a dynamic coprocessor, If the table already has the same class coprocessor, coprocessor fails to load.
Because the same class coprocessor cannot load.
So I should unload old version coprocessor before load new version coprocessor.
But coprocessor has a mission-critical task. So the table always loads the coprocessor.
Here is a problem.
If I should update coprocessor jar file, I think I cannot update coprocessor jar file before coprocessor unload.
If I can use a reload command, It is great for me.
Does there have a reload command?
(HBase version: 1.2.9)
Best regards,
Minwoo Kang
Re: How to reload dynamic Coprocessors
Posted by Kang Minwoo <mi...@outlook.com>.
Thank you!
And I have a question it is not related to this issue.
Do you have an experience about dependency version conflicts when using static loading coprocessor?
Best regards,
Minwoo Kang
________________________________________
보낸 사람: 张铎(Duo Zhang) <pa...@gmail.com>
보낸 날짜: 2019년 5월 15일 수요일 14:43
받는 사람: hbase-user
제목: Re: How to reload dynamic Coprocessors
Based on your usage, that only system admin can access HBase directly then
I think it is fine to use table level coprocessor. Usually, just do not let
end user make use of coprocessor directly.
And HBaseAdmin.modifyTable will override the old configs, so usually the
code will be
HTableDescriptor htd = admin.getTableDescriptor(tableName);
htd.setCoprocessor or htd.setValue
admin.modifyTable(htd);
Kang Minwoo <mi...@outlook.com> 于2019年5月15日周三 上午10:51写道：
> Thanks! I don't know that.
> HBaseAdmin.modifyTable method looks like overwrite the previous
> configuration. Is it correct?
>
> In my case, I provide a service using HBase and only admin access HBase
> directly.
> The reasons why I choose dynamic loading coprocessor.
> 1) I don't want to meet the dependency version conflicts.
> static loading coprocessor's classloader looks like region server's
> classloader.
> But dynamic loading coprocessor's classloader and RegionServer's
> classloader is different.
> I think I can avoid dependency version conflicts using dynamic
> loading coprocessor.
> Because there are using different classloader.
>
> 2) It is a hard task that region server rolling restart.
> Our team wants to avoid region server rolling restart.
>
> Doesn't table coprocessor recommend in real production?
> If it was not recommended, I will consider using system coprocessor.
> I don't have experience using coprocessor before.
> I would appreciate your help.
>
> Best regards,
> Minwoo Kang
>
> ________________________________________
> 보낸 사람: 张铎(Duo Zhang) <pa...@gmail.com>
> 보낸 날짜: 2019년 5월 14일 화요일 17:35
> 받는 사람: hbase-user
> 제목: Re: How to reload dynamic Coprocessors
>
> You have to call HBaseAdmin.modifyTable to trigger a region reopen.
>
> And for me, I haven't made use of table level coprocessor in real
> production, as it is a bit dangerous in a multi-tenant environment. Usually
> we will add coprocessor at cluster level, through config file. So I'm not
> sure why we do not provide such method... Maybe it is a bit difficult to
> control the uploading part?
>
> Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 下午3:41写道：
>
> > Thank you for your reply.
> >
> > I tried to update the table descriptor using set
> > HTableDescriptor#setValue(byte[], byte[]).
> > the table descriptor changed sucessfully.
> > But the region doesn't reopen. so new jar didn't apply.
> >
> > Why don't we provide a coprocessor jar file update method for users?
> > Is it not a good idea?
> >
> > Best regards,
> > Minwoo Kang
> >
> > ________________________________________
> > 보낸 사람: 张铎(Duo Zhang) <pa...@gmail.com>
> > 보낸 날짜: 2019년 5월 14일 화요일 11:44
> > 받는 사람: hbase-user
> > 제목: Re: How to reload dynamic Coprocessors
> >
> > I think the safest way is to disable the table first, then update the
> > coprocessor jar in place, and then enable the table.
> >
> > Or another way is to upload the coprocessor jar to another place, and
> > update the table descriptor to point to the new place. I think this could
> > be done by code, as you can completely replace the old coprocessor
> config.
> > Not sure if this is easy to do through shell.
> >
> > Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 上午10:36写道：
> >
> > > Hello Users,
> > >
> > > When I load a dynamic coprocessor, If the table already has the same
> > class
> > > coprocessor, coprocessor fails to load.
> > > Because the same class coprocessor cannot load.
> > >
> > > So I should unload old version coprocessor before load new version
> > > coprocessor.
> > > But coprocessor has a mission-critical task. So the table always loads
> > the
> > > coprocessor.
> > >
> > > Here is a problem.
> > > If I should update coprocessor jar file, I think I cannot update
> > > coprocessor jar file before coprocessor unload.
> > >
> > > If I can use a reload command, It is great for me.
> > > Does there have a reload command?
> > >
> > > (HBase version: 1.2.9)
> > >
> > > Best regards,
> > > Minwoo Kang
> >
>
Re: How to reload dynamic Coprocessors
Posted by "张铎 (Duo Zhang)" <pa...@gmail.com>.
Based on your usage, that only system admin can access HBase directly then
I think it is fine to use table level coprocessor. Usually, just do not let
end user make use of coprocessor directly.
And HBaseAdmin.modifyTable will override the old configs, so usually the
code will be
HTableDescriptor htd = admin.getTableDescriptor(tableName);
htd.setCoprocessor or htd.setValue
admin.modifyTable(htd);
Kang Minwoo <mi...@outlook.com> 于2019年5月15日周三 上午10:51写道：
> Thanks! I don't know that.
> HBaseAdmin.modifyTable method looks like overwrite the previous
> configuration. Is it correct?
>
> In my case, I provide a service using HBase and only admin access HBase
> directly.
> The reasons why I choose dynamic loading coprocessor.
> 1) I don't want to meet the dependency version conflicts.
> static loading coprocessor's classloader looks like region server's
> classloader.
> But dynamic loading coprocessor's classloader and RegionServer's
> classloader is different.
> I think I can avoid dependency version conflicts using dynamic
> loading coprocessor.
> Because there are using different classloader.
>
> 2) It is a hard task that region server rolling restart.
> Our team wants to avoid region server rolling restart.
>
> Doesn't table coprocessor recommend in real production?
> If it was not recommended, I will consider using system coprocessor.
> I don't have experience using coprocessor before.
> I would appreciate your help.
>
> Best regards,
> Minwoo Kang
>
> ________________________________________
> 보낸 사람: 张铎(Duo Zhang) <pa...@gmail.com>
> 보낸 날짜: 2019년 5월 14일 화요일 17:35
> 받는 사람: hbase-user
> 제목: Re: How to reload dynamic Coprocessors
>
> You have to call HBaseAdmin.modifyTable to trigger a region reopen.
>
> And for me, I haven't made use of table level coprocessor in real
> production, as it is a bit dangerous in a multi-tenant environment. Usually
> we will add coprocessor at cluster level, through config file. So I'm not
> sure why we do not provide such method... Maybe it is a bit difficult to
> control the uploading part?
>
> Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 下午3:41写道：
>
> > Thank you for your reply.
> >
> > I tried to update the table descriptor using set
> > HTableDescriptor#setValue(byte[], byte[]).
> > the table descriptor changed sucessfully.
> > But the region doesn't reopen. so new jar didn't apply.
> >
> > Why don't we provide a coprocessor jar file update method for users?
> > Is it not a good idea?
> >
> > Best regards,
> > Minwoo Kang
> >
> > ________________________________________
> > 보낸 사람: 张铎(Duo Zhang) <pa...@gmail.com>
> > 보낸 날짜: 2019년 5월 14일 화요일 11:44
> > 받는 사람: hbase-user
> > 제목: Re: How to reload dynamic Coprocessors
> >
> > I think the safest way is to disable the table first, then update the
> > coprocessor jar in place, and then enable the table.
> >
> > Or another way is to upload the coprocessor jar to another place, and
> > update the table descriptor to point to the new place. I think this could
> > be done by code, as you can completely replace the old coprocessor
> config.
> > Not sure if this is easy to do through shell.
> >
> > Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 上午10:36写道：
> >
> > > Hello Users,
> > >
> > > When I load a dynamic coprocessor, If the table already has the same
> > class
> > > coprocessor, coprocessor fails to load.
> > > Because the same class coprocessor cannot load.
> > >
> > > So I should unload old version coprocessor before load new version
> > > coprocessor.
> > > But coprocessor has a mission-critical task. So the table always loads
> > the
> > > coprocessor.
> > >
> > > Here is a problem.
> > > If I should update coprocessor jar file, I think I cannot update
> > > coprocessor jar file before coprocessor unload.
> > >
> > > If I can use a reload command, It is great for me.
> > > Does there have a reload command?
> > >
> > > (HBase version: 1.2.9)
> > >
> > > Best regards,
> > > Minwoo Kang
> >
>
Re: How to reload dynamic Coprocessors
Posted by Kang Minwoo <mi...@outlook.com>.
Thanks! I don't know that.
HBaseAdmin.modifyTable method looks like overwrite the previous configuration. Is it correct?
In my case, I provide a service using HBase and only admin access HBase directly.
The reasons why I choose dynamic loading coprocessor.
1) I don't want to meet the dependency version conflicts.
static loading coprocessor's classloader looks like region server's classloader.
But dynamic loading coprocessor's classloader and RegionServer's classloader is different.
I think I can avoid dependency version conflicts using dynamic loading coprocessor.
Because there are using different classloader.
2) It is a hard task that region server rolling restart.
Our team wants to avoid region server rolling restart.
Doesn't table coprocessor recommend in real production?
If it was not recommended, I will consider using system coprocessor.
I don't have experience using coprocessor before.
I would appreciate your help.
Best regards,
Minwoo Kang
________________________________________
보낸 사람: 张铎(Duo Zhang) <pa...@gmail.com>
보낸 날짜: 2019년 5월 14일 화요일 17:35
받는 사람: hbase-user
제목: Re: How to reload dynamic Coprocessors
You have to call HBaseAdmin.modifyTable to trigger a region reopen.
And for me, I haven't made use of table level coprocessor in real
production, as it is a bit dangerous in a multi-tenant environment. Usually
we will add coprocessor at cluster level, through config file. So I'm not
sure why we do not provide such method... Maybe it is a bit difficult to
control the uploading part?
Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 下午3:41写道：
> Thank you for your reply.
>
> I tried to update the table descriptor using set
> HTableDescriptor#setValue(byte[], byte[]).
> the table descriptor changed sucessfully.
> But the region doesn't reopen. so new jar didn't apply.
>
> Why don't we provide a coprocessor jar file update method for users?
> Is it not a good idea?
>
> Best regards,
> Minwoo Kang
>
> ________________________________________
> 보낸 사람: 张铎(Duo Zhang) <pa...@gmail.com>
> 보낸 날짜: 2019년 5월 14일 화요일 11:44
> 받는 사람: hbase-user
> 제목: Re: How to reload dynamic Coprocessors
>
> I think the safest way is to disable the table first, then update the
> coprocessor jar in place, and then enable the table.
>
> Or another way is to upload the coprocessor jar to another place, and
> update the table descriptor to point to the new place. I think this could
> be done by code, as you can completely replace the old coprocessor config.
> Not sure if this is easy to do through shell.
>
> Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 上午10:36写道：
>
> > Hello Users,
> >
> > When I load a dynamic coprocessor, If the table already has the same
> class
> > coprocessor, coprocessor fails to load.
> > Because the same class coprocessor cannot load.
> >
> > So I should unload old version coprocessor before load new version
> > coprocessor.
> > But coprocessor has a mission-critical task. So the table always loads
> the
> > coprocessor.
> >
> > Here is a problem.
> > If I should update coprocessor jar file, I think I cannot update
> > coprocessor jar file before coprocessor unload.
> >
> > If I can use a reload command, It is great for me.
> > Does there have a reload command?
> >
> > (HBase version: 1.2.9)
> >
> > Best regards,
> > Minwoo Kang
>
Re: How to reload dynamic Coprocessors
Posted by "张铎 (Duo Zhang)" <pa...@gmail.com>.
You have to call HBaseAdmin.modifyTable to trigger a region reopen.
And for me, I haven't made use of table level coprocessor in real
production, as it is a bit dangerous in a multi-tenant environment. Usually
we will add coprocessor at cluster level, through config file. So I'm not
sure why we do not provide such method... Maybe it is a bit difficult to
control the uploading part?
Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 下午3:41写道：
> Thank you for your reply.
>
> I tried to update the table descriptor using set
> HTableDescriptor#setValue(byte[], byte[]).
> the table descriptor changed sucessfully.
> But the region doesn't reopen. so new jar didn't apply.
>
> Why don't we provide a coprocessor jar file update method for users?
> Is it not a good idea?
>
> Best regards,
> Minwoo Kang
>
> ________________________________________
> 보낸 사람: 张铎(Duo Zhang) <pa...@gmail.com>
> 보낸 날짜: 2019년 5월 14일 화요일 11:44
> 받는 사람: hbase-user
> 제목: Re: How to reload dynamic Coprocessors
>
> I think the safest way is to disable the table first, then update the
> coprocessor jar in place, and then enable the table.
>
> Or another way is to upload the coprocessor jar to another place, and
> update the table descriptor to point to the new place. I think this could
> be done by code, as you can completely replace the old coprocessor config.
> Not sure if this is easy to do through shell.
>
> Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 上午10:36写道：
>
> > Hello Users,
> >
> > When I load a dynamic coprocessor, If the table already has the same
> class
> > coprocessor, coprocessor fails to load.
> > Because the same class coprocessor cannot load.
> >
> > So I should unload old version coprocessor before load new version
> > coprocessor.
> > But coprocessor has a mission-critical task. So the table always loads
> the
> > coprocessor.
> >
> > Here is a problem.
> > If I should update coprocessor jar file, I think I cannot update
> > coprocessor jar file before coprocessor unload.
> >
> > If I can use a reload command, It is great for me.
> > Does there have a reload command?
> >
> > (HBase version: 1.2.9)
> >
> > Best regards,
> > Minwoo Kang
>
Re: How to reload dynamic Coprocessors
Posted by Kang Minwoo <mi...@outlook.com>.
Thank you for your reply.
I tried to update the table descriptor using set HTableDescriptor#setValue(byte[], byte[]).
the table descriptor changed sucessfully.
But the region doesn't reopen. so new jar didn't apply.
Why don't we provide a coprocessor jar file update method for users?
Is it not a good idea?
Best regards,
Minwoo Kang
________________________________________
보낸 사람: 张铎(Duo Zhang) <pa...@gmail.com>
보낸 날짜: 2019년 5월 14일 화요일 11:44
받는 사람: hbase-user
제목: Re: How to reload dynamic Coprocessors
I think the safest way is to disable the table first, then update the
coprocessor jar in place, and then enable the table.
Or another way is to upload the coprocessor jar to another place, and
update the table descriptor to point to the new place. I think this could
be done by code, as you can completely replace the old coprocessor config.
Not sure if this is easy to do through shell.
Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 上午10:36写道：
> Hello Users,
>
> When I load a dynamic coprocessor, If the table already has the same class
> coprocessor, coprocessor fails to load.
> Because the same class coprocessor cannot load.
>
> So I should unload old version coprocessor before load new version
> coprocessor.
> But coprocessor has a mission-critical task. So the table always loads the
> coprocessor.
>
> Here is a problem.
> If I should update coprocessor jar file, I think I cannot update
> coprocessor jar file before coprocessor unload.
>
> If I can use a reload command, It is great for me.
> Does there have a reload command?
>
> (HBase version: 1.2.9)
>
> Best regards,
> Minwoo Kang
Re: How to reload dynamic Coprocessors
Posted by "张铎 (Duo Zhang)" <pa...@gmail.com>.
I think the safest way is to disable the table first, then update the
coprocessor jar in place, and then enable the table.
Or another way is to upload the coprocessor jar to another place, and
update the table descriptor to point to the new place. I think this could
be done by code, as you can completely replace the old coprocessor config.
Not sure if this is easy to do through shell.
Kang Minwoo <mi...@outlook.com> 于2019年5月14日周二 上午10:36写道：
> Hello Users,
>
> When I load a dynamic coprocessor, If the table already has the same class
> coprocessor, coprocessor fails to load.
> Because the same class coprocessor cannot load.
>
> So I should unload old version coprocessor before load new version
> coprocessor.
> But coprocessor has a mission-critical task. So the table always loads the
> coprocessor.
>
> Here is a problem.
> If I should update coprocessor jar file, I think I cannot update
> coprocessor jar file before coprocessor unload.
>
> If I can use a reload command, It is great for me.
> Does there have a reload command?
>
> (HBase version: 1.2.9)
>
> Best regards,
> Minwoo Kang