You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@skywalking.apache.org by Sheng Wu <wu...@apache.org> on 2020/03/31 03:29:53 UTC
[8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Hi Dev Team
After the experiences of removing endpoint_inventory, I found out this
strategy is successful.
Especially, we totally get rid of register, so I want to do more.
*SkyWalking 8.0.0*
First, it is already unexpected for me, we have to move to 8.0.0 so
quickly, but after the discussion with +高洪涛@skywalking
<ha...@apache.org>, and thinking about this for several days, I think
we have to.
The key chances are following
1. Remove service, service instance, and network address register. The old
register protocols are totally going to be removed.
2. The agent doesn't need to do register anymore. Service name and Service
Instance name are generated by the agent itself, but the extra information,
such as IP, hostname, language, should report to backend separately.
3. Service Traffic should be added just like the endpoint traffic but keep
the time bucket as we need accurate service name in the given duration
4. Service Instance Traffic should be added too, with external information,
such as language, hostname.
5. Trace context propagation context should be changed to accept string in
service instance name, endpoint name and network address. This could ease
the agent logic, but also, requires changes in all language agent and
plugin test tool,
6. Trace report protocol requires to change too, in order to adopt the
string.
7. e2e tests have to ignore PHP and LUA at first, and remove the 6.x
compatibility test(doesn't support anymore).
The benefits we will get are
1. Don't worry about the inventory(s) that has been deleted randomly by end
users. (We received a lot of issue reports about this)
2. The upgrade could be easier erasing the whole storage and reboot the new
one. (Users don't feel comfortable about upgrade)
3. No hot-reboot case in the agent side
4. No cache of network address register information in the agent.
5. No service and service instance cache in the OAP
6. No register lock in the OAP
7. No file buffer mechanism in the OAP too, same as no register happens.
In my mind, I think this totally break upgrade is super meaningful and will
be good change. Even we break many things, they are easy to follow.
+mrproliu@gmail.com <mr...@gmail.com> I think by following this, we need
to change the collaboration header to `sw8` :) As no 7.1.0 release will
happen.
Sheng Wu 吴晟
Twitter, wusheng1108
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory
permanently.
Posted by Yanlong He <ya...@gmail.com>.
A great idea. skywalking-php agent will be the fastest compatible with 8.0.0
> 2020年3月31日 上午11:29，Sheng Wu <wu...@apache.org> 写道：
>
> Hi Dev Team
>
> After the experiences of removing endpoint_inventory, I found out this
> strategy is successful.
> Especially, we totally get rid of register, so I want to do more.
>
> *SkyWalking 8.0.0*
> First, it is already unexpected for me, we have to move to 8.0.0 so
> quickly, but after the discussion with +高洪涛@skywalking
> <ha...@apache.org>, and thinking about this for several days, I think
> we have to.
>
> The key chances are following
> 1. Remove service, service instance, and network address register. The old
> register protocols are totally going to be removed.
> 2. The agent doesn't need to do register anymore. Service name and Service
> Instance name are generated by the agent itself, but the extra information,
> such as IP, hostname, language, should report to backend separately.
> 3. Service Traffic should be added just like the endpoint traffic but keep
> the time bucket as we need accurate service name in the given duration
> 4. Service Instance Traffic should be added too, with external information,
> such as language, hostname.
> 5. Trace context propagation context should be changed to accept string in
> service instance name, endpoint name and network address. This could ease
> the agent logic, but also, requires changes in all language agent and
> plugin test tool,
> 6. Trace report protocol requires to change too, in order to adopt the
> string.
> 7. e2e tests have to ignore PHP and LUA at first, and remove the 6.x
> compatibility test(doesn't support anymore).
>
> The benefits we will get are
> 1. Don't worry about the inventory(s) that has been deleted randomly by end
> users. (We received a lot of issue reports about this)
> 2. The upgrade could be easier erasing the whole storage and reboot the new
> one. (Users don't feel comfortable about upgrade)
> 3. No hot-reboot case in the agent side
> 4. No cache of network address register information in the agent.
> 5. No service and service instance cache in the OAP
> 6. No register lock in the OAP
> 7. No file buffer mechanism in the OAP too, same as no register happens.
>
> In my mind, I think this totally break upgrade is super meaningful and will
> be good change. Even we break many things, they are easy to follow.
> +mrproliu@gmail.com <mr...@gmail.com> I think by following this, we need
> to change the collaboration header to `sw8` :) As no 7.1.0 release will
> happen.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory
permanently.
Posted by Yanlong He <ya...@gmail.com>.
Hi Dev Team,
I upgraded SkyWalking-PHP to protocol v3[1] and it seems to be working well so far, then I will check the details.
[1]: https://github.com/SkyAPM/SkyAPM-php-sdk/pull/213 <
https://github.com/SkyAPM/SkyAPM-php-sdk/pull/213>
Yanlong 何延龙
> 2020年4月7日 下午9:26，Wei Zhang <zh...@apache.org> 写道：
>
> Hi Dev Team,
>
> I upgraded go2sky to protocol v3[1] and it seems to be working well so
> far[2], then I will check the details.
>
> [1]: https://github.com/SkyAPM/go2sky/pull/53 <
> https://github.com/SkyAPM/go2sky/pull/53>
> [2]: https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191 <
> https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191>
> --
>
> Wei Zhang 张伟
> Github, @arugal
>
>
> Daming <zt...@foxmail.com> 于2020年4月7日周二 下午8:23写道：
>
>>
>> Hi,
>> It is good to me to remove the InfluxDB storage E2E Test.
>> I will make it adapt to the new protocol and restore it. After the PR
>> merged.
>>
>>
>> —————
>> Daming(@dmsolr)
>> Apache SkyWalking
>>
>>
>>
>>> 在 2020年4月7日，下午2:56，Sheng Wu <wu...@gmail.com> 写道：
>>>
>>> Haochao Zhuang, Zhenxu Ke, Yanlong He and Han Liu
>>>
>>> I just adjusted the following things in PR tests
>>> 1. Remove influxdb storage, lua agent and PHP agent from e2e tests. Once
>>> you have done in your side, please restore these, we stil expect these in
>>> 8.0 release.
>>> 2. No need to do a reboot test. Remove it permanently.
>>> 3. SkyWalking 6 compatibility tests have been removed permanently. I
>> assume
>>> no one will release a new 6.9 for v3 protocol, right? If we have, please
>>> let me know.
>>>
>>>
>>> Sheng Wu 吴晟
>>> Twitter, wusheng1108
>>>
>>>
>>> Daming <zt...@foxmail.com> 于2020年4月7日周二 下午1:05写道：
>>>
>>>>
>>>> Hi Dev Team,
>>>>
>>>> Currently, I have upgraded the Agent-Test Tools to protocol v3. And it
>> is
>>>> in service on PluginTests of the main repository[1]. As I know,
>> @mrproliu <
>>>> https://github.com/mrproliu> have brought it to lua-nginx
>> repository[2].
>>>>
>>>> [1]:https://github.com/apache/skywalking/pull/4599 <
>>>> https://github.com/apache/skywalking/pull/4599>
>>>> [2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <
>>>> https://github.com/apache/skywalking-nginx-lua/pull/30>
>>>> —————
>>>> Daming(@dmsolr)
>>>> Apache SkyWalking
>>>>
>>>>
>>>>
>>>>> 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
>>>>>
>>>>> Hi Dev Team
>>>>
>>>>
>>
>>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by han liu <mr...@gmail.com>.
Hi Dev Team,
The skywalking-nginx-lua has finished adapting the new v3 protocol[1].
Thank Sheng Wu, Wen Ming, and YuanSheng Wang to review the code.
[1] https://github.com/apache/skywalking-nginx-lua/pull/30
Sheng Wu <wu...@gmail.com> 于2020年4月8日周三 下午8:49写道：
> han liu <mr...@gmail.com> 于2020年4月8日周三 下午8:44写道：
>
> > Hi Dev Team,
> >
> > I upgraded skywalking-nginx-lua to protocol v3[1] and it seems working
> > well. Also, I have finished the lua e2e test in the local environment.
> > I'm still waiting for the Nginx PR review finish, then I can commit the
> e2e
> >
>
> Wen Ming
>
> I will leave you to make the call about that PR.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> > files into the main repo.
> >
> > [1] https://github.com/apache/skywalking-nginx-lua/pull/30
> >
> > Sheng Wu <wu...@gmail.com> 于2020年4月8日周三 下午6:42写道：
> >
> > > Great. Very appreciated for the quick update.
> > >
> > > Wei Zhang <zh...@apache.org>于2020年4月7日 周二下午9:26写道：
> > >
> > > > Hi Dev Team,
> > > >
> > > > I upgraded go2sky to protocol v3[1] and it seems to be working well
> so
> > > > far[2], then I will check the details.
> > > >
> > > > [1]: https://github.com/SkyAPM/go2sky/pull/53 <
> > > > https://github.com/SkyAPM/go2sky/pull/53>
> > > > [2]: https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191
> <
> > > > https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191>
> > > > --
> > > >
> > > > Wei Zhang 张伟
> > > > Github, @arugal
> > > >
> > > >
> > > > Daming <zt...@foxmail.com> 于2020年4月7日周二 下午8:23写道：
> > > >
> > > > >
> > > > > Hi,
> > > > > It is good to me to remove the InfluxDB storage E2E Test.
> > > > > I will make it adapt to the new protocol and restore it. After the
> PR
> > > > > merged.
> > > > >
> > > > >
> > > > > —————
> > > > > Daming(@dmsolr)
> > > > > Apache SkyWalking
> > > > >
> > > > >
> > > > >
> > > > > > 在 2020年4月7日，下午2:56，Sheng Wu <wu...@gmail.com> 写道：
> > > > > >
> > > > > > Haochao Zhuang, Zhenxu Ke, Yanlong He and Han Liu
> > > > > >
> > > > > > I just adjusted the following things in PR tests
> > > > > > 1. Remove influxdb storage, lua agent and PHP agent from e2e
> tests.
> > > > Once
> > > > > > you have done in your side, please restore these, we stil expect
> > > these
> > > > in
> > > > > > 8.0 release.
> > > > > > 2. No need to do a reboot test. Remove it permanently.
> > > > > > 3. SkyWalking 6 compatibility tests have been removed
> permanently.
> > I
> > > > > assume
> > > > > > no one will release a new 6.9 for v3 protocol, right? If we have,
> > > > please
> > > > > > let me know.
> > > > > >
> > > > > >
> > > > > > Sheng Wu 吴晟
> > > > > > Twitter, wusheng1108
> > > > > >
> > > > > >
> > > > > > Daming <zt...@foxmail.com> 于2020年4月7日周二 下午1:05写道：
> > > > > >
> > > > > >>
> > > > > >> Hi Dev Team,
> > > > > >>
> > > > > >> Currently, I have upgraded the Agent-Test Tools to protocol v3.
> > And
> > > it
> > > > > is
> > > > > >> in service on PluginTests of the main repository[1]. As I know,
> > > > > @mrproliu <
> > > > > >> https://github.com/mrproliu> have brought it to lua-nginx
> > > > > repository[2].
> > > > > >>
> > > > > >> [1]:https://github.com/apache/skywalking/pull/4599 <
> > > > > >> https://github.com/apache/skywalking/pull/4599>
> > > > > >> [2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <
> > > > > >> https://github.com/apache/skywalking-nginx-lua/pull/30>
> > > > > >> —————
> > > > > >> Daming(@dmsolr)
> > > > > >> Apache SkyWalking
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >>> 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
> > > > > >>>
> > > > > >>> Hi Dev Team
> > > > > >>
> > > > > >>
> > > > >
> > > > >
> > > >
> > > --
> > > Sheng Wu 吴晟
> > >
> > > Apache SkyWalking
> > > Apache Incubator
> > > Apache ShardingSphere, ECharts, DolphinScheduler podlings
> > > Zipkin
> > > Twitter, wusheng1108
> > >
> >
>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by Sheng Wu <wu...@gmail.com>.
han liu <mr...@gmail.com> 于2020年4月8日周三 下午8:44写道：
> Hi Dev Team,
>
> I upgraded skywalking-nginx-lua to protocol v3[1] and it seems working
> well. Also, I have finished the lua e2e test in the local environment.
> I'm still waiting for the Nginx PR review finish, then I can commit the e2e
>
Wen Ming
I will leave you to make the call about that PR.
Sheng Wu 吴晟
Twitter, wusheng1108
> files into the main repo.
>
> [1] https://github.com/apache/skywalking-nginx-lua/pull/30
>
> Sheng Wu <wu...@gmail.com> 于2020年4月8日周三 下午6:42写道：
>
> > Great. Very appreciated for the quick update.
> >
> > Wei Zhang <zh...@apache.org>于2020年4月7日 周二下午9:26写道：
> >
> > > Hi Dev Team,
> > >
> > > I upgraded go2sky to protocol v3[1] and it seems to be working well so
> > > far[2], then I will check the details.
> > >
> > > [1]: https://github.com/SkyAPM/go2sky/pull/53 <
> > > https://github.com/SkyAPM/go2sky/pull/53>
> > > [2]: https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191 <
> > > https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191>
> > > --
> > >
> > > Wei Zhang 张伟
> > > Github, @arugal
> > >
> > >
> > > Daming <zt...@foxmail.com> 于2020年4月7日周二 下午8:23写道：
> > >
> > > >
> > > > Hi,
> > > > It is good to me to remove the InfluxDB storage E2E Test.
> > > > I will make it adapt to the new protocol and restore it. After the PR
> > > > merged.
> > > >
> > > >
> > > > —————
> > > > Daming(@dmsolr)
> > > > Apache SkyWalking
> > > >
> > > >
> > > >
> > > > > 在 2020年4月7日，下午2:56，Sheng Wu <wu...@gmail.com> 写道：
> > > > >
> > > > > Haochao Zhuang, Zhenxu Ke, Yanlong He and Han Liu
> > > > >
> > > > > I just adjusted the following things in PR tests
> > > > > 1. Remove influxdb storage, lua agent and PHP agent from e2e tests.
> > > Once
> > > > > you have done in your side, please restore these, we stil expect
> > these
> > > in
> > > > > 8.0 release.
> > > > > 2. No need to do a reboot test. Remove it permanently.
> > > > > 3. SkyWalking 6 compatibility tests have been removed permanently.
> I
> > > > assume
> > > > > no one will release a new 6.9 for v3 protocol, right? If we have,
> > > please
> > > > > let me know.
> > > > >
> > > > >
> > > > > Sheng Wu 吴晟
> > > > > Twitter, wusheng1108
> > > > >
> > > > >
> > > > > Daming <zt...@foxmail.com> 于2020年4月7日周二 下午1:05写道：
> > > > >
> > > > >>
> > > > >> Hi Dev Team,
> > > > >>
> > > > >> Currently, I have upgraded the Agent-Test Tools to protocol v3.
> And
> > it
> > > > is
> > > > >> in service on PluginTests of the main repository[1]. As I know,
> > > > @mrproliu <
> > > > >> https://github.com/mrproliu> have brought it to lua-nginx
> > > > repository[2].
> > > > >>
> > > > >> [1]:https://github.com/apache/skywalking/pull/4599 <
> > > > >> https://github.com/apache/skywalking/pull/4599>
> > > > >> [2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <
> > > > >> https://github.com/apache/skywalking-nginx-lua/pull/30>
> > > > >> —————
> > > > >> Daming(@dmsolr)
> > > > >> Apache SkyWalking
> > > > >>
> > > > >>
> > > > >>
> > > > >>> 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
> > > > >>>
> > > > >>> Hi Dev Team
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > --
> > Sheng Wu 吴晟
> >
> > Apache SkyWalking
> > Apache Incubator
> > Apache ShardingSphere, ECharts, DolphinScheduler podlings
> > Zipkin
> > Twitter, wusheng1108
> >
>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by han liu <mr...@gmail.com>.
Hi Dev Team,
I upgraded skywalking-nginx-lua to protocol v3[1] and it seems working
well. Also, I have finished the lua e2e test in the local environment.
I'm still waiting for the Nginx PR review finish, then I can commit the e2e
files into the main repo.
[1] https://github.com/apache/skywalking-nginx-lua/pull/30
Sheng Wu <wu...@gmail.com> 于2020年4月8日周三 下午6:42写道：
> Great. Very appreciated for the quick update.
>
> Wei Zhang <zh...@apache.org>于2020年4月7日 周二下午9:26写道：
>
> > Hi Dev Team,
> >
> > I upgraded go2sky to protocol v3[1] and it seems to be working well so
> > far[2], then I will check the details.
> >
> > [1]: https://github.com/SkyAPM/go2sky/pull/53 <
> > https://github.com/SkyAPM/go2sky/pull/53>
> > [2]: https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191 <
> > https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191>
> > --
> >
> > Wei Zhang 张伟
> > Github, @arugal
> >
> >
> > Daming <zt...@foxmail.com> 于2020年4月7日周二 下午8:23写道：
> >
> > >
> > > Hi,
> > > It is good to me to remove the InfluxDB storage E2E Test.
> > > I will make it adapt to the new protocol and restore it. After the PR
> > > merged.
> > >
> > >
> > > —————
> > > Daming(@dmsolr)
> > > Apache SkyWalking
> > >
> > >
> > >
> > > > 在 2020年4月7日，下午2:56，Sheng Wu <wu...@gmail.com> 写道：
> > > >
> > > > Haochao Zhuang, Zhenxu Ke, Yanlong He and Han Liu
> > > >
> > > > I just adjusted the following things in PR tests
> > > > 1. Remove influxdb storage, lua agent and PHP agent from e2e tests.
> > Once
> > > > you have done in your side, please restore these, we stil expect
> these
> > in
> > > > 8.0 release.
> > > > 2. No need to do a reboot test. Remove it permanently.
> > > > 3. SkyWalking 6 compatibility tests have been removed permanently. I
> > > assume
> > > > no one will release a new 6.9 for v3 protocol, right? If we have,
> > please
> > > > let me know.
> > > >
> > > >
> > > > Sheng Wu 吴晟
> > > > Twitter, wusheng1108
> > > >
> > > >
> > > > Daming <zt...@foxmail.com> 于2020年4月7日周二 下午1:05写道：
> > > >
> > > >>
> > > >> Hi Dev Team,
> > > >>
> > > >> Currently, I have upgraded the Agent-Test Tools to protocol v3. And
> it
> > > is
> > > >> in service on PluginTests of the main repository[1]. As I know,
> > > @mrproliu <
> > > >> https://github.com/mrproliu> have brought it to lua-nginx
> > > repository[2].
> > > >>
> > > >> [1]:https://github.com/apache/skywalking/pull/4599 <
> > > >> https://github.com/apache/skywalking/pull/4599>
> > > >> [2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <
> > > >> https://github.com/apache/skywalking-nginx-lua/pull/30>
> > > >> —————
> > > >> Daming(@dmsolr)
> > > >> Apache SkyWalking
> > > >>
> > > >>
> > > >>
> > > >>> 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
> > > >>>
> > > >>> Hi Dev Team
> > > >>
> > > >>
> > >
> > >
> >
> --
> Sheng Wu 吴晟
>
> Apache SkyWalking
> Apache Incubator
> Apache ShardingSphere, ECharts, DolphinScheduler podlings
> Zipkin
> Twitter, wusheng1108
>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by Sheng Wu <wu...@gmail.com>.
Great. Very appreciated for the quick update.
Wei Zhang <zh...@apache.org>于2020年4月7日 周二下午9:26写道：
> Hi Dev Team,
>
> I upgraded go2sky to protocol v3[1] and it seems to be working well so
> far[2], then I will check the details.
>
> [1]: https://github.com/SkyAPM/go2sky/pull/53 <
> https://github.com/SkyAPM/go2sky/pull/53>
> [2]: https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191 <
> https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191>
> --
>
> Wei Zhang 张伟
> Github, @arugal
>
>
> Daming <zt...@foxmail.com> 于2020年4月7日周二 下午8:23写道：
>
> >
> > Hi,
> > It is good to me to remove the InfluxDB storage E2E Test.
> > I will make it adapt to the new protocol and restore it. After the PR
> > merged.
> >
> >
> > —————
> > Daming(@dmsolr)
> > Apache SkyWalking
> >
> >
> >
> > > 在 2020年4月7日，下午2:56，Sheng Wu <wu...@gmail.com> 写道：
> > >
> > > Haochao Zhuang, Zhenxu Ke, Yanlong He and Han Liu
> > >
> > > I just adjusted the following things in PR tests
> > > 1. Remove influxdb storage, lua agent and PHP agent from e2e tests.
> Once
> > > you have done in your side, please restore these, we stil expect these
> in
> > > 8.0 release.
> > > 2. No need to do a reboot test. Remove it permanently.
> > > 3. SkyWalking 6 compatibility tests have been removed permanently. I
> > assume
> > > no one will release a new 6.9 for v3 protocol, right? If we have,
> please
> > > let me know.
> > >
> > >
> > > Sheng Wu 吴晟
> > > Twitter, wusheng1108
> > >
> > >
> > > Daming <zt...@foxmail.com> 于2020年4月7日周二 下午1:05写道：
> > >
> > >>
> > >> Hi Dev Team,
> > >>
> > >> Currently, I have upgraded the Agent-Test Tools to protocol v3. And it
> > is
> > >> in service on PluginTests of the main repository[1]. As I know,
> > @mrproliu <
> > >> https://github.com/mrproliu> have brought it to lua-nginx
> > repository[2].
> > >>
> > >> [1]:https://github.com/apache/skywalking/pull/4599 <
> > >> https://github.com/apache/skywalking/pull/4599>
> > >> [2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <
> > >> https://github.com/apache/skywalking-nginx-lua/pull/30>
> > >> —————
> > >> Daming(@dmsolr)
> > >> Apache SkyWalking
> > >>
> > >>
> > >>
> > >>> 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
> > >>>
> > >>> Hi Dev Team
> > >>
> > >>
> >
> >
>
--
Sheng Wu 吴晟
Apache SkyWalking
Apache Incubator
Apache ShardingSphere, ECharts, DolphinScheduler podlings
Zipkin
Twitter, wusheng1108
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by Wei Zhang <zh...@apache.org>.
Hi Dev Team,
I upgraded go2sky to protocol v3[1] and it seems to be working well so
far[2], then I will check the details.
[1]: https://github.com/SkyAPM/go2sky/pull/53 <
https://github.com/SkyAPM/go2sky/pull/53>
[2]: https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191 <
https://github.com/SkyAPM/go2sky/pull/53#issuecomment-610384191>
--
Wei Zhang 张伟
Github, @arugal
Daming <zt...@foxmail.com> 于2020年4月7日周二 下午8:23写道：
>
> Hi,
> It is good to me to remove the InfluxDB storage E2E Test.
> I will make it adapt to the new protocol and restore it. After the PR
> merged.
>
>
> —————
> Daming(@dmsolr)
> Apache SkyWalking
>
>
>
> > 在 2020年4月7日，下午2:56，Sheng Wu <wu...@gmail.com> 写道：
> >
> > Haochao Zhuang, Zhenxu Ke, Yanlong He and Han Liu
> >
> > I just adjusted the following things in PR tests
> > 1. Remove influxdb storage, lua agent and PHP agent from e2e tests. Once
> > you have done in your side, please restore these, we stil expect these in
> > 8.0 release.
> > 2. No need to do a reboot test. Remove it permanently.
> > 3. SkyWalking 6 compatibility tests have been removed permanently. I
> assume
> > no one will release a new 6.9 for v3 protocol, right? If we have, please
> > let me know.
> >
> >
> > Sheng Wu 吴晟
> > Twitter, wusheng1108
> >
> >
> > Daming <zt...@foxmail.com> 于2020年4月7日周二 下午1:05写道：
> >
> >>
> >> Hi Dev Team,
> >>
> >> Currently, I have upgraded the Agent-Test Tools to protocol v3. And it
> is
> >> in service on PluginTests of the main repository[1]. As I know,
> @mrproliu <
> >> https://github.com/mrproliu> have brought it to lua-nginx
> repository[2].
> >>
> >> [1]:https://github.com/apache/skywalking/pull/4599 <
> >> https://github.com/apache/skywalking/pull/4599>
> >> [2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <
> >> https://github.com/apache/skywalking-nginx-lua/pull/30>
> >> —————
> >> Daming(@dmsolr)
> >> Apache SkyWalking
> >>
> >>
> >>
> >>> 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
> >>>
> >>> Hi Dev Team
> >>
> >>
>
>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory
permanently.
Posted by Daming <zt...@foxmail.com>.
Hi,
It is good to me to remove the InfluxDB storage E2E Test.
I will make it adapt to the new protocol and restore it. After the PR merged.
—————
Daming(@dmsolr)
Apache SkyWalking
> 在 2020年4月7日，下午2:56，Sheng Wu <wu...@gmail.com> 写道：
>
> Haochao Zhuang, Zhenxu Ke, Yanlong He and Han Liu
>
> I just adjusted the following things in PR tests
> 1. Remove influxdb storage, lua agent and PHP agent from e2e tests. Once
> you have done in your side, please restore these, we stil expect these in
> 8.0 release.
> 2. No need to do a reboot test. Remove it permanently.
> 3. SkyWalking 6 compatibility tests have been removed permanently. I assume
> no one will release a new 6.9 for v3 protocol, right? If we have, please
> let me know.
>
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> Daming <zt...@foxmail.com> 于2020年4月7日周二 下午1:05写道：
>
>>
>> Hi Dev Team,
>>
>> Currently, I have upgraded the Agent-Test Tools to protocol v3. And it is
>> in service on PluginTests of the main repository[1]. As I know, @mrproliu <
>> https://github.com/mrproliu> have brought it to lua-nginx repository[2].
>>
>> [1]:https://github.com/apache/skywalking/pull/4599 <
>> https://github.com/apache/skywalking/pull/4599>
>> [2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <
>> https://github.com/apache/skywalking-nginx-lua/pull/30>
>> —————
>> Daming(@dmsolr)
>> Apache SkyWalking
>>
>>
>>
>>> 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
>>>
>>> Hi Dev Team
>>
>>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory
permanently.
Posted by "kezhenxu94@apache" <ke...@apache.org>.
The removal in E2E part looks good to me
> On Apr 7, 2020, at 14:56, Sheng Wu <wu...@gmail.com> wrote:
>
> Haochao Zhuang, Zhenxu Ke, Yanlong He and Han Liu
>
> I just adjusted the following things in PR tests
> 1. Remove influxdb storage, lua agent and PHP agent from e2e tests. Once
> you have done in your side, please restore these, we stil expect these in
> 8.0 release.
> 2. No need to do a reboot test. Remove it permanently.
> 3. SkyWalking 6 compatibility tests have been removed permanently. I assume
> no one will release a new 6.9 for v3 protocol, right? If we have, please
> let me know.
>
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> Daming <zt...@foxmail.com> 于2020年4月7日周二 下午1:05写道：
>
>>
>> Hi Dev Team,
>>
>> Currently, I have upgraded the Agent-Test Tools to protocol v3. And it is
>> in service on PluginTests of the main repository[1]. As I know, @mrproliu <
>> https://github.com/mrproliu> have brought it to lua-nginx repository[2].
>>
>> [1]:https://github.com/apache/skywalking/pull/4599 <
>> https://github.com/apache/skywalking/pull/4599>
>> [2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <
>> https://github.com/apache/skywalking-nginx-lua/pull/30>
>> —————
>> Daming(@dmsolr)
>> Apache SkyWalking
>>
>>
>>
>>> 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
>>>
>>> Hi Dev Team
>>
>>
GitHub @kezhenxu94
Apache SkyWalking, Apache Dubbo
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by Sheng Wu <wu...@gmail.com>.
Haochao Zhuang, Zhenxu Ke, Yanlong He and Han Liu
I just adjusted the following things in PR tests
1. Remove influxdb storage, lua agent and PHP agent from e2e tests. Once
you have done in your side, please restore these, we stil expect these in
8.0 release.
2. No need to do a reboot test. Remove it permanently.
3. SkyWalking 6 compatibility tests have been removed permanently. I assume
no one will release a new 6.9 for v3 protocol, right? If we have, please
let me know.
Sheng Wu 吴晟
Twitter, wusheng1108
Daming <zt...@foxmail.com> 于2020年4月7日周二 下午1:05写道：
>
> Hi Dev Team,
>
> Currently, I have upgraded the Agent-Test Tools to protocol v3. And it is
> in service on PluginTests of the main repository[1]. As I know, @mrproliu <
> https://github.com/mrproliu> have brought it to lua-nginx repository[2].
>
> [1]:https://github.com/apache/skywalking/pull/4599 <
> https://github.com/apache/skywalking/pull/4599>
> [2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <
> https://github.com/apache/skywalking-nginx-lua/pull/30>
> —————
> Daming(@dmsolr)
> Apache SkyWalking
>
>
>
> > 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
> >
> > Hi Dev Team
>
>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory
permanently.
Posted by Daming <zt...@foxmail.com>.
Hi Dev Team,
Currently, I have upgraded the Agent-Test Tools to protocol v3. And it is in service on PluginTests of the main repository[1]. As I know, @mrproliu <https://github.com/mrproliu> have brought it to lua-nginx repository[2].
[1]:https://github.com/apache/skywalking/pull/4599 <https://github.com/apache/skywalking/pull/4599>
[2]:https://github.com/apache/skywalking-nginx-lua/pull/30 <https://github.com/apache/skywalking-nginx-lua/pull/30>
—————
Daming(@dmsolr)
Apache SkyWalking
> 在 2020年3月31日，上午11:29，Sheng Wu <wu...@apache.org> 写道：
>
> Hi Dev Team
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by Sheng Wu <wu...@gmail.com>.
This PR has been approved by
1. Han Liu
2. Haochao Zhuang
3. Zhenxu Ke
4. Hongtao Gao
5. Yanlong He
6. Wei Zhang
3 PMC members and 3 committers. Tests have been covering agent tests, e2e
tests, language agents of java/php/lua/go.
I am going to merge that PR now.
Sheng Wu 吴晟
Twitter, wusheng1108
Sheng Wu <wu...@gmail.com> 于2020年4月9日周四 下午10:57写道：
> Great.
> If any of you think this PR is ready, please approve(but don’t merge) the
> PR.
>
> han liu <mr...@gmail.com>于2020年4月9日 周四下午10:21写道：
>
>> Hi, Sheng Wu
>>
>> I have test the alarm module, and it works well.
>>
>> Sheng Wu <wu...@gmail.com> 于2020年4月9日周四 下午8:26写道：
>>
>>> Summary
>>>
>> This pull request is open for 10 days, and I worked on this 2 weeks ago.
>>> Thanks everyone to help making this PR and tests work now.
>>>
>>> Here are the list of changes we made in this pull request. Top 3 are the
>>> principle changes, others are following these principles.
>>>
>>> 1. New agent and mesh report protocol.
>>> 2. New agent header protocol.
>>> 3. Service register, instance register and network address register
>>> have been removed permanently.
>>> 4. Service traffic, instance traffic and network alias metrics are
>>> added to replace the service, instance and network address inventory.
>>> 5. Register process has been removed.
>>> 6. Metrics stream process supports insert only mode, especially for
>>> traffic entities.
>>> 7. Metrics stream process supports no-downsampling mode for traffic
>>> entities and network alias.
>>> 8. Remove all register mechanism and cache in the java agent.
>>> 9. Remove MONTH step in GraphQL query.
>>> 10. Update UI to remove MONTH step query, the max query range is 60
>>> days now.
>>> 11. Simplify the TTL to metrics and record. And the unit has been
>>> formatted in Day unit. No specific TTL for ElasticSearch storage.
>>> 12. Buffer mechanism of trace receiver and mesh receiver has been
>>> removed due to no register.
>>> 13. New service id, instance id and endpoint id rules, including
>>> service relation, instance relation and endpoint relation id rules.
>>> 14. Java agent support keep tracing mode, meaning, agent generating
>>> tracing context even the backend is unconnected/unavailable.
>>> 15. Plugin test tool up to date, in order to support new protocol.
>>> 16. Plugin tests expected data files updated.
>>> 17. E2E tests updated.
>>> 18. [TBD] InfluxDB storage implementation is not available, need
>>> @dmsolr <https://github.com/dmsolr> to fix later, in order to reduce
>>> the master change block by this PR.
>>>
>>> If anyone plans to review the codes in the blocking mode(meaning don't
>>> merge the PR), please let me know. Otherwise, I will try to merge this
>>> tomorrow due to unblock the new changes in the master branch.
>>>
>>> Han Liu +mrproliu@gmail.com <mr...@gmail.com>
>>> I just want to wait for your alarm test result, due to e2e doesn't
>>> include this.
>>>
>>> Sheng Wu 吴晟
>>> Twitter, wusheng1108
>>>
>>>
>>> kezhenxu94@apache <ke...@apache.org> 于2020年4月9日周四 下午7:32写道：
>>>
>>>> The E2E should be fixed according the new codes now, good luck
>>>>
>>>>
>>>>
>>>> GitHub @kezhenxu94
>>>> Apache SkyWalking, Apache Dubbo
>>>>
>>>> > On Apr 6, 2020, at 20:45, Sheng Wu <wu...@gmail.com> wrote:
>>>> >
>>>> > Zhenxu Ke
>>>> > My PR is ready locally, but e2e seems still failing. Please help with
>>>> > locating what are the issues.
>>>> >
>>>> > Haochao Zhuang
>>>> > I noticed you have upgraded the test tool for v3 protocol. Please
>>>> move on
>>>> > making the plugin tests passed in the v8-core branch.
>>>> >
>>>> > Sheng Wu 吴晟
>>>> > Twitter, wusheng1108
>>>> >
>>>> >
>>>> > Sheng Wu <wu...@apache.org> 于2020年3月31日周二 上午11:29写道：
>>>> >
>>>> >> Hi Dev Team
>>>> >>
>>>> >> After the experiences of removing endpoint_inventory, I found out
>>>> this
>>>> >> strategy is successful.
>>>> >> Especially, we totally get rid of register, so I want to do more.
>>>> >>
>>>> >> *SkyWalking 8.0.0*
>>>> >> First, it is already unexpected for me, we have to move to 8.0.0 so
>>>> >> quickly, but after the discussion with +高洪涛@skywalking
>>>> >> <ha...@apache.org>, and thinking about this for several days, I
>>>> think
>>>> >> we have to.
>>>> >>
>>>> >> The key chances are following
>>>> >> 1. Remove service, service instance, and network address register.
>>>> The old
>>>> >> register protocols are totally going to be removed.
>>>> >> 2. The agent doesn't need to do register anymore. Service name and
>>>> Service
>>>> >> Instance name are generated by the agent itself, but the extra
>>>> information,
>>>> >> such as IP, hostname, language, should report to backend separately.
>>>> >> 3. Service Traffic should be added just like the endpoint traffic
>>>> but keep
>>>> >> the time bucket as we need accurate service name in the given
>>>> duration
>>>> >> 4. Service Instance Traffic should be added too, with external
>>>> >> information, such as language, hostname.
>>>> >> 5. Trace context propagation context should be changed to accept
>>>> string in
>>>> >> service instance name, endpoint name and network address. This could
>>>> ease
>>>> >> the agent logic, but also, requires changes in all language agent and
>>>> >> plugin test tool,
>>>> >> 6. Trace report protocol requires to change too, in order to adopt
>>>> the
>>>> >> string.
>>>> >> 7. e2e tests have to ignore PHP and LUA at first, and remove the 6.x
>>>> >> compatibility test(doesn't support anymore).
>>>> >>
>>>> >> The benefits we will get are
>>>> >> 1. Don't worry about the inventory(s) that has been deleted randomly
>>>> by
>>>> >> end users. (We received a lot of issue reports about this)
>>>> >> 2. The upgrade could be easier erasing the whole storage and reboot
>>>> the
>>>> >> new one. (Users don't feel comfortable about upgrade)
>>>> >> 3. No hot-reboot case in the agent side
>>>> >> 4. No cache of network address register information in the agent.
>>>> >> 5. No service and service instance cache in the OAP
>>>> >> 6. No register lock in the OAP
>>>> >> 7. No file buffer mechanism in the OAP too, same as no register
>>>> happens.
>>>> >>
>>>> >> In my mind, I think this totally break upgrade is super meaningful
>>>> and
>>>> >> will be good change. Even we break many things, they are easy to
>>>> follow.
>>>> >> +mrproliu@gmail.com <mr...@gmail.com> I think by following this,
>>>> we
>>>> >> need to change the collaboration header to `sw8` :) As no 7.1.0
>>>> release
>>>> >> will happen.
>>>> >>
>>>> >> Sheng Wu 吴晟
>>>> >> Twitter, wusheng1108
>>>> >>
>>>>
>>>> --
> Sheng Wu 吴晟
>
> Apache SkyWalking
> Apache Incubator
> Apache ShardingSphere, ECharts, DolphinScheduler podlings
> Zipkin
> Twitter, wusheng1108
>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by Sheng Wu <wu...@gmail.com>.
Great.
If any of you think this PR is ready, please approve(but don’t merge) the
PR.
han liu <mr...@gmail.com>于2020年4月9日 周四下午10:21写道：
> Hi, Sheng Wu
>
> I have test the alarm module, and it works well.
>
> Sheng Wu <wu...@gmail.com> 于2020年4月9日周四 下午8:26写道：
>
>> Summary
>>
> This pull request is open for 10 days, and I worked on this 2 weeks ago.
>> Thanks everyone to help making this PR and tests work now.
>>
>> Here are the list of changes we made in this pull request. Top 3 are the
>> principle changes, others are following these principles.
>>
>> 1. New agent and mesh report protocol.
>> 2. New agent header protocol.
>> 3. Service register, instance register and network address register
>> have been removed permanently.
>> 4. Service traffic, instance traffic and network alias metrics are
>> added to replace the service, instance and network address inventory.
>> 5. Register process has been removed.
>> 6. Metrics stream process supports insert only mode, especially for
>> traffic entities.
>> 7. Metrics stream process supports no-downsampling mode for traffic
>> entities and network alias.
>> 8. Remove all register mechanism and cache in the java agent.
>> 9. Remove MONTH step in GraphQL query.
>> 10. Update UI to remove MONTH step query, the max query range is 60
>> days now.
>> 11. Simplify the TTL to metrics and record. And the unit has been
>> formatted in Day unit. No specific TTL for ElasticSearch storage.
>> 12. Buffer mechanism of trace receiver and mesh receiver has been
>> removed due to no register.
>> 13. New service id, instance id and endpoint id rules, including
>> service relation, instance relation and endpoint relation id rules.
>> 14. Java agent support keep tracing mode, meaning, agent generating
>> tracing context even the backend is unconnected/unavailable.
>> 15. Plugin test tool up to date, in order to support new protocol.
>> 16. Plugin tests expected data files updated.
>> 17. E2E tests updated.
>> 18. [TBD] InfluxDB storage implementation is not available, need
>> @dmsolr <https://github.com/dmsolr> to fix later, in order to reduce
>> the master change block by this PR.
>>
>> If anyone plans to review the codes in the blocking mode(meaning don't
>> merge the PR), please let me know. Otherwise, I will try to merge this
>> tomorrow due to unblock the new changes in the master branch.
>>
>> Han Liu +mrproliu@gmail.com <mr...@gmail.com>
>> I just want to wait for your alarm test result, due to e2e doesn't
>> include this.
>>
>> Sheng Wu 吴晟
>> Twitter, wusheng1108
>>
>>
>> kezhenxu94@apache <ke...@apache.org> 于2020年4月9日周四 下午7:32写道：
>>
>>> The E2E should be fixed according the new codes now, good luck
>>>
>>>
>>>
>>> GitHub @kezhenxu94
>>> Apache SkyWalking, Apache Dubbo
>>>
>>> > On Apr 6, 2020, at 20:45, Sheng Wu <wu...@gmail.com> wrote:
>>> >
>>> > Zhenxu Ke
>>> > My PR is ready locally, but e2e seems still failing. Please help with
>>> > locating what are the issues.
>>> >
>>> > Haochao Zhuang
>>> > I noticed you have upgraded the test tool for v3 protocol. Please move
>>> on
>>> > making the plugin tests passed in the v8-core branch.
>>> >
>>> > Sheng Wu 吴晟
>>> > Twitter, wusheng1108
>>> >
>>> >
>>> > Sheng Wu <wu...@apache.org> 于2020年3月31日周二 上午11:29写道：
>>> >
>>> >> Hi Dev Team
>>> >>
>>> >> After the experiences of removing endpoint_inventory, I found out this
>>> >> strategy is successful.
>>> >> Especially, we totally get rid of register, so I want to do more.
>>> >>
>>> >> *SkyWalking 8.0.0*
>>> >> First, it is already unexpected for me, we have to move to 8.0.0 so
>>> >> quickly, but after the discussion with +高洪涛@skywalking
>>> >> <ha...@apache.org>, and thinking about this for several days, I
>>> think
>>> >> we have to.
>>> >>
>>> >> The key chances are following
>>> >> 1. Remove service, service instance, and network address register.
>>> The old
>>> >> register protocols are totally going to be removed.
>>> >> 2. The agent doesn't need to do register anymore. Service name and
>>> Service
>>> >> Instance name are generated by the agent itself, but the extra
>>> information,
>>> >> such as IP, hostname, language, should report to backend separately.
>>> >> 3. Service Traffic should be added just like the endpoint traffic but
>>> keep
>>> >> the time bucket as we need accurate service name in the given duration
>>> >> 4. Service Instance Traffic should be added too, with external
>>> >> information, such as language, hostname.
>>> >> 5. Trace context propagation context should be changed to accept
>>> string in
>>> >> service instance name, endpoint name and network address. This could
>>> ease
>>> >> the agent logic, but also, requires changes in all language agent and
>>> >> plugin test tool,
>>> >> 6. Trace report protocol requires to change too, in order to adopt the
>>> >> string.
>>> >> 7. e2e tests have to ignore PHP and LUA at first, and remove the 6.x
>>> >> compatibility test(doesn't support anymore).
>>> >>
>>> >> The benefits we will get are
>>> >> 1. Don't worry about the inventory(s) that has been deleted randomly
>>> by
>>> >> end users. (We received a lot of issue reports about this)
>>> >> 2. The upgrade could be easier erasing the whole storage and reboot
>>> the
>>> >> new one. (Users don't feel comfortable about upgrade)
>>> >> 3. No hot-reboot case in the agent side
>>> >> 4. No cache of network address register information in the agent.
>>> >> 5. No service and service instance cache in the OAP
>>> >> 6. No register lock in the OAP
>>> >> 7. No file buffer mechanism in the OAP too, same as no register
>>> happens.
>>> >>
>>> >> In my mind, I think this totally break upgrade is super meaningful and
>>> >> will be good change. Even we break many things, they are easy to
>>> follow.
>>> >> +mrproliu@gmail.com <mr...@gmail.com> I think by following this,
>>> we
>>> >> need to change the collaboration header to `sw8` :) As no 7.1.0
>>> release
>>> >> will happen.
>>> >>
>>> >> Sheng Wu 吴晟
>>> >> Twitter, wusheng1108
>>> >>
>>>
>>> --
Sheng Wu 吴晟
Apache SkyWalking
Apache Incubator
Apache ShardingSphere, ECharts, DolphinScheduler podlings
Zipkin
Twitter, wusheng1108
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by han liu <mr...@gmail.com>.
Hi, Sheng Wu
I have test the alarm module, and it works well.
Sheng Wu <wu...@gmail.com> 于2020年4月9日周四 下午8:26写道：
> Summary
>
> This pull request is open for 10 days, and I worked on this 2 weeks ago.
> Thanks everyone to help making this PR and tests work now.
>
> Here are the list of changes we made in this pull request. Top 3 are the
> principle changes, others are following these principles.
>
> 1. New agent and mesh report protocol.
> 2. New agent header protocol.
> 3. Service register, instance register and network address register
> have been removed permanently.
> 4. Service traffic, instance traffic and network alias metrics are
> added to replace the service, instance and network address inventory.
> 5. Register process has been removed.
> 6. Metrics stream process supports insert only mode, especially for
> traffic entities.
> 7. Metrics stream process supports no-downsampling mode for traffic
> entities and network alias.
> 8. Remove all register mechanism and cache in the java agent.
> 9. Remove MONTH step in GraphQL query.
> 10. Update UI to remove MONTH step query, the max query range is 60
> days now.
> 11. Simplify the TTL to metrics and record. And the unit has been
> formatted in Day unit. No specific TTL for ElasticSearch storage.
> 12. Buffer mechanism of trace receiver and mesh receiver has been
> removed due to no register.
> 13. New service id, instance id and endpoint id rules, including
> service relation, instance relation and endpoint relation id rules.
> 14. Java agent support keep tracing mode, meaning, agent generating
> tracing context even the backend is unconnected/unavailable.
> 15. Plugin test tool up to date, in order to support new protocol.
> 16. Plugin tests expected data files updated.
> 17. E2E tests updated.
> 18. [TBD] InfluxDB storage implementation is not available, need
> @dmsolr <https://github.com/dmsolr> to fix later, in order to reduce
> the master change block by this PR.
>
> If anyone plans to review the codes in the blocking mode(meaning don't
> merge the PR), please let me know. Otherwise, I will try to merge this
> tomorrow due to unblock the new changes in the master branch.
>
> Han Liu +mrproliu@gmail.com <mr...@gmail.com>
> I just want to wait for your alarm test result, due to e2e doesn't include
> this.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> kezhenxu94@apache <ke...@apache.org> 于2020年4月9日周四 下午7:32写道：
>
>> The E2E should be fixed according the new codes now, good luck
>>
>>
>>
>> GitHub @kezhenxu94
>> Apache SkyWalking, Apache Dubbo
>>
>> > On Apr 6, 2020, at 20:45, Sheng Wu <wu...@gmail.com> wrote:
>> >
>> > Zhenxu Ke
>> > My PR is ready locally, but e2e seems still failing. Please help with
>> > locating what are the issues.
>> >
>> > Haochao Zhuang
>> > I noticed you have upgraded the test tool for v3 protocol. Please move
>> on
>> > making the plugin tests passed in the v8-core branch.
>> >
>> > Sheng Wu 吴晟
>> > Twitter, wusheng1108
>> >
>> >
>> > Sheng Wu <wu...@apache.org> 于2020年3月31日周二 上午11:29写道：
>> >
>> >> Hi Dev Team
>> >>
>> >> After the experiences of removing endpoint_inventory, I found out this
>> >> strategy is successful.
>> >> Especially, we totally get rid of register, so I want to do more.
>> >>
>> >> *SkyWalking 8.0.0*
>> >> First, it is already unexpected for me, we have to move to 8.0.0 so
>> >> quickly, but after the discussion with +高洪涛@skywalking
>> >> <ha...@apache.org>, and thinking about this for several days, I
>> think
>> >> we have to.
>> >>
>> >> The key chances are following
>> >> 1. Remove service, service instance, and network address register. The
>> old
>> >> register protocols are totally going to be removed.
>> >> 2. The agent doesn't need to do register anymore. Service name and
>> Service
>> >> Instance name are generated by the agent itself, but the extra
>> information,
>> >> such as IP, hostname, language, should report to backend separately.
>> >> 3. Service Traffic should be added just like the endpoint traffic but
>> keep
>> >> the time bucket as we need accurate service name in the given duration
>> >> 4. Service Instance Traffic should be added too, with external
>> >> information, such as language, hostname.
>> >> 5. Trace context propagation context should be changed to accept
>> string in
>> >> service instance name, endpoint name and network address. This could
>> ease
>> >> the agent logic, but also, requires changes in all language agent and
>> >> plugin test tool,
>> >> 6. Trace report protocol requires to change too, in order to adopt the
>> >> string.
>> >> 7. e2e tests have to ignore PHP and LUA at first, and remove the 6.x
>> >> compatibility test(doesn't support anymore).
>> >>
>> >> The benefits we will get are
>> >> 1. Don't worry about the inventory(s) that has been deleted randomly by
>> >> end users. (We received a lot of issue reports about this)
>> >> 2. The upgrade could be easier erasing the whole storage and reboot the
>> >> new one. (Users don't feel comfortable about upgrade)
>> >> 3. No hot-reboot case in the agent side
>> >> 4. No cache of network address register information in the agent.
>> >> 5. No service and service instance cache in the OAP
>> >> 6. No register lock in the OAP
>> >> 7. No file buffer mechanism in the OAP too, same as no register
>> happens.
>> >>
>> >> In my mind, I think this totally break upgrade is super meaningful and
>> >> will be good change. Even we break many things, they are easy to
>> follow.
>> >> +mrproliu@gmail.com <mr...@gmail.com> I think by following this, we
>> >> need to change the collaboration header to `sw8` :) As no 7.1.0 release
>> >> will happen.
>> >>
>> >> Sheng Wu 吴晟
>> >> Twitter, wusheng1108
>> >>
>>
>>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by Sheng Wu <wu...@gmail.com>.
If anyone still doesn't know the PR, here is the link,
https://github.com/apache/skywalking/pull/4599
Sheng Wu 吴晟
Twitter, wusheng1108
Sheng Wu <wu...@gmail.com> 于2020年4月9日周四 下午8:26写道：
> Summary
>
> This pull request is open for 10 days, and I worked on this 2 weeks ago.
> Thanks everyone to help making this PR and tests work now.
>
> Here are the list of changes we made in this pull request. Top 3 are the
> principle changes, others are following these principles.
>
> 1. New agent and mesh report protocol.
> 2. New agent header protocol.
> 3. Service register, instance register and network address register
> have been removed permanently.
> 4. Service traffic, instance traffic and network alias metrics are
> added to replace the service, instance and network address inventory.
> 5. Register process has been removed.
> 6. Metrics stream process supports insert only mode, especially for
> traffic entities.
> 7. Metrics stream process supports no-downsampling mode for traffic
> entities and network alias.
> 8. Remove all register mechanism and cache in the java agent.
> 9. Remove MONTH step in GraphQL query.
> 10. Update UI to remove MONTH step query, the max query range is 60
> days now.
> 11. Simplify the TTL to metrics and record. And the unit has been
> formatted in Day unit. No specific TTL for ElasticSearch storage.
> 12. Buffer mechanism of trace receiver and mesh receiver has been
> removed due to no register.
> 13. New service id, instance id and endpoint id rules, including
> service relation, instance relation and endpoint relation id rules.
> 14. Java agent support keep tracing mode, meaning, agent generating
> tracing context even the backend is unconnected/unavailable.
> 15. Plugin test tool up to date, in order to support new protocol.
> 16. Plugin tests expected data files updated.
> 17. E2E tests updated.
> 18. [TBD] InfluxDB storage implementation is not available, need
> @dmsolr <https://github.com/dmsolr> to fix later, in order to reduce
> the master change block by this PR.
>
> If anyone plans to review the codes in the blocking mode(meaning don't
> merge the PR), please let me know. Otherwise, I will try to merge this
> tomorrow due to unblock the new changes in the master branch.
>
> Han Liu +mrproliu@gmail.com <mr...@gmail.com>
> I just want to wait for your alarm test result, due to e2e doesn't include
> this.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> kezhenxu94@apache <ke...@apache.org> 于2020年4月9日周四 下午7:32写道：
>
>> The E2E should be fixed according the new codes now, good luck
>>
>>
>>
>> GitHub @kezhenxu94
>> Apache SkyWalking, Apache Dubbo
>>
>> > On Apr 6, 2020, at 20:45, Sheng Wu <wu...@gmail.com> wrote:
>> >
>> > Zhenxu Ke
>> > My PR is ready locally, but e2e seems still failing. Please help with
>> > locating what are the issues.
>> >
>> > Haochao Zhuang
>> > I noticed you have upgraded the test tool for v3 protocol. Please move
>> on
>> > making the plugin tests passed in the v8-core branch.
>> >
>> > Sheng Wu 吴晟
>> > Twitter, wusheng1108
>> >
>> >
>> > Sheng Wu <wu...@apache.org> 于2020年3月31日周二 上午11:29写道：
>> >
>> >> Hi Dev Team
>> >>
>> >> After the experiences of removing endpoint_inventory, I found out this
>> >> strategy is successful.
>> >> Especially, we totally get rid of register, so I want to do more.
>> >>
>> >> *SkyWalking 8.0.0*
>> >> First, it is already unexpected for me, we have to move to 8.0.0 so
>> >> quickly, but after the discussion with +高洪涛@skywalking
>> >> <ha...@apache.org>, and thinking about this for several days, I
>> think
>> >> we have to.
>> >>
>> >> The key chances are following
>> >> 1. Remove service, service instance, and network address register. The
>> old
>> >> register protocols are totally going to be removed.
>> >> 2. The agent doesn't need to do register anymore. Service name and
>> Service
>> >> Instance name are generated by the agent itself, but the extra
>> information,
>> >> such as IP, hostname, language, should report to backend separately.
>> >> 3. Service Traffic should be added just like the endpoint traffic but
>> keep
>> >> the time bucket as we need accurate service name in the given duration
>> >> 4. Service Instance Traffic should be added too, with external
>> >> information, such as language, hostname.
>> >> 5. Trace context propagation context should be changed to accept
>> string in
>> >> service instance name, endpoint name and network address. This could
>> ease
>> >> the agent logic, but also, requires changes in all language agent and
>> >> plugin test tool,
>> >> 6. Trace report protocol requires to change too, in order to adopt the
>> >> string.
>> >> 7. e2e tests have to ignore PHP and LUA at first, and remove the 6.x
>> >> compatibility test(doesn't support anymore).
>> >>
>> >> The benefits we will get are
>> >> 1. Don't worry about the inventory(s) that has been deleted randomly by
>> >> end users. (We received a lot of issue reports about this)
>> >> 2. The upgrade could be easier erasing the whole storage and reboot the
>> >> new one. (Users don't feel comfortable about upgrade)
>> >> 3. No hot-reboot case in the agent side
>> >> 4. No cache of network address register information in the agent.
>> >> 5. No service and service instance cache in the OAP
>> >> 6. No register lock in the OAP
>> >> 7. No file buffer mechanism in the OAP too, same as no register
>> happens.
>> >>
>> >> In my mind, I think this totally break upgrade is super meaningful and
>> >> will be good change. Even we break many things, they are easy to
>> follow.
>> >> +mrproliu@gmail.com <mr...@gmail.com> I think by following this, we
>> >> need to change the collaboration header to `sw8` :) As no 7.1.0 release
>> >> will happen.
>> >>
>> >> Sheng Wu 吴晟
>> >> Twitter, wusheng1108
>> >>
>>
>>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by Sheng Wu <wu...@gmail.com>.
Summary
This pull request is open for 10 days, and I worked on this 2 weeks ago.
Thanks everyone to help making this PR and tests work now.
Here are the list of changes we made in this pull request. Top 3 are the
principle changes, others are following these principles.
1. New agent and mesh report protocol.
2. New agent header protocol.
3. Service register, instance register and network address register have
been removed permanently.
4. Service traffic, instance traffic and network alias metrics are added
to replace the service, instance and network address inventory.
5. Register process has been removed.
6. Metrics stream process supports insert only mode, especially for
traffic entities.
7. Metrics stream process supports no-downsampling mode for traffic
entities and network alias.
8. Remove all register mechanism and cache in the java agent.
9. Remove MONTH step in GraphQL query.
10. Update UI to remove MONTH step query, the max query range is 60 days
now.
11. Simplify the TTL to metrics and record. And the unit has been
formatted in Day unit. No specific TTL for ElasticSearch storage.
12. Buffer mechanism of trace receiver and mesh receiver has been
removed due to no register.
13. New service id, instance id and endpoint id rules, including service
relation, instance relation and endpoint relation id rules.
14. Java agent support keep tracing mode, meaning, agent generating
tracing context even the backend is unconnected/unavailable.
15. Plugin test tool up to date, in order to support new protocol.
16. Plugin tests expected data files updated.
17. E2E tests updated.
18. [TBD] InfluxDB storage implementation is not available, need @dmsolr
<https://github.com/dmsolr> to fix later, in order to reduce the master
change block by this PR.
If anyone plans to review the codes in the blocking mode(meaning don't
merge the PR), please let me know. Otherwise, I will try to merge this
tomorrow due to unblock the new changes in the master branch.
Han Liu +mrproliu@gmail.com <mr...@gmail.com>
I just want to wait for your alarm test result, due to e2e doesn't include
this.
Sheng Wu 吴晟
Twitter, wusheng1108
kezhenxu94@apache <ke...@apache.org> 于2020年4月9日周四 下午7:32写道：
> The E2E should be fixed according the new codes now, good luck
>
>
>
> GitHub @kezhenxu94
> Apache SkyWalking, Apache Dubbo
>
> > On Apr 6, 2020, at 20:45, Sheng Wu <wu...@gmail.com> wrote:
> >
> > Zhenxu Ke
> > My PR is ready locally, but e2e seems still failing. Please help with
> > locating what are the issues.
> >
> > Haochao Zhuang
> > I noticed you have upgraded the test tool for v3 protocol. Please move on
> > making the plugin tests passed in the v8-core branch.
> >
> > Sheng Wu 吴晟
> > Twitter, wusheng1108
> >
> >
> > Sheng Wu <wu...@apache.org> 于2020年3月31日周二 上午11:29写道：
> >
> >> Hi Dev Team
> >>
> >> After the experiences of removing endpoint_inventory, I found out this
> >> strategy is successful.
> >> Especially, we totally get rid of register, so I want to do more.
> >>
> >> *SkyWalking 8.0.0*
> >> First, it is already unexpected for me, we have to move to 8.0.0 so
> >> quickly, but after the discussion with +高洪涛@skywalking
> >> <ha...@apache.org>, and thinking about this for several days, I
> think
> >> we have to.
> >>
> >> The key chances are following
> >> 1. Remove service, service instance, and network address register. The
> old
> >> register protocols are totally going to be removed.
> >> 2. The agent doesn't need to do register anymore. Service name and
> Service
> >> Instance name are generated by the agent itself, but the extra
> information,
> >> such as IP, hostname, language, should report to backend separately.
> >> 3. Service Traffic should be added just like the endpoint traffic but
> keep
> >> the time bucket as we need accurate service name in the given duration
> >> 4. Service Instance Traffic should be added too, with external
> >> information, such as language, hostname.
> >> 5. Trace context propagation context should be changed to accept string
> in
> >> service instance name, endpoint name and network address. This could
> ease
> >> the agent logic, but also, requires changes in all language agent and
> >> plugin test tool,
> >> 6. Trace report protocol requires to change too, in order to adopt the
> >> string.
> >> 7. e2e tests have to ignore PHP and LUA at first, and remove the 6.x
> >> compatibility test(doesn't support anymore).
> >>
> >> The benefits we will get are
> >> 1. Don't worry about the inventory(s) that has been deleted randomly by
> >> end users. (We received a lot of issue reports about this)
> >> 2. The upgrade could be easier erasing the whole storage and reboot the
> >> new one. (Users don't feel comfortable about upgrade)
> >> 3. No hot-reboot case in the agent side
> >> 4. No cache of network address register information in the agent.
> >> 5. No service and service instance cache in the OAP
> >> 6. No register lock in the OAP
> >> 7. No file buffer mechanism in the OAP too, same as no register happens.
> >>
> >> In my mind, I think this totally break upgrade is super meaningful and
> >> will be good change. Even we break many things, they are easy to follow.
> >> +mrproliu@gmail.com <mr...@gmail.com> I think by following this, we
> >> need to change the collaboration header to `sw8` :) As no 7.1.0 release
> >> will happen.
> >>
> >> Sheng Wu 吴晟
> >> Twitter, wusheng1108
> >>
>
>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory
permanently.
Posted by "kezhenxu94@apache" <ke...@apache.org>.
The E2E should be fixed according the new codes now, good luck
GitHub @kezhenxu94
Apache SkyWalking, Apache Dubbo
> On Apr 6, 2020, at 20:45, Sheng Wu <wu...@gmail.com> wrote:
>
> Zhenxu Ke
> My PR is ready locally, but e2e seems still failing. Please help with
> locating what are the issues.
>
> Haochao Zhuang
> I noticed you have upgraded the test tool for v3 protocol. Please move on
> making the plugin tests passed in the v8-core branch.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>
>
> Sheng Wu <wu...@apache.org> 于2020年3月31日周二 上午11:29写道：
>
>> Hi Dev Team
>>
>> After the experiences of removing endpoint_inventory, I found out this
>> strategy is successful.
>> Especially, we totally get rid of register, so I want to do more.
>>
>> *SkyWalking 8.0.0*
>> First, it is already unexpected for me, we have to move to 8.0.0 so
>> quickly, but after the discussion with +高洪涛@skywalking
>> <ha...@apache.org>, and thinking about this for several days, I think
>> we have to.
>>
>> The key chances are following
>> 1. Remove service, service instance, and network address register. The old
>> register protocols are totally going to be removed.
>> 2. The agent doesn't need to do register anymore. Service name and Service
>> Instance name are generated by the agent itself, but the extra information,
>> such as IP, hostname, language, should report to backend separately.
>> 3. Service Traffic should be added just like the endpoint traffic but keep
>> the time bucket as we need accurate service name in the given duration
>> 4. Service Instance Traffic should be added too, with external
>> information, such as language, hostname.
>> 5. Trace context propagation context should be changed to accept string in
>> service instance name, endpoint name and network address. This could ease
>> the agent logic, but also, requires changes in all language agent and
>> plugin test tool,
>> 6. Trace report protocol requires to change too, in order to adopt the
>> string.
>> 7. e2e tests have to ignore PHP and LUA at first, and remove the 6.x
>> compatibility test(doesn't support anymore).
>>
>> The benefits we will get are
>> 1. Don't worry about the inventory(s) that has been deleted randomly by
>> end users. (We received a lot of issue reports about this)
>> 2. The upgrade could be easier erasing the whole storage and reboot the
>> new one. (Users don't feel comfortable about upgrade)
>> 3. No hot-reboot case in the agent side
>> 4. No cache of network address register information in the agent.
>> 5. No service and service instance cache in the OAP
>> 6. No register lock in the OAP
>> 7. No file buffer mechanism in the OAP too, same as no register happens.
>>
>> In my mind, I think this totally break upgrade is super meaningful and
>> will be good change. Even we break many things, they are easy to follow.
>> +mrproliu@gmail.com <mr...@gmail.com> I think by following this, we
>> need to change the collaboration header to `sw8` :) As no 7.1.0 release
>> will happen.
>>
>> Sheng Wu 吴晟
>> Twitter, wusheng1108
>>
Re: [8.0.0] Move forward to 8.0.0. REMOVE the register and inventory permanently.
Posted by Sheng Wu <wu...@gmail.com>.
Zhenxu Ke
My PR is ready locally, but e2e seems still failing. Please help with
locating what are the issues.
Haochao Zhuang
I noticed you have upgraded the test tool for v3 protocol. Please move on
making the plugin tests passed in the v8-core branch.
Sheng Wu 吴晟
Twitter, wusheng1108
Sheng Wu <wu...@apache.org> 于2020年3月31日周二 上午11:29写道：
> Hi Dev Team
>
> After the experiences of removing endpoint_inventory, I found out this
> strategy is successful.
> Especially, we totally get rid of register, so I want to do more.
>
> *SkyWalking 8.0.0*
> First, it is already unexpected for me, we have to move to 8.0.0 so
> quickly, but after the discussion with +高洪涛@skywalking
> <ha...@apache.org>, and thinking about this for several days, I think
> we have to.
>
> The key chances are following
> 1. Remove service, service instance, and network address register. The old
> register protocols are totally going to be removed.
> 2. The agent doesn't need to do register anymore. Service name and Service
> Instance name are generated by the agent itself, but the extra information,
> such as IP, hostname, language, should report to backend separately.
> 3. Service Traffic should be added just like the endpoint traffic but keep
> the time bucket as we need accurate service name in the given duration
> 4. Service Instance Traffic should be added too, with external
> information, such as language, hostname.
> 5. Trace context propagation context should be changed to accept string in
> service instance name, endpoint name and network address. This could ease
> the agent logic, but also, requires changes in all language agent and
> plugin test tool,
> 6. Trace report protocol requires to change too, in order to adopt the
> string.
> 7. e2e tests have to ignore PHP and LUA at first, and remove the 6.x
> compatibility test(doesn't support anymore).
>
> The benefits we will get are
> 1. Don't worry about the inventory(s) that has been deleted randomly by
> end users. (We received a lot of issue reports about this)
> 2. The upgrade could be easier erasing the whole storage and reboot the
> new one. (Users don't feel comfortable about upgrade)
> 3. No hot-reboot case in the agent side
> 4. No cache of network address register information in the agent.
> 5. No service and service instance cache in the OAP
> 6. No register lock in the OAP
> 7. No file buffer mechanism in the OAP too, same as no register happens.
>
> In my mind, I think this totally break upgrade is super meaningful and
> will be good change. Even we break many things, they are easy to follow.
> +mrproliu@gmail.com <mr...@gmail.com> I think by following this, we
> need to change the collaboration header to `sw8` :) As no 7.1.0 release
> will happen.
>
> Sheng Wu 吴晟
> Twitter, wusheng1108
>