You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2021/04/01 05:11:15 UTC
[GitHub] [skywalking] gtahubb opened a new issue #6664: Skywalking8.4 cannot upload log after authentication
gtahubb opened a new issue #6664:
URL: https://github.com/apache/skywalking/issues/6664
Please answer these questions before submitting your issue.
- Why do you submit this issue?
- [x] Question or discussion
- [ ] Bug
- [ ] Requirement
- [ ] Feature or performance improvement
___
### Question
Skywalking8.4. After setting the authentication, the server log cannot be uploaded to OAP, but the service information can be monitored. After closing the authentication information, the log can be uploaded. Error message of agent log: error 2021-04-01 12:22:46:414 grpc-default-executor-0 grpclogreportserviceclient: try to send 2 log data to collector, with unexpected exception
org.apache.skywalking . apm.dependencies.io . grpc.StatusRuntimeException : PERMISSION_ DENIED
agent:
agent.authentication = ${SW_AGENT_AUTHENTICATION:xxxxxxxx}
plugin.toolkit.log.grpc.reporter.server_host=${SW_GRPC_LOG_SERVER_HOST:192.168.8.121}
plugin.toolkit.log.grpc.reporter.server_port=${SW_GRPC_LOG_SERVER_PORT:11800}
plugin.toolkit.log.grpc.reporter.max_message_size=${SW_GRPC_LOG_MAX_MESSAGE_SIZE:10485760}
plugin.toolkit.log.grpc.reporter.upstream_timeout=${SW_GRPC_LOG_GRPC_UPSTREAM_TIMEOUT:30}
oap:
gRPCHost: ${SW_RECEIVER_GRPC_HOST:0.0.0.0}
gRPCPort: ${SW_RECEIVER_GRPC_PORT:0}
maxConcurrentCallsPerConnection: ${SW_RECEIVER_GRPC_MAX_CONCURRENT_CALL:0}
maxMessageSize: ${SW_RECEIVER_GRPC_MAX_MESSAGE_SIZE:0}
gRPCThreadPoolQueueSize: ${SW_RECEIVER_GRPC_POOL_QUEUE_SIZE:-1}
gRPCThreadPoolSize: ${SW_RECEIVER_GRPC_THREAD_POOL_SIZE:-1}
gRPCSslEnabled: ${SW_RECEIVER_GRPC_SSL_ENABLED:false}
gRPCSslKeyPath: ${SW_RECEIVER_GRPC_SSL_KEY_PATH:""}
gRPCSslCertChainPath: ${SW_RECEIVER_GRPC_SSL_CERT_CHAIN_PATH:""}
authentication: ${SW_AUTHENTICATION:"xxxxxxxx"}
___
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhyyu commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
zhyyu commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-862164901
> > > This is not a correct way to code. You are not looking for tech issue.
> > > The community is working for making satellite bypass all formats, and after that, no extra channel is required.
> >
> >
> > Ok, I know that using satellite in the future. But the problem is some companies use the current or old version of skywalking, and they active the authentication, but their oap cluster cannot receive log. To fix it, these comany only have to deploy additional oap node to receive the log.
>
> As I mentioned, we could accept PR to fix this issue, but in the right way. The codes you are showing are not. This thing should be done by shade or some standard mechanism, rather than using strange reflection.
Well, I had tried to solve this issue in common way, but failed... like followed code
```java
GRPCLogReportServiceClient
channel = ManagedChannelBuilder
.forAddress(
ToolkitConfig.Plugin.Toolkit.Log.GRPC.Reporter.SERVER_HOST,
ToolkitConfig.Plugin.Toolkit.Log.GRPC.Reporter.SERVER_PORT
)
.usePlaintext()
.build();
// decorate the channel with authentication
AuthenticationDecorator authenticationDecorator = new AuthenticationDecorator();
Channel decoratedChannel = authenticationDecorator.build(channel);
asyncStub = LogReportServiceGrpc.newStub(decoratedChannel)
.withMaxOutboundMessageSize(
ToolkitConfig.Plugin.Toolkit.Log.GRPC.Reporter.MAX_MESSAGE_SIZE);
```
It complied error. "不兼容的类型: io.grpc.ManagedChannel无法转换为org.apache.skywalking.apm.dependencies.io.grpc.Channel"
I checked the "apm-toolkit-logging-common" module and found it inherit "apm-agent-core" from it's parent module "apm-toolkit-activation".
The reason why complied error is that when complied "GRPCLogReportServiceClient", it used shaded "apm-agent-core" package, and "AuthenticationDecorator" build method parameter class have already been changend to "org.apache.skywalking.apm.dependencies.io.grpc.Channel" because of shade plugin. So when pass "io.grpc.ManagedChannel" to "AuthenticationDecorator" in "GRPCLogReportServiceClient", it complies error.
So I used the java reflection to cheat the compiler to complie success. And when maven shade worked on "apm-toolkit-logging-common" later on maven package lifecycle , the class import will be changed, so in the java runtime will not occure error.
Maybe we can merge "apm-toolkit-activation" module into "apm-agent-core" module, so that it will looks "standard mechanism" like you said. But it will break the module development traditional....
How to " by shade or some standard mechanism" in a elegant way really confuse me a lot, I'd appreciate a little direction from you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840522519
> I think we can simply decorate the channel created by boot() method with AuthenticationDecorator class.
But we need a different configuration to support this, sadlly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-861390260
This is not a correct way to code. You are not looking for tech issue.
The community is working for making satellite bypass all formats, and after that, no extra channel is required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wallezhang commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wallezhang commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840532342
> > I think we can simply decorate the channel created by boot() method with AuthenticationDecorator class.
>
> But we need a different configuration to support this, sadlly.
Why need a different configuration? Is it possible that token is different?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-862187542
> > The easy answer is, you should make sure the dependency codes are shaded. `org.apache.skywalking.apm.dependencies.io.grpc.Channel` should be used in the compiled reporter plugin.
>
> What's "compiled reporter plugin"?. Is it maven-compiler-plugin? And how to use shade class "org.apache.skywalking.apm.dependencies.io.grpc.Channel" in maven compile lifecycle? The shade plugin seems only worked on maven package lifecycle [https://maven.apache.org/plugins/maven-shade-plugin/index.html](url)
You use the unshaded codes and set up shade configurations. This is not the style we designed. This is how the shade plugin works
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840533230
> > > I think we can simply decorate the channel created by boot() method with AuthenticationDecorator class.
> >
> >
> > But we need a different configuration to support this, sadlly.
>
> Why need a different configuration? Is it possible that token is different?
One to SkyWalking OAP, the other for Satellite. Satellit as a sidecar, it is usually deployed close to the users, rather than like a backend service. So, they wouldn't share. Also, I can see, satellite even doesn't have an authentication
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] humv3 commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
humv3 commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-814578406
> Yes, I think this is missing. Are you going to contribute?
I had a similar problem. The error log is :
ERROR 2021-04-07 11:27:11:826 SkywalkingAgent-7-JVMService-consume-0 JVMMetricsSender : send JVM metrics to Collector fail.
org.apache.skywalking.apm.dependencies.io.grpc.StatusRuntimeException: PERMISSION_DENIED
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-861425741
> > This is not a correct way to code. You are not looking for tech issue.
> > The community is working for making satellite bypass all formats, and after that, no extra channel is required.
>
> Ok, I know that using satellite in the future. But the problem is some companies use the current or old version of skywalking, and they active the authentication, but their oap cluster cannot receive log. To fix it, these comany only have to deploy additional oap node to receive the log.
As I mentioned, we could accept PR to fix this issue, but in the right way. The codes you are showing are not. This thing should be done by shade or some standard mechanism, rather than using strange reflection.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhyyu commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
zhyyu commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-861393493
> This is not a correct way to code. You are not looking for tech issue.
> The community is working for making satellite bypass all formats, and after that, no extra channel is required.
Ok, I know that using satellite in the future. But the problem is some companies use the current or old version of skywalking, and they active the authentication, but their oap cluster cannot receive log. To fix it, these comany only have to deploy additional oap node to receive the log.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wallezhang commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wallezhang commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840519165
I think we can simply decorate the channel created by `boot()` method with `AuthenticationDecorator` class.
I was just imagining a collector project like satellite. The performance written in java is definitely not high enough. 😃 But the language I usually use in my work is Java, and I haven't any experience with go. :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840483535
The background is, we have SkyWalking Satellite to bypass logs, so, we need to another channel and another backend address.
But satellite only supports logs for now. Once it could cover trace, log, and metric, we could merge those.
I see you are active in the community, are you willing to work on satellite project? I wish we could move faster on that project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng edited a comment on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng edited a comment on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-862167505
The easy answer is, you should make sure the dependency codes are shaded. `org.apache.skywalking.apm.dependencies.io.grpc.Channel` should be used in the compiled reporter plugin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng closed issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng closed issue #6664:
URL: https://github.com/apache/skywalking/issues/6664
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840522205
> I was just imagining a collector project like satellite. The performance written in java is definitely not high enough. 😃 But the language I usually use in my work is Java, and I haven't any experience with go. :(
We choose a language based on tech stack. Go stack uses less memory, and the MMAP queue is only available there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-811645162
Yes, I think this is missing. Are you going to contribute?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhyyu commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
zhyyu commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-862180093
> The easy answer is, you should make sure the dependency codes are shaded. `org.apache.skywalking.apm.dependencies.io.grpc.Channel` should be used in the compiled reporter plugin.
What's "compiled reporter plugin"?. Is it maven-compiler-plugin? And how to use shade class "org.apache.skywalking.apm.dependencies.io.grpc.Channel" in maven compile lifecycle? The shade plugin seems only worked on maven package lifecycle [https://maven.apache.org/plugins/maven-shade-plugin/index.html](url)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wallezhang commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wallezhang commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840541729
OK, I see. So this issue is because of uploading the log to the SkyWalking OAP.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] zhyyu commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
zhyyu commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-861385566
> > I think we can simply decorate the channel created by boot() method with AuthenticationDecorator class.
>
> But we need a different configuration to support this, sadlly.
Hi, I try to change the code like this(tested worked), to add authentication on log channel. And there is no need to change configuration. Can you add this feature in the next version? So the OAP backend can receive the log with authentication.
from
```java
GRPCLogReportServiceClient
channel = ManagedChannelBuilder
.forAddress(
ToolkitConfig.Plugin.Toolkit.Log.GRPC.Reporter.SERVER_HOST,
ToolkitConfig.Plugin.Toolkit.Log.GRPC.Reporter.SERVER_PORT
)
.usePlaintext()
.build();
asyncStub = LogReportServiceGrpc.newStub(channel)
.withMaxOutboundMessageSize(
ToolkitConfig.Plugin.Toolkit.Log.GRPC.Reporter.MAX_MESSAGE_SIZE);
```
to
```java
GRPCLogReportServiceClient
channel = ManagedChannelBuilder
.forAddress(
ToolkitConfig.Plugin.Toolkit.Log.GRPC.Reporter.SERVER_HOST,
ToolkitConfig.Plugin.Toolkit.Log.GRPC.Reporter.SERVER_PORT
)
.usePlaintext()
.build();
// use reflection to avoid shade compile error
AuthenticationDecorator authenticationDecorator = new AuthenticationDecorator();
Method method = AuthenticationDecorator.class.getMethod("build", Channel.class);
Channel decoratedChannel = (Channel) method.invoke(authenticationDecorator, channel);
asyncStub = LogReportServiceGrpc.newStub(decoratedChannel)
.withMaxOutboundMessageSize(
ToolkitConfig.Plugin.Toolkit.Log.GRPC.Reporter.MAX_MESSAGE_SIZE);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wallezhang commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wallezhang commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840530874
> > I was just imagining a collector project like satellite. The performance written in java is definitely not high enough. 😃 But the language I usually use in my work is Java, and I haven't any experience with go. :(
>
> We choose a language based on tech stack. Go stack uses less memory, and the MMAP queue is only available there.
Yes I agree. Golang is more suitable for this scenario.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wallezhang commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wallezhang commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840448839
@wu-sheng Why not use `org.apache.skywalking.apm.agent.core.remote.GRPCChannelManager` class to manage the GRPC channel in a unified way. I found `GRPCLogReportServiceClient` class create a new GRPC channel in `boot()` method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840513466
This can't be fixed through sharing channel. Do you want to add duplicated codes? I prefer we don't do that.
https://github.com/apache/skywalking-satellite This is SkyWalking satellite, written in go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wu-sheng commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wu-sheng commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-862167505
The easy answer is, you should make sure the dependency codes should be shaded. `org.apache.skywalking.apm.dependencies.io.grpc.Channel` should be used in the compiled reporter plugin.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [skywalking] wallezhang commented on issue #6664: Skywalking8.4 cannot upload log after authentication
Posted by GitBox <gi...@apache.org>.
wallezhang commented on issue #6664:
URL: https://github.com/apache/skywalking/issues/6664#issuecomment-840511906
yes, I'd like to do that, however I haven't seen satellite project yet. I think maybe I can fix this issue first, and then go to see the satellite project. BTW, is backend of the GRPC log reporter satellite now?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org