You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@servicecomb.apache.org by GitBox <gi...@apache.org> on 2020/04/09 12:23:45 UTC
[GitHub] [servicecomb-pack] lostpillow opened a new issue #649: omega端线程等待
lostpillow opened a new issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649
偶然造成的问题，具体怎么触发的不明，当前现象为，线程进入使用了@Saga注解的方法后，打了一行日志就没动静了，用jstack查看线程状态，情况如下：
"[Ljava.lang.String;@29e63bc3.container-63-C-1" #510 prio=5 os_prio=0 tid=0x000000000ce2c000 nid=0x2ce waiting on condition [0x00007f83f7ce6000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f0bdbc90> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:629)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:123)
at org.apache.servicecomb.pack.contract.grpc.TxEventServiceGrpc$TxEventServiceBlockingStub.onTxEvent(TxEventServiceGrpc.java:273)
at org.apache.servicecomb.pack.omega.connector.grpc.saga.GrpcSagaClientMessageSender.send(GrpcSagaClientMessageSender.java:90)
at org.apache.servicecomb.pack.omega.connector.grpc.saga.SagaLoadBalanceSender$1.apply(SagaLoadBalanceSender.java:45)
at org.apache.servicecomb.pack.omega.connector.grpc.saga.SagaLoadBalanceSender$1.apply(SagaLoadBalanceSender.java:42)
at org.apache.servicecomb.pack.omega.connector.grpc.core.LoadBalanceSenderAdapter.doGrpcSend(LoadBalanceSenderAdapter.java:54)
at org.apache.servicecomb.pack.omega.connector.grpc.saga.SagaLoadBalanceSender.send(SagaLoadBalanceSender.java:42)
at org.apache.servicecomb.pack.omega.transaction.SagaStartAnnotationProcessor.postIntercept(SagaStartAnnotationProcessor.java:44)
at org.apache.servicecomb.pack.omega.transaction.wrapper.SagaStartAnnotationProcessorWrapper.apply(SagaStartAnnotationProcessorWrapper.java:49)
at org.apache.servicecomb.pack.omega.transaction.SagaStartAspect.advise(SagaStartAspect.java:50)
at sun.reflect.GeneratedMethodAccessor563.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:644)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:633)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:175)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
at com.shuyun.kylin.offer.service.GrantService$$EnhancerBySpringCGLIB$$319ee6cd.grant(<generated>)
at com.shuyun.kylin.offer.service.GrantService.grant$default(GrantService.kt:156)
at com.shuyun.kylin.offer.service.GrantService.dispatchGrant(GrantService.kt:120)
at com.shuyun.kylin.offer.service.GrantService$$FastClassBySpringCGLIB$$f07a8e5e.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:749)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688)
at com.shuyun.kylin.offer.service.GrantService$$EnhancerBySpringCGLIB$$319ee6cd.dispatchGrant(<generated>)
at com.shuyun.kylin.offer.kafka.EventReceiver.grant(EventReceiver.kt:59)
at sun.reflect.GeneratedMethodAccessor561.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:171)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:120)
at org.springframework.cloud.stream.binding.StreamListenerMessageHandler.handleRequestMessage(StreamListenerMessageHandler.java:55)
at org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleMessageInternal(AbstractReplyProducingMessageHandler.java:123)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:162)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:115)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:132)
at org.springframework.integration.dispatcher.UnicastingDispatcher.dispatch(UnicastingDispatcher.java:105)
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:73)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:453)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:401)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:187)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:166)
at org.springframework.messaging.core.GenericMessagingTemplate.doSend(GenericMessagingTemplate.java:47)
at org.springframework.messaging.core.AbstractMessageSendingTemplate.send(AbstractMessageSendingTemplate.java:109)
at org.springframework.integration.endpoint.MessageProducerSupport.sendMessage(MessageProducerSupport.java:205)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.sendMessageIfAny(KafkaMessageDrivenChannelAdapter.java:369)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter.access$400(KafkaMessageDrivenChannelAdapter.java:74)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:431)
at org.springframework.integration.kafka.inbound.KafkaMessageDrivenChannelAdapter$IntegrationRecordMessageListener.onMessage(KafkaMessageDrivenChannelAdapter.java:402)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.lambda$onMessage$0(RetryingMessageListenerAdapter.java:120)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter$$Lambda$1434/1504139066.doWithRetry(Unknown Source)
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287)
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:211)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:114)
at org.springframework.kafka.listener.adapter.RetryingMessageListenerAdapter.onMessage(RetryingMessageListenerAdapter.java:40)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:1263)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:1256)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:1217)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:1198)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:1118)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:933)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:749)
at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:698)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Locked ownable synchronizers:
- None
alpha端相应时间段没有报错日志
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] lostpillow commented on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
lostpillow commented on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-611834438
> 看了一下stack trace，这个看起来像是 TxEventServiceBlockingStub. 在等待消息应答。
>
> ```
> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:629)
> at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:123)
> at org.apache.servicecomb.pack.contract.grpc.TxEventServiceGrpc$TxEventServiceBlockingStub.onTxEvent(TxEventServiceGrpc.java:273)
> ```
>
> 因为TxEventServiceBlockingStub是一个同步调用， 是需要等待Alpha的回复的消息。 建议检查一下Alpha运行的情况，看看Alpha是否还在工作。
alpha端是正常的，因为别的请求还在正常处理。
omega端下午两点到三点发出了一些请求，这些请求一直没有被成功处理，处理失败后丢回了kafka不断重试，一直到晚上都没有进度。而后续的其他请求都能正常执行。给你的线程信息是晚上六点重试之前失败请求的。
alpha端在开始出现这个问题的时候，反复出现过以下堆栈，但是报了一段时间后到三点五十就不报了，不知道有没有关系：
2020-04-09 15:50:59.064 WARN 513e292cd20b --- [pool-2-thread-1] o.e.p.s./.5.0.jar_default :
Local Exception Stack:
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.7.1.v20171221-bd47e8f): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "command_eventid_key"
详细：Key (eventid)=(6049337) already exists.
Error Code: 0
Call: INSERT INTO Command (COMPENSATIONMETHOD, EVENTID, GLOBALTXID, INSTANCEID, LASTMODIFIED, LOCALTXID, PARENTTXID, PAYLOADS, SERVICENAME, STATUS, VERSION) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
bind => [11 parameters bound]
Query: InsertObjectQuery(Command{eventId=6049337, serviceName='offer-service', instanceId='offer-service-172.17.0.7', globalTxId='832af507-094c-4d5e-abe4-1a47b0822778', localTxId='cd184060-a1a6-4366-aac8-106bb7bf5d90', parentTxId='832af507-094c-4d5e-abe4-1a47b0822778', compensationMethod='public void com.shuyun.kylin.offer.service.GrantDataService.rollBackAddTotalCountCache(java.lang.String,java.lang.String,long)'})
at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:340)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processExceptionForCommError(DatabaseAccessor.java:1645)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:903)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeNoSelect(DatabaseAccessor.java:967)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:637)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:564)
at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:2093)
at org.eclipse.persistence.sessions.server.ClientSession.executeCall(ClientSession.java:309)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:270)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:256)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.insertObject(DatasourceCallQueryMechanism.java:405)
at org.eclipse.persistence.internal.queries.StatementQueryMechanism.insertObject(StatementQueryMechanism.java:165)
at org.eclipse.persistence.internal.queries.StatementQueryMechanism.insertObject(StatementQueryMechanism.java:180)
at org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.insertObjectForWrite(DatabaseQueryMechanism.java:502)
at org.eclipse.persistence.queries.InsertObjectQuery.executeCommit(InsertObjectQuery.java:80)
at org.eclipse.persistence.queries.InsertObjectQuery.executeCommitWithChangeSet(InsertObjectQuery.java:90)
at org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.executeWriteWithChangeSet(DatabaseQueryMechanism.java:314)
at org.eclipse.persistence.queries.WriteObjectQuery.executeDatabaseQuery(WriteObjectQuery.java:58)
at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:911)
at org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:810)
at org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWorkObjectLevelModifyQuery(ObjectLevelModifyQuery.java:108)
at org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWork(ObjectLevelModifyQuery.java:85)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2979)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1892)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1874)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1824)
at org.eclipse.persistence.internal.sessions.CommitManager.commitNewObjectsForClassWithChangeSet(CommitManager.java:227)
at org.eclipse.persistence.internal.sessions.CommitManager.commitAllObjectsWithChangeSet(CommitManager.java:126)
at org.eclipse.persistence.internal.sessions.AbstractSession.writeAllObjectsWithChangeSet(AbstractSession.java:4384)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabase(UnitOfWorkImpl.java:1491)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabaseWithChangeSet(UnitOfWorkImpl.java:1581)
at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.commitRootUnitOfWork(RepeatableWriteUnitOfWork.java:278)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitAndResume(UnitOfWorkImpl.java:1218)
at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.commit(EntityTransactionImpl.java:134)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:532)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:746)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:534)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:305)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:139)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:138)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:93)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:61)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy145.save(Unknown Source)
at org.apache.servicecomb.pack.alpha.server.SpringCommandRepository.saveCompensationCommands(SpringCommandRepository.java:63)
at org.apache.servicecomb.pack.alpha.server.SpringCommandRepository$$FastClassBySpringCGLIB$$23e94f54.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:684)
at org.apache.servicecomb.pack.alpha.server.SpringCommandRepository$$EnhancerBySpringCGLIB$$124e5f0f.saveCompensationCommands(<generated>)
at org.apache.servicecomb.pack.alpha.core.EventScanner.lambda$saveUncompensatedEventsToCommands$2(EventScanner.java:118)
at java.util.Vector.forEach(Vector.java:1249)
at org.apache.servicecomb.pack.alpha.core.EventScanner.saveUncompensatedEventsToCommands(EventScanner.java:115)
at org.apache.servicecomb.pack.alpha.core.EventScanner.lambda$pollEvents$0(EventScanner.java:89)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "command_eventid_key"
详细：Key (eventid)=(6049337) already exists.
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:120)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:895)
... 64 more
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] coolbeevip commented on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
coolbeevip commented on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-613762819
What version are you using?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] WillemJiang commented on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
WillemJiang commented on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-612580236
看样子你的服务好像是从karaf中取数据，然后在调用后台的服务。
可否给出更详细的示例代码（读取karaf数据，调用后台服务），这样可以让我们帮助你分析为什么Omega会一直在那死等。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] lostpillow edited a comment on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
lostpillow edited a comment on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-611839685
> 看了一下stack trace，这个看起来像是 TxEventServiceBlockingStub. 在等待消息应答。
>
> ```
> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:629)
> at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:123)
> at org.apache.servicecomb.pack.contract.grpc.TxEventServiceGrpc$TxEventServiceBlockingStub.onTxEvent(TxEventServiceGrpc.java:273)
> ```
>
> 因为TxEventServiceBlockingStub是一个同步调用， 是需要等待Alpha的回复的消息。 建议检查一下Alpha运行的情况，看看Alpha是否还在工作。
刚刚重启了omega端，那些堆积的错误请求又开始处理了，所以alpha端肯定没挂
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] lostpillow commented on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
lostpillow commented on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-614386936
> What version are you using?
0.5.0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] lostpillow commented on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
lostpillow commented on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-611839685
> 看了一下stack trace，这个看起来像是 TxEventServiceBlockingStub. 在等待消息应答。
>
> ```
> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
> at io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:629)
> at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:123)
> at org.apache.servicecomb.pack.contract.grpc.TxEventServiceGrpc$TxEventServiceBlockingStub.onTxEvent(TxEventServiceGrpc.java:273)
> ```
>
> 因为TxEventServiceBlockingStub是一个同步调用， 是需要等待Alpha的回复的消息。 建议检查一下Alpha运行的情况，看看Alpha是否还在工作。
刚刚重启了omega端，那些堆积的错误请求又开始处理了
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] WillemJiang commented on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
WillemJiang commented on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-611804769
看了一下stack trace，这个看起来像是 TxEventServiceBlockingStub. 在等待消息应答。
```
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:629)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:123)
at org.apache.servicecomb.pack.contract.grpc.TxEventServiceGrpc$TxEventServiceBlockingStub.onTxEvent(TxEventServiceGrpc.java:273)
```
因为TxEventServiceBlockingStub是一个同步调用， 是需要等待Alpha的回复的消息。 建议检查一下Alpha运行的情况，看看Alpha是否还在工作。
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] lostpillow commented on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
lostpillow commented on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-612772694
> 看样子你的服务好像是从karaf中取数据，然后在调用后台的服务。
> 可否给出更详细的示例代码（读取karaf数据，调用后台服务），这样可以让我们帮助你分析为什么Omega会一直在那死等。
代码不是java，是一个jvm语言，不妨碍阅读：
Kafka消息获取使用了spring-cloud-stream：
```
@StreamListener(KafkaSink.GRANT_INPUT)
fun grant(@Payload event: GrantCommand) {
grantService.grant(event)
}
```
GrantService的grant方法：
```
@SagaStart(timeout = 30)
fun grant(cmd: GrantCommand, prior: Boolean = false): OfferGrantBatchCreatedEvent? {
redisLocker.getReadLock("offer.project.readWrite.${cmd.projectEntityId}").withLock {//获取redisson自带的读写锁的读锁，并调用lock()方法
logger.debug("start processing ${cmd.batchId}")//这一行日志打了
//处理业务，调用多个带有@Compensable注解的方法，整个业务可能会超过sagastart注解的超时时间，带有@Compensable注解的方法可能在别的加锁代码段中
}
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] WillemJiang commented on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
WillemJiang commented on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-612793136
> GrantService的grant方法：
>
> ```
> @SagaStart(timeout = 30)
> fun grant(cmd: GrantCommand, prior: Boolean = false): OfferGrantBatchCreatedEvent? {
> redisLocker.getReadLock("offer.project.readWrite.${cmd.projectEntityId}").withLock {//获取redisson自带的读写锁的读锁，并调用lock()方法
> logger.debug("start processing ${cmd.batchId}")//这一行日志打了
> //处理业务，调用多个带有@Compensable注解的方法，整个业务可能会超过sagastart注解的超时时间，带有@Compensable注解的方法可能在别的加锁代码段中
> }
>
> }
> ```
我们的测试用例没有涉及到你的这个场景。
问个问题， 如果调用过程出现问题， 这个锁如何释放？
还有就是业务超时，锁会不会自动释放？
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [servicecomb-pack] lostpillow commented on issue #649: omega端线程等待
Posted by GitBox <gi...@apache.org>.
lostpillow commented on issue #649: omega端线程等待
URL: https://github.com/apache/servicecomb-pack/issues/649#issuecomment-612828853
> > GrantService的grant方法：
> > ```
> > @SagaStart(timeout = 30)
> > fun grant(cmd: GrantCommand, prior: Boolean = false): OfferGrantBatchCreatedEvent? {
> > redisLocker.getReadLock("offer.project.readWrite.${cmd.projectEntityId}").withLock {//获取redisson自带的读写锁的读锁，并调用lock()方法
> > logger.debug("start processing ${cmd.batchId}")//这一行日志打了
> > //处理业务，调用多个带有@Compensable注解的方法，整个业务可能会超过sagastart注解的超时时间，带有@Compensable注解的方法可能在别的加锁代码段中
> > }
> >
> > }
> > ```
>
> 我们的测试用例没有涉及到你的这个场景。
> 问个问题， 如果调用过程出现问题， 这个锁如何释放？
> 还有就是业务超时，锁会不会自动释放？
```
@SagaStart(timeout = 30)
fun demo(){
lock1.lock()//此处不会超时
try{
lock2.lock()//30s内可能拿不到锁
try{
//业务代码，其中调用了带有@Compensable注解的方法
}finally{
lock2.unlock()
}
//业务代码，其中调用了带有@Compensable注解的方法
}finally{
lock1.unlock()
}
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services