You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/09/04 09:27:52 UTC
[GitHub] [pulsar-dotpulsar] usaguerrilla opened a new issue #55: Producer Executor object gets killed because exception is thrown which isn't handled by Handle method.
usaguerrilla opened a new issue #55:
URL: https://github.com/apache/pulsar-dotpulsar/issues/55
Producer Executor object gets killed because exception is thrown which isn't handled by Handle method. This is happening when server gets disconnected. After it happens producer won't reconnect after service become available again.
Exception:
```
{Name = "PersistenceException" FullName = "DotPulsar.Exceptions.PersistenceException"}
```
Stack trace:
```
DotPulsar.dll!DotPulsar.Internal.Executor.Execute<DotPulsar.Internal.PulsarApi.CommandSendReceipt>(System.Func<System.Threading.Tasks.Task<DotPulsar.Internal.PulsarApi.CommandSendReceipt>> func, System.Threading.CancellationToken cancellationToken) Line 123 C#
[Resuming Async Method]
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<DotPulsar.Internal.PulsarApi.CommandSendReceipt>.AsyncStateMachineBox<DotPulsar.Internal.Executor.<Execute>d__8<DotPulsar.Internal.PulsarApi.CommandSendReceipt>>.MoveNext(System.Threading.Thread threadPoolThread) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Unknown
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Unknown
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Unknown
System.Private.CoreLib.dll!System.Threading.Tasks.Task.FinishSlow(bool userDelegateExecute) Unknown
System.Private.CoreLib.dll!System.Threading.Tasks.Task.TrySetException(object exceptionObject) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<DotPulsar.Internal.PulsarApi.CommandSendReceipt>.SetException(System.Exception exception) Unknown
[Completed] DotPulsar.dll!DotPulsar.Internal.ProducerChannel.SendPackage(DotPulsar.Internal.PulsarApi.MessageMetadata metadata, System.Buffers.ReadOnlySequence<byte> payload, System.Threading.CancellationToken cancellationToken) Line 109 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<DotPulsar.Internal.PulsarApi.CommandSendReceipt>.AsyncStateMachineBox<DotPulsar.Internal.ProducerChannel.<SendPackage>d__9>.MoveNext(System.Threading.Thread threadPoolThread) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Unknown
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action action, bool allowInlining) Unknown
System.Private.CoreLib.dll!System.Threading.Tasks.Task.RunContinuations(object continuationObject) Unknown
System.Private.CoreLib.dll!System.Threading.Tasks.Task<System.__Canon>.TrySetResult(System.__Canon result) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<DotPulsar.Internal.PulsarApi.BaseCommand>.SetResult(DotPulsar.Internal.PulsarApi.BaseCommand result) Unknown
[Completed] DotPulsar.dll!DotPulsar.Internal.Connection.Send(DotPulsar.Internal.SendPackage command, System.Threading.CancellationToken cancellationToken) Line 194 C#
System.Private.CoreLib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder<DotPulsar.Internal.PulsarApi.BaseCommand>.AsyncStateMachineBox<DotPulsar.Internal.Connection.<Send>d__23>.MoveNext(System.Threading.Thread threadPoolThread) Unknown
System.Private.CoreLib.dll!System.Runtime.CompilerServices.TaskAwaiter.OutputWaitEtwEvents.AnonymousMethod__12_0(System.Action innerContinuation, System.Threading.Tasks.Task innerTask) Unknown
System.Private.CoreLib.dll!System.Threading.Tasks.AwaitTaskContinuation.System.Threading.IThreadPoolWorkItem.Execute() Unknown
System.Private.CoreLib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Unknown
[Async Call Stack]
> [Async] DotPulsar.dll!DotPulsar.Internal.Producer.Send(DotPulsar.MessageMetadata metadata, System.Buffers.ReadOnlySequence<byte> data, System.Threading.CancellationToken cancellationToken) Line 117 C#
[Async] DotPulsar.dll!DotPulsar.Internal.MessageBuilder.Send(System.ReadOnlyMemory<byte> data, System.Threading.CancellationToken cancellationToken) Line 89 C#
[Async] PulsarTest.dll!PulsarTest.DotPulsarTest.Send(DotPulsar.Abstractions.IProducer publisher, int messageSize, long index) Line 199 C#
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] usaguerrilla commented on issue #55: Producer Executor object gets killed because exception is thrown which isn't handled by Handle method.
Posted by GitBox <gi...@apache.org>.
usaguerrilla commented on issue #55:
URL: https://github.com/apache/pulsar-dotpulsar/issues/55#issuecomment-687087053
Nope. Need to look into this a bit more. I pasted wrong stack traces. But I do see producers and consumers not reconnecting (some do some do not) after server shut down
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] usaguerrilla closed issue #55: Producer Executor object gets killed because exception is thrown which isn't handled by Handle method.
Posted by GitBox <gi...@apache.org>.
usaguerrilla closed issue #55:
URL: https://github.com/apache/pulsar-dotpulsar/issues/55
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #55: Producer Executor object gets killed because exception is thrown which isn't handled by Handle method.
Posted by GitBox <gi...@apache.org>.
blankensteiner commented on issue #55:
URL: https://github.com/apache/pulsar-dotpulsar/issues/55#issuecomment-687081012
Closed by accident?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] usaguerrilla commented on issue #55: Producer Executor object gets killed because exception is thrown which isn't handled by Handle method.
Posted by GitBox <gi...@apache.org>.
usaguerrilla commented on issue #55:
URL: https://github.com/apache/pulsar-dotpulsar/issues/55#issuecomment-687058121
Another exception which doesn't seem to be caught:
```
System.IO.IOException: Unable to read data from the transport connection: An established connection was aborted by the software in your host machine..
---> System.Net.Sockets.SocketException (10053): An established connection was aborted by the software in your host machine.
--- End of inner exception stack trace ---
at DotPulsar.Internal.PulsarStream.Send(ReadOnlySequence`1 sequence) in C:\Projects.Pulsar\dotpulsar.mp\src\DotPulsar\Internal\PulsarStream.cs:line 61
at DotPulsar.Internal.Connection.Send(SendPackage command, CancellationToken cancellationToken) in C:\Projects.Pulsar\dotpulsar.mp\src\DotPulsar\Internal\Connection.cs:line 190
at DotPulsar.Internal.ProducerChannel.SendPackage(MessageMetadata metadata, ReadOnlySequence`1 payload, CancellationToken cancellationToken) in C:\Projects.Pulsar\dotpulsar.mp\src\DotPulsar\Internal\ProducerChannel.cs:line 102
at DotPulsar.Internal.Executor.Execute[TResult](Func`1 func, CancellationToken cancellationToken) in C:\Projects.Pulsar\dotpulsar.mp\src\DotPulsar\Internal\Executor.cs:line 130
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] usaguerrilla commented on issue #55: Producer Executor object gets killed because exception is thrown which isn't handled by Handle method.
Posted by GitBox <gi...@apache.org>.
usaguerrilla commented on issue #55:
URL: https://github.com/apache/pulsar-dotpulsar/issues/55#issuecomment-687079634
Will test a bit more. For some reason pulsar service won't start after shut down (unless I reboot machine).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] usaguerrilla commented on issue #55: Producer Executor object gets killed because exception is thrown which isn't handled by Handle method.
Posted by GitBox <gi...@apache.org>.
usaguerrilla commented on issue #55:
URL: https://github.com/apache/pulsar-dotpulsar/issues/55#issuecomment-687048396
```
DotPulsar.Exceptions.PersistenceException: fenced
at DotPulsar.Internal.Extensions.CommandExtensions.Throw(ServerError error, String message) in C:\Projects.Pulsar\dotpulsar.mp\src\DotPulsar\Internal\Extensions\CommandExtensions.cs:line 50
at DotPulsar.Internal.Extensions.CommandExtensions.Throw(CommandSendError command) in C:\Projects.Pulsar\dotpulsar.mp\src\DotPulsar\Internal\Extensions\CommandExtensions.cs:line 38
at DotPulsar.Internal.Extensions.CommandExtensions.Expect(BaseCommand command, Type type) in C:\Projects.Pulsar\dotpulsar.mp\src\DotPulsar\Internal\Extensions\CommandExtensions.cs:line 32
at DotPulsar.Internal.ProducerChannel.SendPackage(MessageMetadata metadata, ReadOnlySequence`1 payload, CancellationToken cancellationToken) in C:\Projects.Pulsar\dotpulsar.mp\src\DotPulsar\Internal\ProducerChannel.cs:line 103
at DotPulsar.Internal.Executor.Execute[TResult](Func`1 func, CancellationToken cancellationToken) in C:\Projects.Pulsar\dotpulsar.mp\src\DotPulsar\Internal\Executor.cs:line 130
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar-dotpulsar] usaguerrilla commented on issue #55: Producer Executor object gets killed because exception is thrown which isn't handled by Handle method.
Posted by GitBox <gi...@apache.org>.
usaguerrilla commented on issue #55:
URL: https://github.com/apache/pulsar-dotpulsar/issues/55#issuecomment-687109611
```
On 2020-09-04 12:21:21: Publishing to persistent://public/default/timtestparted...
persistent://public/default/timtestparted-partition-7 -> Connected
persistent://public/default/timtestparted-partition-3 -> Connected
persistent://public/default/timtestparted-partition-1 -> Connected
persistent://public/default/timtestparted-partition-4 -> Connected
persistent://public/default/timtestparted-partition-5 -> Connected
persistent://public/default/timtestparted-partition-0 -> Connected
persistent://public/default/timtestparted-partition-6 -> Connected
persistent://public/default/timtestparted-partition-2 -> Connected
On 2020-09-04 12:21:29: Sent 10000 messages to persistent://public/default/timtestparted so far...
persistent://public/default/timtestparted-partition-1 -> ClosedByServer
persistent://public/default/timtestparted-partition-5 -> ClosedByServer
persistent://public/default/timtestparted-partition-6 -> ClosedByServer
persistent://public/default/timtestparted-partition-2 -> ClosedByServer
persistent://public/default/timtestparted-partition-0 -> ClosedByServer
Connection disposed
-->ExecutorState(persistent://public/default/timtestparted-partition-6) == ExecutorState.Faulted
-->ExecutorState(persistent://public/default/timtestparted-partition-0) == ExecutorState.Faulted
-->_producer(persistent://public/default/timtestparted-partition-6).IsFinalState(Faulted)
persistent://public/default/timtestparted-partition-4 -> ClosedByServer
-->ExecutorState(persistent://public/default/timtestparted-partition-5) == ExecutorState.Faulted
persistent://public/default/timtestparted-partition-7 -> ClosedByServer
-->_producer(persistent://public/default/timtestparted-partition-5).IsFinalState(Faulted)
persistent://public/default/timtestparted-partition-3 -> ClosedByServer
-->ExecutorState(persistent://public/default/timtestparted-partition-2) == ExecutorState.Faulted
-->ExecutorState(persistent://public/default/timtestparted-partition-7) == ExecutorState.Faulted
Connection disposed
-->ExecutorState(persistent://public/default/timtestparted-partition-4) == ExecutorState.Faulted
```
Upon closing connection some producers / consumers end up in faulted state and never try to reconnect.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org