You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/05/08 18:22:07 UTC
[GitHub] [pulsar] xavipen opened a new issue #6924: Exception creating producer on unit test using PulsarStandalone
xavipen opened a new issue #6924:
URL: https://github.com/apache/pulsar/issues/6924
**Describe the bug**
Running a simple unit test to send and received messages using PulsarStandalone running on a Windows 10 machine works with pulsar 2.4.1 and the same code with pulsar 2.5.1 fails. In Pulsar 2.5.1 the producer fails to create the topics with the following error:
> 2020-05-08 19:46:57,076 OrderedExecutor-6-0] WARN areEnsemblePlacementPolicyImpl Failed to find 1 bookies : excludeBookies [<Bookie:127.0.0.1:3181>], allBookies [<Bookie:127.0.0.1:3181>].
2020-05-08 19:46:57,083 OrderedExecutor-6-0] WARN areEnsemblePlacementPolicyImpl Failed to find 1 bookies : excludeBookies [<Bookie:127.0.0.1:3181>], allBookies [<Bookie:127.0.0.1:3181>].
2020-05-08 19:46:57,083 OrderedExecutor-6-0] ERROR >o.a.b.client.LedgerCreateOp Not enough bookies to create ledger
2020-05-08 19:46:57,086 OrderedExecutor-6-0] WARN >o.a.p.b.service.BrokerService Failed to create topic persistent://public/default/hello
org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available
2020-05-08 19:46:57,087 OrderedExecutor-6-0] ERROR >o.a.p.b.service.ServerCnx [/127.0.0.1:64889] Failed to create topic persistent://public/default/hello
java.util.concurrent.CompletionException: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:331)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:346)
at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:632)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
at org.apache.pulsar.broker.service.BrokerService$2.openLedgerFailed(BrokerService.java:930)
at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.lambda$asyncOpen$8(ManagedLedgerFactoryImpl.java:343)
at java.base/java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:986)
at java.base/java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:970)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2088)
at org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl$2.initializeFailed(ManagedLedgerFactoryImpl.java:336)
at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.lambda$null$1(ManagedLedgerImpl.java:404)
at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32)
at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.pulsar.broker.service.BrokerServiceException$PersistenceException: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available
... 14 common frames omitted
Caused by: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available
2020-05-08 19:46:57,097 lsar-client-io-55-1] WARN >o.a.p.client.impl.ClientCnx [id: 0x1b08bc17, L:/127.0.0.1:64889 - R:localhost/127.0.0.1:6650] Received error from server: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available
2020-05-08 19:46:57,098 lsar-client-io-55-1] ERROR >o.a.p.c.impl.ProducerImpl [hello] [null] Failed to create producer: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available
2020-05-08 19:46:57,098 lsar-client-io-55-1] WARN >o.a.p.c.i.ConnectionHandler [hello] [null] Could not get connection to broker: org.apache.bookkeeper.mledger.ManagedLedgerException: Not enough non-faulty bookies available -- Will try again in 0.1 s
**To Reproduce**
Steps to reproduce the behavior:
1. Run the following test (I use TestNg) on a windows 10 machine.
```
@Test
public void testSendReceive() throws PulsarClientException {
File configFile = new File("empty.conf");
try {
Files.touch(configFile);
} catch (IOException e) {
e.printStackTrace();
}
PulsarStandalone broker;
broker = PulsarStandaloneBuilder.instance().withOnlyBroker(false).build();
broker.getConfig().setWebServicePort(Optional.of(8099));
broker.getConfig().setWebSocketServiceEnabled(false);
broker.setNoStreamStorage(true); // NEEDED FOR WINDOWS - otherwise fails at symlink creation
broker.setConfigFile(configFile.getAbsolutePath());
try {
broker.start();
} catch (Exception e) {
e.printStackTrace();
}
final PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://localhost:6650")
.build();
final Producer<String> producer = client.newProducer(Schema.STRING)
.topic(TOPIC)
.enableBatching(false)
.create();
final Consumer<String> consumer = client.newConsumer(Schema.STRING)
.topic(TOPIC)
.subscriptionName("test-subs-1")
.ackTimeout(10, TimeUnit.SECONDS)
.subscriptionType(SubscriptionType.Exclusive)
.subscribe();
for (int i = 1; i <= NUM_OF_MESSAGES; ++i) {
producer.send("Hello_" + i);
}
for (int i = 1; i <= NUM_OF_MESSAGES; ++i) {
final Message<String> message = consumer.receive(1, TimeUnit.SECONDS);
log.info("Message received : {}", message.getValue());
assertThat(message.getValue()).isEqualTo("Hello_" + i);
}
producer.close();
consumer.close();
client.close();
broker.close();
}
```
**Expected behavior**
The same code in Pulsar 2.4.1 works. It fails on 2.5.0 and 2.5.1
**Screenshots**
N/A
**Desktop (please complete the following information):**
- OS: Windows 10
**Additional context**
Add any other context about the problem here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] xavipen commented on issue #6924: Exception creating producer on unit test using PulsarStandalone
Posted by GitBox <gi...@apache.org>.
xavipen commented on issue #6924:
URL: https://github.com/apache/pulsar/issues/6924#issuecomment-626576287
That does not answer why this works in the same machine with same disk space with pulsar version 2.4.1...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] horsteff commented on issue #6924: Exception creating producer on unit test using PulsarStandalone
Posted by GitBox <gi...@apache.org>.
horsteff commented on issue #6924:
URL: https://github.com/apache/pulsar/issues/6924#issuecomment-656214989
@xavipen Recently I discovered that the default value for `managedLedgerDefaultEnsembleSize` in the ServiceConfiguration class was changed from 1 to 2 with version 2.5.0 (together with the values for `managedLedgerDefaultWriteQuorum` and `managedLedgerDefaultAckQuorum`), but the default number of bookies stays at 1. So the ledger needs two bookies by default but gets only one, which leads to the "Not enough non-faulty bookies available" error.
To avoid this problem you can either change the values of `managedLedgerDefaultEnsembleSize`, `managedLedgerDefaultWriteQuorum` and `managedLedgerDefaultAckQuorum` to 1 (with `broker.getConfig().set...`, using the config file does not work) or set the number of bookies to 2 (with `PulsarStandaloneBuilder.instance().withNumOfBk` or `broker.setNumOfBk`). Either way works for me.
@sijie Should the default `numOfBk` in `PulsarStandalone` be changed to 2 according to default `managedLedgerDefaultEnsembleSize` to avoid this problem? I think, it doesn't make sense to have less bookies than required for a ledger.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] xavipen commented on issue #6924: Exception creating producer on unit test using PulsarStandalone
Posted by GitBox <gi...@apache.org>.
xavipen commented on issue #6924:
URL: https://github.com/apache/pulsar/issues/6924#issuecomment-665308051
Thanks for the update
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [pulsar] sijie commented on issue #6924: Exception creating producer on unit test using PulsarStandalone
Posted by GitBox <gi...@apache.org>.
sijie commented on issue #6924:
URL: https://github.com/apache/pulsar/issues/6924#issuecomment-626427735
@xavipen The error message "Not enough non-faulty bookies available" indicates the bookies are not writable. It typically means that your disk usage is larger than 95% so the bookies are forced to run in readonly mode.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org