You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2019/10/11 15:56:11 UTC
[GitHub] [pulsar] liangyuanpeng commented on issue #5262: pulsar SQL
(Presto) can only query data less than 100000
liangyuanpeng commented on issue #5262: pulsar SQL (Presto) can only query data less than 100000
URL: https://github.com/apache/pulsar/issues/5262#issuecomment-541121599
There are may be other reasons.
I can query 100000+ data
``pulsar-admin persistent stats-internal`` result
```
[root@localhost apache-pulsar-2.4.1-normal]# bin/pulsar-admin persistent stats-internal persistent://public/default/topic-3
{
"entriesAddedCounter" : 102062,
"numberOfEntries" : 102062,
"totalSize" : 7623081,
"currentLedgerEntries" : 102062,
"currentLedgerSize" : 7623081,
"lastLedgerCreatedTimestamp" : "2019-10-11T04:23:27.017+08:00",
"waitingCursorsCount" : 0,
"pendingAddEntriesCount" : 0,
"lastConfirmedEntry" : "6:102061",
"state" : "LedgerOpened",
"ledgers" : [ {
"ledgerId" : 6,
"entries" : 0,
"size" : 0,
"offloaded" : false
} ],
"cursors" : { }
}
```
``sql run`` result
```
presto> SELECT COUNT(1) FROM pulsar."public/default"."topic-3";
_col0
--------
102307
(1 row)
Query 20191010_202608_00012_zu36h, FINISHED, 1 node
Splits: 19 total, 19 done (100.00%)
1:10 [102K rows, 7.26MB] [1.47K rows/s, 107KB/s]
```
and my test code
```
Producer<byte[]> producer = client.newProducer().topic("persistent://public/default/topic-3").create();
producer.sendAsync("My message".getBytes());
ExecutorService executorService = Executors.newFixedThreadPool(100);
int count = 102400;
for(int i=0;i<count;i++){
executorService.execute(new Runnable() {
@Override
public void run() {
String msg = ("My message"+System.currentTimeMillis()+ RandomUtils.nextInt(1,10000));
System.out.println(msg);
try {
producer.send(msg.getBytes());
} catch (PulsarClientException e) {
System.out.println(e.getMessage());
}
}
});
}
```
pulsar 2.4.1
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services