You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by lonesomerain <lo...@163.com> on 2023/05/22 11:52:24 UTC
Ignite thin client continuous query listener cannot listen to all events
Hi,
I have a question while using ignite 2.15.0
Problem scenario:
Start the Ignite server of one node, start one thin client and create a continuous query listener, and then use 50 threads to add 500 data to the cache concurrently.
Problem phenomenon:
Through the information printed on the listener, it was found that the number of events listened to each time varies, possibly 496, 499 or 500...
Test Code:
public class StartServer {
public static void main(String[] args) {
Ignite ignite = Ignition.start();
}
}
public class StartThinClient {
public static void main(String[] args) throws InterruptedException {
String addr = "127.0.0.1:10800";
int threadNmu = 50;
ClientConfiguration clientConfiguration = new ClientConfiguration();
clientConfiguration.setAddresses(addr);
IgniteClient client1 = Ignition.startClient(clientConfiguration);
ClientCache<Object, Object> cache1 = client1.getOrCreateCache("test");
ContinuousQuery<Object, Object> query = new ContinuousQuery<>();
query.setLocalListener(new CacheEntryUpdatedListener<Object, Object>() {
@Override
public void onUpdated(Iterable<CacheEntryEvent<?, ?>> cacheEntryEvents) throws CacheEntryListenerException {
Iterator<CacheEntryEvent<?, ?>> iterator = cacheEntryEvents.iterator();
while (iterator.hasNext()) {
CacheEntryEvent<?, ?> next = iterator.next();
System.out.println("----" + next.getKey());
}
}
});
cache1.query(query);
IgniteClient client2 = Ignition.startClient(clientConfiguration);
ClientCache<Object, Object> cache2 = client2.cache("test");
Thread[] threads = new Thread[threadNmu];
for (int i = 0; i < threads.length; ++i) {
threads[i] = new Thread(new OperationInsert(cache2, i, 500, threadNmu));
}
for (int i = 0; i < threads.length; ++i) {
threads[i].start();
}
for (Thread thread : threads) {
thread.join();
}
Thread.sleep(60000);
}
static class OperationInsert implements Runnable {
private ClientCache<Object, Object> cache;
private int k;
private Integer test_rows;
private Integer thread_cnt;
public OperationInsert(ClientCache<Object, Object> cache, int k, Integer test_rows, Integer thread_cnt) {
this.cache = cache;
this.k = k;
this.test_rows = test_rows;
this.thread_cnt = thread_cnt;
}
@Override
public void run() {
for (int i = 1000000 + (test_rows/thread_cnt) * k; i < 1000000 + (test_rows/thread_cnt) * (k + 1); i++) {
cache.put("" + i, "aaa");
}
}
}
}
Version:
The testing program uses Ignite version 2.15.0
I attempted to insert data using one thread and did not observe any event loss. In addition, I also attempted an Ignite cluster with two or three nodes, which can still listen to all 500 events even when inserting data using multiple threads. May I ask if this issue only occurs at a single node? Are there any good solutions?
Re: Ignite thin client continuous query listener cannot listen to all events
Posted by Alex Plehanov <pl...@gmail.com>.
Hello,
Looks like the bug is related to server-side continuous queries
functionality. I've tried to reproduce it using thick-client and got
the same results, in rare cases events are lost.
чт, 25 мая 2023 г. в 06:48, Pavel Tupitsyn <pt...@apache.org>:
>
> Thank you for the bug report, I will have a look.
>
> On Thu, May 25, 2023 at 5:10 AM LonesomeRain <lo...@163.com> wrote:
>>
>> Hi, Jeremy McMillan
>>
>>
>>
>> I have created a Jira about this bug.
>>
>>
>>
>> https://issues.apache.org/jira/browse/IGNITE-19561
>>
>>
>>
>> By executing these two test codes in the description, it is easy to reproduce the problem.
>>
>>
>>
>> What should I do next? Continuously following this Jira?
>>
>>
>>
>>
>>
>> 发件人: Jeremy McMillan
>> 发送时间: 2023年5月24日 23:24
>> 收件人: user@ignite.apache.org
>> 主题: Re: Ignite thin client continuous query listener cannot listen to all events
>>
>>
>>
>> Thanks for bringing this up!
>>
>>
>>
>> https://ignite.apache.org/docs/latest/key-value-api/continuous-queries#events-delivery-guarantees
>>
>>
>>
>> This sounds like you may have found a bug, but the details you've provided are not sufficient to help others recreate and observe it for themselves, and this effort needs to be recorded in a ticket. Would you be able to sign up for a Jira account and detail steps to reproduce this behavior?
>>
>> You may also want to research this:
>> https://issues.apache.org/jira/browse/IGNITE-8035
>>
>>
>>
>> On Mon, May 22, 2023 at 6:52 AM lonesomerain <lo...@163.com> wrote:
>>
>> Hi,
>>
>> I have a question while using ignite 2.15.0
>>
>>
>>
>> Problem scenario:
>>
>> Start the Ignite server of one node, start one thin client and create a continuous query listener, and then use 50 threads to add 500 data to the cache concurrently.
>>
>> Problem phenomenon:
>>
>> Through the information printed on the listener, it was found that the number of events listened to each time varies, possibly 496, 499 or 500...
>>
>> Test Code:
>>
>> public class StartServer {
>>
>> public static void main(String[] args) {
>>
>> Ignite ignite = Ignition.start();
>>
>> }
>>
>> }
>>
>>
>>
>> public class StartThinClient {
>>
>> public static void main(String[] args) throws InterruptedException {
>>
>> String addr = "127.0.0.1:10800";
>>
>>
>>
>> int threadNmu = 50;
>>
>>
>>
>> ClientConfiguration clientConfiguration = new ClientConfiguration();
>>
>> clientConfiguration.setAddresses(addr);
>>
>>
>>
>> IgniteClient client1 = Ignition.startClient(clientConfiguration);
>>
>>
>>
>> ClientCache<Object, Object> cache1 = client1.getOrCreateCache("test");
>>
>>
>>
>> ContinuousQuery<Object, Object> query = new ContinuousQuery<>();
>>
>> query.setLocalListener(new CacheEntryUpdatedListener<Object, Object>() {
>>
>> @Override
>>
>> public void onUpdated(Iterable<CacheEntryEvent<?, ?>> cacheEntryEvents) throws CacheEntryListenerException {
>>
>> Iterator<CacheEntryEvent<?, ?>> iterator = cacheEntryEvents.iterator();
>>
>> while (iterator.hasNext()) {
>>
>> CacheEntryEvent<?, ?> next = iterator.next();
>>
>> System.out.println("----" + next.getKey());
>>
>> }
>>
>> }
>>
>> });
>>
>>
>>
>> cache1.query(query);
>>
>>
>>
>> IgniteClient client2 = Ignition.startClient(clientConfiguration);
>>
>> ClientCache<Object, Object> cache2 = client2.cache("test");
>>
>>
>>
>> Thread[] threads = new Thread[threadNmu];
>>
>> for (int i = 0; i < threads.length; ++i) {
>>
>> threads[i] = new Thread(new OperationInsert(cache2, i, 500, threadNmu));
>>
>> }
>>
>> for (int i = 0; i < threads.length; ++i) {
>>
>> threads[i].start();
>>
>> }
>>
>> for (Thread thread : threads) {
>>
>> thread.join();
>>
>> }
>>
>>
>>
>> Thread.sleep(60000);
>>
>>
>>
>> }
>>
>>
>>
>> static class OperationInsert implements Runnable {
>>
>>
>>
>> private ClientCache<Object, Object> cache;
>>
>> private int k;
>>
>> private Integer test_rows;
>>
>> private Integer thread_cnt;
>>
>>
>>
>> public OperationInsert(ClientCache<Object, Object> cache, int k, Integer test_rows, Integer thread_cnt) {
>>
>> this.cache = cache;
>>
>> this.k = k;
>>
>> this.test_rows = test_rows;
>>
>> this.thread_cnt = thread_cnt;
>>
>> }
>>
>>
>>
>> @Override
>>
>> public void run() {
>>
>> for (int i = 1000000 + (test_rows/thread_cnt) * k; i < 1000000 + (test_rows/thread_cnt) * (k + 1); i++) {
>>
>> cache.put("" + i, "aaa");
>>
>> }
>>
>> }
>>
>> }
>>
>>
>>
>> }
>>
>>
>>
>> Version:
>>
>> The testing program uses Ignite version 2.15.0
>>
>> I attempted to insert data using one thread and did not observe any event loss. In addition, I also attempted an Ignite cluster with two or three nodes, which can still listen to all 500 events even when inserting data using multiple threads. May I ask if this issue only occurs at a single node? Are there any good solutions?
>>
>>
Re: Ignite thin client continuous query listener cannot listen to all events
Posted by Pavel Tupitsyn <pt...@apache.org>.
Thank you for the bug report, I will have a look.
On Thu, May 25, 2023 at 5:10 AM LonesomeRain <lo...@163.com> wrote:
> Hi, Jeremy McMillan
>
>
>
> I have created a Jira about this bug.
>
>
>
> https://issues.apache.org/jira/browse/IGNITE-19561
>
>
>
> By executing these two test codes in the description, it is easy to
> reproduce the problem.
>
>
>
> What should I do next? Continuously following this Jira?
>
>
>
>
>
> *发件人: *Jeremy McMillan <je...@gridgain.com>
> *发送时间: *2023年5月24日 23:24
> *收件人: *user@ignite.apache.org
> *主题: *Re: Ignite thin client continuous query listener cannot listen to
> all events
>
>
>
> Thanks for bringing this up!
>
>
>
>
> https://ignite.apache.org/docs/latest/key-value-api/continuous-queries#events-delivery-guarantees
>
>
>
> This sounds like you may have found a bug, but the details you've provided
> are not sufficient to help others recreate and observe it for themselves,
> and this effort needs to be recorded in a ticket. Would you be able to sign
> up for a Jira account <https://selfserve.apache.org/jira-account.html>
> and detail steps to reproduce this behavior?
>
> You may also want to research this:
> https://issues.apache.org/jira/browse/IGNITE-8035
>
>
>
> On Mon, May 22, 2023 at 6:52 AM lonesomerain <lo...@163.com> wrote:
>
> *Hi,*
>
> *I have a question while using ignite 2.15.0*
>
>
>
> *Problem scenario:*
>
> Start the Ignite server of one node, start one thin client and create a
> continuous query listener, and then use 50 threads to add 500 data to the
> cache concurrently.
>
> *Problem phenomenon:*
>
> Through the information printed on the listener, it was found that the
> number of events listened to each time varies, possibly 496, 499 or 500...
>
> *Test Code:*
>
> public class StartServer {
>
> public static void main(String[] args) {
>
> Ignite ignite = Ignition.start();
>
> }
>
> }
>
>
>
> public class StartThinClient {
>
> public static void main(String[] args) throws InterruptedException {
>
> String addr = "127.0.0.1:10800";
>
>
>
> int threadNmu = 50;
>
>
>
> ClientConfiguration clientConfiguration = new
> ClientConfiguration();
>
> clientConfiguration.setAddresses(addr);
>
>
>
> IgniteClient client1 = Ignition.startClient(clientConfiguration);
>
>
>
> ClientCache<Object, Object> cache1 =
> client1.getOrCreateCache("test");
>
>
>
> ContinuousQuery<Object, Object> query = new ContinuousQuery<>();
>
> query.setLocalListener(new CacheEntryUpdatedListener<Object,
> Object>() {
>
> @Override
>
> public void onUpdated(Iterable<CacheEntryEvent<?, ?>>
> cacheEntryEvents) throws CacheEntryListenerException {
>
> Iterator<CacheEntryEvent<?, ?>> iterator =
> cacheEntryEvents.iterator();
>
> while (iterator.hasNext()) {
>
> CacheEntryEvent<?, ?> next = iterator.next();
>
> System.out.println("----" + next.getKey());
>
> }
>
> }
>
> });
>
>
>
> cache1.query(query);
>
>
>
> IgniteClient client2 = Ignition.startClient(clientConfiguration);
>
> ClientCache<Object, Object> cache2 = client2.cache("test");
>
>
>
> Thread[] threads = new Thread[threadNmu];
>
> for (int i = 0; i < threads.length; ++i) {
>
> threads[i] = new Thread(new OperationInsert(cache2, i, 500,
> threadNmu));
>
> }
>
> for (int i = 0; i < threads.length; ++i) {
>
> threads[i].start();
>
> }
>
> for (Thread thread : threads) {
>
> thread.join();
>
> }
>
>
>
> Thread.sleep(60000);
>
>
>
> }
>
>
>
> static class OperationInsert implements Runnable {
>
>
>
> private ClientCache<Object, Object> cache;
>
> private int k;
>
> private Integer test_rows;
>
> private Integer thread_cnt;
>
>
>
> public OperationInsert(ClientCache<Object, Object> cache, int k,
> Integer test_rows, Integer thread_cnt) {
>
> this.cache = cache;
>
> this.k = k;
>
> this.test_rows = test_rows;
>
> this.thread_cnt = thread_cnt;
>
> }
>
>
>
> @Override
>
> public void run() {
>
> for (int i = 1000000 + (test_rows/thread_cnt) * k; i < 1000000
> + (test_rows/thread_cnt) * (k + 1); i++) {
>
> cache.put("" + i, "aaa");
>
> }
>
> }
>
> }
>
>
>
> }
>
>
>
> *Version:*
>
> The testing program uses Ignite version 2.15.0
>
> I attempted to insert data using one thread and did not observe any event
> loss. In addition, I also attempted an Ignite cluster with two or three
> nodes, which can still listen to all 500 events even when inserting data
> using multiple threads. May I ask if this issue only occurs at a single
> node? Are there any good solutions?
>
>
>
Re: Ignite thin client continuous query listener cannot listen to all events
Posted by LonesomeRain <lo...@163.com>.
Hi, Jeremy McMillan
I have created a Jira about this bug.
https://issues.apache.org/jira/browse/IGNITE-19561
By executing these two test codes in the description, it is easy to reproduce the problem.
What should I do next? Continuously following this Jira?
发件人: Jeremy McMillan
发送时间: 2023年5月24日 23:24
收件人: user@ignite.apache.org
主题: Re: Ignite thin client continuous query listener cannot listen to all events
Thanks for bringing this up!
https://ignite.apache.org/docs/latest/key-value-api/continuous-queries#events-delivery-guarantees
This sounds like you may have found a bug, but the details you've provided are not sufficient to help others recreate and observe it for themselves, and this effort needs to be recorded in a ticket. Would you be able to sign up for a Jira account and detail steps to reproduce this behavior?
You may also want to research this:
https://issues.apache.org/jira/browse/IGNITE-8035
On Mon, May 22, 2023 at 6:52 AM lonesomerain <lo...@163.com> wrote:
Hi,
I have a question while using ignite 2.15.0
Problem scenario:
Start the Ignite server of one node, start one thin client and create a continuous query listener, and then use 50 threads to add 500 data to the cache concurrently.
Problem phenomenon:
Through the information printed on the listener, it was found that the number of events listened to each time varies, possibly 496, 499 or 500...
Test Code:
public class StartServer {
public static void main(String[] args) {
Ignite ignite = Ignition.start();
}
}
public class StartThinClient {
public static void main(String[] args) throws InterruptedException {
String addr = "127.0.0.1:10800";
int threadNmu = 50;
ClientConfiguration clientConfiguration = new ClientConfiguration();
clientConfiguration.setAddresses(addr);
IgniteClient client1 = Ignition.startClient(clientConfiguration);
ClientCache<Object, Object> cache1 = client1.getOrCreateCache("test");
ContinuousQuery<Object, Object> query = new ContinuousQuery<>();
query.setLocalListener(new CacheEntryUpdatedListener<Object, Object>() {
@Override
public void onUpdated(Iterable<CacheEntryEvent<?, ?>> cacheEntryEvents) throws CacheEntryListenerException {
Iterator<CacheEntryEvent<?, ?>> iterator = cacheEntryEvents.iterator();
while (iterator.hasNext()) {
CacheEntryEvent<?, ?> next = iterator.next();
System.out.println("----" + next.getKey());
}
}
});
cache1.query(query);
IgniteClient client2 = Ignition.startClient(clientConfiguration);
ClientCache<Object, Object> cache2 = client2.cache("test");
Thread[] threads = new Thread[threadNmu];
for (int i = 0; i < threads.length; ++i) {
threads[i] = new Thread(new OperationInsert(cache2, i, 500, threadNmu));
}
for (int i = 0; i < threads.length; ++i) {
threads[i].start();
}
for (Thread thread : threads) {
thread.join();
}
Thread.sleep(60000);
}
static class OperationInsert implements Runnable {
private ClientCache<Object, Object> cache;
private int k;
private Integer test_rows;
private Integer thread_cnt;
public OperationInsert(ClientCache<Object, Object> cache, int k, Integer test_rows, Integer thread_cnt) {
this.cache = cache;
this.k = k;
this.test_rows = test_rows;
this.thread_cnt = thread_cnt;
}
@Override
public void run() {
for (int i = 1000000 + (test_rows/thread_cnt) * k; i < 1000000 + (test_rows/thread_cnt) * (k + 1); i++) {
cache.put("" + i, "aaa");
}
}
}
}
Version:
The testing program uses Ignite version 2.15.0
I attempted to insert data using one thread and did not observe any event loss. In addition, I also attempted an Ignite cluster with two or three nodes, which can still listen to all 500 events even when inserting data using multiple threads. May I ask if this issue only occurs at a single node? Are there any good solutions?
Re: Ignite thin client continuous query listener cannot listen to all events
Posted by Jeremy McMillan <je...@gridgain.com>.
Thanks for bringing this up!
https://ignite.apache.org/docs/latest/key-value-api/continuous-queries#events-delivery-guarantees
This sounds like you may have found a bug, but the details you've provided
are not sufficient to help others recreate and observe it for themselves,
and this effort needs to be recorded in a ticket. Would you be able to sign
up for a Jira account <https://selfserve.apache.org/jira-account.html> and
detail steps to reproduce this behavior?
You may also want to research this:
https://issues.apache.org/jira/browse/IGNITE-8035
On Mon, May 22, 2023 at 6:52 AM lonesomerain <lo...@163.com> wrote:
> *Hi,*
> *I have a question while using ignite 2.15.0*
>
> *Problem scenario:*
>
> Start the Ignite server of one node, start one thin client and create a
> continuous query listener, and then use 50 threads to add 500 data to the
> cache concurrently.
>
> *Problem phenomenon:*
>
> Through the information printed on the listener, it was found that the
> number of events listened to each time varies, possibly 496, 499 or 500...
>
> *Test Code:*
>
> public class StartServer {
>
> public static void main(String[] args) {
>
> Ignite ignite = Ignition.start();
>
> }
>
> }
>
>
> public class StartThinClient {
>
> public static void main(String[] args) throws InterruptedException {
>
> String addr = "127.0.0.1:10800";
>
>
> int threadNmu = 50;
>
>
> ClientConfiguration clientConfiguration = new
> ClientConfiguration();
>
> clientConfiguration.setAddresses(addr);
>
>
> IgniteClient client1 = Ignition.startClient(clientConfiguration);
>
>
> ClientCache<Object, Object> cache1 =
> client1.getOrCreateCache("test");
>
>
> ContinuousQuery<Object, Object> query = new ContinuousQuery<>();
>
> query.setLocalListener(new CacheEntryUpdatedListener<Object,
> Object>() {
>
> @Override
>
> public void onUpdated(Iterable<CacheEntryEvent<?, ?>>
> cacheEntryEvents) throws CacheEntryListenerException {
>
> Iterator<CacheEntryEvent<?, ?>> iterator =
> cacheEntryEvents.iterator();
>
> while (iterator.hasNext()) {
>
> CacheEntryEvent<?, ?> next = iterator.next();
>
> System.out.println("----" + next.getKey());
>
> }
>
> }
>
> });
>
>
> cache1.query(query);
>
>
> IgniteClient client2 = Ignition.startClient(clientConfiguration);
>
> ClientCache<Object, Object> cache2 = client2.cache("test");
>
>
> Thread[] threads = new Thread[threadNmu];
>
> for (int i = 0; i < threads.length; ++i) {
>
> threads[i] = new Thread(new OperationInsert(cache2, i, 500,
> threadNmu));
>
> }
>
> for (int i = 0; i < threads.length; ++i) {
>
> threads[i].start();
>
> }
>
> for (Thread thread : threads) {
>
> thread.join();
>
> }
>
>
> Thread.sleep(60000);
>
>
> }
>
>
> static class OperationInsert implements Runnable {
>
>
> private ClientCache<Object, Object> cache;
>
> private int k;
>
> private Integer test_rows;
>
> private Integer thread_cnt;
>
>
> public OperationInsert(ClientCache<Object, Object> cache, int k,
> Integer test_rows, Integer thread_cnt) {
>
> this.cache = cache;
>
> this.k = k;
>
> this.test_rows = test_rows;
>
> this.thread_cnt = thread_cnt;
>
> }
>
>
> @Override
>
> public void run() {
>
> for (int i = 1000000 + (test_rows/thread_cnt) * k; i < 1000000
> + (test_rows/thread_cnt) * (k + 1); i++) {
>
> cache.put("" + i, "aaa");
>
> }
>
> }
>
> }
>
>
> }
>
>
> *Version:*
>
> The testing program uses Ignite version 2.15.0
>
> I attempted to insert data using one thread and did not observe any event
> loss. In addition, I also attempted an Ignite cluster with two or three
> nodes, which can still listen to all 500 events even when inserting data
> using multiple threads. May I ask if this issue only occurs at a single
> node? Are there any good solutions?
>