You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Anil <an...@gmail.com> on 2017/03/08 12:04:29 UTC
IGNITE-4106
Hi,
Does queryparallelism uses system pool or separate thread pool other than
system and public thread pool ? please clarify.
1.
https://apacheignite.readme.io/docs/sql-performance-and-debugging#sql-performance-and-usability-considerations
Thanks.
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Hi Andrey,
Did you get a chance to look into the reproducer ? thanks.
Thanks.
On 11 March 2017 at 17:37, Andrey Mashenkov <an...@gmail.com>
wrote:
> Thanks Anil.
>
> I'll take a look.
>
> 11 марта 2017 г. 9:58 пользователь "Anil" <an...@gmail.com> написал:
>
> Hi Andrey,
>
> i am able to reproduce the issue with following reproducer.
>
> https://github.com/adasari/test-ignite-parallelism.git
>
> Please let me know if you see any issue with reproducer.
>
> Thanks
>
> On 11 March 2017 at 11:46, Anil <an...@gmail.com> wrote:
>
>> Hi Andrey,
>>
>> i have created test project which pushes the entries to drastreamer using
>> for loop and it is working without any issue.
>>
>> But my application which does the same thing but loads the data from
>> hbase and it is failing with *"java.lang.IllegalStateException: Data
>> streamer has been closed." *
>>
>>
>>
>> *java.lang.IllegalStateException: Data streamer has been closed.
>> at org.apache.ignite.internal.pro
>> <http://org.apache.ignite.internal.pro>cessors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:406)
>> at org.apache.ignite.internal.pro
>> <http://org.apache.ignite.internal.pro>cessors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:613)
>> at org.apache.ignite.internal.pro
>> <http://org.apache.ignite.internal.pro>cessors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:667)*
>>
>> i set the system and public thread pool as 32 for an 8 core machine and
>> using 4 node cluster.
>>
>> and i see that loading data into cache for which queryParallelism
>> configured is failing and other cache loads working fine.
>>
>> Thanks
>>
>>
>> On 10 March 2017 at 05:05, Andrey Mashenkov <an...@gmail.com>
>> wrote:
>>
>>> Hi Anil,
>>>
>>> I don't think it is a thead starvation due to it work fine for 1 thread.
>>> Whould you please share a repro?
>>>
>>> On Thu, Mar 9, 2017 at 9:09 PM, Anil <an...@gmail.com> wrote:
>>>
>>>>
>>>> Hi Andrey,
>>>>
>>>> i tried to set the parallelism to 2, 4 on 4 node cluster (8 core
>>>> machines) and initiated the data load (using compute job). Data streamers
>>>> are getting closed and data load is failing.
>>>>
>>>> When I set the parallelism to 1, data load working as expected. the
>>>> issue could be threads starvation/ non availability of resources and not
>>>> ignite issue. Do you think any other issue ?
>>>>
>>>> I will try on 16 core machines cluster tomorrow.
>>>>
>>>> Thanks.
>>>>
>>>> On 8 March 2017 at 18:04, Andrey Mashenkov <an...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Anil,
>>>>>
>>>>> SQL queries are run in system pool.
>>>>>
>>>>> On Wed, Mar 8, 2017 at 3:04 PM, Anil <an...@gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Does queryparallelism uses system pool or separate thread pool other
>>>>>> than system and public thread pool ? please clarify.
>>>>>>
>>>>>> 1. https://apacheignite.readme.io/docs/sql-performance-and-d
>>>>>> ebugging#sql-performance-and-usability-considerations
>>>>>>
>>>>>> Thanks.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Andrey V. Mashenkov
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>
>
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Thanks Anil.
I'll take a look.
11 марта 2017 г. 9:58 пользователь "Anil" <an...@gmail.com> написал:
Hi Andrey,
i am able to reproduce the issue with following reproducer.
https://github.com/adasari/test-ignite-parallelism.git
Please let me know if you see any issue with reproducer.
Thanks
On 11 March 2017 at 11:46, Anil <an...@gmail.com> wrote:
> Hi Andrey,
>
> i have created test project which pushes the entries to drastreamer using
> for loop and it is working without any issue.
>
> But my application which does the same thing but loads the data from hbase
> and it is failing with *"java.lang.IllegalStateException: Data streamer
> has been closed." *
>
>
>
> *java.lang.IllegalStateException: Data streamer has been closed. at
> org.apache.ignite.internal.pro
> <http://org.apache.ignite.internal.pro>cessors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:406)
> at org.apache.ignite.internal.pro
> <http://org.apache.ignite.internal.pro>cessors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:613)
> at org.apache.ignite.internal.pro
> <http://org.apache.ignite.internal.pro>cessors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:667)*
>
> i set the system and public thread pool as 32 for an 8 core machine and
> using 4 node cluster.
>
> and i see that loading data into cache for which queryParallelism
> configured is failing and other cache loads working fine.
>
> Thanks
>
>
> On 10 March 2017 at 05:05, Andrey Mashenkov <an...@gmail.com>
> wrote:
>
>> Hi Anil,
>>
>> I don't think it is a thead starvation due to it work fine for 1 thread.
>> Whould you please share a repro?
>>
>> On Thu, Mar 9, 2017 at 9:09 PM, Anil <an...@gmail.com> wrote:
>>
>>>
>>> Hi Andrey,
>>>
>>> i tried to set the parallelism to 2, 4 on 4 node cluster (8 core
>>> machines) and initiated the data load (using compute job). Data streamers
>>> are getting closed and data load is failing.
>>>
>>> When I set the parallelism to 1, data load working as expected. the
>>> issue could be threads starvation/ non availability of resources and not
>>> ignite issue. Do you think any other issue ?
>>>
>>> I will try on 16 core machines cluster tomorrow.
>>>
>>> Thanks.
>>>
>>> On 8 March 2017 at 18:04, Andrey Mashenkov <an...@gmail.com>
>>> wrote:
>>>
>>>> Hi Anil,
>>>>
>>>> SQL queries are run in system pool.
>>>>
>>>> On Wed, Mar 8, 2017 at 3:04 PM, Anil <an...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Does queryparallelism uses system pool or separate thread pool other
>>>>> than system and public thread pool ? please clarify.
>>>>>
>>>>> 1. https://apacheignite.readme.io/docs/sql-performance-and-d
>>>>> ebugging#sql-performance-and-usability-considerations
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Andrey V. Mashenkov
>>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Hi Andrey,
i am able to reproduce the issue with following reproducer.
https://github.com/adasari/test-ignite-parallelism.git
Please let me know if you see any issue with reproducer.
Thanks
On 11 March 2017 at 11:46, Anil <an...@gmail.com> wrote:
> Hi Andrey,
>
> i have created test project which pushes the entries to drastreamer using
> for loop and it is working without any issue.
>
> But my application which does the same thing but loads the data from hbase
> and it is failing with *"java.lang.IllegalStateException: Data streamer
> has been closed." *
>
>
>
> *java.lang.IllegalStateException: Data streamer has been closed. at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:406)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:613)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:667)*
>
> i set the system and public thread pool as 32 for an 8 core machine and
> using 4 node cluster.
>
> and i see that loading data into cache for which queryParallelism
> configured is failing and other cache loads working fine.
>
> Thanks
>
>
> On 10 March 2017 at 05:05, Andrey Mashenkov <an...@gmail.com>
> wrote:
>
>> Hi Anil,
>>
>> I don't think it is a thead starvation due to it work fine for 1 thread.
>> Whould you please share a repro?
>>
>> On Thu, Mar 9, 2017 at 9:09 PM, Anil <an...@gmail.com> wrote:
>>
>>>
>>> Hi Andrey,
>>>
>>> i tried to set the parallelism to 2, 4 on 4 node cluster (8 core
>>> machines) and initiated the data load (using compute job). Data streamers
>>> are getting closed and data load is failing.
>>>
>>> When I set the parallelism to 1, data load working as expected. the
>>> issue could be threads starvation/ non availability of resources and not
>>> ignite issue. Do you think any other issue ?
>>>
>>> I will try on 16 core machines cluster tomorrow.
>>>
>>> Thanks.
>>>
>>> On 8 March 2017 at 18:04, Andrey Mashenkov <an...@gmail.com>
>>> wrote:
>>>
>>>> Hi Anil,
>>>>
>>>> SQL queries are run in system pool.
>>>>
>>>> On Wed, Mar 8, 2017 at 3:04 PM, Anil <an...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Does queryparallelism uses system pool or separate thread pool other
>>>>> than system and public thread pool ? please clarify.
>>>>>
>>>>> 1. https://apacheignite.readme.io/docs/sql-performance-and-d
>>>>> ebugging#sql-performance-and-usability-considerations
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Andrey V. Mashenkov
>>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
I will try with sample program and share with you. Thanks.
On 23 March 2017 at 17:53, Andrey Mashenkov <an...@gmail.com>
wrote:
> Hi Anil,
>
> I can't reproduce this issue. Would you please share a repro?
>
> On Wed, Mar 22, 2017 at 9:04 AM, Anil <an...@gmail.com> wrote:
>
>> HI Andrey,
>>
>> i have two records for my query.
>> i did not see same results if i hit the same query number times. Results
>> in number of records are empty, 1, 2.
>>
>> Thanks
>>
>>
>>
>>
>> On 22 March 2017 at 10:49, Andrey Mashenkov <an...@gmail.com>
>> wrote:
>>
>>> Hi Anil,
>>>
>>> What do you mean "the results are not same"? It looks like query should
>>> return a single row.
>>> If there would be more than one row in result and order is not specified
>>> in query, then it is possible to get rows in different order due to data
>>> transferred from other nodes asynchronously.
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Mar 21, 2017 at 7:02 AM, Anil <an...@gmail.com> wrote:
>>>
>>>> Hi Andrew,
>>>>
>>>> #1 - it is very simple select query - select * from person hwere
>>>> personid = 'something';
>>>> i just ran the query in for loop and noticed the results are not same.
>>>>
>>>> #2 - it is stable topology. swap is configured. but this test was done
>>>> when full load is completed and some compute job going on for other cache.
>>>>
>>>> Please let me know if you have any questions. thanks.
>>>>
>>>> Thanks.
>>>>
>>>> On 20 March 2017 at 21:07, Andrey Mashenkov <andrey.mashenkov@gmail.com
>>>> > wrote:
>>>>
>>>>> Hi Anil,
>>>>>
>>>>> 1. Would you please share sql query text?
>>>>>
>>>>> 2. Is it happening on unstable topology or during rebalancing? Or may
>>>>> be eviction\expire policy or swap is configured?
>>>>>
>>>>> On Mon, Mar 20, 2017 at 5:41 PM, Anil <an...@gmail.com> wrote:
>>>>>
>>>>>> Yes. i am using partition cache only with no joins :)
>>>>>>
>>>>>> how about #2 ?
>>>>>>
>>>>>> On 20 March 2017 at 19:20, Andrey Mashenkov <
>>>>>> andrey.mashenkov@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Anil,
>>>>>>>
>>>>>>> I should although mention that Replicated caches can participate in
>>>>>>> same query with partitioned caches regardless a degree of parallelizm.
>>>>>>> This limitation relates to partitioned caches only.
>>>>>>>
>>>>>>> On Mon, Mar 20, 2017 at 3:54 PM, Andrey Mashenkov <
>>>>>>> andrey.mashenkov@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Anil,
>>>>>>>>
>>>>>>>> It is ok. Doc says *"If a query contains JOINs, then all the
>>>>>>>> participating caches must have the same degree of parallelism.".*
>>>>>>>> Possibly, it is easy to fix but there can be unobvious limitations,
>>>>>>>> so we need a time to make a POC.
>>>>>>>> I believe, it will be fixed in future releases.
>>>>>>>>
>>>>>>>> On Mon, Mar 20, 2017 at 1:11 PM, Anil <an...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Andrey,
>>>>>>>>>
>>>>>>>>> I see few more issues with IGNITE-4826
>>>>>>>>>
>>>>>>>>> 1. queryParallelism should be used for all caches for which
>>>>>>>>> queries are used other it throws following exception.
>>>>>>>>>
>>>>>>>>> Caused by: java.sql.SQLException: Failed to query Ignite.
>>>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:131)
>>>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>>>> c2.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.
>>>>>>>>> java:76)
>>>>>>>>> at org.apache.commons.dbcp2.Deleg
>>>>>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>>>>>> ent.java:83)
>>>>>>>>> at org.apache.commons.dbcp2.Deleg
>>>>>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>>>>>> ent.java:83)
>>>>>>>>>
>>>>>>>>> Caused by: javax.cache.CacheException: class
>>>>>>>>> org.apache.ignite.IgniteException: Using indexes with different
>>>>>>>>> parallelism levels in same query is forbidden.
>>>>>>>>> at org.apache.ignite.internal.pro
>>>>>>>>> cessors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:760)
>>>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>>>> c2.JdbcQueryTask.call(JdbcQueryTask.java:161)
>>>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:116)
>>>>>>>>> ... 13 more
>>>>>>>>> 2. query is not returning same result if it is hit number of times.
>>>>>>>>>
>>>>>>>>> please let me know if these are known issues.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Best regards,
>>>>>>>> Andrey V. Mashenkov
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best regards,
>>>>>>> Andrey V. Mashenkov
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Andrey V. Mashenkov
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Anil,
I can't reproduce this issue. Would you please share a repro?
On Wed, Mar 22, 2017 at 9:04 AM, Anil <an...@gmail.com> wrote:
> HI Andrey,
>
> i have two records for my query.
> i did not see same results if i hit the same query number times. Results
> in number of records are empty, 1, 2.
>
> Thanks
>
>
>
>
> On 22 March 2017 at 10:49, Andrey Mashenkov <an...@gmail.com>
> wrote:
>
>> Hi Anil,
>>
>> What do you mean "the results are not same"? It looks like query should
>> return a single row.
>> If there would be more than one row in result and order is not specified
>> in query, then it is possible to get rows in different order due to data
>> transferred from other nodes asynchronously.
>>
>>
>>
>>
>>
>> On Tue, Mar 21, 2017 at 7:02 AM, Anil <an...@gmail.com> wrote:
>>
>>> Hi Andrew,
>>>
>>> #1 - it is very simple select query - select * from person hwere
>>> personid = 'something';
>>> i just ran the query in for loop and noticed the results are not same.
>>>
>>> #2 - it is stable topology. swap is configured. but this test was done
>>> when full load is completed and some compute job going on for other cache.
>>>
>>> Please let me know if you have any questions. thanks.
>>>
>>> Thanks.
>>>
>>> On 20 March 2017 at 21:07, Andrey Mashenkov <an...@gmail.com>
>>> wrote:
>>>
>>>> Hi Anil,
>>>>
>>>> 1. Would you please share sql query text?
>>>>
>>>> 2. Is it happening on unstable topology or during rebalancing? Or may
>>>> be eviction\expire policy or swap is configured?
>>>>
>>>> On Mon, Mar 20, 2017 at 5:41 PM, Anil <an...@gmail.com> wrote:
>>>>
>>>>> Yes. i am using partition cache only with no joins :)
>>>>>
>>>>> how about #2 ?
>>>>>
>>>>> On 20 March 2017 at 19:20, Andrey Mashenkov <
>>>>> andrey.mashenkov@gmail.com> wrote:
>>>>>
>>>>>> Hi Anil,
>>>>>>
>>>>>> I should although mention that Replicated caches can participate in
>>>>>> same query with partitioned caches regardless a degree of parallelizm.
>>>>>> This limitation relates to partitioned caches only.
>>>>>>
>>>>>> On Mon, Mar 20, 2017 at 3:54 PM, Andrey Mashenkov <
>>>>>> andrey.mashenkov@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Anil,
>>>>>>>
>>>>>>> It is ok. Doc says *"If a query contains JOINs, then all the
>>>>>>> participating caches must have the same degree of parallelism.".*
>>>>>>> Possibly, it is easy to fix but there can be unobvious limitations,
>>>>>>> so we need a time to make a POC.
>>>>>>> I believe, it will be fixed in future releases.
>>>>>>>
>>>>>>> On Mon, Mar 20, 2017 at 1:11 PM, Anil <an...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Andrey,
>>>>>>>>
>>>>>>>> I see few more issues with IGNITE-4826
>>>>>>>>
>>>>>>>> 1. queryParallelism should be used for all caches for which queries
>>>>>>>> are used other it throws following exception.
>>>>>>>>
>>>>>>>> Caused by: java.sql.SQLException: Failed to query Ignite.
>>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:131)
>>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>>> c2.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.
>>>>>>>> java:76)
>>>>>>>> at org.apache.commons.dbcp2.Deleg
>>>>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>>>>> ent.java:83)
>>>>>>>> at org.apache.commons.dbcp2.Deleg
>>>>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>>>>> ent.java:83)
>>>>>>>>
>>>>>>>> Caused by: javax.cache.CacheException: class
>>>>>>>> org.apache.ignite.IgniteException: Using indexes with different
>>>>>>>> parallelism levels in same query is forbidden.
>>>>>>>> at org.apache.ignite.internal.pro
>>>>>>>> cessors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:760)
>>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>>> c2.JdbcQueryTask.call(JdbcQueryTask.java:161)
>>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:116)
>>>>>>>> ... 13 more
>>>>>>>> 2. query is not returning same result if it is hit number of times.
>>>>>>>>
>>>>>>>> please let me know if these are known issues.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best regards,
>>>>>>> Andrey V. Mashenkov
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best regards,
>>>>>> Andrey V. Mashenkov
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Andrey V. Mashenkov
>>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
--
Best regards,
Andrey V. Mashenkov
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
HI Andrey,
i have two records for my query.
i did not see same results if i hit the same query number times. Results in
number of records are empty, 1, 2.
Thanks
On 22 March 2017 at 10:49, Andrey Mashenkov <an...@gmail.com>
wrote:
> Hi Anil,
>
> What do you mean "the results are not same"? It looks like query should
> return a single row.
> If there would be more than one row in result and order is not specified
> in query, then it is possible to get rows in different order due to data
> transferred from other nodes asynchronously.
>
>
>
>
>
> On Tue, Mar 21, 2017 at 7:02 AM, Anil <an...@gmail.com> wrote:
>
>> Hi Andrew,
>>
>> #1 - it is very simple select query - select * from person hwere personid
>> = 'something';
>> i just ran the query in for loop and noticed the results are not same.
>>
>> #2 - it is stable topology. swap is configured. but this test was done
>> when full load is completed and some compute job going on for other cache.
>>
>> Please let me know if you have any questions. thanks.
>>
>> Thanks.
>>
>> On 20 March 2017 at 21:07, Andrey Mashenkov <an...@gmail.com>
>> wrote:
>>
>>> Hi Anil,
>>>
>>> 1. Would you please share sql query text?
>>>
>>> 2. Is it happening on unstable topology or during rebalancing? Or may be
>>> eviction\expire policy or swap is configured?
>>>
>>> On Mon, Mar 20, 2017 at 5:41 PM, Anil <an...@gmail.com> wrote:
>>>
>>>> Yes. i am using partition cache only with no joins :)
>>>>
>>>> how about #2 ?
>>>>
>>>> On 20 March 2017 at 19:20, Andrey Mashenkov <andrey.mashenkov@gmail.com
>>>> > wrote:
>>>>
>>>>> Hi Anil,
>>>>>
>>>>> I should although mention that Replicated caches can participate in
>>>>> same query with partitioned caches regardless a degree of parallelizm.
>>>>> This limitation relates to partitioned caches only.
>>>>>
>>>>> On Mon, Mar 20, 2017 at 3:54 PM, Andrey Mashenkov <
>>>>> andrey.mashenkov@gmail.com> wrote:
>>>>>
>>>>>> Hi Anil,
>>>>>>
>>>>>> It is ok. Doc says *"If a query contains JOINs, then all the
>>>>>> participating caches must have the same degree of parallelism.".*
>>>>>> Possibly, it is easy to fix but there can be unobvious limitations,
>>>>>> so we need a time to make a POC.
>>>>>> I believe, it will be fixed in future releases.
>>>>>>
>>>>>> On Mon, Mar 20, 2017 at 1:11 PM, Anil <an...@gmail.com> wrote:
>>>>>>
>>>>>>> Hi Andrey,
>>>>>>>
>>>>>>> I see few more issues with IGNITE-4826
>>>>>>>
>>>>>>> 1. queryParallelism should be used for all caches for which queries
>>>>>>> are used other it throws following exception.
>>>>>>>
>>>>>>> Caused by: java.sql.SQLException: Failed to query Ignite.
>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:131)
>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>> c2.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:76)
>>>>>>> at org.apache.commons.dbcp2.Deleg
>>>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>>>> ent.java:83)
>>>>>>> at org.apache.commons.dbcp2.Deleg
>>>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>>>> ent.java:83)
>>>>>>>
>>>>>>> Caused by: javax.cache.CacheException: class
>>>>>>> org.apache.ignite.IgniteException: Using indexes with different
>>>>>>> parallelism levels in same query is forbidden.
>>>>>>> at org.apache.ignite.internal.pro
>>>>>>> cessors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:760)
>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>> c2.JdbcQueryTask.call(JdbcQueryTask.java:161)
>>>>>>> at org.apache.ignite.internal.jdb
>>>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:116)
>>>>>>> ... 13 more
>>>>>>> 2. query is not returning same result if it is hit number of times.
>>>>>>>
>>>>>>> please let me know if these are known issues.
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best regards,
>>>>>> Andrey V. Mashenkov
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Andrey V. Mashenkov
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Anil,
What do you mean "the results are not same"? It looks like query should
return a single row.
If there would be more than one row in result and order is not specified in
query, then it is possible to get rows in different order due to data
transferred from other nodes asynchronously.
On Tue, Mar 21, 2017 at 7:02 AM, Anil <an...@gmail.com> wrote:
> Hi Andrew,
>
> #1 - it is very simple select query - select * from person hwere personid
> = 'something';
> i just ran the query in for loop and noticed the results are not same.
>
> #2 - it is stable topology. swap is configured. but this test was done
> when full load is completed and some compute job going on for other cache.
>
> Please let me know if you have any questions. thanks.
>
> Thanks.
>
> On 20 March 2017 at 21:07, Andrey Mashenkov <an...@gmail.com>
> wrote:
>
>> Hi Anil,
>>
>> 1. Would you please share sql query text?
>>
>> 2. Is it happening on unstable topology or during rebalancing? Or may be
>> eviction\expire policy or swap is configured?
>>
>> On Mon, Mar 20, 2017 at 5:41 PM, Anil <an...@gmail.com> wrote:
>>
>>> Yes. i am using partition cache only with no joins :)
>>>
>>> how about #2 ?
>>>
>>> On 20 March 2017 at 19:20, Andrey Mashenkov <an...@gmail.com>
>>> wrote:
>>>
>>>> Hi Anil,
>>>>
>>>> I should although mention that Replicated caches can participate in
>>>> same query with partitioned caches regardless a degree of parallelizm.
>>>> This limitation relates to partitioned caches only.
>>>>
>>>> On Mon, Mar 20, 2017 at 3:54 PM, Andrey Mashenkov <
>>>> andrey.mashenkov@gmail.com> wrote:
>>>>
>>>>> Hi Anil,
>>>>>
>>>>> It is ok. Doc says *"If a query contains JOINs, then all the
>>>>> participating caches must have the same degree of parallelism.".*
>>>>> Possibly, it is easy to fix but there can be unobvious limitations, so
>>>>> we need a time to make a POC.
>>>>> I believe, it will be fixed in future releases.
>>>>>
>>>>> On Mon, Mar 20, 2017 at 1:11 PM, Anil <an...@gmail.com> wrote:
>>>>>
>>>>>> Hi Andrey,
>>>>>>
>>>>>> I see few more issues with IGNITE-4826
>>>>>>
>>>>>> 1. queryParallelism should be used for all caches for which queries
>>>>>> are used other it throws following exception.
>>>>>>
>>>>>> Caused by: java.sql.SQLException: Failed to query Ignite.
>>>>>> at org.apache.ignite.internal.jdb
>>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:131)
>>>>>> at org.apache.ignite.internal.jdb
>>>>>> c2.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:76)
>>>>>> at org.apache.commons.dbcp2.Deleg
>>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>>> ent.java:83)
>>>>>> at org.apache.commons.dbcp2.Deleg
>>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>>> ent.java:83)
>>>>>>
>>>>>> Caused by: javax.cache.CacheException: class
>>>>>> org.apache.ignite.IgniteException: Using indexes with different
>>>>>> parallelism levels in same query is forbidden.
>>>>>> at org.apache.ignite.internal.pro
>>>>>> cessors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:760)
>>>>>> at org.apache.ignite.internal.jdb
>>>>>> c2.JdbcQueryTask.call(JdbcQueryTask.java:161)
>>>>>> at org.apache.ignite.internal.jdb
>>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:116)
>>>>>> ... 13 more
>>>>>> 2. query is not returning same result if it is hit number of times.
>>>>>>
>>>>>> please let me know if these are known issues.
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>> Andrey V. Mashenkov
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Andrey V. Mashenkov
>>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
--
Best regards,
Andrey V. Mashenkov
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Hi Andrew,
#1 - it is very simple select query - select * from person hwere personid =
'something';
i just ran the query in for loop and noticed the results are not same.
#2 - it is stable topology. swap is configured. but this test was done when
full load is completed and some compute job going on for other cache.
Please let me know if you have any questions. thanks.
Thanks.
On 20 March 2017 at 21:07, Andrey Mashenkov <an...@gmail.com>
wrote:
> Hi Anil,
>
> 1. Would you please share sql query text?
>
> 2. Is it happening on unstable topology or during rebalancing? Or may be
> eviction\expire policy or swap is configured?
>
> On Mon, Mar 20, 2017 at 5:41 PM, Anil <an...@gmail.com> wrote:
>
>> Yes. i am using partition cache only with no joins :)
>>
>> how about #2 ?
>>
>> On 20 March 2017 at 19:20, Andrey Mashenkov <an...@gmail.com>
>> wrote:
>>
>>> Hi Anil,
>>>
>>> I should although mention that Replicated caches can participate in same
>>> query with partitioned caches regardless a degree of parallelizm.
>>> This limitation relates to partitioned caches only.
>>>
>>> On Mon, Mar 20, 2017 at 3:54 PM, Andrey Mashenkov <
>>> andrey.mashenkov@gmail.com> wrote:
>>>
>>>> Hi Anil,
>>>>
>>>> It is ok. Doc says *"If a query contains JOINs, then all the
>>>> participating caches must have the same degree of parallelism.".*
>>>> Possibly, it is easy to fix but there can be unobvious limitations, so
>>>> we need a time to make a POC.
>>>> I believe, it will be fixed in future releases.
>>>>
>>>> On Mon, Mar 20, 2017 at 1:11 PM, Anil <an...@gmail.com> wrote:
>>>>
>>>>> Hi Andrey,
>>>>>
>>>>> I see few more issues with IGNITE-4826
>>>>>
>>>>> 1. queryParallelism should be used for all caches for which queries
>>>>> are used other it throws following exception.
>>>>>
>>>>> Caused by: java.sql.SQLException: Failed to query Ignite.
>>>>> at org.apache.ignite.internal.jdb
>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:131)
>>>>> at org.apache.ignite.internal.jdb
>>>>> c2.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:76)
>>>>> at org.apache.commons.dbcp2.Deleg
>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>> ent.java:83)
>>>>> at org.apache.commons.dbcp2.Deleg
>>>>> atingPreparedStatement.executeQuery(DelegatingPreparedStatem
>>>>> ent.java:83)
>>>>>
>>>>> Caused by: javax.cache.CacheException: class
>>>>> org.apache.ignite.IgniteException: Using indexes with different
>>>>> parallelism levels in same query is forbidden.
>>>>> at org.apache.ignite.internal.pro
>>>>> cessors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:760)
>>>>> at org.apache.ignite.internal.jdb
>>>>> c2.JdbcQueryTask.call(JdbcQueryTask.java:161)
>>>>> at org.apache.ignite.internal.jdb
>>>>> c2.JdbcStatement.executeQuery(JdbcStatement.java:116)
>>>>> ... 13 more
>>>>> 2. query is not returning same result if it is hit number of times.
>>>>>
>>>>> please let me know if these are known issues.
>>>>>
>>>>> Thanks
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Andrey V. Mashenkov
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Anil,
1. Would you please share sql query text?
2. Is it happening on unstable topology or during rebalancing? Or may be
eviction\expire policy or swap is configured?
On Mon, Mar 20, 2017 at 5:41 PM, Anil <an...@gmail.com> wrote:
> Yes. i am using partition cache only with no joins :)
>
> how about #2 ?
>
> On 20 March 2017 at 19:20, Andrey Mashenkov <an...@gmail.com>
> wrote:
>
>> Hi Anil,
>>
>> I should although mention that Replicated caches can participate in same
>> query with partitioned caches regardless a degree of parallelizm.
>> This limitation relates to partitioned caches only.
>>
>> On Mon, Mar 20, 2017 at 3:54 PM, Andrey Mashenkov <
>> andrey.mashenkov@gmail.com> wrote:
>>
>>> Hi Anil,
>>>
>>> It is ok. Doc says *"If a query contains JOINs, then all the
>>> participating caches must have the same degree of parallelism.".*
>>> Possibly, it is easy to fix but there can be unobvious limitations, so
>>> we need a time to make a POC.
>>> I believe, it will be fixed in future releases.
>>>
>>> On Mon, Mar 20, 2017 at 1:11 PM, Anil <an...@gmail.com> wrote:
>>>
>>>> Hi Andrey,
>>>>
>>>> I see few more issues with IGNITE-4826
>>>>
>>>> 1. queryParallelism should be used for all caches for which queries are
>>>> used other it throws following exception.
>>>>
>>>> Caused by: java.sql.SQLException: Failed to query Ignite.
>>>> at org.apache.ignite.internal.jdbc2.JdbcStatement.executeQuery(
>>>> JdbcStatement.java:131)
>>>> at org.apache.ignite.internal.jdbc2.JdbcPreparedStatement.execu
>>>> teQuery(JdbcPreparedStatement.java:76)
>>>> at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute
>>>> Query(DelegatingPreparedStatement.java:83)
>>>> at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute
>>>> Query(DelegatingPreparedStatement.java:83)
>>>>
>>>> Caused by: javax.cache.CacheException: class
>>>> org.apache.ignite.IgniteException: Using indexes with different
>>>> parallelism levels in same query is forbidden.
>>>> at org.apache.ignite.internal.processors.cache.IgniteCacheProxy
>>>> .query(IgniteCacheProxy.java:760)
>>>> at org.apache.ignite.internal.jdbc2.JdbcQueryTask.call(JdbcQuer
>>>> yTask.java:161)
>>>> at org.apache.ignite.internal.jdbc2.JdbcStatement.executeQuery(
>>>> JdbcStatement.java:116)
>>>> ... 13 more
>>>> 2. query is not returning same result if it is hit number of times.
>>>>
>>>> please let me know if these are known issues.
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
--
Best regards,
Andrey V. Mashenkov
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Yes. i am using partition cache only with no joins :)
how about #2 ?
On 20 March 2017 at 19:20, Andrey Mashenkov <an...@gmail.com>
wrote:
> Hi Anil,
>
> I should although mention that Replicated caches can participate in same
> query with partitioned caches regardless a degree of parallelizm.
> This limitation relates to partitioned caches only.
>
> On Mon, Mar 20, 2017 at 3:54 PM, Andrey Mashenkov <
> andrey.mashenkov@gmail.com> wrote:
>
>> Hi Anil,
>>
>> It is ok. Doc says *"If a query contains JOINs, then all the
>> participating caches must have the same degree of parallelism.".*
>> Possibly, it is easy to fix but there can be unobvious limitations, so we
>> need a time to make a POC.
>> I believe, it will be fixed in future releases.
>>
>> On Mon, Mar 20, 2017 at 1:11 PM, Anil <an...@gmail.com> wrote:
>>
>>> Hi Andrey,
>>>
>>> I see few more issues with IGNITE-4826
>>>
>>> 1. queryParallelism should be used for all caches for which queries are
>>> used other it throws following exception.
>>>
>>> Caused by: java.sql.SQLException: Failed to query Ignite.
>>> at org.apache.ignite.internal.jdbc2.JdbcStatement.executeQuery(
>>> JdbcStatement.java:131)
>>> at org.apache.ignite.internal.jdbc2.JdbcPreparedStatement.execu
>>> teQuery(JdbcPreparedStatement.java:76)
>>> at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute
>>> Query(DelegatingPreparedStatement.java:83)
>>> at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute
>>> Query(DelegatingPreparedStatement.java:83)
>>>
>>> Caused by: javax.cache.CacheException: class
>>> org.apache.ignite.IgniteException: Using indexes with different
>>> parallelism levels in same query is forbidden.
>>> at org.apache.ignite.internal.processors.cache.IgniteCacheProxy
>>> .query(IgniteCacheProxy.java:760)
>>> at org.apache.ignite.internal.jdbc2.JdbcQueryTask.call(JdbcQuer
>>> yTask.java:161)
>>> at org.apache.ignite.internal.jdbc2.JdbcStatement.executeQuery(
>>> JdbcStatement.java:116)
>>> ... 13 more
>>> 2. query is not returning same result if it is hit number of times.
>>>
>>> please let me know if these are known issues.
>>>
>>> Thanks
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Anil,
I should although mention that Replicated caches can participate in same
query with partitioned caches regardless a degree of parallelizm.
This limitation relates to partitioned caches only.
On Mon, Mar 20, 2017 at 3:54 PM, Andrey Mashenkov <
andrey.mashenkov@gmail.com> wrote:
> Hi Anil,
>
> It is ok. Doc says *"If a query contains JOINs, then all the
> participating caches must have the same degree of parallelism.".*
> Possibly, it is easy to fix but there can be unobvious limitations, so we
> need a time to make a POC.
> I believe, it will be fixed in future releases.
>
> On Mon, Mar 20, 2017 at 1:11 PM, Anil <an...@gmail.com> wrote:
>
>> Hi Andrey,
>>
>> I see few more issues with IGNITE-4826
>>
>> 1. queryParallelism should be used for all caches for which queries are
>> used other it throws following exception.
>>
>> Caused by: java.sql.SQLException: Failed to query Ignite.
>> at org.apache.ignite.internal.jdbc2.JdbcStatement.executeQuery(
>> JdbcStatement.java:131)
>> at org.apache.ignite.internal.jdbc2.JdbcPreparedStatement.execu
>> teQuery(JdbcPreparedStatement.java:76)
>> at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute
>> Query(DelegatingPreparedStatement.java:83)
>> at org.apache.commons.dbcp2.DelegatingPreparedStatement.execute
>> Query(DelegatingPreparedStatement.java:83)
>>
>> Caused by: javax.cache.CacheException: class
>> org.apache.ignite.IgniteException: Using indexes with different
>> parallelism levels in same query is forbidden.
>> at org.apache.ignite.internal.processors.cache.IgniteCacheProxy
>> .query(IgniteCacheProxy.java:760)
>> at org.apache.ignite.internal.jdbc2.JdbcQueryTask.call(JdbcQuer
>> yTask.java:161)
>> at org.apache.ignite.internal.jdbc2.JdbcStatement.executeQuery(
>> JdbcStatement.java:116)
>> ... 13 more
>> 2. query is not returning same result if it is hit number of times.
>>
>> please let me know if these are known issues.
>>
>> Thanks
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
--
Best regards,
Andrey V. Mashenkov
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Anil,
It is ok. Doc says *"If a query contains JOINs, then all the participating
caches must have the same degree of parallelism.".*
Possibly, it is easy to fix but there can be unobvious limitations, so we
need a time to make a POC.
I believe, it will be fixed in future releases.
On Mon, Mar 20, 2017 at 1:11 PM, Anil <an...@gmail.com> wrote:
> Hi Andrey,
>
> I see few more issues with IGNITE-4826
>
> 1. queryParallelism should be used for all caches for which queries are
> used other it throws following exception.
>
> Caused by: java.sql.SQLException: Failed to query Ignite.
> at org.apache.ignite.internal.jdbc2.JdbcStatement.
> executeQuery(JdbcStatement.java:131)
> at org.apache.ignite.internal.jdbc2.JdbcPreparedStatement.
> executeQuery(JdbcPreparedStatement.java:76)
> at org.apache.commons.dbcp2.DelegatingPreparedStatement.
> executeQuery(DelegatingPreparedStatement.java:83)
> at org.apache.commons.dbcp2.DelegatingPreparedStatement.
> executeQuery(DelegatingPreparedStatement.java:83)
>
> Caused by: javax.cache.CacheException: class org.apache.ignite.IgniteException:
> Using indexes with different parallelism levels in same query is forbidden.
> at org.apache.ignite.internal.processors.cache.
> IgniteCacheProxy.query(IgniteCacheProxy.java:760)
> at org.apache.ignite.internal.jdbc2.JdbcQueryTask.call(
> JdbcQueryTask.java:161)
> at org.apache.ignite.internal.jdbc2.JdbcStatement.
> executeQuery(JdbcStatement.java:116)
> ... 13 more
> 2. query is not returning same result if it is hit number of times.
>
> please let me know if these are known issues.
>
> Thanks
>
--
Best regards,
Andrey V. Mashenkov
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Hi Andrey,
I see few more issues with IGNITE-4826
1. queryParallelism should be used for all caches for which queries are
used other it throws following exception.
Caused by: java.sql.SQLException: Failed to query Ignite.
at
org.apache.ignite.internal.jdbc2.JdbcStatement.executeQuery(JdbcStatement.java:131)
at
org.apache.ignite.internal.jdbc2.JdbcPreparedStatement.executeQuery(JdbcPreparedStatement.java:76)
at
org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
at
org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:83)
Caused by: javax.cache.CacheException: class
org.apache.ignite.IgniteException: Using indexes with different parallelism
levels in same query is forbidden.
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxy.query(IgniteCacheProxy.java:760)
at
org.apache.ignite.internal.jdbc2.JdbcQueryTask.call(JdbcQueryTask.java:161)
at
org.apache.ignite.internal.jdbc2.JdbcStatement.executeQuery(JdbcStatement.java:116)
... 13 more
2. query is not returning same result if it is hit number of times.
please let me know if these are known issues.
Thanks
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Hi ,
May I know the ignite-1.9.0 branch ? i want to add the fix given in the
jira and test it.
ignite-1.9.0 pom says 2.0.0-SNAPSHOT.
Thanks
On 15 March 2017 at 23:26, Anil <an...@gmail.com> wrote:
> Hi Andrey,
>
> Thank you.
>
> I see it as Path Available. You guys are quick. I will test the fix
> tomorrow.
>
> Thanks.
>
> On 15 March 2017 at 20:58, Andrey Mashenkov <an...@gmail.com>
> wrote:
>
>> Hi Anil,
>>
>> It is a bug. Error occurs when entry has evicted from cache.
>> I've create a ticket IGNITE-4826 [1].
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-4826
>>
>>
>> On Wed, Mar 15, 2017 at 10:22 AM, Anil <an...@gmail.com> wrote:
>>
>>> Hi Val and Andrey,
>>>
>>> I am seeing exception with following code as well. Not sure why is not
>>> reproduced at your end.,
>>>
>>> Ignite ignite = Ignition.start(new File("/workspace/cache-manager
>>> /test-parallelism/src/main/resources/ignite.xml").toURI().toURL());
>>> IgniteCache<String, Test> cache = ignite.cache("TEST_CACHE");
>>> IgniteDataStreamer<String, Test> streamer =
>>> ignite.dataStreamer("TEST_CACHE");
>>> for (int i =1; i< 100000; i++){
>>> streamer.addData(String.valueOf(i), new Test("1", "1"));
>>> }
>>>
>>> Exception :
>>>
>>> 2017-03-15 12:46:43 ERROR DataStreamerImpl:495 - DataStreamer operation
>>> failed.
>>> class org.apache.ignite.IgniteCheckedException: Failed to finish
>>> operation (too many remaps): 32
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$5.apply(DataStreamerImpl.java:863)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$5.apply(DataStreamerImpl.java:828)
>>> at org.apache.ignite.internal.util.future.GridFutureAdapter$Arr
>>> ayListener.apply(GridFutureAdapter.java:456)
>>> at org.apache.ignite.internal.util.future.GridFutureAdapter$Arr
>>> ayListener.apply(GridFutureAdapter.java:439)
>>> at org.apache.ignite.internal.util.future.GridFutureAdapter.not
>>> ifyListener(GridFutureAdapter.java:271)
>>> at org.apache.ignite.internal.util.future.GridFutureAdapter.not
>>> ifyListeners(GridFutureAdapter.java:259)
>>> at org.apache.ignite.internal.util.future.GridFutureAdapter.onD
>>> one(GridFutureAdapter.java:389)
>>> at org.apache.ignite.internal.util.future.GridFutureAdapter.onD
>>> one(GridFutureAdapter.java:355)
>>> at org.apache.ignite.internal.util.future.GridFutureAdapter.onD
>>> one(GridFutureAdapter.java:343)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$Buffer$2.apply(DataStreamerImpl.java:1564)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$Buffer$2.apply(DataStreamerImpl.java:1554)
>>> at org.apache.ignite.internal.util.future.GridFutureAdapter.not
>>> ifyListener(GridFutureAdapter.java:271)
>>> at org.apache.ignite.internal.util.future.GridFutureAdapter.lis
>>> ten(GridFutureAdapter.java:228)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$Buffer.localUpdate(DataStreamerImpl.java:1554)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$Buffer.submit(DataStreamerImpl.java:1626)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$Buffer.update(DataStreamerImpl.java:1416)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl.load0(DataStreamerImpl.java:932)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl.access$1100(DataStreamerImpl.java:121)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$5$1.run(DataStreamerImpl.java:876)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$5$2.call(DataStreamerImpl.java:903)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$5$2.call(DataStreamerImpl.java:891)
>>> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader
>>> (IgniteUtils.java:6618)
>>> at org.apache.ignite.internal.processors.closure.GridClosurePro
>>> cessor$2.body(GridClosureProcessor.java:925)
>>> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWo
>>> rker.java:110)
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>>> Executor.java:1142)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>>> lExecutor.java:617)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: class org.apache.ignite.IgniteCheckedException:
>>> GridH2QueryContext is not initialized.
>>> at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils
>>> .java:7239)
>>> at org.apache.ignite.internal.processors.closure.GridClosurePro
>>> cessor$2.body(GridClosureProcessor.java:933)
>>> ... 4 more
>>> Caused by: java.lang.IllegalStateException: GridH2QueryContext is not
>>> initialized.
>>> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Ind
>>> exBase.threadLocalSegment(GridH2IndexBase.java:197)
>>> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Tre
>>> eIndex.findOne(GridH2TreeIndex.java:290)
>>> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Tab
>>> le.onSwapUnswap(GridH2Table.java:232)
>>> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Tab
>>> le.onSwap(GridH2Table.java:189)
>>> at org.apache.ignite.internal.processors.query.h2.IgniteH2Index
>>> ing.onSwap(IgniteH2Indexing.java:737)
>>> at org.apache.ignite.internal.processors.query.GridQueryProcess
>>> or.onSwap(GridQueryProcessor.java:1183)
>>> at org.apache.ignite.internal.processors.cache.query.GridCacheQ
>>> ueryManager.onSwap(GridCacheQueryManager.java:398)
>>> at org.apache.ignite.internal.processors.cache.GridCacheSwapMan
>>> ager.write(GridCacheSwapManager.java:1345)
>>> at org.apache.ignite.internal.processors.cache.GridCacheMapEntr
>>> y.swap(GridCacheMapEntry.java:681)
>>> at org.apache.ignite.internal.processors.cache.GridCacheMapEntr
>>> y.evictInternal(GridCacheMapEntry.java:4310)
>>> at org.apache.ignite.internal.processors.cache.GridCacheEvictio
>>> nManager.evict0(GridCacheEvictionManager.java:709)
>>> at org.apache.ignite.internal.processors.cache.GridCacheEvictio
>>> nManager.touch(GridCacheEvictionManager.java:798)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl$IsolatedUpdater.receive(DataStreamerImpl.java:1957)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merUpdateJob.call(DataStreamerUpdateJob.java:140)
>>> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader
>>> (IgniteUtils.java:6618)
>>> at org.apache.ignite.internal.processors.closure.GridClosurePro
>>> cessor$2.body(GridClosureProcessor.java:925)
>>> ... 4 more
>>> Exception in thread "main" java.lang.IllegalStateException: Data
>>> streamer has been closed.
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl.enterBusy(DataStreamerImpl.java:406)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl.addDataInternal(DataStreamerImpl.java:613)
>>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>>> merImpl.addData(DataStreamerImpl.java:667)
>>> at com.test.cache.loader.TestManager.main(TestManager.java:19)
>>>
>>>
>>> attached the ignite.xml that is used for test.
>>>
>>> Please let me know if you have any questions. thanks.
>>>
>>> Thanks
>>>
>>> On 15 March 2017 at 11:59, Anil <an...@gmail.com> wrote:
>>>
>>>> Sure Val. let me try again.
>>>>
>>>> Thanks.
>>>>
>>>> On 14 March 2017 at 20:28, vkulichenko <va...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Anil,
>>>>>
>>>>> I tried to run your project and also didn't get the exception. Please
>>>>> provide exact steps how to run it in order to reproduce the behavior.
>>>>>
>>>>> -Val
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context: http://apache-ignite-users.705
>>>>> 18.x6.nabble.com/IGNITE-4106-tp11073p11169.html
>>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Hi Andrey,
Thank you.
I see it as Path Available. You guys are quick. I will test the fix
tomorrow.
Thanks.
On 15 March 2017 at 20:58, Andrey Mashenkov <an...@gmail.com>
wrote:
> Hi Anil,
>
> It is a bug. Error occurs when entry has evicted from cache.
> I've create a ticket IGNITE-4826 [1].
>
> [1] https://issues.apache.org/jira/browse/IGNITE-4826
>
>
> On Wed, Mar 15, 2017 at 10:22 AM, Anil <an...@gmail.com> wrote:
>
>> Hi Val and Andrey,
>>
>> I am seeing exception with following code as well. Not sure why is not
>> reproduced at your end.,
>>
>> Ignite ignite = Ignition.start(new File("/workspace/cache-manager
>> /test-parallelism/src/main/resources/ignite.xml").toURI().toURL());
>> IgniteCache<String, Test> cache = ignite.cache("TEST_CACHE");
>> IgniteDataStreamer<String, Test> streamer = ignite.dataStreamer("TEST_CACH
>> E");
>> for (int i =1; i< 100000; i++){
>> streamer.addData(String.valueOf(i), new Test("1", "1"));
>> }
>>
>> Exception :
>>
>> 2017-03-15 12:46:43 ERROR DataStreamerImpl:495 - DataStreamer operation
>> failed.
>> class org.apache.ignite.IgniteCheckedException: Failed to finish
>> operation (too many remaps): 32
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$5.apply(DataStreamerImpl.java:863)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$5.apply(DataStreamerImpl.java:828)
>> at org.apache.ignite.internal.util.future.GridFutureAdapter$Arr
>> ayListener.apply(GridFutureAdapter.java:456)
>> at org.apache.ignite.internal.util.future.GridFutureAdapter$Arr
>> ayListener.apply(GridFutureAdapter.java:439)
>> at org.apache.ignite.internal.util.future.GridFutureAdapter.not
>> ifyListener(GridFutureAdapter.java:271)
>> at org.apache.ignite.internal.util.future.GridFutureAdapter.not
>> ifyListeners(GridFutureAdapter.java:259)
>> at org.apache.ignite.internal.util.future.GridFutureAdapter.onD
>> one(GridFutureAdapter.java:389)
>> at org.apache.ignite.internal.util.future.GridFutureAdapter.onD
>> one(GridFutureAdapter.java:355)
>> at org.apache.ignite.internal.util.future.GridFutureAdapter.onD
>> one(GridFutureAdapter.java:343)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$Buffer$2.apply(DataStreamerImpl.java:1564)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$Buffer$2.apply(DataStreamerImpl.java:1554)
>> at org.apache.ignite.internal.util.future.GridFutureAdapter.not
>> ifyListener(GridFutureAdapter.java:271)
>> at org.apache.ignite.internal.util.future.GridFutureAdapter.lis
>> ten(GridFutureAdapter.java:228)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$Buffer.localUpdate(DataStreamerImpl.java:1554)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$Buffer.submit(DataStreamerImpl.java:1626)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$Buffer.update(DataStreamerImpl.java:1416)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl.load0(DataStreamerImpl.java:932)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl.access$1100(DataStreamerImpl.java:121)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$5$1.run(DataStreamerImpl.java:876)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$5$2.call(DataStreamerImpl.java:903)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$5$2.call(DataStreamerImpl.java:891)
>> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader
>> (IgniteUtils.java:6618)
>> at org.apache.ignite.internal.processors.closure.GridClosurePro
>> cessor$2.body(GridClosureProcessor.java:925)
>> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWo
>> rker.java:110)
>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPool
>> Executor.java:1142)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoo
>> lExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: class org.apache.ignite.IgniteCheckedException:
>> GridH2QueryContext is not initialized.
>> at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils
>> .java:7239)
>> at org.apache.ignite.internal.processors.closure.GridClosurePro
>> cessor$2.body(GridClosureProcessor.java:933)
>> ... 4 more
>> Caused by: java.lang.IllegalStateException: GridH2QueryContext is not
>> initialized.
>> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Ind
>> exBase.threadLocalSegment(GridH2IndexBase.java:197)
>> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Tre
>> eIndex.findOne(GridH2TreeIndex.java:290)
>> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Tab
>> le.onSwapUnswap(GridH2Table.java:232)
>> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Tab
>> le.onSwap(GridH2Table.java:189)
>> at org.apache.ignite.internal.processors.query.h2.IgniteH2Index
>> ing.onSwap(IgniteH2Indexing.java:737)
>> at org.apache.ignite.internal.processors.query.GridQueryProcess
>> or.onSwap(GridQueryProcessor.java:1183)
>> at org.apache.ignite.internal.processors.cache.query.GridCacheQ
>> ueryManager.onSwap(GridCacheQueryManager.java:398)
>> at org.apache.ignite.internal.processors.cache.GridCacheSwapMan
>> ager.write(GridCacheSwapManager.java:1345)
>> at org.apache.ignite.internal.processors.cache.GridCacheMapEntr
>> y.swap(GridCacheMapEntry.java:681)
>> at org.apache.ignite.internal.processors.cache.GridCacheMapEntr
>> y.evictInternal(GridCacheMapEntry.java:4310)
>> at org.apache.ignite.internal.processors.cache.GridCacheEvictio
>> nManager.evict0(GridCacheEvictionManager.java:709)
>> at org.apache.ignite.internal.processors.cache.GridCacheEvictio
>> nManager.touch(GridCacheEvictionManager.java:798)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl$IsolatedUpdater.receive(DataStreamerImpl.java:1957)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merUpdateJob.call(DataStreamerUpdateJob.java:140)
>> at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader
>> (IgniteUtils.java:6618)
>> at org.apache.ignite.internal.processors.closure.GridClosurePro
>> cessor$2.body(GridClosureProcessor.java:925)
>> ... 4 more
>> Exception in thread "main" java.lang.IllegalStateException: Data
>> streamer has been closed.
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl.enterBusy(DataStreamerImpl.java:406)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl.addDataInternal(DataStreamerImpl.java:613)
>> at org.apache.ignite.internal.processors.datastreamer.DataStrea
>> merImpl.addData(DataStreamerImpl.java:667)
>> at com.test.cache.loader.TestManager.main(TestManager.java:19)
>>
>>
>> attached the ignite.xml that is used for test.
>>
>> Please let me know if you have any questions. thanks.
>>
>> Thanks
>>
>> On 15 March 2017 at 11:59, Anil <an...@gmail.com> wrote:
>>
>>> Sure Val. let me try again.
>>>
>>> Thanks.
>>>
>>> On 14 March 2017 at 20:28, vkulichenko <va...@gmail.com>
>>> wrote:
>>>
>>>> Hi Anil,
>>>>
>>>> I tried to run your project and also didn't get the exception. Please
>>>> provide exact steps how to run it in order to reproduce the behavior.
>>>>
>>>> -Val
>>>>
>>>>
>>>>
>>>> --
>>>> View this message in context: http://apache-ignite-users.705
>>>> 18.x6.nabble.com/IGNITE-4106-tp11073p11169.html
>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>>
>>>
>>>
>>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Anil,
It is a bug. Error occurs when entry has evicted from cache.
I've create a ticket IGNITE-4826 [1].
[1] https://issues.apache.org/jira/browse/IGNITE-4826
On Wed, Mar 15, 2017 at 10:22 AM, Anil <an...@gmail.com> wrote:
> Hi Val and Andrey,
>
> I am seeing exception with following code as well. Not sure why is not
> reproduced at your end.,
>
> Ignite ignite = Ignition.start(new File("/workspace/cache-
> manager/test-parallelism/src/main/resources/ignite.xml").toURI().toURL());
> IgniteCache<String, Test> cache = ignite.cache("TEST_CACHE");
> IgniteDataStreamer<String, Test> streamer = ignite.dataStreamer("TEST_
> CACHE");
> for (int i =1; i< 100000; i++){
> streamer.addData(String.valueOf(i), new Test("1", "1"));
> }
>
> Exception :
>
> 2017-03-15 12:46:43 ERROR DataStreamerImpl:495 - DataStreamer operation
> failed.
> class org.apache.ignite.IgniteCheckedException: Failed to finish
> operation (too many remaps): 32
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$5.apply(DataStreamerImpl.java:863)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$5.apply(DataStreamerImpl.java:828)
> at org.apache.ignite.internal.util.future.GridFutureAdapter$
> ArrayListener.apply(GridFutureAdapter.java:456)
> at org.apache.ignite.internal.util.future.GridFutureAdapter$
> ArrayListener.apply(GridFutureAdapter.java:439)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.
> notifyListener(GridFutureAdapter.java:271)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.
> notifyListeners(GridFutureAdapter.java:259)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.
> onDone(GridFutureAdapter.java:389)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.
> onDone(GridFutureAdapter.java:355)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.
> onDone(GridFutureAdapter.java:343)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$Buffer$2.apply(DataStreamerImpl.java:1564)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$Buffer$2.apply(DataStreamerImpl.java:1554)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.
> notifyListener(GridFutureAdapter.java:271)
> at org.apache.ignite.internal.util.future.GridFutureAdapter.
> listen(GridFutureAdapter.java:228)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$Buffer.localUpdate(DataStreamerImpl.java:1554)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$Buffer.submit(DataStreamerImpl.java:1626)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$Buffer.update(DataStreamerImpl.java:1416)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl.load0(DataStreamerImpl.java:932)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl.access$1100(DataStreamerImpl.java:121)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$5$1.run(DataStreamerImpl.java:876)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$5$2.call(DataStreamerImpl.java:903)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl$5$2.call(DataStreamerImpl.java:891)
> at org.apache.ignite.internal.util.IgniteUtils.
> wrapThreadLoader(IgniteUtils.java:6618)
> at org.apache.ignite.internal.processors.closure.
> GridClosureProcessor$2.body(GridClosureProcessor.java:925)
> at org.apache.ignite.internal.util.worker.GridWorker.run(
> GridWorker.java:110)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: class org.apache.ignite.IgniteCheckedException:
> GridH2QueryContext is not initialized.
> at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7239)
> at org.apache.ignite.internal.processors.closure.
> GridClosureProcessor$2.body(GridClosureProcessor.java:933)
> ... 4 more
> Caused by: java.lang.IllegalStateException: GridH2QueryContext is not
> initialized.
> at org.apache.ignite.internal.processors.query.h2.opt.GridH2IndexBase.
> threadLocalSegment(GridH2IndexBase.java:197)
> at org.apache.ignite.internal.processors.query.h2.opt.
> GridH2TreeIndex.findOne(GridH2TreeIndex.java:290)
> at org.apache.ignite.internal.processors.query.h2.opt.
> GridH2Table.onSwapUnswap(GridH2Table.java:232)
> at org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.onSwap(
> GridH2Table.java:189)
> at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.onSwap(
> IgniteH2Indexing.java:737)
> at org.apache.ignite.internal.processors.query.GridQueryProcessor.onSwap(
> GridQueryProcessor.java:1183)
> at org.apache.ignite.internal.processors.cache.query.
> GridCacheQueryManager.onSwap(GridCacheQueryManager.java:398)
> at org.apache.ignite.internal.processors.cache.GridCacheSwapManager.write(
> GridCacheSwapManager.java:1345)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.swap(
> GridCacheMapEntry.java:681)
> at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.
> evictInternal(GridCacheMapEntry.java:4310)
> at org.apache.ignite.internal.processors.cache.GridCacheEvictionManager.
> evict0(GridCacheEvictionManager.java:709)
> at org.apache.ignite.internal.processors.cache.GridCacheEvictionManager.
> touch(GridCacheEvictionManager.java:798)
> at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$
> IsolatedUpdater.receive(DataStreamerImpl.java:1957)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140)
> at org.apache.ignite.internal.util.IgniteUtils.
> wrapThreadLoader(IgniteUtils.java:6618)
> at org.apache.ignite.internal.processors.closure.
> GridClosureProcessor$2.body(GridClosureProcessor.java:925)
> ... 4 more
> Exception in thread "main" java.lang.IllegalStateException: Data streamer
> has been closed.
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl.enterBusy(DataStreamerImpl.java:406)
> at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.
> addDataInternal(DataStreamerImpl.java:613)
> at org.apache.ignite.internal.processors.datastreamer.
> DataStreamerImpl.addData(DataStreamerImpl.java:667)
> at com.test.cache.loader.TestManager.main(TestManager.java:19)
>
>
> attached the ignite.xml that is used for test.
>
> Please let me know if you have any questions. thanks.
>
> Thanks
>
> On 15 March 2017 at 11:59, Anil <an...@gmail.com> wrote:
>
>> Sure Val. let me try again.
>>
>> Thanks.
>>
>> On 14 March 2017 at 20:28, vkulichenko <va...@gmail.com>
>> wrote:
>>
>>> Hi Anil,
>>>
>>> I tried to run your project and also didn't get the exception. Please
>>> provide exact steps how to run it in order to reproduce the behavior.
>>>
>>> -Val
>>>
>>>
>>>
>>> --
>>> View this message in context: http://apache-ignite-users.705
>>> 18.x6.nabble.com/IGNITE-4106-tp11073p11169.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>>
>>
>>
>
--
Best regards,
Andrey V. Mashenkov
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Hi Val and Andrey,
I am seeing exception with following code as well. Not sure why is not
reproduced at your end.,
Ignite ignite = Ignition.start(new
File("/workspace/cache-manager/test-parallelism/src/main/resources/ignite.xml").toURI().toURL());
IgniteCache<String, Test> cache = ignite.cache("TEST_CACHE");
IgniteDataStreamer<String, Test> streamer =
ignite.dataStreamer("TEST_CACHE");
for (int i =1; i< 100000; i++){
streamer.addData(String.valueOf(i), new Test("1", "1"));
}
Exception :
2017-03-15 12:46:43 ERROR DataStreamerImpl:495 - DataStreamer operation
failed.
class org.apache.ignite.IgniteCheckedException: Failed to finish operation
(too many remaps): 32
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:863)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5.apply(DataStreamerImpl.java:828)
at
org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:456)
at
org.apache.ignite.internal.util.future.GridFutureAdapter$ArrayListener.apply(GridFutureAdapter.java:439)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:271)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListeners(GridFutureAdapter.java:259)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:389)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:355)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:343)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer$2.apply(DataStreamerImpl.java:1564)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer$2.apply(DataStreamerImpl.java:1554)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:271)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.listen(GridFutureAdapter.java:228)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.localUpdate(DataStreamerImpl.java:1554)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.submit(DataStreamerImpl.java:1626)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.update(DataStreamerImpl.java:1416)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.load0(DataStreamerImpl.java:932)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.access$1100(DataStreamerImpl.java:121)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5$1.run(DataStreamerImpl.java:876)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5$2.call(DataStreamerImpl.java:903)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$5$2.call(DataStreamerImpl.java:891)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6618)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:925)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: class org.apache.ignite.IgniteCheckedException:
GridH2QueryContext is not initialized.
at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7239)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:933)
... 4 more
Caused by: java.lang.IllegalStateException: GridH2QueryContext is not
initialized.
at
org.apache.ignite.internal.processors.query.h2.opt.GridH2IndexBase.threadLocalSegment(GridH2IndexBase.java:197)
at
org.apache.ignite.internal.processors.query.h2.opt.GridH2TreeIndex.findOne(GridH2TreeIndex.java:290)
at
org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.onSwapUnswap(GridH2Table.java:232)
at
org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.onSwap(GridH2Table.java:189)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.onSwap(IgniteH2Indexing.java:737)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onSwap(GridQueryProcessor.java:1183)
at
org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.onSwap(GridCacheQueryManager.java:398)
at
org.apache.ignite.internal.processors.cache.GridCacheSwapManager.write(GridCacheSwapManager.java:1345)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.swap(GridCacheMapEntry.java:681)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.evictInternal(GridCacheMapEntry.java:4310)
at
org.apache.ignite.internal.processors.cache.GridCacheEvictionManager.evict0(GridCacheEvictionManager.java:709)
at
org.apache.ignite.internal.processors.cache.GridCacheEvictionManager.touch(GridCacheEvictionManager.java:798)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:1957)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140)
at
org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:6618)
at
org.apache.ignite.internal.processors.closure.GridClosureProcessor$2.body(GridClosureProcessor.java:925)
... 4 more
Exception in thread "main" java.lang.IllegalStateException: Data streamer
has been closed.
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:406)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:613)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:667)
at com.test.cache.loader.TestManager.main(TestManager.java:19)
attached the ignite.xml that is used for test.
Please let me know if you have any questions. thanks.
Thanks
On 15 March 2017 at 11:59, Anil <an...@gmail.com> wrote:
> Sure Val. let me try again.
>
> Thanks.
>
> On 14 March 2017 at 20:28, vkulichenko <va...@gmail.com>
> wrote:
>
>> Hi Anil,
>>
>> I tried to run your project and also didn't get the exception. Please
>> provide exact steps how to run it in order to reproduce the behavior.
>>
>> -Val
>>
>>
>>
>> --
>> View this message in context: http://apache-ignite-users.705
>> 18.x6.nabble.com/IGNITE-4106-tp11073p11169.html
>> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>>
>
>
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Sure Val. let me try again.
Thanks.
On 14 March 2017 at 20:28, vkulichenko <va...@gmail.com>
wrote:
> Hi Anil,
>
> I tried to run your project and also didn't get the exception. Please
> provide exact steps how to run it in order to reproduce the behavior.
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/IGNITE-4106-tp11073p11169.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>
Re: IGNITE-4106
Posted by vkulichenko <va...@gmail.com>.
Hi Anil,
I tried to run your project and also didn't get the exception. Please
provide exact steps how to run it in order to reproduce the behavior.
-Val
--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/IGNITE-4106-tp11073p11169.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Anil,
I can't reproduce the error.
On Sat, Mar 11, 2017 at 9:16 AM, Anil <an...@gmail.com> wrote:
> Hi Andrey,
>
> i have created test project which pushes the entries to drastreamer using
> for loop and it is working without any issue.
>
> But my application which does the same thing but loads the data from hbase
> and it is failing with *"java.lang.IllegalStateException: Data streamer
> has been closed." *
>
>
>
> *java.lang.IllegalStateException: Data streamer has been closed. at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:406)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:613)
> at
> org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:667)*
>
> i set the system and public thread pool as 32 for an 8 core machine and
> using 4 node cluster.
>
> and i see that loading data into cache for which queryParallelism
> configured is failing and other cache loads working fine.
>
> Thanks
>
>
> On 10 March 2017 at 05:05, Andrey Mashenkov <an...@gmail.com>
> wrote:
>
>> Hi Anil,
>>
>> I don't think it is a thead starvation due to it work fine for 1 thread.
>> Whould you please share a repro?
>>
>> On Thu, Mar 9, 2017 at 9:09 PM, Anil <an...@gmail.com> wrote:
>>
>>>
>>> Hi Andrey,
>>>
>>> i tried to set the parallelism to 2, 4 on 4 node cluster (8 core
>>> machines) and initiated the data load (using compute job). Data streamers
>>> are getting closed and data load is failing.
>>>
>>> When I set the parallelism to 1, data load working as expected. the
>>> issue could be threads starvation/ non availability of resources and not
>>> ignite issue. Do you think any other issue ?
>>>
>>> I will try on 16 core machines cluster tomorrow.
>>>
>>> Thanks.
>>>
>>> On 8 March 2017 at 18:04, Andrey Mashenkov <an...@gmail.com>
>>> wrote:
>>>
>>>> Hi Anil,
>>>>
>>>> SQL queries are run in system pool.
>>>>
>>>> On Wed, Mar 8, 2017 at 3:04 PM, Anil <an...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Does queryparallelism uses system pool or separate thread pool other
>>>>> than system and public thread pool ? please clarify.
>>>>>
>>>>> 1. https://apacheignite.readme.io/docs/sql-performance-and-d
>>>>> ebugging#sql-performance-and-usability-considerations
>>>>>
>>>>> Thanks.
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Best regards,
>>>> Andrey V. Mashenkov
>>>>
>>>
>>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
--
Best regards,
Andrey V. Mashenkov
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Hi Andrey,
i have created test project which pushes the entries to drastreamer using
for loop and it is working without any issue.
But my application which does the same thing but loads the data from hbase
and it is failing with *"java.lang.IllegalStateException: Data streamer has
been closed." *
*java.lang.IllegalStateException: Data streamer has been closed. at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.enterBusy(DataStreamerImpl.java:406)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addDataInternal(DataStreamerImpl.java:613)
at
org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl.addData(DataStreamerImpl.java:667)*
i set the system and public thread pool as 32 for an 8 core machine and
using 4 node cluster.
and i see that loading data into cache for which queryParallelism
configured is failing and other cache loads working fine.
Thanks
On 10 March 2017 at 05:05, Andrey Mashenkov <an...@gmail.com>
wrote:
> Hi Anil,
>
> I don't think it is a thead starvation due to it work fine for 1 thread.
> Whould you please share a repro?
>
> On Thu, Mar 9, 2017 at 9:09 PM, Anil <an...@gmail.com> wrote:
>
>>
>> Hi Andrey,
>>
>> i tried to set the parallelism to 2, 4 on 4 node cluster (8 core
>> machines) and initiated the data load (using compute job). Data streamers
>> are getting closed and data load is failing.
>>
>> When I set the parallelism to 1, data load working as expected. the issue
>> could be threads starvation/ non availability of resources and not ignite
>> issue. Do you think any other issue ?
>>
>> I will try on 16 core machines cluster tomorrow.
>>
>> Thanks.
>>
>> On 8 March 2017 at 18:04, Andrey Mashenkov <an...@gmail.com>
>> wrote:
>>
>>> Hi Anil,
>>>
>>> SQL queries are run in system pool.
>>>
>>> On Wed, Mar 8, 2017 at 3:04 PM, Anil <an...@gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Does queryparallelism uses system pool or separate thread pool other
>>>> than system and public thread pool ? please clarify.
>>>>
>>>> 1. https://apacheignite.readme.io/docs/sql-performance-and-d
>>>> ebugging#sql-performance-and-usability-considerations
>>>>
>>>> Thanks.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>
>>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Anil,
I don't think it is a thead starvation due to it work fine for 1 thread.
Whould you please share a repro?
On Thu, Mar 9, 2017 at 9:09 PM, Anil <an...@gmail.com> wrote:
>
> Hi Andrey,
>
> i tried to set the parallelism to 2, 4 on 4 node cluster (8 core machines)
> and initiated the data load (using compute job). Data streamers are getting
> closed and data load is failing.
>
> When I set the parallelism to 1, data load working as expected. the issue
> could be threads starvation/ non availability of resources and not ignite
> issue. Do you think any other issue ?
>
> I will try on 16 core machines cluster tomorrow.
>
> Thanks.
>
> On 8 March 2017 at 18:04, Andrey Mashenkov <an...@gmail.com>
> wrote:
>
>> Hi Anil,
>>
>> SQL queries are run in system pool.
>>
>> On Wed, Mar 8, 2017 at 3:04 PM, Anil <an...@gmail.com> wrote:
>>
>>> Hi,
>>>
>>> Does queryparallelism uses system pool or separate thread pool other
>>> than system and public thread pool ? please clarify.
>>>
>>> 1. https://apacheignite.readme.io/docs/sql-performance-and-d
>>> ebugging#sql-performance-and-usability-considerations
>>>
>>> Thanks.
>>>
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrey V. Mashenkov
>>
>
>
--
Best regards,
Andrey V. Mashenkov
Re: IGNITE-4106
Posted by Anil <an...@gmail.com>.
Hi Andrey,
i tried to set the parallelism to 2, 4 on 4 node cluster (8 core machines)
and initiated the data load (using compute job). Data streamers are getting
closed and data load is failing.
When I set the parallelism to 1, data load working as expected. the issue
could be threads starvation/ non availability of resources and not ignite
issue. Do you think any other issue ?
I will try on 16 core machines cluster tomorrow.
Thanks.
On 8 March 2017 at 18:04, Andrey Mashenkov <an...@gmail.com>
wrote:
> Hi Anil,
>
> SQL queries are run in system pool.
>
> On Wed, Mar 8, 2017 at 3:04 PM, Anil <an...@gmail.com> wrote:
>
>> Hi,
>>
>> Does queryparallelism uses system pool or separate thread pool other than
>> system and public thread pool ? please clarify.
>>
>> 1. https://apacheignite.readme.io/docs/sql-performance-and-
>> debugging#sql-performance-and-usability-considerations
>>
>> Thanks.
>>
>>
>
>
>
> --
> Best regards,
> Andrey V. Mashenkov
>
Re: IGNITE-4106
Posted by Andrey Mashenkov <an...@gmail.com>.
Hi Anil,
SQL queries are run in system pool.
On Wed, Mar 8, 2017 at 3:04 PM, Anil <an...@gmail.com> wrote:
> Hi,
>
> Does queryparallelism uses system pool or separate thread pool other than
> system and public thread pool ? please clarify.
>
> 1. https://apacheignite.readme.io/docs/sql-performance-and-debugging#sql-
> performance-and-usability-considerations
>
> Thanks.
>
>
--
Best regards,
Andrey V. Mashenkov