You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@ignite.apache.org by Hemambara <ko...@gmail.com> on 2019/10/03 19:19:44 UTC
Issue with adding nested index dynamically
We have to add indexes on cache dynamically on java pojo with nested objects.
In the below example we do not have @QuerySqlField. In this case if I try to
add index on "username" dynamically using "CREATE INDEX" it worked. But if I
want to add index on "Address.zipcode" - this is not working as we are NOT
creating aliases for these fields which are getting added dynamically. I see
there is a bug in this implementation. I fixed the bug (please see below)
and rebuilt the jar in my local and it worked fine. Proposing this bug to
this open source community so that it can be fixed in ignite and we can
start using the jar from central repo. We have just started using ignite and
planning to take this to production. So it would be helpful if we can
implement this bug fix.
User{
String userName;
Address address;
}
Address{
String streetName;
String zipcode;
}
Bug Fix: QueryUtils.class
------------------------------
Method name :
------------------
QueryBinaryProperty buildBinaryProperty(GridKernalContext ctx, String
pathStr,
Class<?> resType, Map<String, String> aliases, @Nullable Boolean
isKeyField, boolean notNull, Object dlftVal,
int precision, int scale)
--------
Change from :
-----------------
String alias = (String)aliases.get(fullName.toString());
to :
-----
String alias = aliases.get(fullName.toString()) == null ?
fullName.toString() : aliases.get(fullName.toString());
With this way we will always make sure we have default alias (if not
provided), otherwise aliases on dynamic query fields and dynamic indexes are
not working properly
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
I have tested with adding QuerySqlFields as well, it is enabling to add
dynamic field and indexes on nested objects but if I restart the client
again it is not able to join the grid because it is not able to find this
column. But with the fix and maintaining proper aliases we are able to add
fields and indexes dynamically on direct/ nested attribute. This
functionality will be out of the box with this fix. Please let me know if
you still see any issue.
Ignite does provide dynamic addition of columns and indexes but it is just
that there is a bug on adding indexes dynamically on QuerySqlFields nested
objects. With this small fix it will run with out any issues.
SEVERE: Failed to reinitialize local partitions (rebalancing will be
stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion
[topVer=3, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=7af3d442-946f-4572-9fb7-08971b8d82cc, addrs=[0:0:0:0:0:0:0:1,
114.18.146.93, 127.0.0.1], sockAddrs=[/0:0:0:0:0:0:0:1:0, /127.0.0.1:0,
DTC4C4A8C85A622.ent.wfb.bank.corp/114.18.146.93:0], discPort=0, order=3,
intOrder=0, lastExchangeTime=1570647484111, loc=true,
ver=2.7.6#20190911-sha1:21f7ca41, isClient=true], topVer=3,
nodeId8=7af3d442, msg=null, type=NODE_JOINED, tstamp=1570647484265],
nodeId=7af3d442, evt=NODE_JOINED]
class org.apache.ignite.IgniteCheckedException: Field not found:
Users.userName
at
org.apache.ignite.internal.processors.query.QueryIndexDescriptorImpl.addField(QueryIndexDescriptorImpl.java:124)
at
org.apache.ignite.internal.processors.query.QueryUtils.createIndexDescriptor(QueryUtils.java:735)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndex(QueryUtils.java:752)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndexes(QueryUtils.java:688)
at
org.apache.ignite.internal.processors.query.QueryUtils.processBinaryMeta(QueryUtils.java:612)
at
org.apache.ignite.internal.processors.query.QueryUtils.typeForQueryEntity(QueryUtils.java:480)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:706)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:866)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1337)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2172)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:2030)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:927)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:769)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2681)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2553)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
ignite_bug_share.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/t2625/ignite_bug_share.zip>
I have tested by adding QuerySqlFields as well, with or with out it, it is
working when I add it, but if I restart the client it is not able to
identify nested columns as its alias is null or not equal to whole path.
Please see below exception. If we can make alias with whole nested object
path we will be able to add indexes and fields dynamically with out any
issues. Ignite, out of the box provides dynamic field and index alterations.
But it has a bug when we add nested field / index dynamically. This small
fix will fix this bug.
PFA for the code.
SEVERE: Failed to reinitialize local partitions (rebalancing will be
stopped): GridDhtPartitionExchangeId [topVer=AffinityTopologyVersion
[topVer=5, minorTopVer=0], discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=a5379438-868b-43e4-b402-02edf2913795, addrs=[0:0:0:0:0:0:0:1,
114.18.146.93, 127.0.0.1], sockAddrs=[/0:0:0:0:0:0:0:1:0, /127.0.0.1:0,
DTC4C4A8C85A622.ent.wfb.bank.corp/114.18.146.93:0], discPort=0, order=5,
intOrder=0, lastExchangeTime=1570648315703, loc=true,
ver=2.7.6#20190911-sha1:21f7ca41, isClient=true], topVer=5,
nodeId8=a5379438, msg=null, type=NODE_JOINED, tstamp=1570648318843],
nodeId=a5379438, evt=NODE_JOINED]
class org.apache.ignite.IgniteCheckedException: Field not found:
Users.userName
at
org.apache.ignite.internal.processors.query.QueryIndexDescriptorImpl.addField(QueryIndexDescriptorImpl.java:124)
at
org.apache.ignite.internal.processors.query.QueryUtils.createIndexDescriptor(QueryUtils.java:735)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndex(QueryUtils.java:752)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndexes(QueryUtils.java:688)
at
org.apache.ignite.internal.processors.query.QueryUtils.processBinaryMeta(QueryUtils.java:612)
at
org.apache.ignite.internal.processors.query.QueryUtils.typeForQueryEntity(QueryUtils.java:480)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:706)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:866)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1337)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2172)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:2030)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:927)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:769)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2681)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2553)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I think you should move this discussion over to developers list.
Regards,
--
Ilya Kasnacheev
вт, 26 нояб. 2019 г. в 15:01, Hemambara <ko...@gmail.com>:
> Can anyone please check above comment and update. Thanks
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Can anyone please check above comment and update. Thanks
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Hello, I tried to go to internals and override GridQueryProcessor but somehow
I do not feel like clean solution. Just had another thought...whatever code
I have written and created a pull request for, shall is surround it with -D
arg like nested_field_experimental_feature or with better system property
name. So that my logic will be enabled for the guys that need this feature,
else default to existing logic. Any way as it is mentioned as experimental
feature it leaves the testing part to individuals for thorough testing and
whenever you have time you can get back on this.. if this is fine I will
change code and raise pull request
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
You can a write an email to dev@ignite.apache.org with reference to
the issue and a description of the fix. It might be that someone will
be ready to do a review and merge.
вт, 5 нояб. 2019 г. в 15:57, Hemambara <ko...@gmail.com>:
>
> Okay, so the issue you are facing with is incorrect data type which is valid,
> so its not an issue then.
>
> Yes agreed that it requires more testing, but I feel the fix that is going
> in, is safe and good to do. This fix is really important for us to proceed
> further. I have tested few other scnearios and its working fine for me. Is
> there any way we can plan to merge ? or you do not want to do it now ?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Okay, so the issue you are facing with is incorrect data type which is valid,
so its not an issue then.
Yes agreed that it requires more testing, but I feel the fix that is going
in, is safe and good to do. This fix is really important for us to proceed
further. I have tested few other scnearios and its working fine for me. Is
there any way we can plan to merge ? or you do not want to do it now ?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
Check my example [1].
1. Launch step1.
2. Uncomment code and enable Address.number field.
3. Launch step2.
Actually the problem here is type mismatch for "number" int vs
varchar. Work with nested fields could be really tricky. Subsequently
there is not much activity to improve their support. Applying proposed
fix requires thorough testing and review. A lot of end to end
scenarios should be covered by tests.
[1] https://gist.github.com/pavlukhin/53d8a23b48ca0018481a203ceb06065f
пт, 1 нояб. 2019 г. в 11:28, Ivan Pavlukhin <vo...@gmail.com>:
>
> Hi Hemambara,
>
> I appologize but I will be able to share a problematic example only on
> next week.
>
> чт, 31 окт. 2019 г. в 19:49, Hemambara <ko...@gmail.com>:
> >
> > I did not face any issue. Its working fine for me. Can you share your code
> > and exception that you are getting
> >
> > I tried like below and it worked for me.
> > ((Person)cache.get(1)).address.community)
> >
> >
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
I appologize but I will be able to share a problematic example only on
next week.
чт, 31 окт. 2019 г. в 19:49, Hemambara <ko...@gmail.com>:
>
> I did not face any issue. Its working fine for me. Can you share your code
> and exception that you are getting
>
> I tried like below and it worked for me.
> ((Person)cache.get(1)).address.community)
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
I did not face any issue. Its working fine for me. Can you share your code
and exception that you are getting
I tried like below and it worked for me.
((Person)cache.get(1)).address.community)
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
I checked some more scenarios with your fix. I a scenario:
1. Register QueryEntity with classes
class Person { String name; Address address; }
class Address { String street; }
2. cache.put(1, new Person(...))
3. Add column dynamically
alter table add column "Address.number"
4. UPDATE Person set "Address.number" = 666
5. Update class Address to be
class Address { String street; int number; }
6. Retrieving a value from cache
cache.get(1).address.number
results in a weird exception.
I checked the same scenario with flat objects (without nesting) and it
finished successfully.
I am afraid I cannot merge the proposed fix as is. The thing with
nesting SQL fields is quite complicated and requires thorough
analysis. Unfortunately, I do not have enough time frame to do a
sufficient analysis.
ср, 30 окт. 2019 г. в 13:31, Hemambara <ko...@gmail.com>:
>
> Hello Ivan, please let me know if you get a chance to check the code and
> merge it..
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Hello Ivan, please let me know if you get a chance to check the code and
merge it..
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Sure. Have created one https://github.com/apache/ignite/pull/7016
Can you please check this. It has all the changes at one place. please let
me know for any issues. Thank you.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
It really good that we have common understanding now.
> I could not merge pull request as it is saying that only users with "Write access" can merge pull requests
By merge I meant merging branches in your fork repository. It is
really inconvenient to work with bunch of PRs simultaneously. It will
be great if there is only one PR.
пт, 25 окт. 2019 г. в 14:43, Hemambara <ko...@gmail.com>:
>
> Ohh yes you are right. Without the fix, it is failing to query cache. With
> the other approach that I have attached earlier it failed to join node,
> there I have CacheConfiguration in xml file for both server and client node.
> Not sure if that does matters. Either way it is failing
>
> Also this is my bad, my apologies, that I forget to check in one change in
> GridQueryProcessor.java where it is using QueryBinaryProperty constructor,
> instead I am using QUeryUtils.buildBinaryProperty to build it. Now I have
> checked in all the changes along with test cases.
>
> Can you please review pull requests:
> https://github.com/apache/ignite/pull/7008 - Change in QueryUtils
> https://github.com/apache/ignite/pull/7009 - Test case for QueryUtilsTest
> https://github.com/apache/ignite/pull/7010 - Change in
> GridQueryProcessor.java
> https://github.com/apache/ignite/pull/7011 - Test case to make sure it is
> adding nested field
>
> I could not merge pull request as it is saying that only users with "Write
> access" can merge pull requests. Can you please merge these pull requests
> and review. Please let me know if you see any issues.
>
> Appreciate your help on this.
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Ohh yes you are right. Without the fix, it is failing to query cache. With
the other approach that I have attached earlier it failed to join node,
there I have CacheConfiguration in xml file for both server and client node.
Not sure if that does matters. Either way it is failing
Also this is my bad, my apologies, that I forget to check in one change in
GridQueryProcessor.java where it is using QueryBinaryProperty constructor,
instead I am using QUeryUtils.buildBinaryProperty to build it. Now I have
checked in all the changes along with test cases.
Can you please review pull requests:
https://github.com/apache/ignite/pull/7008 - Change in QueryUtils
https://github.com/apache/ignite/pull/7009 - Test case for QueryUtilsTest
https://github.com/apache/ignite/pull/7010 - Change in
GridQueryProcessor.java
https://github.com/apache/ignite/pull/7011 - Test case to make sure it is
adding nested field
I could not merge pull request as it is saying that only users with "Write
access" can merge pull requests. Can you please merge these pull requests
and review. Please let me know if you see any issues.
Appreciate your help on this.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
I tried adding 2nd node. I saw a following:
1. Without the fix 2nd is able to join cluster but failed to execute "select *".
2. With and without the fix I always got null for "Address.street".
Could you please consolidate all changes in one pull request and
remove links for irrelevant one from the ticket [1]? Please add
end-to-end test ensuring that added column has non-null value to that
PR as well.
[1] https://issues.apache.org/jira/browse/IGNITE-12261
чт, 24 окт. 2019 г. в 21:30, Hemambara <ko...@gmail.com>:
>
> Result should not be same, could you please share your logs. Couple of issues
> here.
>
> 1) *With out fix *- I got null instead of "Address.street" value
> [[1, Person{name='john', address=Address{street='baker', number=221}}, john,
> Address{street='baker', number=221}, *null*]]
>
> *With fix*
> [[1, Person{name='john', address=Address{street='baker', number=221}},
> john, Address{street='baker', number=221},* baker*]]
>
> 2) Also you are trying with only one node. Could you please try to *bring
> another node*. Using below code. This one will just start another node and
> try to join existing. *With out fix - it cannot join and with fix - it will
> join*
>
>
> @Test
> public void test(){
> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500..47509")));
>
> IgniteConfiguration igniteConfig = new
> IgniteConfiguration().setDiscoverySpi(discoverySpi);
>
> try(Ignite ignite = Ignition.start(igniteConfig)){
> IgniteCache<Object, Object> cache = ignite.getOrCreateCache(new
> CacheConfiguration<>("cache")
> .setIndexedTypes(Integer.class, Person.class));
>
> System.err.println(cache.query(new SqlFieldsQuery("Select * from
> Person")).getAll());
>
> }
>
> }
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Result should not be same, could you please share your logs. Couple of issues
here.
1) *With out fix *- I got null instead of "Address.street" value
[[1, Person{name='john', address=Address{street='baker', number=221}}, john,
Address{street='baker', number=221}, *null*]]
*With fix*
[[1, Person{name='john', address=Address{street='baker', number=221}},
john, Address{street='baker', number=221},* baker*]]
2) Also you are trying with only one node. Could you please try to *bring
another node*. Using below code. This one will just start another node and
try to join existing. *With out fix - it cannot join and with fix - it will
join*
@Test
public void test(){
TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi()
.setIpFinder(new
TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500..47509")));
IgniteConfiguration igniteConfig = new
IgniteConfiguration().setDiscoverySpi(discoverySpi);
try(Ignite ignite = Ignition.start(igniteConfig)){
IgniteCache<Object, Object> cache = ignite.getOrCreateCache(new
CacheConfiguration<>("cache")
.setIndexedTypes(Integer.class, Person.class));
System.err.println(cache.query(new SqlFieldsQuery("Select * from
Person")).getAll());
}
}
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
I rewrote my example in form of a junit test [1]. And I suppose that
it should pass to make the feature useful. Could please check it and
tell me if my understanding is wrong. I tried this test with proposed
fix and without it. Result was the same.
[1] https://gist.githubusercontent.com/pavlukhin/3ac27c2def1731a5158222a0d542dfa6/raw/0fd0b0ee7d51649c8b79d36358ce5a80940254b8/NestedFieldUnitTest.java
ср, 23 окт. 2019 г. в 23:36, Hemambara <ko...@gmail.com>:
>
> HI Ivan, thanks for the quick reply.
>
> Yes it perfectly works as needed with the fix.
>
> Person.Address.Street will not work because
> person.getPerson().getAddress().getStreet() does not exist. It has to be
> person.getAddress().getStreet(). So column name should be "Address.street"
>
> Any other name does the trick as long as there is alias option provided in
> SQL, but in SQLFieldQuery I cannot provide alias option so the only option
> that we are left with is use field name as alias name by default. That way
> we will be able to rejoin the node.
>
> Also just creating field name as "street" will not work as with the same
> name there could be another column
>
> Ex: Person.Address.Street and Person.College.Address.Street. If we use
> street here then which street we we are referring to ? This fix will take
> the whole name and use it query to avoid complete ambiguity
>
>
> Please follow the instructions that I have mentioned in my previous email.
> If you execute those two programs with and with out fix, you can get better
> idea.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
HI Ivan, thanks for the quick reply.
Yes it perfectly works as needed with the fix.
Person.Address.Street will not work because
person.getPerson().getAddress().getStreet() does not exist. It has to be
person.getAddress().getStreet(). So column name should be "Address.street"
Any other name does the trick as long as there is alias option provided in
SQL, but in SQLFieldQuery I cannot provide alias option so the only option
that we are left with is use field name as alias name by default. That way
we will be able to rejoin the node.
Also just creating field name as "street" will not work as with the same
name there could be another column
Ex: Person.Address.Street and Person.College.Address.Street. If we use
street here then which street we we are referring to ? This fix will take
the whole name and use it query to avoid complete ambiguity
Please follow the instructions that I have mentioned in my previous email.
If you execute those two programs with and with out fix, you can get better
idea.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
Yep, I left some trash in gist [1] initially ("as address"), cleaned up.
But does it really works as you need? In previous email I meant that
the behavior is not plausible even in one node case. And the problem
here is that added column "Address.street" is not mapped to
Person.address.street Java object field! It will be a separate column
instead. If separate column is what you need then I do not understand
why you need the name "Address.street", any other same seems to do the
trick.
Did I miss something?
ср, 23 окт. 2019 г. в 00:31, Hemambara <ko...@gmail.com>:
>
> I see there are couple of issues with the test
>
> 1) cache.query(new SqlFieldsQuery("alter table Person add column
> \"Address.street\" as street varchar"));
> - "as" is not supported in sql. So we should remove "as street" and it
> should be like
> cache.query(new SqlFieldsQuery("alter table Person add column
> \"Address.street\" varchar"));
>
> 2) After making above change, it will work with / with out fix. But the
> problem is with out the fix, if you have any other node joining this node,
> it will not be able to join / query
>
> I just tweaked your test code. PLease find the below one. main() is your
> test class which will start one node.
> a) Take this example and execute main() - this will start one node
> b) now execute @Test method, this will start another node. You will get
> attached exception ( exception.txt
> <http://apache-ignite-users.70518.x6.nabble.com/file/t2625/exception.txt> )
> with out fix as by default it take alias as "street". But with fix it will
> pass through, as it will create alias as "Address.street"
>
> Please let me know if you need more clarification. Any quick help would be
> appreciated. Thank you!
>
>
> package my.ignite.cache.poc.index;
>
> import org.apache.ignite.Ignite;
> import org.apache.ignite.IgniteCache;
> import org.apache.ignite.Ignition;
> import org.apache.ignite.cache.query.SqlFieldsQuery;
> import org.apache.ignite.configuration.CacheConfiguration;
> import org.apache.ignite.configuration.IgniteConfiguration;
> import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
> import
> org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
> import org.junit.Test;
>
> import java.util.Collections;
>
> public class NestedIndexFieldTest {
>
> public static void main(String args[]){
> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500..47509")));
>
> IgniteConfiguration igniteConfig = new
> IgniteConfiguration().setDiscoverySpi(discoverySpi);
> Ignite ignite = Ignition.start(igniteConfig);
> try{
> IgniteCache<Object, Object> cache = ignite.createCache(new
> CacheConfiguration<>("cache")
> .setIndexedTypes(Integer.class, Person.class));
>
> cache.put(1, new Person("john", new Address("baker", 221)));
>
> System.err.println(cache.query(new SqlFieldsQuery("Select * from
> Person")).getAll());
>
> cache.query(new SqlFieldsQuery("alter table Person add column
> name varchar"));
>
> cache.query(new SqlFieldsQuery("alter table Person add column
> address other"));
>
> System.err.println(cache.query(new SqlFieldsQuery("Select * from
> Person")).getAll());
>
> cache.query(new SqlFieldsQuery("alter table Person add column
> \"Address.street\" varchar"));
>
> System.err.println(cache.query(new SqlFieldsQuery("Select * from
> Person")).getAll());
>
> cache.put(2, new Person("bill", new Address("green", 223)));
>
> System.err.println(cache.query(new SqlFieldsQuery("Select * from
> Person")).getAll());
> } catch (Exception anyEx){
>
> }
>
> }
>
> @Test
> public void test(){
> TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi()
> .setIpFinder(new
> TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500..47509")));
>
> IgniteConfiguration igniteConfig = new
> IgniteConfiguration().setDiscoverySpi(discoverySpi);
>
> try(Ignite ignite = Ignition.start(igniteConfig)){
> IgniteCache<Object, Object> cache = ignite.getOrCreateCache(new
> CacheConfiguration<>("cache")
> .setIndexedTypes(Integer.class, Person.class));
>
> System.err.println(cache.query(new SqlFieldsQuery("Select * from
> Person")).getAll());
>
> }
>
> }
> }
>
> class Person {
> String name;
> Address address;
>
> Person(String name, Address address){
> this.name = name;
> this.address = address;
> }
>
> @Override
> public String toString() {
> return "Person{" +
> "name='" + name + '\'' +
> ", address=" + address +
> '}';
> }
> }
>
> class Address {
> String street;
> int number;
>
> public Address(String street, int number){
> this.street = street;
> this.number = number;
> }
>
> @Override
> public String toString() {
> return "Address{" +
> "street='" + street + '\'' +
> ", number=" + number +
> '}';
> }
> }
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
I see there are couple of issues with the test
1) cache.query(new SqlFieldsQuery("alter table Person add column
\"Address.street\" as street varchar"));
- "as" is not supported in sql. So we should remove "as street" and it
should be like
cache.query(new SqlFieldsQuery("alter table Person add column
\"Address.street\" varchar"));
2) After making above change, it will work with / with out fix. But the
problem is with out the fix, if you have any other node joining this node,
it will not be able to join / query
I just tweaked your test code. PLease find the below one. main() is your
test class which will start one node.
a) Take this example and execute main() - this will start one node
b) now execute @Test method, this will start another node. You will get
attached exception ( exception.txt
<http://apache-ignite-users.70518.x6.nabble.com/file/t2625/exception.txt> )
with out fix as by default it take alias as "street". But with fix it will
pass through, as it will create alias as "Address.street"
Please let me know if you need more clarification. Any quick help would be
appreciated. Thank you!
package my.ignite.cache.poc.index;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.query.SqlFieldsQuery;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
import
org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder;
import org.junit.Test;
import java.util.Collections;
public class NestedIndexFieldTest {
public static void main(String args[]){
TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi()
.setIpFinder(new
TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500..47509")));
IgniteConfiguration igniteConfig = new
IgniteConfiguration().setDiscoverySpi(discoverySpi);
Ignite ignite = Ignition.start(igniteConfig);
try{
IgniteCache<Object, Object> cache = ignite.createCache(new
CacheConfiguration<>("cache")
.setIndexedTypes(Integer.class, Person.class));
cache.put(1, new Person("john", new Address("baker", 221)));
System.err.println(cache.query(new SqlFieldsQuery("Select * from
Person")).getAll());
cache.query(new SqlFieldsQuery("alter table Person add column
name varchar"));
cache.query(new SqlFieldsQuery("alter table Person add column
address other"));
System.err.println(cache.query(new SqlFieldsQuery("Select * from
Person")).getAll());
cache.query(new SqlFieldsQuery("alter table Person add column
\"Address.street\" varchar"));
System.err.println(cache.query(new SqlFieldsQuery("Select * from
Person")).getAll());
cache.put(2, new Person("bill", new Address("green", 223)));
System.err.println(cache.query(new SqlFieldsQuery("Select * from
Person")).getAll());
} catch (Exception anyEx){
}
}
@Test
public void test(){
TcpDiscoverySpi discoverySpi = new TcpDiscoverySpi()
.setIpFinder(new
TcpDiscoveryVmIpFinder().setAddresses(Collections.singleton("127.0.0.1:47500..47509")));
IgniteConfiguration igniteConfig = new
IgniteConfiguration().setDiscoverySpi(discoverySpi);
try(Ignite ignite = Ignition.start(igniteConfig)){
IgniteCache<Object, Object> cache = ignite.getOrCreateCache(new
CacheConfiguration<>("cache")
.setIndexedTypes(Integer.class, Person.class));
System.err.println(cache.query(new SqlFieldsQuery("Select * from
Person")).getAll());
}
}
}
class Person {
String name;
Address address;
Person(String name, Address address){
this.name = name;
this.address = address;
}
@Override
public String toString() {
return "Person{" +
"name='" + name + '\'' +
", address=" + address +
'}';
}
}
class Address {
String street;
int number;
public Address(String street, int number){
this.street = street;
this.number = number;
}
@Override
public String toString() {
return "Address{" +
"street='" + street + '\'' +
", number=" + number +
'}';
}
}
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
I created an example [1] and checked with the proposed fix and without
it. It seems the thing does not work in both cases.
And think that supporting such dynamic nested fields properly is not
an easy thing in general. My first thought is that you need to
implement object relational mapping in your application code.
[1] https://gist.github.com/pavlukhin/3ac27c2def1731a5158222a0d542dfa6
вт, 22 окт. 2019 г. в 01:16, Hemambara <ko...@gmail.com>:
>
> Can you please provide an update
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Can you please provide an update
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
ignite_bug_share.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/t2625/ignite_bug_share.zip>
Thanks for the reply. Please find the attached.
1) IndextestIgniteServer1 - starts server
2) IndexTestIgniteClient1 - starts the client
3) I did not add QueryEntities - we have usecase where we should be able to
add fields and indexes on the fly and ignite does support it.
This program works when we run for the first time. But if we restart client,
it is not able to join the server again because when we use "Users.userName"
field added dynamically, alias is being added as username and when we
restart due to mismatch in configuration it is not able to join
- With this fix, both field and index name will be "Users.userName" and
there wont be any mismatch. I built in local and it looks good.
January we are going with first prod release, if we can have interim patch
for this, that should be fine. Please let us know if that works.
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Thank you. One more thing to add - it is able to add index dynamically only
thing is, when I restart the client it is not able join the cluster due to
incorrect alias
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
I created a gist with my code [1] as it renders not so good inline.
[1] https://gist.github.com/pavlukhin/362c0e40d4a010a8f7c0795368e53efb
чт, 17 окт. 2019 г. в 15:01, Ivan Pavlukhin <vo...@gmail.com>:
>
> Hi Hemambara,
>
> Could you bring a little bit more details about your problem?
> I would like to see you java classes and QueryEntity configuration.
> Ideally it would be to have a runnable reproducer (junit test or class
> with main method).
>
> Also I would like to realize how soon are you expecting a fix in Ignite release?
> As far as I know next Ignite release is planned in the beginning of next year.
>
> I tried following and it worked for me:
>
> class User {
> String userName;
> Address address;
>
> public User(String userName, Address address) {
> this.userName = userName;
> this.address = address;
> }
> }
>
> class Address {
> String streetName;
> String zipcode;
>
> public Address(String streetName, String zipcode) {
> this.streetName = streetName;
> this.zipcode = zipcode;
> }
> }
>
> @Test
> public void testSqlPojoWithoutAnnotations() throws Exception {
> IgniteEx grid = startGrid(0);
>
> LinkedHashMap<String, String> fields = new LinkedHashMap<>();
> fields.put("userName", "String");
> fields.put("address.zipcode", "String");
>
> IgniteCache<Object, Object> cache = grid.createCache(new
> CacheConfiguration<>(DEFAULT_CACHE_NAME)
> .setQueryEntities(Collections.singleton(new
> QueryEntity(Integer.class, User.class)
> .setFields(fields))));
>
> cache.put(1, new User("ivan", new Address("a", "123")));
>
> System.err.println(cache.query(new SqlFieldsQuery("select * from
> User")).getAll());
>
> System.err.println(cache.query(new SqlFieldsQuery("select zipcode
> from User")).getAll());
>
> cache.query(new SqlFieldsQuery("create index on User(zipcode)"));
>
> System.err.println(cache.query(new SqlFieldsQuery("select zipcode
> from User")).getAll());
> }
>
> ср, 16 окт. 2019 г. в 10:59, Hemambara <ko...@gmail.com>:
> >
> > Hello Ivan, can you please check and provide updates on this
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hi Hemambara,
Could you bring a little bit more details about your problem?
I would like to see you java classes and QueryEntity configuration.
Ideally it would be to have a runnable reproducer (junit test or class
with main method).
Also I would like to realize how soon are you expecting a fix in Ignite release?
As far as I know next Ignite release is planned in the beginning of next year.
I tried following and it worked for me:
class User {
String userName;
Address address;
public User(String userName, Address address) {
this.userName = userName;
this.address = address;
}
}
class Address {
String streetName;
String zipcode;
public Address(String streetName, String zipcode) {
this.streetName = streetName;
this.zipcode = zipcode;
}
}
@Test
public void testSqlPojoWithoutAnnotations() throws Exception {
IgniteEx grid = startGrid(0);
LinkedHashMap<String, String> fields = new LinkedHashMap<>();
fields.put("userName", "String");
fields.put("address.zipcode", "String");
IgniteCache<Object, Object> cache = grid.createCache(new
CacheConfiguration<>(DEFAULT_CACHE_NAME)
.setQueryEntities(Collections.singleton(new
QueryEntity(Integer.class, User.class)
.setFields(fields))));
cache.put(1, new User("ivan", new Address("a", "123")));
System.err.println(cache.query(new SqlFieldsQuery("select * from
User")).getAll());
System.err.println(cache.query(new SqlFieldsQuery("select zipcode
from User")).getAll());
cache.query(new SqlFieldsQuery("create index on User(zipcode)"));
System.err.println(cache.query(new SqlFieldsQuery("select zipcode
from User")).getAll());
}
ср, 16 окт. 2019 г. в 10:59, Hemambara <ko...@gmail.com>:
>
> Hello Ivan, can you please check and provide updates on this
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
--
Best regards,
Ivan Pavlukhin
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Hello Ivan, can you please check and provide updates on this
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Denis Magda <dm...@apache.org>.
Please check Ivan's latest response
-
Denis
On Thu, Oct 17, 2019 at 5:16 AM Hemambara <ko...@gmail.com> wrote:
> Hello Denis/Ivan, can you please check this and if there are no issues can
> you merge the code. Sorry for the rush. We are getting ready for prod in
> next 3 months and this is really important for us and we need to test all
> our functionality as well. It would be a great help if we have this fixed
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Hello Denis/Ivan, can you please check this and if there are no issues can
you merge the code. Sorry for the rush. We are getting ready for prod in
next 3 months and this is really important for us and we need to test all
our functionality as well. It would be a great help if we have this fixed
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Can you please provide any update in this...
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Hello Ivan, to be more specific, if I want add field
"Users.userName" - current ignite version 2.7.6 is setting the field name as
"Users.userName" and default alias name as "userName" with which makes it
non-queryable, non -indexable and issues during restart due to mismatch in
configuration . But with this fix for fieldname "Users.userName" it will set
alias as "Users.userName" and the same can be used to add index and
everywhere and there wont be any issues
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Denis Magda <dm...@apache.org>.
Ivan,
As a member of the Ignite SQL group, could you please check this thread and
help Hemambara with a proper fix?
-
Denis
On Thu, Oct 10, 2019 at 11:50 AM Hemambara <ko...@gmail.com> wrote:
> Sorry for the push. This is a major blocker for us. We have coherence
> clients
> where they can add indexes dynamically and we want to move them to ignite
> in
> next 3 months. Coherence has a way to add indexes dynamically and ignite
> also does supporting it. This issue exists event with QuerySqlField. If
> this
> fix cannot be merged, I am not sure how to proceed further. Do you see any
> issue putting this fix ? or it is just that we are not ready yet ?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Sorry for the push. This is a major blocker for us. We have coherence clients
where they can add indexes dynamically and we want to move them to ignite in
next 3 months. Coherence has a way to add indexes dynamically and ignite
also does supporting it. This issue exists event with QuerySqlField. If this
fix cannot be merged, I am not sure how to proceed further. Do you see any
issue putting this fix ? or it is just that we are not ready yet ?
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Currently we do not support altering tables which are created with
indexedTypes/queryEntities.
There are plans to implement it, but your fix, while solving your
particular problem, isn't fit in this scheme IMHO.
Regards,
--
Ilya Kasnacheev
ср, 9 окт. 2019 г. в 22:45, Hemambara <ko...@gmail.com>:
> My apologies for multiple replies. Please consider latest reply
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
My apologies for multiple replies. Please consider latest reply
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
ignite_bug_share.zip
<http://apache-ignite-users.70518.x6.nabble.com/file/t2625/ignite_bug_share.zip>
I have tested by adding QuerySqlFields as well, with or with out it, it is
working when I add it, but if I restart the client it is not able to
identify nested columns as its alias is null or not equal to whole path.
Please see below exception. If we can make alias with whole nested object
path we will be able to add indexes and fields dynamically with out any
issues. Ignite, out of the box provides dynamic field and index alterations.
But it has a bug when we add nested field / index dynamically. This small
fix will fix this bug. PFA for the code. SEVERE: Failed to reinitialize
local partitions (rebalancing will be stopped): GridDhtPartitionExchangeId
[topVer=AffinityTopologyVersion [topVer=5, minorTopVer=0],
discoEvt=DiscoveryEvent [evtNode=TcpDiscoveryNode
[id=a5379438-868b-43e4-b402-02edf2913795, addrs=[0:0:0:0:0:0:0:1,
114.18.146.93, 127.0.0.1], sockAddrs=[/0:0:0:0:0:0:0:1:0, /127.0.0.1:0,
DTC4C4A8C85A622.ent.wfb.bank.corp/114.18.146.93:0], discPort=0, order=5,
intOrder=0, lastExchangeTime=1570648315703, loc=true,
ver=2.7.6#20190911-sha1:21f7ca41, isClient=true], topVer=5,
nodeId8=a5379438, msg=null, type=NODE_JOINED, tstamp=1570648318843],
nodeId=a5379438, evt=NODE_JOINED] class
org.apache.ignite.IgniteCheckedException: Field not found: Users.userName
at
org.apache.ignite.internal.processors.query.QueryIndexDescriptorImpl.addField(QueryIndexDescriptorImpl.java:124)
at
org.apache.ignite.internal.processors.query.QueryUtils.createIndexDescriptor(QueryUtils.java:735)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndex(QueryUtils.java:752)
at
org.apache.ignite.internal.processors.query.QueryUtils.processIndexes(QueryUtils.java:688)
at
org.apache.ignite.internal.processors.query.QueryUtils.processBinaryMeta(QueryUtils.java:612)
at
org.apache.ignite.internal.processors.query.QueryUtils.typeForQueryEntity(QueryUtils.java:480)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:706)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:866)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1337)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:2172)
at
org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:2030)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:927)
at
org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:769)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:2681)
at
org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2553)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at java.lang.Thread.run(Thread.java:748)
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Frankly speaking, I don't think this is a scenario that we want to support.
If you are interested, you may like to start a discussion in dev@ list,
but: working with nested objects is not a priority, and implicit
configuration of a corner case is discouraged.
In this case we recommend you to have custom serialization of some sort
(via BinaryTypeConfiguration for example).
Regards,
--
Ilya Kasnacheev
ср, 9 окт. 2019 г. в 06:03, Hemambara <ko...@gmail.com>:
> Added test case and created pull request. Please check both the code
> changes
> under pull request #6949 and test case pull request #6955 and let me know
> for any suggestions. Thank you.
>
> IGNITE-12261 - Issue with adding nested index dynamically - Adding tests
> #6955
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Added test case and created pull request. Please check both the code changes
under pull request #6949 and test case pull request #6955 and let me know
for any suggestions. Thank you.
IGNITE-12261 - Issue with adding nested index dynamically - Adding tests
#6955
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
Sure, will add test case.
My domain model is existing domain model and I cannot change or add
@QuerySqlField to my domain model. I am trying to achieve dynamic index
creation on any domain model object (pojo) irrespective @QuerySqlField's
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Can you add test case to this change?
Otherwise, can you explain when it is useful? Can't zipcode just be
annotated with @QuerySqlField in your use case?
Regards,
--
Ilya Kasnacheev
вт, 8 окт. 2019 г. в 13:57, Hemambara <ko...@gmail.com>:
> https://github.com/apache/ignite/pull/6949
>
> Created pull request with suggestion. Can you please check and close
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
https://github.com/apache/ignite/pull/6949
Created pull request with suggestion. Can you please check and close
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
I have made changes and created pull request. Can you please check and let me
know if there are any issues with the commit. Otherwise can you please let
me know when it can it be available in maven repo so that we can start
pulling it in our organization
https://github.com/apache/ignite/pull/6947
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I have added you to Contributor role. Now you should be able to assign
issue to yourself.
Please read
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute
Regards,
--
Ilya Kasnacheev
пт, 4 окт. 2019 г. в 20:49, Hemambara <ko...@gmail.com>:
> https://issues.apache.org/jira/browse/IGNITE-12261
>
> I have created JIRA. Please let me know how I can assign to myself. My
> username on jira board is "kotari"
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Issue with adding nested index dynamically
Posted by Hemambara <ko...@gmail.com>.
https://issues.apache.org/jira/browse/IGNITE-12261
I have created JIRA. Please let me know how I can assign to myself. My
username on jira board is "kotari"
--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/
Re: Issue with adding nested index dynamically
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Please find/create a ticket about this issue and attach your patch to this
ticket (in plain text or pull request form).
Then it can be tested and merged.
Regards,
--
Ilya Kasnacheev
чт, 3 окт. 2019 г. в 22:19, Hemambara <ko...@gmail.com>:
> We have to add indexes on cache dynamically on java pojo with nested
> objects.
> In the below example we do not have @QuerySqlField. In this case if I try
> to
> add index on "username" dynamically using "CREATE INDEX" it worked. But if
> I
> want to add index on "Address.zipcode" - this is not working as we are NOT
> creating aliases for these fields which are getting added dynamically. I
> see
> there is a bug in this implementation. I fixed the bug (please see below)
> and rebuilt the jar in my local and it worked fine. Proposing this bug to
> this open source community so that it can be fixed in ignite and we can
> start using the jar from central repo. We have just started using ignite
> and
> planning to take this to production. So it would be helpful if we can
> implement this bug fix.
>
> User{
> String userName;
> Address address;
>
> }
>
> Address{
> String streetName;
> String zipcode;
> }
>
> Bug Fix: QueryUtils.class
> ------------------------------
> Method name :
> ------------------
> QueryBinaryProperty buildBinaryProperty(GridKernalContext ctx, String
> pathStr,
> Class<?> resType, Map<String, String> aliases, @Nullable Boolean
> isKeyField, boolean notNull, Object dlftVal,
> int precision, int scale)
>
> --------
>
> Change from :
> -----------------
> String alias = (String)aliases.get(fullName.toString());
>
> to :
> -----
> String alias = aliases.get(fullName.toString()) == null ?
> fullName.toString() : aliases.get(fullName.toString());
>
> With this way we will always make sure we have default alias (if not
> provided), otherwise aliases on dynamic query fields and dynamic indexes
> are
> not working properly
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Issue with adding nested index dynamically
Posted by Denis Magda <dm...@apache.org>.
Hi, thanks for starting this thread!
SQL folks, could you please confirm that's neither known nor expected
limitation? If that's a real issue then we will just need to go through a
standard contribution process.
-
Denis
On Thu, Oct 3, 2019 at 12:19 PM Hemambara <ko...@gmail.com> wrote:
> We have to add indexes on cache dynamically on java pojo with nested
> objects.
> In the below example we do not have @QuerySqlField. In this case if I try
> to
> add index on "username" dynamically using "CREATE INDEX" it worked. But if
> I
> want to add index on "Address.zipcode" - this is not working as we are NOT
> creating aliases for these fields which are getting added dynamically. I
> see
> there is a bug in this implementation. I fixed the bug (please see below)
> and rebuilt the jar in my local and it worked fine. Proposing this bug to
> this open source community so that it can be fixed in ignite and we can
> start using the jar from central repo. We have just started using ignite
> and
> planning to take this to production. So it would be helpful if we can
> implement this bug fix.
>
> User{
> String userName;
> Address address;
>
> }
>
> Address{
> String streetName;
> String zipcode;
> }
>
> Bug Fix: QueryUtils.class
> ------------------------------
> Method name :
> ------------------
> QueryBinaryProperty buildBinaryProperty(GridKernalContext ctx, String
> pathStr,
> Class<?> resType, Map<String, String> aliases, @Nullable Boolean
> isKeyField, boolean notNull, Object dlftVal,
> int precision, int scale)
>
> --------
>
> Change from :
> -----------------
> String alias = (String)aliases.get(fullName.toString());
>
> to :
> -----
> String alias = aliases.get(fullName.toString()) == null ?
> fullName.toString() : aliases.get(fullName.toString());
>
> With this way we will always make sure we have default alias (if not
> provided), otherwise aliases on dynamic query fields and dynamic indexes
> are
> not working properly
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
Re: Issue with adding nested index dynamically
Posted by Denis Magda <dm...@apache.org>.
Hi, thanks for starting this thread!
SQL folks, could you please confirm that's neither known nor expected
limitation? If that's a real issue then we will just need to go through a
standard contribution process.
-
Denis
On Thu, Oct 3, 2019 at 12:19 PM Hemambara <ko...@gmail.com> wrote:
> We have to add indexes on cache dynamically on java pojo with nested
> objects.
> In the below example we do not have @QuerySqlField. In this case if I try
> to
> add index on "username" dynamically using "CREATE INDEX" it worked. But if
> I
> want to add index on "Address.zipcode" - this is not working as we are NOT
> creating aliases for these fields which are getting added dynamically. I
> see
> there is a bug in this implementation. I fixed the bug (please see below)
> and rebuilt the jar in my local and it worked fine. Proposing this bug to
> this open source community so that it can be fixed in ignite and we can
> start using the jar from central repo. We have just started using ignite
> and
> planning to take this to production. So it would be helpful if we can
> implement this bug fix.
>
> User{
> String userName;
> Address address;
>
> }
>
> Address{
> String streetName;
> String zipcode;
> }
>
> Bug Fix: QueryUtils.class
> ------------------------------
> Method name :
> ------------------
> QueryBinaryProperty buildBinaryProperty(GridKernalContext ctx, String
> pathStr,
> Class<?> resType, Map<String, String> aliases, @Nullable Boolean
> isKeyField, boolean notNull, Object dlftVal,
> int precision, int scale)
>
> --------
>
> Change from :
> -----------------
> String alias = (String)aliases.get(fullName.toString());
>
> to :
> -----
> String alias = aliases.get(fullName.toString()) == null ?
> fullName.toString() : aliases.get(fullName.toString());
>
> With this way we will always make sure we have default alias (if not
> provided), otherwise aliases on dynamic query fields and dynamic indexes
> are
> not working properly
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>