You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@syncope.apache.org by Francesco Chicchiriccò <il...@apache.org> on 2017/07/03 07:05:54 UTC
Re: AnyObjects query
On 30/06/2017 18:23, Colm O hEigeartaigh wrote:
> Hi Francesco,
>
> On Fri, Jun 30, 2017 at 3:42 PM, Francesco Chicchiriccò <il...@apache.org> wrote:
>> At persistence level, Relationships have leftEnd and rightEnd:
>>
>> https://github.com/apache/syncope/blob/2_0_X/core/persistence-api/src/main/java/org/apache/syncope/core/persistence/api/
>> entity/Relationship.java
>>
>> However, since Relationships can only be read via REST through one of
>> their ends (User <-> AnyObject or AnyObject <-> AnyObject), then it would
>> make more sense to always report the other end via RelationshipTO, rather
>> than reporting the right end, as doing currently:
>>
>> https://github.com/apache/syncope/blob/2_0_X/common/lib/src/main/java/org/apache/syncope/common/lib/to/RelationshipTO.java
>>
>> I would say the current implementation is anyway not the best, and worths
>> an improvement, maybe for 2.0.5 as there are ATM 0 issues open for 2.0.4
>> and CXF 3.1.12 was just released - meaning that we could start our own
>> release process on Monday.
> OK great! I will create a JIRA for this for 2.0.5. One other related query
> - let's say I get back my list of blue cartridges who are in a relationship
> with a printer. I'll be able to extract (I guess) the leftKey from the
> relationships, so I'll know the "key" of the printers. Now how do I
> retrieve all printers that match any of the given keys? Looking at
> SpecialAttr there is not a "$key" available, or is there another way of
> search for this?
The key is not a special attribute, so it would simply be "key", not
"$key" in FIQL.
FYI all the special attributes are
https://github.com/apache/syncope/blob/2_0_X/common/lib/src/main/java/org/apache/syncope/common/lib/search/SpecialAttr.java
Regards.
> On Thu, Jun 29, 2017 at 10:19 AM, Francesco Chicchiriccò <
>>> ilgrosso@apache.org> wrote:
>>>
>>> On 29/06/2017 11:04, Sergey Beryozkin wrote:
>>>> Hi Francesco
>>>>> OK. So, as far as the Syncope returning the list of printers to the
>>>>> client code issuing the query, how would it guess that when someone
>>>>> searches for the blue cartridges it is actually a list of printers is
>>>>> expected ? Where will this code which next goes over the list of all the
>>>>> cartridges and prepare a list of Printers will sit ?
>>>>>
>>>>> As far as the demand is concerned: it is driven by a too long URI
>>>>> production issue Colm referred to
>>>>>
>>>>> The URI too long is generated if you do the search the way Colm
>>>> describes
>>>> it.
>>>>
>>>> If instead, by following Fabio's suggestion, you search for all
>>>> Cartdriges
>>>> with "colour==blue" having a certain RelationshipType (which you have
>>>> prior
>>>> defined, with Printers, say it is named 'PRINTER_CARTDRIGE_ASSIGNMENT'),
>>>> then you need a single query with the following FIQL:
>>>>
>>>> $type==CARTDRIGE;color==blue;$relationshipTypes==PRINTER_CAR
>>>> TDRIGE_ASSIGNMENT
>>>>
>>>> Such query will return a list of AnyObjectTO instances, each of which
>>>> containing a list of RelationshipTO, where each RelationshipTO instance
>>>> has
>>>> a type field.
>>>>
>>>> Hence, to get "the printers with a blue cartridge", it will be enough to
>>>> iterate over the returned list of AnyObjectTO instances (of type
>>>> CARTDRIGE), find the RelationshipTO instances of type
>>>> PRINTER_CARTDRIGE_ASSIGNMENT, and finally get the other side of the
>>>> relationship, e.g. the printer.
>>>> Code-wise:
>>>>
>>>> PagedResult<AnyObjectTO> cartdriges = ...; // returned by the
>>>> search
>>>> List<String> printers = new ArrayList<>();
>>>> for (AnyObjectTO cartdrige: cartdriges.getResult()) {
>>>> for (RelationshipTO relationship:
>>>> cartdrige.getRelationships()) {
>>>> if ("PRINTER_CARTDRIGE_ASSIGNMENT
>>>> ".equals(relationship.getType())) {
>>>> printers.add(relationship.getRightKey());
>>>> }
>>>> }
>>>> }
>>>>
>>>> Anyway, as said, there is always room to improve.
>>>> Regards.
>>>>
>>>>
>>>> On 29/06/17 09:53, Francesco Chicchiriccò wrote:
>>>>
>>>>> On 29/06/2017 10:52, Sergey Beryozkin wrote:
>>>>>> Hi Fabio
>>>>>>> That would not work at the generic level as it would require a 2-way
>>>>>>> relationship (cartridge - printer for ex) which is not always
>>>>>>> realistic to
>>>>>>> expect,
>>>>>>>
>>>>>>> In the Syncope data model, all Relationships are 2-way.
>>>>>> I agree with Fabio: the use case proposed by Colm is doable with
>>>>>> current
>>>>>> implementation.
>>>>>>
>>>>>> Naturally, there is always room for improvements, but I don't see much
>>>>>> demand ATM.
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>>> On 29/06/17 09:07, Fabio Martelli wrote:
>>>>>>
>>>>>>>> Il 28/06/2017 18:34, Colm O hEigeartaigh ha scritto:
>>>>>>>>
>>>>>>>> Thanks for the feedback guys! Let me just expand a bit on the
>>>>>>>>> motivation
>>>>>>>>> behind my previous example....
>>>>>>>>>
>>>>>>>>> Let's say I'm managing hundreds of printers each of which have a
>>>>>>>>> relationship to a cartridge (of which there are many hundreds) with
>>>>>>>>> a
>>>>>>>>> colour attribute. I want to find the printers with a blue cartridge.
>>>>>>>>>
>>>>>>>>> So I first make a search for a list of "blue" cartridges. Then I
>>>>>>>>> search for
>>>>>>>>> the printers that have a relationship with these cartridges. The
>>>>>>>>> problem is
>>>>>>>>> on the second search I end up with a ginormous search expression
>>>>>>>>> "relationships%3D%3D7db4512-ad25-40e8-bc78-63ad25c0e894%2C%
>>>>>>>>> 24relationships%3D%3D16dc6acd-6.....".
>>>>>>>>> " that could be an invalid URL.
>>>>>>>>>
>>>>>>>>> Is there a better way of handling it than this?
>>>>>>>>>
>>>>>>>>> Hi Colm, maybe you can search all the cartridges blue with a
>>>>>>>> relationship with a printer.
>>>>>>>> The result will be a list of cartridges. Each item will be reported
>>>>>>>> including a specific field relationships (if I well remember) about
>>>>>>>> all the
>>>>>>>> printers related to the item.
>>>>>>>> In this case, you will have all the info you need retrieved by a
>>>>>>>> single short query. Don't you agree?
>>>>>>>>
>>>>>>>> BR,
>>>>>>>> F.
>>>>>>>>
>>>>>>>> Colm.
>>>>>>>>> On Wed, Jun 28, 2017 at 10:52 AM, Sergey Beryozkin <
>>>>>>>>> sberyozkin@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hi Francesco
>>>>>>>>>
>>>>>>>>>> One thing I can point to is this code:
>>>>>>>>>>
>>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>>>>>>>> ractJPATypedQueryVisitor.java#L167
>>>>>>>>>>
>>>>>>>>>> There, in the end,
>>>>>>>>>>
>>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>>> earch/src/main/java/org/apache/cxf/jaxrs/ext/search/jpa/Abst
>>>>>>>>>> ractJPATypedQueryVisitor.java#L181
>>>>>>>>>>
>>>>>>>>>> it branches to either doBuildPredicate() (==> similar to Syncope
>>>>>>>>>> SearchCondVisitor.visitPrimitive) or doBuildCollectionPredicate()
>>>>>>>>>>
>>>>>>>>>> When we have "a.b.c" then if 'b' is a collection then it would
>>>>>>>>>> branch to
>>>>>>>>>> doBuildCollectionPredicate.
>>>>>>>>>>
>>>>>>>>>> It was awhile back since I played with the typed JPA2 code, Native
>>>>>>>>>> one is
>>>>>>>>>> a mystery...
>>>>>>>>>>
>>>>>>>>>> I agree supporting such queries is not easy...but supporting then
>>>>>>>>>> can
>>>>>>>>>> offer an ultimate search experience :-)
>>>>>>>>>>
>>>>>>>>>> Cheers, Sergey
>>>>>>>>>>
>>>>>>>>>> On 28/06/17 10:26, Francesco Chicchiriccò wrote:
>>>>>>>>>>
>>>>>>>>>> On 28/06/2017 10:59, Sergey Beryozkin wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Francesco
>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the explanation.
>>>>>>>>>>>>
>>>>>>>>>>>> I see why the example I pointed to won't be applicable to
>>>>>>>>>>>> Syncope.
>>>>>>>>>>>> In that case when the linked beans are available, CXF
>>>>>>>>>>>> AbstractSearchParser will prepare a bean tree which would be
>>>>>>>>>>>> initialized
>>>>>>>>>>>> with the values from the expression like "a.b.c" and the OOB
>>>>>>>>>>>> visitor like
>>>>>>>>>>>> JPA2 one takes care of dealing with these linked beans.
>>>>>>>>>>>>
>>>>>>>>>>>> In the SearchBean case it is up to the custom visitor whether to
>>>>>>>>>>>> react
>>>>>>>>>>>> to the '.'s or not where a '.' indicates that for ex 'a' needs to
>>>>>>>>>>>> have 'b'
>>>>>>>>>>>> with a property 'c'.
>>>>>>>>>>>>
>>>>>>>>>>>> Do you reckon Syncope custom visitors can be updated to support
>>>>>>>>>>>> such
>>>>>>>>>>>> queries ? Not sure about ElasticSearch but for SQL it should
>>>>>>>>>>>> probably be
>>>>>>>>>>>> possible...
>>>>>>>>>>>>
>>>>>>>>>>>> Maybe in principle yes, it could be possible to support such
>>>>>>>>>>>>
>>>>>>>>>>> queries but:
>>>>>>>>>>>
>>>>>>>>>>> 1. implementation would be rather complex as the query logic is
>>>>>>>>>>> already
>>>>>>>>>>> quite involved
>>>>>>>>>>> 2. we haven't had may requests for such complex queries so far
>>>>>>>>>>>
>>>>>>>>>>> ...anyway, as you know, volunteers are welcome :-)
>>>>>>>>>>>
>>>>>>>>>>> Regards.
>>>>>>>>>>>
>>>>>>>>>>> On 28/06/17 09:46, Francesco Chicchiriccò wrote:
>>>>>>>>>>>
>>>>>>>>>>> On 28/06/2017 10:41, Sergey Beryozkin wrote:
>>>>>>>>>>>>> Hi
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think something similar works for a CXF FIQL JPA2 visitor,
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>> example:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://github.com/apache/cxf/blob/master/rt/rs/extensions/s
>>>>>>>>>>>>>> earch/src/test/java/org/apache/cxf/jaxrs/ext/search/jpa/JPAT
>>>>>>>>>>>>>> ypedQueryVisitorFiqlTest.java#L65
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (find the books which have been revied done by Ted)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Sergey,
>>>>>>>>>>>>>>
>>>>>>>>>>>>> that would work if we had straight beans as in the linked
>>>>>>>>>>>>> sample.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Syncope data model is instead much more involved as new schema
>>>>>>>>>>>>> for
>>>>>>>>>>>>> attributes can be defined at runtime: this is the reason why we
>>>>>>>>>>>>> have
>>>>>>>>>>>>> SearchCondVisitor [1] translating FIQL into our internal search
>>>>>>>>>>>>> conditions,
>>>>>>>>>>>>> which serve as input to one of available implementations of
>>>>>>>>>>>>> AnySearchDAO
>>>>>>>>>>>>> like as the default one based on SQL views [2] and another
>>>>>>>>>>>>> relying on
>>>>>>>>>>>>> Elasticsearch-based [3].
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://github.com/apache/sync
>>>>>>>>>>>>> ope/blob/2_0_X/core/persistenc
>>>>>>>>>>>>> e-api/src/main/java/org/apache/syncope/core/persistenc
>>>>>>>>>>>>> e/api/search/SearchCondVisitor.java
>>>>>>>>>>>>> [2] https://github.com/apache/sync
>>>>>>>>>>>>> ope/blob/2_0_X/core/persistenc
>>>>>>>>>>>>> e-jpa/src/main/java/org/apache/syncope/core/persistenc
>>>>>>>>>>>>> e/jpa/dao/JPAAnySearchDAO.java
>>>>>>>>>>>>> [3] https://github.com/apache/sync
>>>>>>>>>>>>> ope/blob/2_0_X/ext/elasticsear
>>>>>>>>>>>>> ch/persistence-jpa/src/main/java/org/apache/syncope/core/
>>>>>>>>>>>>> persistence/jpa/dao/ElasticsearchAnySearchDAO.java
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 28/06/17 08:54, Francesco Chicchiriccò wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 27/06/2017 18:18, Colm O hEigeartaigh wrote:
>>>>>>>>>>>>>>> Thanks Francesco! On a related note, let's say I have some
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> AnyObjects
>>>>>>>>>>>>>>>> (Printer) with a relationship to other AnyObjects
>>>>>>>>>>>>>>>> (Cartridge).
>>>>>>>>>>>>>>>> Now I
>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>> to search for a Printer which has a relationship with a
>>>>>>>>>>>>>>>> Cartridge
>>>>>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>> "colour" attribute of "blue". Is there a way to do this via a
>>>>>>>>>>>>>>>> FIQL
>>>>>>>>>>>>>>>> expression?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> No, you cannot express such condition ATM; you could do for
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> example:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>>>>>>>> inRelationships("ce75249b-76e4-44b6-88ae-0841846faceb").
>>>>>>>>>>>>>>> and().is("colour").equalTo("blue").query();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> which translates to FIQL
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> $type==PRINTER;$relationships==ce75249b-76e4-44b6-88ae-08418
>>>>>>>>>>>>>>> 46faceb;colour==blue
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> but this would rather search for blue printers having a
>>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>> with an any object with key 'ce75249b-76e4-44b6-88ae-08418
>>>>>>>>>>>>>>> 46faceb'.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> or alternatively
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> SyncopeClient.getAnyObjectSearchConditionBuilder("PRINTER").
>>>>>>>>>>>>>>> inRelationshipTypes("WITH_CARTDRIGE").
>>>>>>>>>>>>>>> and().is("color").equalTo("blue").query();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> which translates to FIQL
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> $type==PRINTER;$relationshipTypes==WITH_CARTDRIGE;color==blue
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> but this would rather search for blue printers having a
>>>>>>>>>>>>>>> relationship
>>>>>>>>>>>>>>> on type WITH_CARTDRIGE.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jun 27, 2017 at 4:29 PM, Francesco Chicchiriccò <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ilgrosso@apache.org> wrote:
>>>>>>>>>>>>>>>> On 27/06/2017 17:24, Colm O hEigeartaigh wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi all,
>>>>>>>>>>>>>>>>> How can I retrieve a list of AnyObjects? The following
>>>>>>>>>>>>>>>>>> returns a
>>>>>>>>>>>>>>>>>> 400:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> curl -I -X GET -u admin:password
>>>>>>>>>>>>>>>>>> http://localhost:9080/syncope/rest/anyObjects
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> You must at least provide the AnyType, e.g.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> http://localhost:9080/syncope/
>>>>>>>>>>>>>>>>> rest/anyObjects;fiql=%24type%3
>>>>>>>>>>>>>>>>> D%3DPRINTER
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>
>> --
>> Francesco Chicchiriccò
>>
>> Tirasa - Open Source Excellence
>> http://www.tirasa.net/
>>
>> Member at The Apache Software Foundation
>> Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
>> http://home.apache.org/~ilgrosso/
>>
>>
>
--
Francesco Chicchiriccò
Tirasa - Open Source Excellence
http://www.tirasa.net/
Member at The Apache Software Foundation
Syncope, Cocoon, Olingo, CXF, OpenJPA, PonyMail
http://home.apache.org/~ilgrosso/