You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@archiva.apache.org by Carlos Sanchez <ca...@apache.org> on 2014/11/03 20:33:48 UTC
Searching pom elements and generic metadata
Hi,
Two things I wasn't able to do and was going to implement (unless I missed
something):
* search for generic metadata
* search for pom contents
The way the search queries are built in SearchService only looks for the
typical fields (artifactId, groupId,...), so that needs to be changed, but
I don't know if the pom and generic metadata fields are even indexed, any
pointers?
Looking to be able to search for "myMetadata:myValue" and something like
"developer:carlos*" through the rest api and ui.
Thanks
Re: Searching pom elements and generic metadata
Posted by Carlos Sanchez <ca...@csanchez.org>.
Thanks, got an initial implementation of getArtifactsByMetadata at
https://github.com/carlossg/archiva/compare/metadata-search
I added the repositoryId param, like other methods, but I'm not sure if
that would be limiting the feature, searching in all repos would be more
interesting.
On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <ol...@apache.org> wrote:
> basic Cassandra requests tru Hector client won't work.
> You will have to write some manual cql queries.
>
> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org> wrote:
> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org> wrote:
> >
> >> I would add that in the interface MetadataRepository, add a test in
> >> the tck and implements it for all providers :-)
> >> Maybe having 2 methods:
> >> * generic metadata: getArtifactsByMetadata( key, value) ( maybe a Map
> >> for multiple value pair? will you test the full value or a matching?)
> >>
> >
> > is the matching syntax for cassandra an jcr different?
> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to do matching
> is
> > it going to work only in jcr?
> >
> >
> >
> >> * pom contents: getArtifactsByProperty( key, value ) ( same questions )
> >>
> >> To start I won't go with a Map and exact matching :-) (I reckon pom
> >> content can be a pain with Cassandra impl)
> >>
> >> WDYT?
> >>
> >> Olivier
> >>
> >>
> >> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org> wrote:
> >> > Thanks for the pointers, so if I add a method to JcrMetadataRepository
> >> >
> >> > getArtifactsByProperty(key, value)
> >> >
> >> > similar to getArtifactsByDateRange, writing the correct JCR query, I
> >> could
> >> > access all that metadata?
> >> >
> >> >
> >> > In which of the rest services would it make sense to add such a
> search?
> >> >
> >> >
> >> >
> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <ol...@apache.org>
> wrote:
> >> >
> >> >> Hi,
> >> >> The current SearchService search in maven indexer available datas.
> >> >> All the data you want to search are not available here but in the
> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
> >> >> And currently there is no search methods for that!
> >> >> So really good news to see you as a volunteer to implement this! :-)
> >> >> Let me know if you need any pointers.
> >> >>
> >> >> Cheers
> >> >> Olivier
> >> >>
> >> >>
> >> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > Two things I wasn't able to do and was going to implement (unless I
> >> >> missed
> >> >> > something):
> >> >> >
> >> >> > * search for generic metadata
> >> >> > * search for pom contents
> >> >> >
> >> >> > The way the search queries are built in SearchService only looks
> for
> >> the
> >> >> > typical fields (artifactId, groupId,...), so that needs to be
> changed,
> >> >> but
> >> >> > I don't know if the pom and generic metadata fields are even
> indexed,
> >> any
> >> >> > pointers?
> >> >> >
> >> >> > Looking to be able to search for "myMetadata:myValue" and something
> >> like
> >> >> > "developer:carlos*" through the rest api and ui.
> >> >> >
> >> >> > Thanks
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Olivier Lamy
> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >>
> >>
> >>
> >>
> >> --
> >> Olivier Lamy
> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>
>
>
>
> --
> Olivier Lamy
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
Re: Searching pom elements and generic metadata
Posted by Olivier Lamy <ol...@apache.org>.
I did a bulk update of issue marked fixed for 2.1.2 to 2.2.0 so we are good :-)
On 26 November 2014 at 15:54, Olivier Lamy <ol...@apache.org> wrote:
> release notes page is @ ./archiva-docs/src/site/apt/release-notes.apt.vm
>
> Not renaming but maybe copy values to the new column?
> Anyway I'm not sure if any users use cassandra :-)
> So maybe worth to mention in release note but not more?
>
> On 26 November 2014 at 09:49, Carlos Sanchez <ca...@apache.org> wrote:
>> Where would I put release notes?
>> For those using Cassandra they need to rename a column, not sure if it is
>> easy programmatically
>>
>> On Tue, Nov 25, 2014 at 10:40 PM, Olivier Lamy <ol...@apache.org> wrote:
>>
>>> Looks good.
>>> Ship it! :-)
>>> As you bump version to 2.2 there are some bulk updates to do in jira.
>>>
>>> --
>>> Olivier
>>> On 25 Nov 2014 06:06, "Carlos Sanchez" <ca...@apache.org> wrote:
>>>
>>> > PR submitted as https://github.com/apache/archiva/pull/14
>>> >
>>> > feedback welcome, there are some new methods there
>>> >
>>> > On Thu, Nov 20, 2014 at 11:09 AM, Carlos Sanchez <ca...@apache.org>
>>> > wrote:
>>> >
>>> > > I have added these REST methods with JCR implementations
>>> > >
>>> > > artifactsByProjectVersionMetadata/{key}/{value}
>>> > > artifactsByMetadata/{key}/{value}
>>> > > artifactsByProperty/{key}/{value}
>>> > > searchArtifacts/{text}
>>> > > searchArtifacts/{key}/{text}
>>> > >
>>> > >
>>> > > Docs here
>>> > >
>>> >
>>> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search
>>> > >
>>> > > Any suggestions on how to run these types of queries in Cassandra?
>>> > >
>>> > >
>>> >
>>> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search#diff-9de24dcf873b85547a60cd92cc2f6379R764
>>> > >
>>> > >
>>> > >
>>> > > On Fri, Nov 7, 2014 at 11:19 PM, Carlos Sanchez <ca...@csanchez.org>
>>> > > wrote:
>>> > >
>>> > >> I have added search methods for any property, including facets
>>> > >>
>>> > >> restServices/archivaServices/browseService/searchArtifacts/key/text
>>> > >> restServices/archivaServices/browseService/searchArtifacts/text
>>> > >>
>>> > >> that search in JCR
>>> > >>
>>> > >> SELECT * FROM [archiva:projectVersion] AS projectVersion
>>> > >> LEFT OUTER JOIN [archiva:artifact] AS artifact ON
>>> > ISCHILDNODE(artifact,
>>> > >> projectVersion)
>>> > >> LEFT OUTER JOIN [archiva:facet] AS facet ON ISCHILDNODE(facet,
>>> > >> projectVersion)
>>> > >> WHERE (contains([projectVersion].*, $value) OR contains([facet].*,
>>> > >> $value))
>>> > >> AND ISDESCENDANTNODE(artifact,'/repositories/internal/content/')
>>> > >>
>>> > >> But seems that performance is bad for a "normal" sized repo. Any
>>> > >> suggestions?
>>> > >> I'll try with subselects and tuning jcr, I would imagine that it
>>> should
>>> > >> work decently and there's something wrong with archiva.
>>> > >>
>>> > >>
>>> > >>
>>> > >> On Fri, Nov 7, 2014 at 11:17 AM, Olivier Lamy <ol...@apache.org>
>>> wrote:
>>> > >>
>>> > >>> On 7 November 2014 07:42, Carlos Sanchez <ca...@apache.org> wrote:
>>> > >>> > I got both getArtifactsByMetadata and getArtifactsByProperty
>>> working,
>>> > >>> > passing the repository
>>> > >>> >
>>> > >>> >
>>> > >>>
>>> >
>>> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
>>> > >>> >
>>> > >>> > The problem I hit with not passing the repo is that
>>> > >>> > AbstractTestService.buildArtifacts needs it. Will look later.
>>> > >>> >
>>> > >>> > About adding the other pom fields to the search index, from what I
>>> > saw
>>> > >>> I
>>> > >>> > need to add them to MinimalArtifactInfoIndexCreator (or a new one)
>>> in
>>> > >>> the
>>> > >>> > maven-indexer project, and then archiva would just use that?
>>> > >>>
>>> > >>> In such case you will have to modify the lucene model in the maven
>>> > >>> indexer project ( can certainly breaks a lot :-) ).
>>> > >>>
>>> > >>> >
>>> > >>> > Are the fields indexed already searchable somewhere in archiva with
>>> > >>> field
>>> > >>> > selection? e.g. g:myGroupId or v:1.0 ?
>>> > >>>
>>> > >>> Not associated at this level but that's already in our repository.
>>> > >>>
>>> > >>>
>>> > >>> >
>>> > >>> > Thanks
>>> > >>> >
>>> > >>> >
>>> > >>> >
>>> > >>> >
>>> > >>> > On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <ol...@apache.org>
>>> > >>> wrote:
>>> > >>> >
>>> > >>> >> Hi,
>>> > >>> >> I would make repositoryId as a query param so will make it
>>> optional.
>>> > >>> >> If empty/null use the users repositories?
>>> > >>> >>
>>> > >>> >> On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org>
>>> wrote:
>>> > >>> >> > Thanks, got an initial implementation of getArtifactsByMetadata
>>> at
>>> > >>> >> > https://github.com/carlossg/archiva/compare/metadata-search
>>> > >>> >> >
>>> > >>> >> > I added the repositoryId param, like other methods, but I'm not
>>> > >>> sure if
>>> > >>> >> > that would be limiting the feature, searching in all repos would
>>> > be
>>> > >>> more
>>> > >>> >> > interesting.
>>> > >>> >> >
>>> > >>> >> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <olamy@apache.org
>>> >
>>> > >>> wrote:
>>> > >>> >> >
>>> > >>> >> >> basic Cassandra requests tru Hector client won't work.
>>> > >>> >> >> You will have to write some manual cql queries.
>>> > >>> >> >>
>>> > >>> >> >> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org>
>>> > >>> wrote:
>>> > >>> >> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <
>>> > olamy@apache.org>
>>> > >>> >> wrote:
>>> > >>> >> >> >
>>> > >>> >> >> >> I would add that in the interface MetadataRepository, add a
>>> > >>> test in
>>> > >>> >> >> >> the tck and implements it for all providers :-)
>>> > >>> >> >> >> Maybe having 2 methods:
>>> > >>> >> >> >> * generic metadata: getArtifactsByMetadata( key, value) (
>>> > maybe
>>> > >>> a Map
>>> > >>> >> >> >> for multiple value pair? will you test the full value or a
>>> > >>> matching?)
>>> > >>> >> >> >>
>>> > >>> >> >> >
>>> > >>> >> >> > is the matching syntax for cassandra an jcr different?
>>> > >>> >> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to
>>> do
>>> > >>> >> matching
>>> > >>> >> >> is
>>> > >>> >> >> > it going to work only in jcr?
>>> > >>> >> >> >
>>> > >>> >> >> >
>>> > >>> >> >> >
>>> > >>> >> >> >> * pom contents: getArtifactsByProperty( key, value ) ( same
>>> > >>> >> questions )
>>> > >>> >> >> >>
>>> > >>> >> >> >> To start I won't go with a Map and exact matching :-) (I
>>> > reckon
>>> > >>> pom
>>> > >>> >> >> >> content can be a pain with Cassandra impl)
>>> > >>> >> >> >>
>>> > >>> >> >> >> WDYT?
>>> > >>> >> >> >>
>>> > >>> >> >> >> Olivier
>>> > >>> >> >> >>
>>> > >>> >> >> >>
>>> > >>> >> >> >> On 5 November 2014 21:45, Carlos Sanchez <carlos@apache.org
>>> >
>>> > >>> wrote:
>>> > >>> >> >> >> > Thanks for the pointers, so if I add a method to
>>> > >>> >> JcrMetadataRepository
>>> > >>> >> >> >> >
>>> > >>> >> >> >> > getArtifactsByProperty(key, value)
>>> > >>> >> >> >> >
>>> > >>> >> >> >> > similar to getArtifactsByDateRange, writing the correct
>>> JCR
>>> > >>> query,
>>> > >>> >> I
>>> > >>> >> >> >> could
>>> > >>> >> >> >> > access all that metadata?
>>> > >>> >> >> >> >
>>> > >>> >> >> >> >
>>> > >>> >> >> >> > In which of the rest services would it make sense to add
>>> > such
>>> > >>> a
>>> > >>> >> >> search?
>>> > >>> >> >> >> >
>>> > >>> >> >> >> >
>>> > >>> >> >> >> >
>>> > >>> >> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <
>>> > >>> olamy@apache.org>
>>> > >>> >> >> wrote:
>>> > >>> >> >> >> >
>>> > >>> >> >> >> >> Hi,
>>> > >>> >> >> >> >> The current SearchService search in maven indexer
>>> available
>>> > >>> datas.
>>> > >>> >> >> >> >> All the data you want to search are not available here
>>> but
>>> > >>> in the
>>> > >>> >> >> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
>>> > >>> >> >> >> >> And currently there is no search methods for that!
>>> > >>> >> >> >> >> So really good news to see you as a volunteer to
>>> implement
>>> > >>> this!
>>> > >>> >> :-)
>>> > >>> >> >> >> >> Let me know if you need any pointers.
>>> > >>> >> >> >> >>
>>> > >>> >> >> >> >> Cheers
>>> > >>> >> >> >> >> Olivier
>>> > >>> >> >> >> >>
>>> > >>> >> >> >> >>
>>> > >>> >> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <
>>> > carlos@apache.org>
>>> > >>> >> wrote:
>>> > >>> >> >> >> >> > Hi,
>>> > >>> >> >> >> >> >
>>> > >>> >> >> >> >> > Two things I wasn't able to do and was going to
>>> implement
>>> > >>> >> (unless I
>>> > >>> >> >> >> >> missed
>>> > >>> >> >> >> >> > something):
>>> > >>> >> >> >> >> >
>>> > >>> >> >> >> >> > * search for generic metadata
>>> > >>> >> >> >> >> > * search for pom contents
>>> > >>> >> >> >> >> >
>>> > >>> >> >> >> >> > The way the search queries are built in SearchService
>>> > only
>>> > >>> looks
>>> > >>> >> >> for
>>> > >>> >> >> >> the
>>> > >>> >> >> >> >> > typical fields (artifactId, groupId,...), so that needs
>>> > to
>>> > >>> be
>>> > >>> >> >> changed,
>>> > >>> >> >> >> >> but
>>> > >>> >> >> >> >> > I don't know if the pom and generic metadata fields are
>>> > >>> even
>>> > >>> >> >> indexed,
>>> > >>> >> >> >> any
>>> > >>> >> >> >> >> > pointers?
>>> > >>> >> >> >> >> >
>>> > >>> >> >> >> >> > Looking to be able to search for "myMetadata:myValue"
>>> and
>>> > >>> >> something
>>> > >>> >> >> >> like
>>> > >>> >> >> >> >> > "developer:carlos*" through the rest api and ui.
>>> > >>> >> >> >> >> >
>>> > >>> >> >> >> >> > Thanks
>>> > >>> >> >> >> >>
>>> > >>> >> >> >> >>
>>> > >>> >> >> >> >>
>>> > >>> >> >> >> >> --
>>> > >>> >> >> >> >> Olivier Lamy
>>> > >>> >> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> > >>> >> >> >> >>
>>> > >>> >> >> >>
>>> > >>> >> >> >>
>>> > >>> >> >> >>
>>> > >>> >> >> >> --
>>> > >>> >> >> >> Olivier Lamy
>>> > >>> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> > >>> >> >> >>
>>> > >>> >> >>
>>> > >>> >> >>
>>> > >>> >> >>
>>> > >>> >> >> --
>>> > >>> >> >> Olivier Lamy
>>> > >>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> > >>> >> >>
>>> > >>> >>
>>> > >>> >>
>>> > >>> >>
>>> > >>> >> --
>>> > >>> >> Olivier Lamy
>>> > >>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> > >>> >>
>>> > >>>
>>> > >>>
>>> > >>>
>>> > >>> --
>>> > >>> Olivier Lamy
>>> > >>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> > >>>
>>> > >>
>>> > >>
>>> > >
>>> >
>>>
>
>
>
> --
> Olivier Lamy
> http://twitter.com/olamy | http://linkedin.com/in/olamy
--
Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Searching pom elements and generic metadata
Posted by Olivier Lamy <ol...@apache.org>.
release notes page is @ ./archiva-docs/src/site/apt/release-notes.apt.vm
Not renaming but maybe copy values to the new column?
Anyway I'm not sure if any users use cassandra :-)
So maybe worth to mention in release note but not more?
On 26 November 2014 at 09:49, Carlos Sanchez <ca...@apache.org> wrote:
> Where would I put release notes?
> For those using Cassandra they need to rename a column, not sure if it is
> easy programmatically
>
> On Tue, Nov 25, 2014 at 10:40 PM, Olivier Lamy <ol...@apache.org> wrote:
>
>> Looks good.
>> Ship it! :-)
>> As you bump version to 2.2 there are some bulk updates to do in jira.
>>
>> --
>> Olivier
>> On 25 Nov 2014 06:06, "Carlos Sanchez" <ca...@apache.org> wrote:
>>
>> > PR submitted as https://github.com/apache/archiva/pull/14
>> >
>> > feedback welcome, there are some new methods there
>> >
>> > On Thu, Nov 20, 2014 at 11:09 AM, Carlos Sanchez <ca...@apache.org>
>> > wrote:
>> >
>> > > I have added these REST methods with JCR implementations
>> > >
>> > > artifactsByProjectVersionMetadata/{key}/{value}
>> > > artifactsByMetadata/{key}/{value}
>> > > artifactsByProperty/{key}/{value}
>> > > searchArtifacts/{text}
>> > > searchArtifacts/{key}/{text}
>> > >
>> > >
>> > > Docs here
>> > >
>> >
>> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search
>> > >
>> > > Any suggestions on how to run these types of queries in Cassandra?
>> > >
>> > >
>> >
>> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search#diff-9de24dcf873b85547a60cd92cc2f6379R764
>> > >
>> > >
>> > >
>> > > On Fri, Nov 7, 2014 at 11:19 PM, Carlos Sanchez <ca...@csanchez.org>
>> > > wrote:
>> > >
>> > >> I have added search methods for any property, including facets
>> > >>
>> > >> restServices/archivaServices/browseService/searchArtifacts/key/text
>> > >> restServices/archivaServices/browseService/searchArtifacts/text
>> > >>
>> > >> that search in JCR
>> > >>
>> > >> SELECT * FROM [archiva:projectVersion] AS projectVersion
>> > >> LEFT OUTER JOIN [archiva:artifact] AS artifact ON
>> > ISCHILDNODE(artifact,
>> > >> projectVersion)
>> > >> LEFT OUTER JOIN [archiva:facet] AS facet ON ISCHILDNODE(facet,
>> > >> projectVersion)
>> > >> WHERE (contains([projectVersion].*, $value) OR contains([facet].*,
>> > >> $value))
>> > >> AND ISDESCENDANTNODE(artifact,'/repositories/internal/content/')
>> > >>
>> > >> But seems that performance is bad for a "normal" sized repo. Any
>> > >> suggestions?
>> > >> I'll try with subselects and tuning jcr, I would imagine that it
>> should
>> > >> work decently and there's something wrong with archiva.
>> > >>
>> > >>
>> > >>
>> > >> On Fri, Nov 7, 2014 at 11:17 AM, Olivier Lamy <ol...@apache.org>
>> wrote:
>> > >>
>> > >>> On 7 November 2014 07:42, Carlos Sanchez <ca...@apache.org> wrote:
>> > >>> > I got both getArtifactsByMetadata and getArtifactsByProperty
>> working,
>> > >>> > passing the repository
>> > >>> >
>> > >>> >
>> > >>>
>> >
>> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
>> > >>> >
>> > >>> > The problem I hit with not passing the repo is that
>> > >>> > AbstractTestService.buildArtifacts needs it. Will look later.
>> > >>> >
>> > >>> > About adding the other pom fields to the search index, from what I
>> > saw
>> > >>> I
>> > >>> > need to add them to MinimalArtifactInfoIndexCreator (or a new one)
>> in
>> > >>> the
>> > >>> > maven-indexer project, and then archiva would just use that?
>> > >>>
>> > >>> In such case you will have to modify the lucene model in the maven
>> > >>> indexer project ( can certainly breaks a lot :-) ).
>> > >>>
>> > >>> >
>> > >>> > Are the fields indexed already searchable somewhere in archiva with
>> > >>> field
>> > >>> > selection? e.g. g:myGroupId or v:1.0 ?
>> > >>>
>> > >>> Not associated at this level but that's already in our repository.
>> > >>>
>> > >>>
>> > >>> >
>> > >>> > Thanks
>> > >>> >
>> > >>> >
>> > >>> >
>> > >>> >
>> > >>> > On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <ol...@apache.org>
>> > >>> wrote:
>> > >>> >
>> > >>> >> Hi,
>> > >>> >> I would make repositoryId as a query param so will make it
>> optional.
>> > >>> >> If empty/null use the users repositories?
>> > >>> >>
>> > >>> >> On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org>
>> wrote:
>> > >>> >> > Thanks, got an initial implementation of getArtifactsByMetadata
>> at
>> > >>> >> > https://github.com/carlossg/archiva/compare/metadata-search
>> > >>> >> >
>> > >>> >> > I added the repositoryId param, like other methods, but I'm not
>> > >>> sure if
>> > >>> >> > that would be limiting the feature, searching in all repos would
>> > be
>> > >>> more
>> > >>> >> > interesting.
>> > >>> >> >
>> > >>> >> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <olamy@apache.org
>> >
>> > >>> wrote:
>> > >>> >> >
>> > >>> >> >> basic Cassandra requests tru Hector client won't work.
>> > >>> >> >> You will have to write some manual cql queries.
>> > >>> >> >>
>> > >>> >> >> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org>
>> > >>> wrote:
>> > >>> >> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <
>> > olamy@apache.org>
>> > >>> >> wrote:
>> > >>> >> >> >
>> > >>> >> >> >> I would add that in the interface MetadataRepository, add a
>> > >>> test in
>> > >>> >> >> >> the tck and implements it for all providers :-)
>> > >>> >> >> >> Maybe having 2 methods:
>> > >>> >> >> >> * generic metadata: getArtifactsByMetadata( key, value) (
>> > maybe
>> > >>> a Map
>> > >>> >> >> >> for multiple value pair? will you test the full value or a
>> > >>> matching?)
>> > >>> >> >> >>
>> > >>> >> >> >
>> > >>> >> >> > is the matching syntax for cassandra an jcr different?
>> > >>> >> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to
>> do
>> > >>> >> matching
>> > >>> >> >> is
>> > >>> >> >> > it going to work only in jcr?
>> > >>> >> >> >
>> > >>> >> >> >
>> > >>> >> >> >
>> > >>> >> >> >> * pom contents: getArtifactsByProperty( key, value ) ( same
>> > >>> >> questions )
>> > >>> >> >> >>
>> > >>> >> >> >> To start I won't go with a Map and exact matching :-) (I
>> > reckon
>> > >>> pom
>> > >>> >> >> >> content can be a pain with Cassandra impl)
>> > >>> >> >> >>
>> > >>> >> >> >> WDYT?
>> > >>> >> >> >>
>> > >>> >> >> >> Olivier
>> > >>> >> >> >>
>> > >>> >> >> >>
>> > >>> >> >> >> On 5 November 2014 21:45, Carlos Sanchez <carlos@apache.org
>> >
>> > >>> wrote:
>> > >>> >> >> >> > Thanks for the pointers, so if I add a method to
>> > >>> >> JcrMetadataRepository
>> > >>> >> >> >> >
>> > >>> >> >> >> > getArtifactsByProperty(key, value)
>> > >>> >> >> >> >
>> > >>> >> >> >> > similar to getArtifactsByDateRange, writing the correct
>> JCR
>> > >>> query,
>> > >>> >> I
>> > >>> >> >> >> could
>> > >>> >> >> >> > access all that metadata?
>> > >>> >> >> >> >
>> > >>> >> >> >> >
>> > >>> >> >> >> > In which of the rest services would it make sense to add
>> > such
>> > >>> a
>> > >>> >> >> search?
>> > >>> >> >> >> >
>> > >>> >> >> >> >
>> > >>> >> >> >> >
>> > >>> >> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <
>> > >>> olamy@apache.org>
>> > >>> >> >> wrote:
>> > >>> >> >> >> >
>> > >>> >> >> >> >> Hi,
>> > >>> >> >> >> >> The current SearchService search in maven indexer
>> available
>> > >>> datas.
>> > >>> >> >> >> >> All the data you want to search are not available here
>> but
>> > >>> in the
>> > >>> >> >> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
>> > >>> >> >> >> >> And currently there is no search methods for that!
>> > >>> >> >> >> >> So really good news to see you as a volunteer to
>> implement
>> > >>> this!
>> > >>> >> :-)
>> > >>> >> >> >> >> Let me know if you need any pointers.
>> > >>> >> >> >> >>
>> > >>> >> >> >> >> Cheers
>> > >>> >> >> >> >> Olivier
>> > >>> >> >> >> >>
>> > >>> >> >> >> >>
>> > >>> >> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <
>> > carlos@apache.org>
>> > >>> >> wrote:
>> > >>> >> >> >> >> > Hi,
>> > >>> >> >> >> >> >
>> > >>> >> >> >> >> > Two things I wasn't able to do and was going to
>> implement
>> > >>> >> (unless I
>> > >>> >> >> >> >> missed
>> > >>> >> >> >> >> > something):
>> > >>> >> >> >> >> >
>> > >>> >> >> >> >> > * search for generic metadata
>> > >>> >> >> >> >> > * search for pom contents
>> > >>> >> >> >> >> >
>> > >>> >> >> >> >> > The way the search queries are built in SearchService
>> > only
>> > >>> looks
>> > >>> >> >> for
>> > >>> >> >> >> the
>> > >>> >> >> >> >> > typical fields (artifactId, groupId,...), so that needs
>> > to
>> > >>> be
>> > >>> >> >> changed,
>> > >>> >> >> >> >> but
>> > >>> >> >> >> >> > I don't know if the pom and generic metadata fields are
>> > >>> even
>> > >>> >> >> indexed,
>> > >>> >> >> >> any
>> > >>> >> >> >> >> > pointers?
>> > >>> >> >> >> >> >
>> > >>> >> >> >> >> > Looking to be able to search for "myMetadata:myValue"
>> and
>> > >>> >> something
>> > >>> >> >> >> like
>> > >>> >> >> >> >> > "developer:carlos*" through the rest api and ui.
>> > >>> >> >> >> >> >
>> > >>> >> >> >> >> > Thanks
>> > >>> >> >> >> >>
>> > >>> >> >> >> >>
>> > >>> >> >> >> >>
>> > >>> >> >> >> >> --
>> > >>> >> >> >> >> Olivier Lamy
>> > >>> >> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> > >>> >> >> >> >>
>> > >>> >> >> >>
>> > >>> >> >> >>
>> > >>> >> >> >>
>> > >>> >> >> >> --
>> > >>> >> >> >> Olivier Lamy
>> > >>> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> > >>> >> >> >>
>> > >>> >> >>
>> > >>> >> >>
>> > >>> >> >>
>> > >>> >> >> --
>> > >>> >> >> Olivier Lamy
>> > >>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> > >>> >> >>
>> > >>> >>
>> > >>> >>
>> > >>> >>
>> > >>> >> --
>> > >>> >> Olivier Lamy
>> > >>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> > >>> >>
>> > >>>
>> > >>>
>> > >>>
>> > >>> --
>> > >>> Olivier Lamy
>> > >>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> > >>>
>> > >>
>> > >>
>> > >
>> >
>>
--
Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Searching pom elements and generic metadata
Posted by Carlos Sanchez <ca...@apache.org>.
Where would I put release notes?
For those using Cassandra they need to rename a column, not sure if it is
easy programmatically
On Tue, Nov 25, 2014 at 10:40 PM, Olivier Lamy <ol...@apache.org> wrote:
> Looks good.
> Ship it! :-)
> As you bump version to 2.2 there are some bulk updates to do in jira.
>
> --
> Olivier
> On 25 Nov 2014 06:06, "Carlos Sanchez" <ca...@apache.org> wrote:
>
> > PR submitted as https://github.com/apache/archiva/pull/14
> >
> > feedback welcome, there are some new methods there
> >
> > On Thu, Nov 20, 2014 at 11:09 AM, Carlos Sanchez <ca...@apache.org>
> > wrote:
> >
> > > I have added these REST methods with JCR implementations
> > >
> > > artifactsByProjectVersionMetadata/{key}/{value}
> > > artifactsByMetadata/{key}/{value}
> > > artifactsByProperty/{key}/{value}
> > > searchArtifacts/{text}
> > > searchArtifacts/{key}/{text}
> > >
> > >
> > > Docs here
> > >
> >
> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search
> > >
> > > Any suggestions on how to run these types of queries in Cassandra?
> > >
> > >
> >
> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search#diff-9de24dcf873b85547a60cd92cc2f6379R764
> > >
> > >
> > >
> > > On Fri, Nov 7, 2014 at 11:19 PM, Carlos Sanchez <ca...@csanchez.org>
> > > wrote:
> > >
> > >> I have added search methods for any property, including facets
> > >>
> > >> restServices/archivaServices/browseService/searchArtifacts/key/text
> > >> restServices/archivaServices/browseService/searchArtifacts/text
> > >>
> > >> that search in JCR
> > >>
> > >> SELECT * FROM [archiva:projectVersion] AS projectVersion
> > >> LEFT OUTER JOIN [archiva:artifact] AS artifact ON
> > ISCHILDNODE(artifact,
> > >> projectVersion)
> > >> LEFT OUTER JOIN [archiva:facet] AS facet ON ISCHILDNODE(facet,
> > >> projectVersion)
> > >> WHERE (contains([projectVersion].*, $value) OR contains([facet].*,
> > >> $value))
> > >> AND ISDESCENDANTNODE(artifact,'/repositories/internal/content/')
> > >>
> > >> But seems that performance is bad for a "normal" sized repo. Any
> > >> suggestions?
> > >> I'll try with subselects and tuning jcr, I would imagine that it
> should
> > >> work decently and there's something wrong with archiva.
> > >>
> > >>
> > >>
> > >> On Fri, Nov 7, 2014 at 11:17 AM, Olivier Lamy <ol...@apache.org>
> wrote:
> > >>
> > >>> On 7 November 2014 07:42, Carlos Sanchez <ca...@apache.org> wrote:
> > >>> > I got both getArtifactsByMetadata and getArtifactsByProperty
> working,
> > >>> > passing the repository
> > >>> >
> > >>> >
> > >>>
> >
> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
> > >>> >
> > >>> > The problem I hit with not passing the repo is that
> > >>> > AbstractTestService.buildArtifacts needs it. Will look later.
> > >>> >
> > >>> > About adding the other pom fields to the search index, from what I
> > saw
> > >>> I
> > >>> > need to add them to MinimalArtifactInfoIndexCreator (or a new one)
> in
> > >>> the
> > >>> > maven-indexer project, and then archiva would just use that?
> > >>>
> > >>> In such case you will have to modify the lucene model in the maven
> > >>> indexer project ( can certainly breaks a lot :-) ).
> > >>>
> > >>> >
> > >>> > Are the fields indexed already searchable somewhere in archiva with
> > >>> field
> > >>> > selection? e.g. g:myGroupId or v:1.0 ?
> > >>>
> > >>> Not associated at this level but that's already in our repository.
> > >>>
> > >>>
> > >>> >
> > >>> > Thanks
> > >>> >
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <ol...@apache.org>
> > >>> wrote:
> > >>> >
> > >>> >> Hi,
> > >>> >> I would make repositoryId as a query param so will make it
> optional.
> > >>> >> If empty/null use the users repositories?
> > >>> >>
> > >>> >> On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org>
> wrote:
> > >>> >> > Thanks, got an initial implementation of getArtifactsByMetadata
> at
> > >>> >> > https://github.com/carlossg/archiva/compare/metadata-search
> > >>> >> >
> > >>> >> > I added the repositoryId param, like other methods, but I'm not
> > >>> sure if
> > >>> >> > that would be limiting the feature, searching in all repos would
> > be
> > >>> more
> > >>> >> > interesting.
> > >>> >> >
> > >>> >> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <olamy@apache.org
> >
> > >>> wrote:
> > >>> >> >
> > >>> >> >> basic Cassandra requests tru Hector client won't work.
> > >>> >> >> You will have to write some manual cql queries.
> > >>> >> >>
> > >>> >> >> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org>
> > >>> wrote:
> > >>> >> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <
> > olamy@apache.org>
> > >>> >> wrote:
> > >>> >> >> >
> > >>> >> >> >> I would add that in the interface MetadataRepository, add a
> > >>> test in
> > >>> >> >> >> the tck and implements it for all providers :-)
> > >>> >> >> >> Maybe having 2 methods:
> > >>> >> >> >> * generic metadata: getArtifactsByMetadata( key, value) (
> > maybe
> > >>> a Map
> > >>> >> >> >> for multiple value pair? will you test the full value or a
> > >>> matching?)
> > >>> >> >> >>
> > >>> >> >> >
> > >>> >> >> > is the matching syntax for cassandra an jcr different?
> > >>> >> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to
> do
> > >>> >> matching
> > >>> >> >> is
> > >>> >> >> > it going to work only in jcr?
> > >>> >> >> >
> > >>> >> >> >
> > >>> >> >> >
> > >>> >> >> >> * pom contents: getArtifactsByProperty( key, value ) ( same
> > >>> >> questions )
> > >>> >> >> >>
> > >>> >> >> >> To start I won't go with a Map and exact matching :-) (I
> > reckon
> > >>> pom
> > >>> >> >> >> content can be a pain with Cassandra impl)
> > >>> >> >> >>
> > >>> >> >> >> WDYT?
> > >>> >> >> >>
> > >>> >> >> >> Olivier
> > >>> >> >> >>
> > >>> >> >> >>
> > >>> >> >> >> On 5 November 2014 21:45, Carlos Sanchez <carlos@apache.org
> >
> > >>> wrote:
> > >>> >> >> >> > Thanks for the pointers, so if I add a method to
> > >>> >> JcrMetadataRepository
> > >>> >> >> >> >
> > >>> >> >> >> > getArtifactsByProperty(key, value)
> > >>> >> >> >> >
> > >>> >> >> >> > similar to getArtifactsByDateRange, writing the correct
> JCR
> > >>> query,
> > >>> >> I
> > >>> >> >> >> could
> > >>> >> >> >> > access all that metadata?
> > >>> >> >> >> >
> > >>> >> >> >> >
> > >>> >> >> >> > In which of the rest services would it make sense to add
> > such
> > >>> a
> > >>> >> >> search?
> > >>> >> >> >> >
> > >>> >> >> >> >
> > >>> >> >> >> >
> > >>> >> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <
> > >>> olamy@apache.org>
> > >>> >> >> wrote:
> > >>> >> >> >> >
> > >>> >> >> >> >> Hi,
> > >>> >> >> >> >> The current SearchService search in maven indexer
> available
> > >>> datas.
> > >>> >> >> >> >> All the data you want to search are not available here
> but
> > >>> in the
> > >>> >> >> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
> > >>> >> >> >> >> And currently there is no search methods for that!
> > >>> >> >> >> >> So really good news to see you as a volunteer to
> implement
> > >>> this!
> > >>> >> :-)
> > >>> >> >> >> >> Let me know if you need any pointers.
> > >>> >> >> >> >>
> > >>> >> >> >> >> Cheers
> > >>> >> >> >> >> Olivier
> > >>> >> >> >> >>
> > >>> >> >> >> >>
> > >>> >> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <
> > carlos@apache.org>
> > >>> >> wrote:
> > >>> >> >> >> >> > Hi,
> > >>> >> >> >> >> >
> > >>> >> >> >> >> > Two things I wasn't able to do and was going to
> implement
> > >>> >> (unless I
> > >>> >> >> >> >> missed
> > >>> >> >> >> >> > something):
> > >>> >> >> >> >> >
> > >>> >> >> >> >> > * search for generic metadata
> > >>> >> >> >> >> > * search for pom contents
> > >>> >> >> >> >> >
> > >>> >> >> >> >> > The way the search queries are built in SearchService
> > only
> > >>> looks
> > >>> >> >> for
> > >>> >> >> >> the
> > >>> >> >> >> >> > typical fields (artifactId, groupId,...), so that needs
> > to
> > >>> be
> > >>> >> >> changed,
> > >>> >> >> >> >> but
> > >>> >> >> >> >> > I don't know if the pom and generic metadata fields are
> > >>> even
> > >>> >> >> indexed,
> > >>> >> >> >> any
> > >>> >> >> >> >> > pointers?
> > >>> >> >> >> >> >
> > >>> >> >> >> >> > Looking to be able to search for "myMetadata:myValue"
> and
> > >>> >> something
> > >>> >> >> >> like
> > >>> >> >> >> >> > "developer:carlos*" through the rest api and ui.
> > >>> >> >> >> >> >
> > >>> >> >> >> >> > Thanks
> > >>> >> >> >> >>
> > >>> >> >> >> >>
> > >>> >> >> >> >>
> > >>> >> >> >> >> --
> > >>> >> >> >> >> Olivier Lamy
> > >>> >> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> > >>> >> >> >> >>
> > >>> >> >> >>
> > >>> >> >> >>
> > >>> >> >> >>
> > >>> >> >> >> --
> > >>> >> >> >> Olivier Lamy
> > >>> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> > >>> >> >> >>
> > >>> >> >>
> > >>> >> >>
> > >>> >> >>
> > >>> >> >> --
> > >>> >> >> Olivier Lamy
> > >>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> > >>> >> >>
> > >>> >>
> > >>> >>
> > >>> >>
> > >>> >> --
> > >>> >> Olivier Lamy
> > >>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> > >>> >>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Olivier Lamy
> > >>> http://twitter.com/olamy | http://linkedin.com/in/olamy
> > >>>
> > >>
> > >>
> > >
> >
>
Re: Searching pom elements and generic metadata
Posted by Olivier Lamy <ol...@apache.org>.
Looks good.
Ship it! :-)
As you bump version to 2.2 there are some bulk updates to do in jira.
--
Olivier
On 25 Nov 2014 06:06, "Carlos Sanchez" <ca...@apache.org> wrote:
> PR submitted as https://github.com/apache/archiva/pull/14
>
> feedback welcome, there are some new methods there
>
> On Thu, Nov 20, 2014 at 11:09 AM, Carlos Sanchez <ca...@apache.org>
> wrote:
>
> > I have added these REST methods with JCR implementations
> >
> > artifactsByProjectVersionMetadata/{key}/{value}
> > artifactsByMetadata/{key}/{value}
> > artifactsByProperty/{key}/{value}
> > searchArtifacts/{text}
> > searchArtifacts/{key}/{text}
> >
> >
> > Docs here
> >
> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search
> >
> > Any suggestions on how to run these types of queries in Cassandra?
> >
> >
> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search#diff-9de24dcf873b85547a60cd92cc2f6379R764
> >
> >
> >
> > On Fri, Nov 7, 2014 at 11:19 PM, Carlos Sanchez <ca...@csanchez.org>
> > wrote:
> >
> >> I have added search methods for any property, including facets
> >>
> >> restServices/archivaServices/browseService/searchArtifacts/key/text
> >> restServices/archivaServices/browseService/searchArtifacts/text
> >>
> >> that search in JCR
> >>
> >> SELECT * FROM [archiva:projectVersion] AS projectVersion
> >> LEFT OUTER JOIN [archiva:artifact] AS artifact ON
> ISCHILDNODE(artifact,
> >> projectVersion)
> >> LEFT OUTER JOIN [archiva:facet] AS facet ON ISCHILDNODE(facet,
> >> projectVersion)
> >> WHERE (contains([projectVersion].*, $value) OR contains([facet].*,
> >> $value))
> >> AND ISDESCENDANTNODE(artifact,'/repositories/internal/content/')
> >>
> >> But seems that performance is bad for a "normal" sized repo. Any
> >> suggestions?
> >> I'll try with subselects and tuning jcr, I would imagine that it should
> >> work decently and there's something wrong with archiva.
> >>
> >>
> >>
> >> On Fri, Nov 7, 2014 at 11:17 AM, Olivier Lamy <ol...@apache.org> wrote:
> >>
> >>> On 7 November 2014 07:42, Carlos Sanchez <ca...@apache.org> wrote:
> >>> > I got both getArtifactsByMetadata and getArtifactsByProperty working,
> >>> > passing the repository
> >>> >
> >>> >
> >>>
> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
> >>> >
> >>> > The problem I hit with not passing the repo is that
> >>> > AbstractTestService.buildArtifacts needs it. Will look later.
> >>> >
> >>> > About adding the other pom fields to the search index, from what I
> saw
> >>> I
> >>> > need to add them to MinimalArtifactInfoIndexCreator (or a new one) in
> >>> the
> >>> > maven-indexer project, and then archiva would just use that?
> >>>
> >>> In such case you will have to modify the lucene model in the maven
> >>> indexer project ( can certainly breaks a lot :-) ).
> >>>
> >>> >
> >>> > Are the fields indexed already searchable somewhere in archiva with
> >>> field
> >>> > selection? e.g. g:myGroupId or v:1.0 ?
> >>>
> >>> Not associated at this level but that's already in our repository.
> >>>
> >>>
> >>> >
> >>> > Thanks
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <ol...@apache.org>
> >>> wrote:
> >>> >
> >>> >> Hi,
> >>> >> I would make repositoryId as a query param so will make it optional.
> >>> >> If empty/null use the users repositories?
> >>> >>
> >>> >> On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org> wrote:
> >>> >> > Thanks, got an initial implementation of getArtifactsByMetadata at
> >>> >> > https://github.com/carlossg/archiva/compare/metadata-search
> >>> >> >
> >>> >> > I added the repositoryId param, like other methods, but I'm not
> >>> sure if
> >>> >> > that would be limiting the feature, searching in all repos would
> be
> >>> more
> >>> >> > interesting.
> >>> >> >
> >>> >> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <ol...@apache.org>
> >>> wrote:
> >>> >> >
> >>> >> >> basic Cassandra requests tru Hector client won't work.
> >>> >> >> You will have to write some manual cql queries.
> >>> >> >>
> >>> >> >> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org>
> >>> wrote:
> >>> >> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <
> olamy@apache.org>
> >>> >> wrote:
> >>> >> >> >
> >>> >> >> >> I would add that in the interface MetadataRepository, add a
> >>> test in
> >>> >> >> >> the tck and implements it for all providers :-)
> >>> >> >> >> Maybe having 2 methods:
> >>> >> >> >> * generic metadata: getArtifactsByMetadata( key, value) (
> maybe
> >>> a Map
> >>> >> >> >> for multiple value pair? will you test the full value or a
> >>> matching?)
> >>> >> >> >>
> >>> >> >> >
> >>> >> >> > is the matching syntax for cassandra an jcr different?
> >>> >> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to do
> >>> >> matching
> >>> >> >> is
> >>> >> >> > it going to work only in jcr?
> >>> >> >> >
> >>> >> >> >
> >>> >> >> >
> >>> >> >> >> * pom contents: getArtifactsByProperty( key, value ) ( same
> >>> >> questions )
> >>> >> >> >>
> >>> >> >> >> To start I won't go with a Map and exact matching :-) (I
> reckon
> >>> pom
> >>> >> >> >> content can be a pain with Cassandra impl)
> >>> >> >> >>
> >>> >> >> >> WDYT?
> >>> >> >> >>
> >>> >> >> >> Olivier
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org>
> >>> wrote:
> >>> >> >> >> > Thanks for the pointers, so if I add a method to
> >>> >> JcrMetadataRepository
> >>> >> >> >> >
> >>> >> >> >> > getArtifactsByProperty(key, value)
> >>> >> >> >> >
> >>> >> >> >> > similar to getArtifactsByDateRange, writing the correct JCR
> >>> query,
> >>> >> I
> >>> >> >> >> could
> >>> >> >> >> > access all that metadata?
> >>> >> >> >> >
> >>> >> >> >> >
> >>> >> >> >> > In which of the rest services would it make sense to add
> such
> >>> a
> >>> >> >> search?
> >>> >> >> >> >
> >>> >> >> >> >
> >>> >> >> >> >
> >>> >> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <
> >>> olamy@apache.org>
> >>> >> >> wrote:
> >>> >> >> >> >
> >>> >> >> >> >> Hi,
> >>> >> >> >> >> The current SearchService search in maven indexer available
> >>> datas.
> >>> >> >> >> >> All the data you want to search are not available here but
> >>> in the
> >>> >> >> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
> >>> >> >> >> >> And currently there is no search methods for that!
> >>> >> >> >> >> So really good news to see you as a volunteer to implement
> >>> this!
> >>> >> :-)
> >>> >> >> >> >> Let me know if you need any pointers.
> >>> >> >> >> >>
> >>> >> >> >> >> Cheers
> >>> >> >> >> >> Olivier
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <
> carlos@apache.org>
> >>> >> wrote:
> >>> >> >> >> >> > Hi,
> >>> >> >> >> >> >
> >>> >> >> >> >> > Two things I wasn't able to do and was going to implement
> >>> >> (unless I
> >>> >> >> >> >> missed
> >>> >> >> >> >> > something):
> >>> >> >> >> >> >
> >>> >> >> >> >> > * search for generic metadata
> >>> >> >> >> >> > * search for pom contents
> >>> >> >> >> >> >
> >>> >> >> >> >> > The way the search queries are built in SearchService
> only
> >>> looks
> >>> >> >> for
> >>> >> >> >> the
> >>> >> >> >> >> > typical fields (artifactId, groupId,...), so that needs
> to
> >>> be
> >>> >> >> changed,
> >>> >> >> >> >> but
> >>> >> >> >> >> > I don't know if the pom and generic metadata fields are
> >>> even
> >>> >> >> indexed,
> >>> >> >> >> any
> >>> >> >> >> >> > pointers?
> >>> >> >> >> >> >
> >>> >> >> >> >> > Looking to be able to search for "myMetadata:myValue" and
> >>> >> something
> >>> >> >> >> like
> >>> >> >> >> >> > "developer:carlos*" through the rest api and ui.
> >>> >> >> >> >> >
> >>> >> >> >> >> > Thanks
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >>
> >>> >> >> >> >> --
> >>> >> >> >> >> Olivier Lamy
> >>> >> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>> >> >> >> >>
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >>
> >>> >> >> >> --
> >>> >> >> >> Olivier Lamy
> >>> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>> >> >> >>
> >>> >> >>
> >>> >> >>
> >>> >> >>
> >>> >> >> --
> >>> >> >> Olivier Lamy
> >>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>> >> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Olivier Lamy
> >>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>> >>
> >>>
> >>>
> >>>
> >>> --
> >>> Olivier Lamy
> >>> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>>
> >>
> >>
> >
>
Re: Searching pom elements and generic metadata
Posted by Carlos Sanchez <ca...@apache.org>.
PR submitted as https://github.com/apache/archiva/pull/14
feedback welcome, there are some new methods there
On Thu, Nov 20, 2014 at 11:09 AM, Carlos Sanchez <ca...@apache.org> wrote:
> I have added these REST methods with JCR implementations
>
> artifactsByProjectVersionMetadata/{key}/{value}
> artifactsByMetadata/{key}/{value}
> artifactsByProperty/{key}/{value}
> searchArtifacts/{text}
> searchArtifacts/{key}/{text}
>
>
> Docs here
> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search
>
> Any suggestions on how to run these types of queries in Cassandra?
>
> https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search#diff-9de24dcf873b85547a60cd92cc2f6379R764
>
>
>
> On Fri, Nov 7, 2014 at 11:19 PM, Carlos Sanchez <ca...@csanchez.org>
> wrote:
>
>> I have added search methods for any property, including facets
>>
>> restServices/archivaServices/browseService/searchArtifacts/key/text
>> restServices/archivaServices/browseService/searchArtifacts/text
>>
>> that search in JCR
>>
>> SELECT * FROM [archiva:projectVersion] AS projectVersion
>> LEFT OUTER JOIN [archiva:artifact] AS artifact ON ISCHILDNODE(artifact,
>> projectVersion)
>> LEFT OUTER JOIN [archiva:facet] AS facet ON ISCHILDNODE(facet,
>> projectVersion)
>> WHERE (contains([projectVersion].*, $value) OR contains([facet].*,
>> $value))
>> AND ISDESCENDANTNODE(artifact,'/repositories/internal/content/')
>>
>> But seems that performance is bad for a "normal" sized repo. Any
>> suggestions?
>> I'll try with subselects and tuning jcr, I would imagine that it should
>> work decently and there's something wrong with archiva.
>>
>>
>>
>> On Fri, Nov 7, 2014 at 11:17 AM, Olivier Lamy <ol...@apache.org> wrote:
>>
>>> On 7 November 2014 07:42, Carlos Sanchez <ca...@apache.org> wrote:
>>> > I got both getArtifactsByMetadata and getArtifactsByProperty working,
>>> > passing the repository
>>> >
>>> >
>>> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
>>> >
>>> > The problem I hit with not passing the repo is that
>>> > AbstractTestService.buildArtifacts needs it. Will look later.
>>> >
>>> > About adding the other pom fields to the search index, from what I saw
>>> I
>>> > need to add them to MinimalArtifactInfoIndexCreator (or a new one) in
>>> the
>>> > maven-indexer project, and then archiva would just use that?
>>>
>>> In such case you will have to modify the lucene model in the maven
>>> indexer project ( can certainly breaks a lot :-) ).
>>>
>>> >
>>> > Are the fields indexed already searchable somewhere in archiva with
>>> field
>>> > selection? e.g. g:myGroupId or v:1.0 ?
>>>
>>> Not associated at this level but that's already in our repository.
>>>
>>>
>>> >
>>> > Thanks
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <ol...@apache.org>
>>> wrote:
>>> >
>>> >> Hi,
>>> >> I would make repositoryId as a query param so will make it optional.
>>> >> If empty/null use the users repositories?
>>> >>
>>> >> On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org> wrote:
>>> >> > Thanks, got an initial implementation of getArtifactsByMetadata at
>>> >> > https://github.com/carlossg/archiva/compare/metadata-search
>>> >> >
>>> >> > I added the repositoryId param, like other methods, but I'm not
>>> sure if
>>> >> > that would be limiting the feature, searching in all repos would be
>>> more
>>> >> > interesting.
>>> >> >
>>> >> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <ol...@apache.org>
>>> wrote:
>>> >> >
>>> >> >> basic Cassandra requests tru Hector client won't work.
>>> >> >> You will have to write some manual cql queries.
>>> >> >>
>>> >> >> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org>
>>> wrote:
>>> >> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org>
>>> >> wrote:
>>> >> >> >
>>> >> >> >> I would add that in the interface MetadataRepository, add a
>>> test in
>>> >> >> >> the tck and implements it for all providers :-)
>>> >> >> >> Maybe having 2 methods:
>>> >> >> >> * generic metadata: getArtifactsByMetadata( key, value) ( maybe
>>> a Map
>>> >> >> >> for multiple value pair? will you test the full value or a
>>> matching?)
>>> >> >> >>
>>> >> >> >
>>> >> >> > is the matching syntax for cassandra an jcr different?
>>> >> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to do
>>> >> matching
>>> >> >> is
>>> >> >> > it going to work only in jcr?
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >> * pom contents: getArtifactsByProperty( key, value ) ( same
>>> >> questions )
>>> >> >> >>
>>> >> >> >> To start I won't go with a Map and exact matching :-) (I reckon
>>> pom
>>> >> >> >> content can be a pain with Cassandra impl)
>>> >> >> >>
>>> >> >> >> WDYT?
>>> >> >> >>
>>> >> >> >> Olivier
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org>
>>> wrote:
>>> >> >> >> > Thanks for the pointers, so if I add a method to
>>> >> JcrMetadataRepository
>>> >> >> >> >
>>> >> >> >> > getArtifactsByProperty(key, value)
>>> >> >> >> >
>>> >> >> >> > similar to getArtifactsByDateRange, writing the correct JCR
>>> query,
>>> >> I
>>> >> >> >> could
>>> >> >> >> > access all that metadata?
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > In which of the rest services would it make sense to add such
>>> a
>>> >> >> search?
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <
>>> olamy@apache.org>
>>> >> >> wrote:
>>> >> >> >> >
>>> >> >> >> >> Hi,
>>> >> >> >> >> The current SearchService search in maven indexer available
>>> datas.
>>> >> >> >> >> All the data you want to search are not available here but
>>> in the
>>> >> >> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
>>> >> >> >> >> And currently there is no search methods for that!
>>> >> >> >> >> So really good news to see you as a volunteer to implement
>>> this!
>>> >> :-)
>>> >> >> >> >> Let me know if you need any pointers.
>>> >> >> >> >>
>>> >> >> >> >> Cheers
>>> >> >> >> >> Olivier
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org>
>>> >> wrote:
>>> >> >> >> >> > Hi,
>>> >> >> >> >> >
>>> >> >> >> >> > Two things I wasn't able to do and was going to implement
>>> >> (unless I
>>> >> >> >> >> missed
>>> >> >> >> >> > something):
>>> >> >> >> >> >
>>> >> >> >> >> > * search for generic metadata
>>> >> >> >> >> > * search for pom contents
>>> >> >> >> >> >
>>> >> >> >> >> > The way the search queries are built in SearchService only
>>> looks
>>> >> >> for
>>> >> >> >> the
>>> >> >> >> >> > typical fields (artifactId, groupId,...), so that needs to
>>> be
>>> >> >> changed,
>>> >> >> >> >> but
>>> >> >> >> >> > I don't know if the pom and generic metadata fields are
>>> even
>>> >> >> indexed,
>>> >> >> >> any
>>> >> >> >> >> > pointers?
>>> >> >> >> >> >
>>> >> >> >> >> > Looking to be able to search for "myMetadata:myValue" and
>>> >> something
>>> >> >> >> like
>>> >> >> >> >> > "developer:carlos*" through the rest api and ui.
>>> >> >> >> >> >
>>> >> >> >> >> > Thanks
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >>
>>> >> >> >> >> --
>>> >> >> >> >> Olivier Lamy
>>> >> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> --
>>> >> >> >> Olivier Lamy
>>> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >> >> >>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> Olivier Lamy
>>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Olivier Lamy
>>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>> >>
>>>
>>>
>>>
>>> --
>>> Olivier Lamy
>>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>>
>>
>>
>
Re: Searching pom elements and generic metadata
Posted by Carlos Sanchez <ca...@apache.org>.
I have added these REST methods with JCR implementations
artifactsByProjectVersionMetadata/{key}/{value}
artifactsByMetadata/{key}/{value}
artifactsByProperty/{key}/{value}
searchArtifacts/{text}
searchArtifacts/{key}/{text}
Docs here
https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search
Any suggestions on how to run these types of queries in Cassandra?
https://github.com/carlossg/archiva/compare/carlossg:master...metadata-search#diff-9de24dcf873b85547a60cd92cc2f6379R764
On Fri, Nov 7, 2014 at 11:19 PM, Carlos Sanchez <ca...@csanchez.org> wrote:
> I have added search methods for any property, including facets
>
> restServices/archivaServices/browseService/searchArtifacts/key/text
> restServices/archivaServices/browseService/searchArtifacts/text
>
> that search in JCR
>
> SELECT * FROM [archiva:projectVersion] AS projectVersion
> LEFT OUTER JOIN [archiva:artifact] AS artifact ON ISCHILDNODE(artifact,
> projectVersion)
> LEFT OUTER JOIN [archiva:facet] AS facet ON ISCHILDNODE(facet,
> projectVersion)
> WHERE (contains([projectVersion].*, $value) OR contains([facet].*,
> $value))
> AND ISDESCENDANTNODE(artifact,'/repositories/internal/content/')
>
> But seems that performance is bad for a "normal" sized repo. Any
> suggestions?
> I'll try with subselects and tuning jcr, I would imagine that it should
> work decently and there's something wrong with archiva.
>
>
>
> On Fri, Nov 7, 2014 at 11:17 AM, Olivier Lamy <ol...@apache.org> wrote:
>
>> On 7 November 2014 07:42, Carlos Sanchez <ca...@apache.org> wrote:
>> > I got both getArtifactsByMetadata and getArtifactsByProperty working,
>> > passing the repository
>> >
>> >
>> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
>> >
>> > The problem I hit with not passing the repo is that
>> > AbstractTestService.buildArtifacts needs it. Will look later.
>> >
>> > About adding the other pom fields to the search index, from what I saw I
>> > need to add them to MinimalArtifactInfoIndexCreator (or a new one) in
>> the
>> > maven-indexer project, and then archiva would just use that?
>>
>> In such case you will have to modify the lucene model in the maven
>> indexer project ( can certainly breaks a lot :-) ).
>>
>> >
>> > Are the fields indexed already searchable somewhere in archiva with
>> field
>> > selection? e.g. g:myGroupId or v:1.0 ?
>>
>> Not associated at this level but that's already in our repository.
>>
>>
>> >
>> > Thanks
>> >
>> >
>> >
>> >
>> > On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <ol...@apache.org> wrote:
>> >
>> >> Hi,
>> >> I would make repositoryId as a query param so will make it optional.
>> >> If empty/null use the users repositories?
>> >>
>> >> On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org> wrote:
>> >> > Thanks, got an initial implementation of getArtifactsByMetadata at
>> >> > https://github.com/carlossg/archiva/compare/metadata-search
>> >> >
>> >> > I added the repositoryId param, like other methods, but I'm not sure
>> if
>> >> > that would be limiting the feature, searching in all repos would be
>> more
>> >> > interesting.
>> >> >
>> >> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <ol...@apache.org>
>> wrote:
>> >> >
>> >> >> basic Cassandra requests tru Hector client won't work.
>> >> >> You will have to write some manual cql queries.
>> >> >>
>> >> >> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org>
>> wrote:
>> >> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org>
>> >> wrote:
>> >> >> >
>> >> >> >> I would add that in the interface MetadataRepository, add a test
>> in
>> >> >> >> the tck and implements it for all providers :-)
>> >> >> >> Maybe having 2 methods:
>> >> >> >> * generic metadata: getArtifactsByMetadata( key, value) ( maybe
>> a Map
>> >> >> >> for multiple value pair? will you test the full value or a
>> matching?)
>> >> >> >>
>> >> >> >
>> >> >> > is the matching syntax for cassandra an jcr different?
>> >> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to do
>> >> matching
>> >> >> is
>> >> >> > it going to work only in jcr?
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >> * pom contents: getArtifactsByProperty( key, value ) ( same
>> >> questions )
>> >> >> >>
>> >> >> >> To start I won't go with a Map and exact matching :-) (I reckon
>> pom
>> >> >> >> content can be a pain with Cassandra impl)
>> >> >> >>
>> >> >> >> WDYT?
>> >> >> >>
>> >> >> >> Olivier
>> >> >> >>
>> >> >> >>
>> >> >> >> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org>
>> wrote:
>> >> >> >> > Thanks for the pointers, so if I add a method to
>> >> JcrMetadataRepository
>> >> >> >> >
>> >> >> >> > getArtifactsByProperty(key, value)
>> >> >> >> >
>> >> >> >> > similar to getArtifactsByDateRange, writing the correct JCR
>> query,
>> >> I
>> >> >> >> could
>> >> >> >> > access all that metadata?
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > In which of the rest services would it make sense to add such a
>> >> >> search?
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <
>> olamy@apache.org>
>> >> >> wrote:
>> >> >> >> >
>> >> >> >> >> Hi,
>> >> >> >> >> The current SearchService search in maven indexer available
>> datas.
>> >> >> >> >> All the data you want to search are not available here but in
>> the
>> >> >> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
>> >> >> >> >> And currently there is no search methods for that!
>> >> >> >> >> So really good news to see you as a volunteer to implement
>> this!
>> >> :-)
>> >> >> >> >> Let me know if you need any pointers.
>> >> >> >> >>
>> >> >> >> >> Cheers
>> >> >> >> >> Olivier
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org>
>> >> wrote:
>> >> >> >> >> > Hi,
>> >> >> >> >> >
>> >> >> >> >> > Two things I wasn't able to do and was going to implement
>> >> (unless I
>> >> >> >> >> missed
>> >> >> >> >> > something):
>> >> >> >> >> >
>> >> >> >> >> > * search for generic metadata
>> >> >> >> >> > * search for pom contents
>> >> >> >> >> >
>> >> >> >> >> > The way the search queries are built in SearchService only
>> looks
>> >> >> for
>> >> >> >> the
>> >> >> >> >> > typical fields (artifactId, groupId,...), so that needs to
>> be
>> >> >> changed,
>> >> >> >> >> but
>> >> >> >> >> > I don't know if the pom and generic metadata fields are even
>> >> >> indexed,
>> >> >> >> any
>> >> >> >> >> > pointers?
>> >> >> >> >> >
>> >> >> >> >> > Looking to be able to search for "myMetadata:myValue" and
>> >> something
>> >> >> >> like
>> >> >> >> >> > "developer:carlos*" through the rest api and ui.
>> >> >> >> >> >
>> >> >> >> >> > Thanks
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> --
>> >> >> >> >> Olivier Lamy
>> >> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Olivier Lamy
>> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Olivier Lamy
>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Olivier Lamy
>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >>
>>
>>
>>
>> --
>> Olivier Lamy
>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
>
>
Re: Searching pom elements and generic metadata
Posted by Carlos Sanchez <ca...@csanchez.org>.
I have added search methods for any property, including facets
restServices/archivaServices/browseService/searchArtifacts/key/text
restServices/archivaServices/browseService/searchArtifacts/text
that search in JCR
SELECT * FROM [archiva:projectVersion] AS projectVersion
LEFT OUTER JOIN [archiva:artifact] AS artifact ON ISCHILDNODE(artifact,
projectVersion)
LEFT OUTER JOIN [archiva:facet] AS facet ON ISCHILDNODE(facet,
projectVersion)
WHERE (contains([projectVersion].*, $value) OR contains([facet].*,
$value))
AND ISDESCENDANTNODE(artifact,'/repositories/internal/content/')
But seems that performance is bad for a "normal" sized repo. Any
suggestions?
I'll try with subselects and tuning jcr, I would imagine that it should
work decently and there's something wrong with archiva.
On Fri, Nov 7, 2014 at 11:17 AM, Olivier Lamy <ol...@apache.org> wrote:
> On 7 November 2014 07:42, Carlos Sanchez <ca...@apache.org> wrote:
> > I got both getArtifactsByMetadata and getArtifactsByProperty working,
> > passing the repository
> >
> >
> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
> >
> > The problem I hit with not passing the repo is that
> > AbstractTestService.buildArtifacts needs it. Will look later.
> >
> > About adding the other pom fields to the search index, from what I saw I
> > need to add them to MinimalArtifactInfoIndexCreator (or a new one) in the
> > maven-indexer project, and then archiva would just use that?
>
> In such case you will have to modify the lucene model in the maven
> indexer project ( can certainly breaks a lot :-) ).
>
> >
> > Are the fields indexed already searchable somewhere in archiva with field
> > selection? e.g. g:myGroupId or v:1.0 ?
>
> Not associated at this level but that's already in our repository.
>
>
> >
> > Thanks
> >
> >
> >
> >
> > On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <ol...@apache.org> wrote:
> >
> >> Hi,
> >> I would make repositoryId as a query param so will make it optional.
> >> If empty/null use the users repositories?
> >>
> >> On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org> wrote:
> >> > Thanks, got an initial implementation of getArtifactsByMetadata at
> >> > https://github.com/carlossg/archiva/compare/metadata-search
> >> >
> >> > I added the repositoryId param, like other methods, but I'm not sure
> if
> >> > that would be limiting the feature, searching in all repos would be
> more
> >> > interesting.
> >> >
> >> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <ol...@apache.org>
> wrote:
> >> >
> >> >> basic Cassandra requests tru Hector client won't work.
> >> >> You will have to write some manual cql queries.
> >> >>
> >> >> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org>
> wrote:
> >> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org>
> >> wrote:
> >> >> >
> >> >> >> I would add that in the interface MetadataRepository, add a test
> in
> >> >> >> the tck and implements it for all providers :-)
> >> >> >> Maybe having 2 methods:
> >> >> >> * generic metadata: getArtifactsByMetadata( key, value) ( maybe a
> Map
> >> >> >> for multiple value pair? will you test the full value or a
> matching?)
> >> >> >>
> >> >> >
> >> >> > is the matching syntax for cassandra an jcr different?
> >> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to do
> >> matching
> >> >> is
> >> >> > it going to work only in jcr?
> >> >> >
> >> >> >
> >> >> >
> >> >> >> * pom contents: getArtifactsByProperty( key, value ) ( same
> >> questions )
> >> >> >>
> >> >> >> To start I won't go with a Map and exact matching :-) (I reckon
> pom
> >> >> >> content can be a pain with Cassandra impl)
> >> >> >>
> >> >> >> WDYT?
> >> >> >>
> >> >> >> Olivier
> >> >> >>
> >> >> >>
> >> >> >> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org>
> wrote:
> >> >> >> > Thanks for the pointers, so if I add a method to
> >> JcrMetadataRepository
> >> >> >> >
> >> >> >> > getArtifactsByProperty(key, value)
> >> >> >> >
> >> >> >> > similar to getArtifactsByDateRange, writing the correct JCR
> query,
> >> I
> >> >> >> could
> >> >> >> > access all that metadata?
> >> >> >> >
> >> >> >> >
> >> >> >> > In which of the rest services would it make sense to add such a
> >> >> search?
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <olamy@apache.org
> >
> >> >> wrote:
> >> >> >> >
> >> >> >> >> Hi,
> >> >> >> >> The current SearchService search in maven indexer available
> datas.
> >> >> >> >> All the data you want to search are not available here but in
> the
> >> >> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
> >> >> >> >> And currently there is no search methods for that!
> >> >> >> >> So really good news to see you as a volunteer to implement
> this!
> >> :-)
> >> >> >> >> Let me know if you need any pointers.
> >> >> >> >>
> >> >> >> >> Cheers
> >> >> >> >> Olivier
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org>
> >> wrote:
> >> >> >> >> > Hi,
> >> >> >> >> >
> >> >> >> >> > Two things I wasn't able to do and was going to implement
> >> (unless I
> >> >> >> >> missed
> >> >> >> >> > something):
> >> >> >> >> >
> >> >> >> >> > * search for generic metadata
> >> >> >> >> > * search for pom contents
> >> >> >> >> >
> >> >> >> >> > The way the search queries are built in SearchService only
> looks
> >> >> for
> >> >> >> the
> >> >> >> >> > typical fields (artifactId, groupId,...), so that needs to be
> >> >> changed,
> >> >> >> >> but
> >> >> >> >> > I don't know if the pom and generic metadata fields are even
> >> >> indexed,
> >> >> >> any
> >> >> >> >> > pointers?
> >> >> >> >> >
> >> >> >> >> > Looking to be able to search for "myMetadata:myValue" and
> >> something
> >> >> >> like
> >> >> >> >> > "developer:carlos*" through the rest api and ui.
> >> >> >> >> >
> >> >> >> >> > Thanks
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> Olivier Lamy
> >> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Olivier Lamy
> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Olivier Lamy
> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >>
> >>
> >>
> >>
> >> --
> >> Olivier Lamy
> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>
>
>
>
> --
> Olivier Lamy
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
Re: Searching pom elements and generic metadata
Posted by Olivier Lamy <ol...@apache.org>.
On 7 November 2014 07:42, Carlos Sanchez <ca...@apache.org> wrote:
> I got both getArtifactsByMetadata and getArtifactsByProperty working,
> passing the repository
>
> https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
>
> The problem I hit with not passing the repo is that
> AbstractTestService.buildArtifacts needs it. Will look later.
>
> About adding the other pom fields to the search index, from what I saw I
> need to add them to MinimalArtifactInfoIndexCreator (or a new one) in the
> maven-indexer project, and then archiva would just use that?
In such case you will have to modify the lucene model in the maven
indexer project ( can certainly breaks a lot :-) ).
>
> Are the fields indexed already searchable somewhere in archiva with field
> selection? e.g. g:myGroupId or v:1.0 ?
Not associated at this level but that's already in our repository.
>
> Thanks
>
>
>
>
> On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <ol...@apache.org> wrote:
>
>> Hi,
>> I would make repositoryId as a query param so will make it optional.
>> If empty/null use the users repositories?
>>
>> On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org> wrote:
>> > Thanks, got an initial implementation of getArtifactsByMetadata at
>> > https://github.com/carlossg/archiva/compare/metadata-search
>> >
>> > I added the repositoryId param, like other methods, but I'm not sure if
>> > that would be limiting the feature, searching in all repos would be more
>> > interesting.
>> >
>> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <ol...@apache.org> wrote:
>> >
>> >> basic Cassandra requests tru Hector client won't work.
>> >> You will have to write some manual cql queries.
>> >>
>> >> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org> wrote:
>> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org>
>> wrote:
>> >> >
>> >> >> I would add that in the interface MetadataRepository, add a test in
>> >> >> the tck and implements it for all providers :-)
>> >> >> Maybe having 2 methods:
>> >> >> * generic metadata: getArtifactsByMetadata( key, value) ( maybe a Map
>> >> >> for multiple value pair? will you test the full value or a matching?)
>> >> >>
>> >> >
>> >> > is the matching syntax for cassandra an jcr different?
>> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to do
>> matching
>> >> is
>> >> > it going to work only in jcr?
>> >> >
>> >> >
>> >> >
>> >> >> * pom contents: getArtifactsByProperty( key, value ) ( same
>> questions )
>> >> >>
>> >> >> To start I won't go with a Map and exact matching :-) (I reckon pom
>> >> >> content can be a pain with Cassandra impl)
>> >> >>
>> >> >> WDYT?
>> >> >>
>> >> >> Olivier
>> >> >>
>> >> >>
>> >> >> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org> wrote:
>> >> >> > Thanks for the pointers, so if I add a method to
>> JcrMetadataRepository
>> >> >> >
>> >> >> > getArtifactsByProperty(key, value)
>> >> >> >
>> >> >> > similar to getArtifactsByDateRange, writing the correct JCR query,
>> I
>> >> >> could
>> >> >> > access all that metadata?
>> >> >> >
>> >> >> >
>> >> >> > In which of the rest services would it make sense to add such a
>> >> search?
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <ol...@apache.org>
>> >> wrote:
>> >> >> >
>> >> >> >> Hi,
>> >> >> >> The current SearchService search in maven indexer available datas.
>> >> >> >> All the data you want to search are not available here but in the
>> >> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
>> >> >> >> And currently there is no search methods for that!
>> >> >> >> So really good news to see you as a volunteer to implement this!
>> :-)
>> >> >> >> Let me know if you need any pointers.
>> >> >> >>
>> >> >> >> Cheers
>> >> >> >> Olivier
>> >> >> >>
>> >> >> >>
>> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org>
>> wrote:
>> >> >> >> > Hi,
>> >> >> >> >
>> >> >> >> > Two things I wasn't able to do and was going to implement
>> (unless I
>> >> >> >> missed
>> >> >> >> > something):
>> >> >> >> >
>> >> >> >> > * search for generic metadata
>> >> >> >> > * search for pom contents
>> >> >> >> >
>> >> >> >> > The way the search queries are built in SearchService only looks
>> >> for
>> >> >> the
>> >> >> >> > typical fields (artifactId, groupId,...), so that needs to be
>> >> changed,
>> >> >> >> but
>> >> >> >> > I don't know if the pom and generic metadata fields are even
>> >> indexed,
>> >> >> any
>> >> >> >> > pointers?
>> >> >> >> >
>> >> >> >> > Looking to be able to search for "myMetadata:myValue" and
>> something
>> >> >> like
>> >> >> >> > "developer:carlos*" through the rest api and ui.
>> >> >> >> >
>> >> >> >> > Thanks
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> --
>> >> >> >> Olivier Lamy
>> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Olivier Lamy
>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Olivier Lamy
>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >>
>>
>>
>>
>> --
>> Olivier Lamy
>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
--
Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Searching pom elements and generic metadata
Posted by Carlos Sanchez <ca...@apache.org>.
I got both getArtifactsByMetadata and getArtifactsByProperty working,
passing the repository
https://github.com/carlossg/archiva/compare/carlossg:abstract-metadata...metadata-search
The problem I hit with not passing the repo is that
AbstractTestService.buildArtifacts needs it. Will look later.
About adding the other pom fields to the search index, from what I saw I
need to add them to MinimalArtifactInfoIndexCreator (or a new one) in the
maven-indexer project, and then archiva would just use that?
Are the fields indexed already searchable somewhere in archiva with field
selection? e.g. g:myGroupId or v:1.0 ?
Thanks
On Wed, Nov 5, 2014 at 11:42 PM, Olivier Lamy <ol...@apache.org> wrote:
> Hi,
> I would make repositoryId as a query param so will make it optional.
> If empty/null use the users repositories?
>
> On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org> wrote:
> > Thanks, got an initial implementation of getArtifactsByMetadata at
> > https://github.com/carlossg/archiva/compare/metadata-search
> >
> > I added the repositoryId param, like other methods, but I'm not sure if
> > that would be limiting the feature, searching in all repos would be more
> > interesting.
> >
> > On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <ol...@apache.org> wrote:
> >
> >> basic Cassandra requests tru Hector client won't work.
> >> You will have to write some manual cql queries.
> >>
> >> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org> wrote:
> >> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org>
> wrote:
> >> >
> >> >> I would add that in the interface MetadataRepository, add a test in
> >> >> the tck and implements it for all providers :-)
> >> >> Maybe having 2 methods:
> >> >> * generic metadata: getArtifactsByMetadata( key, value) ( maybe a Map
> >> >> for multiple value pair? will you test the full value or a matching?)
> >> >>
> >> >
> >> > is the matching syntax for cassandra an jcr different?
> >> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to do
> matching
> >> is
> >> > it going to work only in jcr?
> >> >
> >> >
> >> >
> >> >> * pom contents: getArtifactsByProperty( key, value ) ( same
> questions )
> >> >>
> >> >> To start I won't go with a Map and exact matching :-) (I reckon pom
> >> >> content can be a pain with Cassandra impl)
> >> >>
> >> >> WDYT?
> >> >>
> >> >> Olivier
> >> >>
> >> >>
> >> >> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org> wrote:
> >> >> > Thanks for the pointers, so if I add a method to
> JcrMetadataRepository
> >> >> >
> >> >> > getArtifactsByProperty(key, value)
> >> >> >
> >> >> > similar to getArtifactsByDateRange, writing the correct JCR query,
> I
> >> >> could
> >> >> > access all that metadata?
> >> >> >
> >> >> >
> >> >> > In which of the rest services would it make sense to add such a
> >> search?
> >> >> >
> >> >> >
> >> >> >
> >> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <ol...@apache.org>
> >> wrote:
> >> >> >
> >> >> >> Hi,
> >> >> >> The current SearchService search in maven indexer available datas.
> >> >> >> All the data you want to search are not available here but in the
> >> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
> >> >> >> And currently there is no search methods for that!
> >> >> >> So really good news to see you as a volunteer to implement this!
> :-)
> >> >> >> Let me know if you need any pointers.
> >> >> >>
> >> >> >> Cheers
> >> >> >> Olivier
> >> >> >>
> >> >> >>
> >> >> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org>
> wrote:
> >> >> >> > Hi,
> >> >> >> >
> >> >> >> > Two things I wasn't able to do and was going to implement
> (unless I
> >> >> >> missed
> >> >> >> > something):
> >> >> >> >
> >> >> >> > * search for generic metadata
> >> >> >> > * search for pom contents
> >> >> >> >
> >> >> >> > The way the search queries are built in SearchService only looks
> >> for
> >> >> the
> >> >> >> > typical fields (artifactId, groupId,...), so that needs to be
> >> changed,
> >> >> >> but
> >> >> >> > I don't know if the pom and generic metadata fields are even
> >> indexed,
> >> >> any
> >> >> >> > pointers?
> >> >> >> >
> >> >> >> > Looking to be able to search for "myMetadata:myValue" and
> something
> >> >> like
> >> >> >> > "developer:carlos*" through the rest api and ui.
> >> >> >> >
> >> >> >> > Thanks
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Olivier Lamy
> >> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Olivier Lamy
> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >>
> >>
> >>
> >>
> >> --
> >> Olivier Lamy
> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>
>
>
>
> --
> Olivier Lamy
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
Re: Searching pom elements and generic metadata
Posted by Olivier Lamy <ol...@apache.org>.
Hi,
I would make repositoryId as a query param so will make it optional.
If empty/null use the users repositories?
On 6 November 2014 06:22, Carlos Sanchez <ca...@apache.org> wrote:
> Thanks, got an initial implementation of getArtifactsByMetadata at
> https://github.com/carlossg/archiva/compare/metadata-search
>
> I added the repositoryId param, like other methods, but I'm not sure if
> that would be limiting the feature, searching in all repos would be more
> interesting.
>
> On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <ol...@apache.org> wrote:
>
>> basic Cassandra requests tru Hector client won't work.
>> You will have to write some manual cql queries.
>>
>> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org> wrote:
>> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org> wrote:
>> >
>> >> I would add that in the interface MetadataRepository, add a test in
>> >> the tck and implements it for all providers :-)
>> >> Maybe having 2 methods:
>> >> * generic metadata: getArtifactsByMetadata( key, value) ( maybe a Map
>> >> for multiple value pair? will you test the full value or a matching?)
>> >>
>> >
>> > is the matching syntax for cassandra an jcr different?
>> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to do matching
>> is
>> > it going to work only in jcr?
>> >
>> >
>> >
>> >> * pom contents: getArtifactsByProperty( key, value ) ( same questions )
>> >>
>> >> To start I won't go with a Map and exact matching :-) (I reckon pom
>> >> content can be a pain with Cassandra impl)
>> >>
>> >> WDYT?
>> >>
>> >> Olivier
>> >>
>> >>
>> >> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org> wrote:
>> >> > Thanks for the pointers, so if I add a method to JcrMetadataRepository
>> >> >
>> >> > getArtifactsByProperty(key, value)
>> >> >
>> >> > similar to getArtifactsByDateRange, writing the correct JCR query, I
>> >> could
>> >> > access all that metadata?
>> >> >
>> >> >
>> >> > In which of the rest services would it make sense to add such a
>> search?
>> >> >
>> >> >
>> >> >
>> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <ol...@apache.org>
>> wrote:
>> >> >
>> >> >> Hi,
>> >> >> The current SearchService search in maven indexer available datas.
>> >> >> All the data you want to search are not available here but in the
>> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
>> >> >> And currently there is no search methods for that!
>> >> >> So really good news to see you as a volunteer to implement this! :-)
>> >> >> Let me know if you need any pointers.
>> >> >>
>> >> >> Cheers
>> >> >> Olivier
>> >> >>
>> >> >>
>> >> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org> wrote:
>> >> >> > Hi,
>> >> >> >
>> >> >> > Two things I wasn't able to do and was going to implement (unless I
>> >> >> missed
>> >> >> > something):
>> >> >> >
>> >> >> > * search for generic metadata
>> >> >> > * search for pom contents
>> >> >> >
>> >> >> > The way the search queries are built in SearchService only looks
>> for
>> >> the
>> >> >> > typical fields (artifactId, groupId,...), so that needs to be
>> changed,
>> >> >> but
>> >> >> > I don't know if the pom and generic metadata fields are even
>> indexed,
>> >> any
>> >> >> > pointers?
>> >> >> >
>> >> >> > Looking to be able to search for "myMetadata:myValue" and something
>> >> like
>> >> >> > "developer:carlos*" through the rest api and ui.
>> >> >> >
>> >> >> > Thanks
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Olivier Lamy
>> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Olivier Lamy
>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >>
>>
>>
>>
>> --
>> Olivier Lamy
>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
--
Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Searching pom elements and generic metadata
Posted by Carlos Sanchez <ca...@apache.org>.
Thanks, got an initial implementation of getArtifactsByMetadata at
https://github.com/carlossg/archiva/compare/metadata-search
I added the repositoryId param, like other methods, but I'm not sure if
that would be limiting the feature, searching in all repos would be more
interesting.
On Wed, Nov 5, 2014 at 12:28 PM, Olivier Lamy <ol...@apache.org> wrote:
> basic Cassandra requests tru Hector client won't work.
> You will have to write some manual cql queries.
>
> On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org> wrote:
> > On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org> wrote:
> >
> >> I would add that in the interface MetadataRepository, add a test in
> >> the tck and implements it for all providers :-)
> >> Maybe having 2 methods:
> >> * generic metadata: getArtifactsByMetadata( key, value) ( maybe a Map
> >> for multiple value pair? will you test the full value or a matching?)
> >>
> >
> > is the matching syntax for cassandra an jcr different?
> > I mean if I pass ('foo', 'bar*') -or some sort of regex- to do matching
> is
> > it going to work only in jcr?
> >
> >
> >
> >> * pom contents: getArtifactsByProperty( key, value ) ( same questions )
> >>
> >> To start I won't go with a Map and exact matching :-) (I reckon pom
> >> content can be a pain with Cassandra impl)
> >>
> >> WDYT?
> >>
> >> Olivier
> >>
> >>
> >> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org> wrote:
> >> > Thanks for the pointers, so if I add a method to JcrMetadataRepository
> >> >
> >> > getArtifactsByProperty(key, value)
> >> >
> >> > similar to getArtifactsByDateRange, writing the correct JCR query, I
> >> could
> >> > access all that metadata?
> >> >
> >> >
> >> > In which of the rest services would it make sense to add such a
> search?
> >> >
> >> >
> >> >
> >> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <ol...@apache.org>
> wrote:
> >> >
> >> >> Hi,
> >> >> The current SearchService search in maven indexer available datas.
> >> >> All the data you want to search are not available here but in the
> >> >> MetadataRepository ( so jcr, file or the new Cassandra).
> >> >> And currently there is no search methods for that!
> >> >> So really good news to see you as a volunteer to implement this! :-)
> >> >> Let me know if you need any pointers.
> >> >>
> >> >> Cheers
> >> >> Olivier
> >> >>
> >> >>
> >> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > Two things I wasn't able to do and was going to implement (unless I
> >> >> missed
> >> >> > something):
> >> >> >
> >> >> > * search for generic metadata
> >> >> > * search for pom contents
> >> >> >
> >> >> > The way the search queries are built in SearchService only looks
> for
> >> the
> >> >> > typical fields (artifactId, groupId,...), so that needs to be
> changed,
> >> >> but
> >> >> > I don't know if the pom and generic metadata fields are even
> indexed,
> >> any
> >> >> > pointers?
> >> >> >
> >> >> > Looking to be able to search for "myMetadata:myValue" and something
> >> like
> >> >> > "developer:carlos*" through the rest api and ui.
> >> >> >
> >> >> > Thanks
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Olivier Lamy
> >> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >> >>
> >>
> >>
> >>
> >> --
> >> Olivier Lamy
> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>
>
>
>
> --
> Olivier Lamy
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
Re: Searching pom elements and generic metadata
Posted by Olivier Lamy <ol...@apache.org>.
basic Cassandra requests tru Hector client won't work.
You will have to write some manual cql queries.
On 5 November 2014 22:07, Carlos Sanchez <ca...@csanchez.org> wrote:
> On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org> wrote:
>
>> I would add that in the interface MetadataRepository, add a test in
>> the tck and implements it for all providers :-)
>> Maybe having 2 methods:
>> * generic metadata: getArtifactsByMetadata( key, value) ( maybe a Map
>> for multiple value pair? will you test the full value or a matching?)
>>
>
> is the matching syntax for cassandra an jcr different?
> I mean if I pass ('foo', 'bar*') -or some sort of regex- to do matching is
> it going to work only in jcr?
>
>
>
>> * pom contents: getArtifactsByProperty( key, value ) ( same questions )
>>
>> To start I won't go with a Map and exact matching :-) (I reckon pom
>> content can be a pain with Cassandra impl)
>>
>> WDYT?
>>
>> Olivier
>>
>>
>> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org> wrote:
>> > Thanks for the pointers, so if I add a method to JcrMetadataRepository
>> >
>> > getArtifactsByProperty(key, value)
>> >
>> > similar to getArtifactsByDateRange, writing the correct JCR query, I
>> could
>> > access all that metadata?
>> >
>> >
>> > In which of the rest services would it make sense to add such a search?
>> >
>> >
>> >
>> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <ol...@apache.org> wrote:
>> >
>> >> Hi,
>> >> The current SearchService search in maven indexer available datas.
>> >> All the data you want to search are not available here but in the
>> >> MetadataRepository ( so jcr, file or the new Cassandra).
>> >> And currently there is no search methods for that!
>> >> So really good news to see you as a volunteer to implement this! :-)
>> >> Let me know if you need any pointers.
>> >>
>> >> Cheers
>> >> Olivier
>> >>
>> >>
>> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org> wrote:
>> >> > Hi,
>> >> >
>> >> > Two things I wasn't able to do and was going to implement (unless I
>> >> missed
>> >> > something):
>> >> >
>> >> > * search for generic metadata
>> >> > * search for pom contents
>> >> >
>> >> > The way the search queries are built in SearchService only looks for
>> the
>> >> > typical fields (artifactId, groupId,...), so that needs to be changed,
>> >> but
>> >> > I don't know if the pom and generic metadata fields are even indexed,
>> any
>> >> > pointers?
>> >> >
>> >> > Looking to be able to search for "myMetadata:myValue" and something
>> like
>> >> > "developer:carlos*" through the rest api and ui.
>> >> >
>> >> > Thanks
>> >>
>> >>
>> >>
>> >> --
>> >> Olivier Lamy
>> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
>> >>
>>
>>
>>
>> --
>> Olivier Lamy
>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
--
Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Searching pom elements and generic metadata
Posted by Carlos Sanchez <ca...@csanchez.org>.
On Wed, Nov 5, 2014 at 11:54 AM, Olivier Lamy <ol...@apache.org> wrote:
> I would add that in the interface MetadataRepository, add a test in
> the tck and implements it for all providers :-)
> Maybe having 2 methods:
> * generic metadata: getArtifactsByMetadata( key, value) ( maybe a Map
> for multiple value pair? will you test the full value or a matching?)
>
is the matching syntax for cassandra an jcr different?
I mean if I pass ('foo', 'bar*') -or some sort of regex- to do matching is
it going to work only in jcr?
> * pom contents: getArtifactsByProperty( key, value ) ( same questions )
>
> To start I won't go with a Map and exact matching :-) (I reckon pom
> content can be a pain with Cassandra impl)
>
> WDYT?
>
> Olivier
>
>
> On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org> wrote:
> > Thanks for the pointers, so if I add a method to JcrMetadataRepository
> >
> > getArtifactsByProperty(key, value)
> >
> > similar to getArtifactsByDateRange, writing the correct JCR query, I
> could
> > access all that metadata?
> >
> >
> > In which of the rest services would it make sense to add such a search?
> >
> >
> >
> > On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <ol...@apache.org> wrote:
> >
> >> Hi,
> >> The current SearchService search in maven indexer available datas.
> >> All the data you want to search are not available here but in the
> >> MetadataRepository ( so jcr, file or the new Cassandra).
> >> And currently there is no search methods for that!
> >> So really good news to see you as a volunteer to implement this! :-)
> >> Let me know if you need any pointers.
> >>
> >> Cheers
> >> Olivier
> >>
> >>
> >> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org> wrote:
> >> > Hi,
> >> >
> >> > Two things I wasn't able to do and was going to implement (unless I
> >> missed
> >> > something):
> >> >
> >> > * search for generic metadata
> >> > * search for pom contents
> >> >
> >> > The way the search queries are built in SearchService only looks for
> the
> >> > typical fields (artifactId, groupId,...), so that needs to be changed,
> >> but
> >> > I don't know if the pom and generic metadata fields are even indexed,
> any
> >> > pointers?
> >> >
> >> > Looking to be able to search for "myMetadata:myValue" and something
> like
> >> > "developer:carlos*" through the rest api and ui.
> >> >
> >> > Thanks
> >>
> >>
> >>
> >> --
> >> Olivier Lamy
> >> http://twitter.com/olamy | http://linkedin.com/in/olamy
> >>
>
>
>
> --
> Olivier Lamy
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
Re: Searching pom elements and generic metadata
Posted by Olivier Lamy <ol...@apache.org>.
I would add that in the interface MetadataRepository, add a test in
the tck and implements it for all providers :-)
Maybe having 2 methods:
* generic metadata: getArtifactsByMetadata( key, value) ( maybe a Map
for multiple value pair? will you test the full value or a matching?)
* pom contents: getArtifactsByProperty( key, value ) ( same questions )
To start I won't go with a Map and exact matching :-) (I reckon pom
content can be a pain with Cassandra impl)
WDYT?
Olivier
On 5 November 2014 21:45, Carlos Sanchez <ca...@apache.org> wrote:
> Thanks for the pointers, so if I add a method to JcrMetadataRepository
>
> getArtifactsByProperty(key, value)
>
> similar to getArtifactsByDateRange, writing the correct JCR query, I could
> access all that metadata?
>
>
> In which of the rest services would it make sense to add such a search?
>
>
>
> On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <ol...@apache.org> wrote:
>
>> Hi,
>> The current SearchService search in maven indexer available datas.
>> All the data you want to search are not available here but in the
>> MetadataRepository ( so jcr, file or the new Cassandra).
>> And currently there is no search methods for that!
>> So really good news to see you as a volunteer to implement this! :-)
>> Let me know if you need any pointers.
>>
>> Cheers
>> Olivier
>>
>>
>> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org> wrote:
>> > Hi,
>> >
>> > Two things I wasn't able to do and was going to implement (unless I
>> missed
>> > something):
>> >
>> > * search for generic metadata
>> > * search for pom contents
>> >
>> > The way the search queries are built in SearchService only looks for the
>> > typical fields (artifactId, groupId,...), so that needs to be changed,
>> but
>> > I don't know if the pom and generic metadata fields are even indexed, any
>> > pointers?
>> >
>> > Looking to be able to search for "myMetadata:myValue" and something like
>> > "developer:carlos*" through the rest api and ui.
>> >
>> > Thanks
>>
>>
>>
>> --
>> Olivier Lamy
>> http://twitter.com/olamy | http://linkedin.com/in/olamy
>>
--
Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy
Re: Searching pom elements and generic metadata
Posted by Carlos Sanchez <ca...@apache.org>.
Thanks for the pointers, so if I add a method to JcrMetadataRepository
getArtifactsByProperty(key, value)
similar to getArtifactsByDateRange, writing the correct JCR query, I could
access all that metadata?
In which of the rest services would it make sense to add such a search?
On Wed, Nov 5, 2014 at 11:28 AM, Olivier Lamy <ol...@apache.org> wrote:
> Hi,
> The current SearchService search in maven indexer available datas.
> All the data you want to search are not available here but in the
> MetadataRepository ( so jcr, file or the new Cassandra).
> And currently there is no search methods for that!
> So really good news to see you as a volunteer to implement this! :-)
> Let me know if you need any pointers.
>
> Cheers
> Olivier
>
>
> On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org> wrote:
> > Hi,
> >
> > Two things I wasn't able to do and was going to implement (unless I
> missed
> > something):
> >
> > * search for generic metadata
> > * search for pom contents
> >
> > The way the search queries are built in SearchService only looks for the
> > typical fields (artifactId, groupId,...), so that needs to be changed,
> but
> > I don't know if the pom and generic metadata fields are even indexed, any
> > pointers?
> >
> > Looking to be able to search for "myMetadata:myValue" and something like
> > "developer:carlos*" through the rest api and ui.
> >
> > Thanks
>
>
>
> --
> Olivier Lamy
> http://twitter.com/olamy | http://linkedin.com/in/olamy
>
Re: Searching pom elements and generic metadata
Posted by Olivier Lamy <ol...@apache.org>.
Hi,
The current SearchService search in maven indexer available datas.
All the data you want to search are not available here but in the
MetadataRepository ( so jcr, file or the new Cassandra).
And currently there is no search methods for that!
So really good news to see you as a volunteer to implement this! :-)
Let me know if you need any pointers.
Cheers
Olivier
On 4 November 2014 06:33, Carlos Sanchez <ca...@apache.org> wrote:
> Hi,
>
> Two things I wasn't able to do and was going to implement (unless I missed
> something):
>
> * search for generic metadata
> * search for pom contents
>
> The way the search queries are built in SearchService only looks for the
> typical fields (artifactId, groupId,...), so that needs to be changed, but
> I don't know if the pom and generic metadata fields are even indexed, any
> pointers?
>
> Looking to be able to search for "myMetadata:myValue" and something like
> "developer:carlos*" through the rest api and ui.
>
> Thanks
--
Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy