You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jesper Nøhr <jn...@gmail.com> on 2009/03/25 17:52:28 UTC
How do I accomplish this (semi-)complicated setup?
Hi list,
I've finally settled on Solr, seeing as it has almost everything I
could want out of the box.
My setup is a complicated one. It will serve as the search backend on
Bitbucket.org, a mercurial hosting site. We have literally thousands
of code repositories, as well as users and other data. All this needs
to be indexed.
The complication comes in when we have private repositories. Only
select users have access to these, but we still need to index them.
How would I go about accomplishing this? I can't think of a clean way to do it.
Any pointers much appreciated.
Jesper
Re: How do I accomplish this (semi-)complicated setup?
Posted by Alejandro Gonzalez <al...@gmail.com>.
you can even create separated indexes for private or public access if u need
(and place them in separated machines), but i think Eric's suggestion is the
best and easier
On Wed, Mar 25, 2009 at 5:52 PM, Jesper Nøhr <jn...@gmail.com> wrote:
> Hi list,
>
> I've finally settled on Solr, seeing as it has almost everything I
> could want out of the box.
>
> My setup is a complicated one. It will serve as the search backend on
> Bitbucket.org, a mercurial hosting site. We have literally thousands
> of code repositories, as well as users and other data. All this needs
> to be indexed.
>
> The complication comes in when we have private repositories. Only
> select users have access to these, but we still need to index them.
>
> How would I go about accomplishing this? I can't think of a clean way to do
> it.
>
> Any pointers much appreciated.
>
>
> Jesper
>
Re: How do I accomplish this (semi-)complicated setup?
Posted by "Aleksander M. Stensby" <al...@integrasco.no>.
If you are saying that the number of private repos for the user is limited
to say less than 10 or something like that, the query wouldn't be very
long...
Something like public:true OR (repo_id:(1 OR 2 OR 3 OR 4) etc.
- Aleks
On Thu, 26 Mar 2009 09:33:14 +0100, Jesper Nøhr <je...@noehr.org> wrote:
> Ah. Well that's what I thought, and that's where I get confused.
>
> Realistically speaking, we have, say, 10.000 public repositories and
> any given user may have 2 or 3 private repositories. This means that
> when the user searches, he should search among all those 10.000 public
> ones, but also his 2 or 3 private ones. That means we would have to
> query for repo_id:1 OR repo_id:2 OR .... repo_id:10000. This is going
> to get very long...
>
>
> Jesper
>
>
>
> On Wed, Mar 25, 2009 at 9:10 PM, Alejandro Gonzalez
> <al...@gmail.com> wrote:
>> try using db for permission management and when u want to make a rep
>> public
>> u just have to add it's id or name to everyuser permissions field. i
>> think
>> you don't need to add any "is_public" field to index, just an id or name
>> field in wich the indexed doc is.So you can pre-filter the reps quering
>> the
>> db obtaining the reps for wich user has permissions and adding this
>> restrictions to the solr query. this way you can't change
>> reps'permissions
>> without re-indexing. so the query for solr if the current user is
>> allowed
>> for search in the 1 and 2 reps should be something like
>> ...rep_id:1OR2...
>>
>>
>> Alex
>>
>>
>> On Wed, Mar 25, 2009 at 8:06 PM, Jesper Nøhr <je...@noehr.org> wrote:
>>
>>> OK, we're getting closer. I just have two final questions regarding
>>> this
>>> then:
>>>
>>> 1. This would also include all the public repositories, right? If so,
>>> how would such a query look? Some kind of is_public:true AND ...?
>>>
>>> 2. When a repository is made public, the is_public property in the
>>> Solr index needs to reflect this. How can such an update be made
>>> without having to purge and re-index?
>>>
>>>
>>> Jesper
>>>
>>>
>>> On Wed, Mar 25, 2009 at 6:29 PM, Alejandro Gonzalez
>>> <al...@gmail.com> wrote:
>>> > ok so u can create a table in a DB where you have a row foreach user
>>> and
>>> a
>>> > field with the reps he/she can access. Then you just have to take a
>>> look
>>> on
>>> > the db and include the repository name in the index. so you just
>>> have to
>>> > control (using query parameters) if the query is done for the right
>>> reps
>>> for
>>> > that user.
>>> >
>>> > is it good for u?
>>> >
>>> >
>>> >
>>> > On Wed, Mar 25, 2009 at 6:20 PM, Jesper Nøhr <je...@noehr.org>
>>> wrote:
>>> >
>>> >> Hm, I must be missing something, then.
>>> >>
>>> >> Consider this.
>>> >>
>>> >> There are three repositories, A and B, C. There are two users, U1
>>> and
>>> U2.
>>> >>
>>> >> Repository A is public, while B and C are private. Only U1 can
>>> access
>>> >> B. No one can access C.
>>> >>
>>> >> I index this data, such that Is_Private is true for B.
>>> >>
>>> >> Now, when U2 searches, he will only see data for repo A. This is
>>> correct.
>>> >>
>>> >> When U1 searches, what happens? AFAIK, he will also only see data
>>> for
>>> >> A, unless we specify Is_Private:True, but then he will only see data
>>> >> for B (and C, which he doesn't have access to.)
>>> >>
>>> >> Secondly, say we grant U2 access to B. How do we tell Solr that he
>>> can
>>> >> see it, then?
>>> >>
>>> >> Sorry if I'm not making much sense here, but I'm quite confused.
>>> >>
>>> >>
>>> >> Jesper
>>> >>
>>> >>
>>> >>
>>> >> On Wed, Mar 25, 2009 at 6:13 PM, Alejandro Gonzalez
>>> >> <al...@gmail.com> wrote:
>>> >> > i can't see the problem about that. you can manage your users
>>> using a
>>> DB
>>> >> and
>>> >> > keep there the permissions they could have, and create or erase
>>> users
>>> >> > without problems. you just have to manage a "working index" field
>>> for
>>> >> each
>>> >> > user with repositories' ids he can access. or u can create several
>>> >> indexes
>>> >> > and a users solr index with a multi-valued field with the indexes
>>> the
>>> >> user
>>> >> > can access.
>>> >> >
>>> >> > if then u want to turn a private repository into public u just
>>> have to
>>> >> > change the permissions field in your DB or users' index.
>>> >> >
>>> >> > On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr <je...@noehr.org>
>>> wrote:
>>> >> >
>>> >> >> On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh
>>> >> >> <ep...@opensourceconnections.com> wrote:
>>> >> >> > You could index the user name or ID, and then in your
>>> application
>>> add
>>> >> as
>>> >> >> > filter the username as you pass the query back to Solr. Maybe
>>> have
>>> a
>>> >> >> > access_type that is Public or Private, and then for public
>>> searches
>>> >> only
>>> >> >> > include the ones that meet the access_type of Public.
>>> >> >>
>>> >> >> That makes sense. Two questions on that:
>>> >> >>
>>> >> >> 1. More than one user can have access to a repository, so how
>>> would
>>> >> >> that work? Also, if a user is added/removed, what's the best way
>>> to
>>> >> >> keep that in sync?
>>> >> >>
>>> >> >> 2. In the event that a repository that is private, is made
>>> public,
>>> how
>>> >> >> easy would it be to run an "UPDATE" so to speak?
>>> >> >>
>>> >> >>
>>> >> >> Jesper
>>> >> >>
>>> >> >> > On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote:
>>> >> >> >
>>> >> >> >> Hi list,
>>> >> >> >>
>>> >> >> >> I've finally settled on Solr, seeing as it has almost
>>> everything I
>>> >> >> >> could want out of the box.
>>> >> >> >>
>>> >> >> >> My setup is a complicated one. It will serve as the search
>>> backend
>>> on
>>> >> >> >> Bitbucket.org, a mercurial hosting site. We have literally
>>> thousands
>>> >> >> >> of code repositories, as well as users and other data. All
>>> this
>>> needs
>>> >> >> >> to be indexed.
>>> >> >> >>
>>> >> >> >> The complication comes in when we have private repositories.
>>> Only
>>> >> >> >> select users have access to these, but we still need to index
>>> them.
>>> >> >> >>
>>> >> >> >> How would I go about accomplishing this? I can't think of a
>>> clean
>>> way
>>> >> to
>>> >> >> >> do it.
>>> >> >> >>
>>> >> >> >> Any pointers much appreciated.
>>> >> >> >>
>>> >> >> >>
>>> >> >> >> Jesper
>>> >> >> >
>>> >> >> > -----------------------------------------------------
>>> >> >> > Eric Pugh | Principal | OpenSource Connections, LLC |
>>> 434.466.1467
>>> |
>>> >> >> > http://www.opensourceconnections.com
>>> >> >> > Free/Busy: http://tinyurl.com/eric-cal
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >>
>>> >> >
>>> >>
>>> >
>>>
>>
>
--
Aleksander M. Stensby
Senior software developer
Integrasco A/S
www.integrasco.no
Please consider the environment before printing all or any of this e-mail
Re: How do I accomplish this (semi-)complicated setup?
Posted by Jesper Nøhr <je...@noehr.org>.
Ah. Well that's what I thought, and that's where I get confused.
Realistically speaking, we have, say, 10.000 public repositories and
any given user may have 2 or 3 private repositories. This means that
when the user searches, he should search among all those 10.000 public
ones, but also his 2 or 3 private ones. That means we would have to
query for repo_id:1 OR repo_id:2 OR .... repo_id:10000. This is going
to get very long...
Jesper
On Wed, Mar 25, 2009 at 9:10 PM, Alejandro Gonzalez
<al...@gmail.com> wrote:
> try using db for permission management and when u want to make a rep public
> u just have to add it's id or name to everyuser permissions field. i think
> you don't need to add any "is_public" field to index, just an id or name
> field in wich the indexed doc is.So you can pre-filter the reps quering the
> db obtaining the reps for wich user has permissions and adding this
> restrictions to the solr query. this way you can't change reps'permissions
> without re-indexing. so the query for solr if the current user is allowed
> for search in the 1 and 2 reps should be something like ...rep_id:1OR2...
>
>
> Alex
>
>
> On Wed, Mar 25, 2009 at 8:06 PM, Jesper Nøhr <je...@noehr.org> wrote:
>
>> OK, we're getting closer. I just have two final questions regarding this
>> then:
>>
>> 1. This would also include all the public repositories, right? If so,
>> how would such a query look? Some kind of is_public:true AND ...?
>>
>> 2. When a repository is made public, the is_public property in the
>> Solr index needs to reflect this. How can such an update be made
>> without having to purge and re-index?
>>
>>
>> Jesper
>>
>>
>> On Wed, Mar 25, 2009 at 6:29 PM, Alejandro Gonzalez
>> <al...@gmail.com> wrote:
>> > ok so u can create a table in a DB where you have a row foreach user and
>> a
>> > field with the reps he/she can access. Then you just have to take a look
>> on
>> > the db and include the repository name in the index. so you just have to
>> > control (using query parameters) if the query is done for the right reps
>> for
>> > that user.
>> >
>> > is it good for u?
>> >
>> >
>> >
>> > On Wed, Mar 25, 2009 at 6:20 PM, Jesper Nøhr <je...@noehr.org> wrote:
>> >
>> >> Hm, I must be missing something, then.
>> >>
>> >> Consider this.
>> >>
>> >> There are three repositories, A and B, C. There are two users, U1 and
>> U2.
>> >>
>> >> Repository A is public, while B and C are private. Only U1 can access
>> >> B. No one can access C.
>> >>
>> >> I index this data, such that Is_Private is true for B.
>> >>
>> >> Now, when U2 searches, he will only see data for repo A. This is
>> correct.
>> >>
>> >> When U1 searches, what happens? AFAIK, he will also only see data for
>> >> A, unless we specify Is_Private:True, but then he will only see data
>> >> for B (and C, which he doesn't have access to.)
>> >>
>> >> Secondly, say we grant U2 access to B. How do we tell Solr that he can
>> >> see it, then?
>> >>
>> >> Sorry if I'm not making much sense here, but I'm quite confused.
>> >>
>> >>
>> >> Jesper
>> >>
>> >>
>> >>
>> >> On Wed, Mar 25, 2009 at 6:13 PM, Alejandro Gonzalez
>> >> <al...@gmail.com> wrote:
>> >> > i can't see the problem about that. you can manage your users using a
>> DB
>> >> and
>> >> > keep there the permissions they could have, and create or erase users
>> >> > without problems. you just have to manage a "working index" field for
>> >> each
>> >> > user with repositories' ids he can access. or u can create several
>> >> indexes
>> >> > and a users solr index with a multi-valued field with the indexes the
>> >> user
>> >> > can access.
>> >> >
>> >> > if then u want to turn a private repository into public u just have to
>> >> > change the permissions field in your DB or users' index.
>> >> >
>> >> > On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr <je...@noehr.org>
>> wrote:
>> >> >
>> >> >> On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh
>> >> >> <ep...@opensourceconnections.com> wrote:
>> >> >> > You could index the user name or ID, and then in your application
>> add
>> >> as
>> >> >> > filter the username as you pass the query back to Solr. Maybe have
>> a
>> >> >> > access_type that is Public or Private, and then for public searches
>> >> only
>> >> >> > include the ones that meet the access_type of Public.
>> >> >>
>> >> >> That makes sense. Two questions on that:
>> >> >>
>> >> >> 1. More than one user can have access to a repository, so how would
>> >> >> that work? Also, if a user is added/removed, what's the best way to
>> >> >> keep that in sync?
>> >> >>
>> >> >> 2. In the event that a repository that is private, is made public,
>> how
>> >> >> easy would it be to run an "UPDATE" so to speak?
>> >> >>
>> >> >>
>> >> >> Jesper
>> >> >>
>> >> >> > On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote:
>> >> >> >
>> >> >> >> Hi list,
>> >> >> >>
>> >> >> >> I've finally settled on Solr, seeing as it has almost everything I
>> >> >> >> could want out of the box.
>> >> >> >>
>> >> >> >> My setup is a complicated one. It will serve as the search backend
>> on
>> >> >> >> Bitbucket.org, a mercurial hosting site. We have literally
>> thousands
>> >> >> >> of code repositories, as well as users and other data. All this
>> needs
>> >> >> >> to be indexed.
>> >> >> >>
>> >> >> >> The complication comes in when we have private repositories. Only
>> >> >> >> select users have access to these, but we still need to index
>> them.
>> >> >> >>
>> >> >> >> How would I go about accomplishing this? I can't think of a clean
>> way
>> >> to
>> >> >> >> do it.
>> >> >> >>
>> >> >> >> Any pointers much appreciated.
>> >> >> >>
>> >> >> >>
>> >> >> >> Jesper
>> >> >> >
>> >> >> > -----------------------------------------------------
>> >> >> > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467
>> |
>> >> >> > http://www.opensourceconnections.com
>> >> >> > Free/Busy: http://tinyurl.com/eric-cal
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >>
>> >> >
>> >>
>> >
>>
>
Re: How do I accomplish this (semi-)complicated setup?
Posted by Alejandro Gonzalez <al...@gmail.com>.
try using db for permission management and when u want to make a rep public
u just have to add it's id or name to everyuser permissions field. i think
you don't need to add any "is_public" field to index, just an id or name
field in wich the indexed doc is.So you can pre-filter the reps quering the
db obtaining the reps for wich user has permissions and adding this
restrictions to the solr query. this way you can't change reps'permissions
without re-indexing. so the query for solr if the current user is allowed
for search in the 1 and 2 reps should be something like ...rep_id:1OR2...
Alex
On Wed, Mar 25, 2009 at 8:06 PM, Jesper Nøhr <je...@noehr.org> wrote:
> OK, we're getting closer. I just have two final questions regarding this
> then:
>
> 1. This would also include all the public repositories, right? If so,
> how would such a query look? Some kind of is_public:true AND ...?
>
> 2. When a repository is made public, the is_public property in the
> Solr index needs to reflect this. How can such an update be made
> without having to purge and re-index?
>
>
> Jesper
>
>
> On Wed, Mar 25, 2009 at 6:29 PM, Alejandro Gonzalez
> <al...@gmail.com> wrote:
> > ok so u can create a table in a DB where you have a row foreach user and
> a
> > field with the reps he/she can access. Then you just have to take a look
> on
> > the db and include the repository name in the index. so you just have to
> > control (using query parameters) if the query is done for the right reps
> for
> > that user.
> >
> > is it good for u?
> >
> >
> >
> > On Wed, Mar 25, 2009 at 6:20 PM, Jesper Nøhr <je...@noehr.org> wrote:
> >
> >> Hm, I must be missing something, then.
> >>
> >> Consider this.
> >>
> >> There are three repositories, A and B, C. There are two users, U1 and
> U2.
> >>
> >> Repository A is public, while B and C are private. Only U1 can access
> >> B. No one can access C.
> >>
> >> I index this data, such that Is_Private is true for B.
> >>
> >> Now, when U2 searches, he will only see data for repo A. This is
> correct.
> >>
> >> When U1 searches, what happens? AFAIK, he will also only see data for
> >> A, unless we specify Is_Private:True, but then he will only see data
> >> for B (and C, which he doesn't have access to.)
> >>
> >> Secondly, say we grant U2 access to B. How do we tell Solr that he can
> >> see it, then?
> >>
> >> Sorry if I'm not making much sense here, but I'm quite confused.
> >>
> >>
> >> Jesper
> >>
> >>
> >>
> >> On Wed, Mar 25, 2009 at 6:13 PM, Alejandro Gonzalez
> >> <al...@gmail.com> wrote:
> >> > i can't see the problem about that. you can manage your users using a
> DB
> >> and
> >> > keep there the permissions they could have, and create or erase users
> >> > without problems. you just have to manage a "working index" field for
> >> each
> >> > user with repositories' ids he can access. or u can create several
> >> indexes
> >> > and a users solr index with a multi-valued field with the indexes the
> >> user
> >> > can access.
> >> >
> >> > if then u want to turn a private repository into public u just have to
> >> > change the permissions field in your DB or users' index.
> >> >
> >> > On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr <je...@noehr.org>
> wrote:
> >> >
> >> >> On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh
> >> >> <ep...@opensourceconnections.com> wrote:
> >> >> > You could index the user name or ID, and then in your application
> add
> >> as
> >> >> > filter the username as you pass the query back to Solr. Maybe have
> a
> >> >> > access_type that is Public or Private, and then for public searches
> >> only
> >> >> > include the ones that meet the access_type of Public.
> >> >>
> >> >> That makes sense. Two questions on that:
> >> >>
> >> >> 1. More than one user can have access to a repository, so how would
> >> >> that work? Also, if a user is added/removed, what's the best way to
> >> >> keep that in sync?
> >> >>
> >> >> 2. In the event that a repository that is private, is made public,
> how
> >> >> easy would it be to run an "UPDATE" so to speak?
> >> >>
> >> >>
> >> >> Jesper
> >> >>
> >> >> > On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote:
> >> >> >
> >> >> >> Hi list,
> >> >> >>
> >> >> >> I've finally settled on Solr, seeing as it has almost everything I
> >> >> >> could want out of the box.
> >> >> >>
> >> >> >> My setup is a complicated one. It will serve as the search backend
> on
> >> >> >> Bitbucket.org, a mercurial hosting site. We have literally
> thousands
> >> >> >> of code repositories, as well as users and other data. All this
> needs
> >> >> >> to be indexed.
> >> >> >>
> >> >> >> The complication comes in when we have private repositories. Only
> >> >> >> select users have access to these, but we still need to index
> them.
> >> >> >>
> >> >> >> How would I go about accomplishing this? I can't think of a clean
> way
> >> to
> >> >> >> do it.
> >> >> >>
> >> >> >> Any pointers much appreciated.
> >> >> >>
> >> >> >>
> >> >> >> Jesper
> >> >> >
> >> >> > -----------------------------------------------------
> >> >> > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467
> |
> >> >> > http://www.opensourceconnections.com
> >> >> > Free/Busy: http://tinyurl.com/eric-cal
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >>
> >> >
> >>
> >
>
Re: How do I accomplish this (semi-)complicated setup?
Posted by Jesper Nøhr <je...@noehr.org>.
OK, we're getting closer. I just have two final questions regarding this then:
1. This would also include all the public repositories, right? If so,
how would such a query look? Some kind of is_public:true AND ...?
2. When a repository is made public, the is_public property in the
Solr index needs to reflect this. How can such an update be made
without having to purge and re-index?
Jesper
On Wed, Mar 25, 2009 at 6:29 PM, Alejandro Gonzalez
<al...@gmail.com> wrote:
> ok so u can create a table in a DB where you have a row foreach user and a
> field with the reps he/she can access. Then you just have to take a look on
> the db and include the repository name in the index. so you just have to
> control (using query parameters) if the query is done for the right reps for
> that user.
>
> is it good for u?
>
>
>
> On Wed, Mar 25, 2009 at 6:20 PM, Jesper Nøhr <je...@noehr.org> wrote:
>
>> Hm, I must be missing something, then.
>>
>> Consider this.
>>
>> There are three repositories, A and B, C. There are two users, U1 and U2.
>>
>> Repository A is public, while B and C are private. Only U1 can access
>> B. No one can access C.
>>
>> I index this data, such that Is_Private is true for B.
>>
>> Now, when U2 searches, he will only see data for repo A. This is correct.
>>
>> When U1 searches, what happens? AFAIK, he will also only see data for
>> A, unless we specify Is_Private:True, but then he will only see data
>> for B (and C, which he doesn't have access to.)
>>
>> Secondly, say we grant U2 access to B. How do we tell Solr that he can
>> see it, then?
>>
>> Sorry if I'm not making much sense here, but I'm quite confused.
>>
>>
>> Jesper
>>
>>
>>
>> On Wed, Mar 25, 2009 at 6:13 PM, Alejandro Gonzalez
>> <al...@gmail.com> wrote:
>> > i can't see the problem about that. you can manage your users using a DB
>> and
>> > keep there the permissions they could have, and create or erase users
>> > without problems. you just have to manage a "working index" field for
>> each
>> > user with repositories' ids he can access. or u can create several
>> indexes
>> > and a users solr index with a multi-valued field with the indexes the
>> user
>> > can access.
>> >
>> > if then u want to turn a private repository into public u just have to
>> > change the permissions field in your DB or users' index.
>> >
>> > On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr <je...@noehr.org> wrote:
>> >
>> >> On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh
>> >> <ep...@opensourceconnections.com> wrote:
>> >> > You could index the user name or ID, and then in your application add
>> as
>> >> > filter the username as you pass the query back to Solr. Maybe have a
>> >> > access_type that is Public or Private, and then for public searches
>> only
>> >> > include the ones that meet the access_type of Public.
>> >>
>> >> That makes sense. Two questions on that:
>> >>
>> >> 1. More than one user can have access to a repository, so how would
>> >> that work? Also, if a user is added/removed, what's the best way to
>> >> keep that in sync?
>> >>
>> >> 2. In the event that a repository that is private, is made public, how
>> >> easy would it be to run an "UPDATE" so to speak?
>> >>
>> >>
>> >> Jesper
>> >>
>> >> > On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote:
>> >> >
>> >> >> Hi list,
>> >> >>
>> >> >> I've finally settled on Solr, seeing as it has almost everything I
>> >> >> could want out of the box.
>> >> >>
>> >> >> My setup is a complicated one. It will serve as the search backend on
>> >> >> Bitbucket.org, a mercurial hosting site. We have literally thousands
>> >> >> of code repositories, as well as users and other data. All this needs
>> >> >> to be indexed.
>> >> >>
>> >> >> The complication comes in when we have private repositories. Only
>> >> >> select users have access to these, but we still need to index them.
>> >> >>
>> >> >> How would I go about accomplishing this? I can't think of a clean way
>> to
>> >> >> do it.
>> >> >>
>> >> >> Any pointers much appreciated.
>> >> >>
>> >> >>
>> >> >> Jesper
>> >> >
>> >> > -----------------------------------------------------
>> >> > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 |
>> >> > http://www.opensourceconnections.com
>> >> > Free/Busy: http://tinyurl.com/eric-cal
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >>
>> >
>>
>
Re: How do I accomplish this (semi-)complicated setup?
Posted by Alejandro Gonzalez <al...@gmail.com>.
ok so u can create a table in a DB where you have a row foreach user and a
field with the reps he/she can access. Then you just have to take a look on
the db and include the repository name in the index. so you just have to
control (using query parameters) if the query is done for the right reps for
that user.
is it good for u?
On Wed, Mar 25, 2009 at 6:20 PM, Jesper Nøhr <je...@noehr.org> wrote:
> Hm, I must be missing something, then.
>
> Consider this.
>
> There are three repositories, A and B, C. There are two users, U1 and U2.
>
> Repository A is public, while B and C are private. Only U1 can access
> B. No one can access C.
>
> I index this data, such that Is_Private is true for B.
>
> Now, when U2 searches, he will only see data for repo A. This is correct.
>
> When U1 searches, what happens? AFAIK, he will also only see data for
> A, unless we specify Is_Private:True, but then he will only see data
> for B (and C, which he doesn't have access to.)
>
> Secondly, say we grant U2 access to B. How do we tell Solr that he can
> see it, then?
>
> Sorry if I'm not making much sense here, but I'm quite confused.
>
>
> Jesper
>
>
>
> On Wed, Mar 25, 2009 at 6:13 PM, Alejandro Gonzalez
> <al...@gmail.com> wrote:
> > i can't see the problem about that. you can manage your users using a DB
> and
> > keep there the permissions they could have, and create or erase users
> > without problems. you just have to manage a "working index" field for
> each
> > user with repositories' ids he can access. or u can create several
> indexes
> > and a users solr index with a multi-valued field with the indexes the
> user
> > can access.
> >
> > if then u want to turn a private repository into public u just have to
> > change the permissions field in your DB or users' index.
> >
> > On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr <je...@noehr.org> wrote:
> >
> >> On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh
> >> <ep...@opensourceconnections.com> wrote:
> >> > You could index the user name or ID, and then in your application add
> as
> >> > filter the username as you pass the query back to Solr. Maybe have a
> >> > access_type that is Public or Private, and then for public searches
> only
> >> > include the ones that meet the access_type of Public.
> >>
> >> That makes sense. Two questions on that:
> >>
> >> 1. More than one user can have access to a repository, so how would
> >> that work? Also, if a user is added/removed, what's the best way to
> >> keep that in sync?
> >>
> >> 2. In the event that a repository that is private, is made public, how
> >> easy would it be to run an "UPDATE" so to speak?
> >>
> >>
> >> Jesper
> >>
> >> > On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote:
> >> >
> >> >> Hi list,
> >> >>
> >> >> I've finally settled on Solr, seeing as it has almost everything I
> >> >> could want out of the box.
> >> >>
> >> >> My setup is a complicated one. It will serve as the search backend on
> >> >> Bitbucket.org, a mercurial hosting site. We have literally thousands
> >> >> of code repositories, as well as users and other data. All this needs
> >> >> to be indexed.
> >> >>
> >> >> The complication comes in when we have private repositories. Only
> >> >> select users have access to these, but we still need to index them.
> >> >>
> >> >> How would I go about accomplishing this? I can't think of a clean way
> to
> >> >> do it.
> >> >>
> >> >> Any pointers much appreciated.
> >> >>
> >> >>
> >> >> Jesper
> >> >
> >> > -----------------------------------------------------
> >> > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 |
> >> > http://www.opensourceconnections.com
> >> > Free/Busy: http://tinyurl.com/eric-cal
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >
>
Re: How do I accomplish this (semi-)complicated setup?
Posted by Jesper Nøhr <je...@noehr.org>.
Hm, I must be missing something, then.
Consider this.
There are three repositories, A and B, C. There are two users, U1 and U2.
Repository A is public, while B and C are private. Only U1 can access
B. No one can access C.
I index this data, such that Is_Private is true for B.
Now, when U2 searches, he will only see data for repo A. This is correct.
When U1 searches, what happens? AFAIK, he will also only see data for
A, unless we specify Is_Private:True, but then he will only see data
for B (and C, which he doesn't have access to.)
Secondly, say we grant U2 access to B. How do we tell Solr that he can
see it, then?
Sorry if I'm not making much sense here, but I'm quite confused.
Jesper
On Wed, Mar 25, 2009 at 6:13 PM, Alejandro Gonzalez
<al...@gmail.com> wrote:
> i can't see the problem about that. you can manage your users using a DB and
> keep there the permissions they could have, and create or erase users
> without problems. you just have to manage a "working index" field for each
> user with repositories' ids he can access. or u can create several indexes
> and a users solr index with a multi-valued field with the indexes the user
> can access.
>
> if then u want to turn a private repository into public u just have to
> change the permissions field in your DB or users' index.
>
> On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr <je...@noehr.org> wrote:
>
>> On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh
>> <ep...@opensourceconnections.com> wrote:
>> > You could index the user name or ID, and then in your application add as
>> > filter the username as you pass the query back to Solr. Maybe have a
>> > access_type that is Public or Private, and then for public searches only
>> > include the ones that meet the access_type of Public.
>>
>> That makes sense. Two questions on that:
>>
>> 1. More than one user can have access to a repository, so how would
>> that work? Also, if a user is added/removed, what's the best way to
>> keep that in sync?
>>
>> 2. In the event that a repository that is private, is made public, how
>> easy would it be to run an "UPDATE" so to speak?
>>
>>
>> Jesper
>>
>> > On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote:
>> >
>> >> Hi list,
>> >>
>> >> I've finally settled on Solr, seeing as it has almost everything I
>> >> could want out of the box.
>> >>
>> >> My setup is a complicated one. It will serve as the search backend on
>> >> Bitbucket.org, a mercurial hosting site. We have literally thousands
>> >> of code repositories, as well as users and other data. All this needs
>> >> to be indexed.
>> >>
>> >> The complication comes in when we have private repositories. Only
>> >> select users have access to these, but we still need to index them.
>> >>
>> >> How would I go about accomplishing this? I can't think of a clean way to
>> >> do it.
>> >>
>> >> Any pointers much appreciated.
>> >>
>> >>
>> >> Jesper
>> >
>> > -----------------------------------------------------
>> > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 |
>> > http://www.opensourceconnections.com
>> > Free/Busy: http://tinyurl.com/eric-cal
>> >
>> >
>> >
>> >
>> >
>>
>
Re: How do I accomplish this (semi-)complicated setup?
Posted by Alejandro Gonzalez <al...@gmail.com>.
i can't see the problem about that. you can manage your users using a DB and
keep there the permissions they could have, and create or erase users
without problems. you just have to manage a "working index" field for each
user with repositories' ids he can access. or u can create several indexes
and a users solr index with a multi-valued field with the indexes the user
can access.
if then u want to turn a private repository into public u just have to
change the permissions field in your DB or users' index.
On Wed, Mar 25, 2009 at 6:02 PM, Jesper Nøhr <je...@noehr.org> wrote:
> On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh
> <ep...@opensourceconnections.com> wrote:
> > You could index the user name or ID, and then in your application add as
> > filter the username as you pass the query back to Solr. Maybe have a
> > access_type that is Public or Private, and then for public searches only
> > include the ones that meet the access_type of Public.
>
> That makes sense. Two questions on that:
>
> 1. More than one user can have access to a repository, so how would
> that work? Also, if a user is added/removed, what's the best way to
> keep that in sync?
>
> 2. In the event that a repository that is private, is made public, how
> easy would it be to run an "UPDATE" so to speak?
>
>
> Jesper
>
> > On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote:
> >
> >> Hi list,
> >>
> >> I've finally settled on Solr, seeing as it has almost everything I
> >> could want out of the box.
> >>
> >> My setup is a complicated one. It will serve as the search backend on
> >> Bitbucket.org, a mercurial hosting site. We have literally thousands
> >> of code repositories, as well as users and other data. All this needs
> >> to be indexed.
> >>
> >> The complication comes in when we have private repositories. Only
> >> select users have access to these, but we still need to index them.
> >>
> >> How would I go about accomplishing this? I can't think of a clean way to
> >> do it.
> >>
> >> Any pointers much appreciated.
> >>
> >>
> >> Jesper
> >
> > -----------------------------------------------------
> > Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 |
> > http://www.opensourceconnections.com
> > Free/Busy: http://tinyurl.com/eric-cal
> >
> >
> >
> >
> >
>
Re: How do I accomplish this (semi-)complicated setup?
Posted by Jesper Nøhr <je...@noehr.org>.
On Wed, Mar 25, 2009 at 5:57 PM, Eric Pugh
<ep...@opensourceconnections.com> wrote:
> You could index the user name or ID, and then in your application add as
> filter the username as you pass the query back to Solr. Maybe have a
> access_type that is Public or Private, and then for public searches only
> include the ones that meet the access_type of Public.
That makes sense. Two questions on that:
1. More than one user can have access to a repository, so how would
that work? Also, if a user is added/removed, what's the best way to
keep that in sync?
2. In the event that a repository that is private, is made public, how
easy would it be to run an "UPDATE" so to speak?
Jesper
> On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote:
>
>> Hi list,
>>
>> I've finally settled on Solr, seeing as it has almost everything I
>> could want out of the box.
>>
>> My setup is a complicated one. It will serve as the search backend on
>> Bitbucket.org, a mercurial hosting site. We have literally thousands
>> of code repositories, as well as users and other data. All this needs
>> to be indexed.
>>
>> The complication comes in when we have private repositories. Only
>> select users have access to these, but we still need to index them.
>>
>> How would I go about accomplishing this? I can't think of a clean way to
>> do it.
>>
>> Any pointers much appreciated.
>>
>>
>> Jesper
>
> -----------------------------------------------------
> Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 |
> http://www.opensourceconnections.com
> Free/Busy: http://tinyurl.com/eric-cal
>
>
>
>
>
Re: How do I accomplish this (semi-)complicated setup?
Posted by Eric Pugh <ep...@opensourceconnections.com>.
You could index the user name or ID, and then in your application add
as filter the username as you pass the query back to Solr. Maybe have
a access_type that is Public or Private, and then for public searches
only include the ones that meet the access_type of Public.
Eric
On Mar 25, 2009, at 12:52 PM, Jesper Nøhr wrote:
> Hi list,
>
> I've finally settled on Solr, seeing as it has almost everything I
> could want out of the box.
>
> My setup is a complicated one. It will serve as the search backend on
> Bitbucket.org, a mercurial hosting site. We have literally thousands
> of code repositories, as well as users and other data. All this needs
> to be indexed.
>
> The complication comes in when we have private repositories. Only
> select users have access to these, but we still need to index them.
>
> How would I go about accomplishing this? I can't think of a clean
> way to do it.
>
> Any pointers much appreciated.
>
>
> Jesper
-----------------------------------------------------
Eric Pugh | Principal | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com
Free/Busy: http://tinyurl.com/eric-cal