You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@manifoldcf.apache.org by Phillip Rhodes <mo...@gmail.com> on 2017/10/29 00:36:42 UTC
Problem with Solr/ManifoldCF security filtering
MCF Gang:
I've followed the instructions in the "ManifoldCF in Action" docs to
setup security integration between ManifoldCF and Solr. I've added
the ManifoldCF SearchComponent to Solr, and I see that my indexed
documents are getting allow_token_share, allow_token_parent,
allow_token_share, etc. tokens added.
But when I query with the MCF plugin added and the
AuthenticatedUserName parameter added, I never get any results.
I tried just with with username "Fred" and I see this in the solr logs:
2017-10-29 00:18:51.527 INFO (qtp834133664-16) [ ]
o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for
2147483647\
transient cores
2017-10-29 00:18:52.742 INFO (qtp834133664-15) [ ]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={indexInfo=fa\
lse&wt=json&_=1509236332203} status=0 QTime=6
2017-10-29 00:18:53.009 INFO (qtp834133664-11) [ ]
o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system
params={wt=jso\
n&_=1509236332206} status=0 QTime=201
2017-10-29 00:19:14.349 INFO (qtp834133664-16) [ x:gettingstarted]
o.a.s.m.ManifoldCFSearchComponent Trying to match docs for user\
'[:Fred]'
2017-10-29 00:19:14.476 INFO (qtp834133664-16) [ x:gettingstarted]
o.a.s.m.ManifoldCFSearchComponent Saw authority response AUTHOR\
IZED:Null+authority+connection+for+testing
2017-10-29 00:19:14.529 INFO (qtp834133664-16) [ x:gettingstarted]
o.a.s.c.S.Request [gettingstarted] webapp=/solr path=/select p\
arams={q=*:*&AuthenticatedUserName=Fred&indent=on&wt=xml&_=1509236332558}
hits=0 status=0 QTime=228
I can tell Solr is talking to the MCF authority service, because
"Null+authority+connection+for+testing" is the description I used on
the Manifold side.
There are documents in the index that include fields like this:
<doc> <arr name="allow_token_document"> <str>Null:Fred</str> </arr>
<arr name="title"> <str/> </arr> <str
name="id">http://rss.cnn.com/~r/rss/cnn_world/~3/iTYAcfUavzM/orig-burger-king-bullying.cnn</str>
<arr name="deny_token_document"> <str>Null:DEAD_AUTHORITY</str> </arr>
<str name="stream_content_type">text/html; charset=utf-8</str> <str
name="keywords">world, Burger King stands up to bullying - CNN
Video</str> <str name="description">Burger King creates a PSA that
asks their customers to take a closer look at bullying. </str> <str
name="stream_name">docname</str> <str name="dc_title">Burger King
stands up to bullying - CNN Video</str> <arr name="content_type">
<str>text/html; charset=UTF-8</str> </arr> <long
name="stream_size">489145</long> <str
name="x_parsed_by">org.apache.tika.parser.DefaultParser
org.apache.tika.parser.html.HtmlParser</str> <str
name="stream_source_info">docname</str> <str
name="resourcename">docname</str> <str
name="fb_app_id">80401312489</str> <arr name="deny_token_parent">
<str>__no_security__</str> </arr> <arr name="allow_token_share">
<str>__no_security__</str> </arr> <arr name="deny_token_share">
<str>__no_security__</str> </arr> <arr name="allow_token_parent">
<str>__no_security__</str> </arr>
...
...
</doc>
But nonetheless, no results are returned. I'm sure I'm missing
something obvious here, but whatever it is is defeating me at the
moment.
The only thing I see that looks a little dodgy is this "Trying to
match docs for user '[:Fred]'" given that the tokens look like
"Null:Fred".
Any ideas what the problem could be?
Thanks,
Phil
Re: Problem with Solr/ManifoldCF security filtering
Posted by Karl Wright <da...@gmail.com>.
Ok, it's a little hard to follow your log snippets at this point, but let's
review the way this is supposed to work.
(1) The authority tokens get qualified by the name of the authority group.
So, both your tokens and your authority MUST be within the same authority
group for this to work. That's the most common error users make, since
authority groups were added later (after the book was written). That
probably accounts for the mismatch between what you are querying for and
how your tokens look.
(2) The Solr plugin simply wraps the incoming query with a boolean query
that matches the authorization fields. So if those fields are missing from
the Solr schema, or have the wrong default values, it won't work right.
There are SIX fields you need. The README for the plug describes what they
need to be and what the defaults need to be. If you set it up with only
four fields, you're using old instructions again.
Hope this helps...
Karl
On Sat, Oct 28, 2017 at 9:05 PM, Phillip Rhodes <mo...@gmail.com>
wrote:
> FWIW, I tried adding an explicit "AuthenticatedUserDomain=Null" to my
> initial query and I now see this kind of business in the Solr logs:
>
> 2017-10-29 01:02:27.991 INFO (qtp834133664-18) [ x:gettingstarted]
> o.a.s.m.ManifoldCFSearchComponent Trying to match docs for user
> '[Null:George]'
> 2017-10-29 01:02:27.997 INFO (qtp834133664-18) [ x:gettingstarted]
> o.a.s.c.S.Request [gettingstarted] webapp=/solr path=/select pa
> rams={q=*&AuthenticatedUserDomain=Null&AuthenticatedUserName=George&
> indent=on&wt=xml}
> hits=0 status=0 QTime=5
> 2017-10-29 01:02:43.786 INFO (qtp834133664-14) [ x:gettingstarted]
> o.a.s.m.ManifoldCFSearchComponent Trying to match docs for user
> '[Null:George]'
> 2017-10-29 01:02:43.794 INFO (qtp834133664-14) [ x:gettingstarted]
> o.a.s.c.S.Request [gettingstarted] webapp=/solr path=/select pa
> rams={q=afghanistan&AuthenticatedUserDomain=Null&
> AuthenticatedUserName=George&indent=on&wt=xml}
> hits=0 status=0 QTime=50
>
> but still no results are returned. :-(
>
>
> Phil
>
> This message optimized for indexing by NSA PRISM
>
>
> On Sat, Oct 28, 2017 at 8:39 PM, Phillip Rhodes
> <mo...@gmail.com> wrote:
> > Just to follow up on this: if I hand craft a query to the MCF
> > authority service that looks like this:
> >
> > http://manifoldcf.aws:8345/mcf-authority-service/UserACLs?username=Fred
> >
> > I get back
> >
> > AUTHORIZED:Null+authority+connection+for+testing
> > TOKEN:Null:Fred
> >
> > which looks right to me, given what I know about this so far.
> >
> > And "Null:Fred" matches what is getting put into the Solr documents.
> >
> >
> > Thanks,
> >
> >
> > Phil
> >
> >
> > This message optimized for indexing by NSA PRISM
> >
> >
> > On Sat, Oct 28, 2017 at 8:36 PM, Phillip Rhodes
> > <mo...@gmail.com> wrote:
> >> MCF Gang:
> >>
> >> I've followed the instructions in the "ManifoldCF in Action" docs to
> >> setup security integration between ManifoldCF and Solr. I've added
> >> the ManifoldCF SearchComponent to Solr, and I see that my indexed
> >> documents are getting allow_token_share, allow_token_parent,
> >> allow_token_share, etc. tokens added.
> >>
> >> But when I query with the MCF plugin added and the
> >> AuthenticatedUserName parameter added, I never get any results.
> >>
> >> I tried just with with username "Fred" and I see this in the solr logs:
> >>
> >> 2017-10-29 00:18:51.527 INFO (qtp834133664-16) [ ]
> >> o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for
> >> 2147483647\
> >> transient cores
> >> 2017-10-29 00:18:52.742 INFO (qtp834133664-15) [ ]
> >> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
> >> params={indexInfo=fa\
> >> lse&wt=json&_=1509236332203} status=0 QTime=6
> >> 2017-10-29 00:18:53.009 INFO (qtp834133664-11) [ ]
> >> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system
> >> params={wt=jso\
> >> n&_=1509236332206} status=0 QTime=201
> >> 2017-10-29 00:19:14.349 INFO (qtp834133664-16) [ x:gettingstarted]
> >> o.a.s.m.ManifoldCFSearchComponent Trying to match docs for user\
> >> '[:Fred]'
> >> 2017-10-29 00:19:14.476 INFO (qtp834133664-16) [ x:gettingstarted]
> >> o.a.s.m.ManifoldCFSearchComponent Saw authority response AUTHOR\
> >> IZED:Null+authority+connection+for+testing
> >> 2017-10-29 00:19:14.529 INFO (qtp834133664-16) [ x:gettingstarted]
> >> o.a.s.c.S.Request [gettingstarted] webapp=/solr path=/select p\
> >> arams={q=*:*&AuthenticatedUserName=Fred&indent=on&wt=xml&_=
> 1509236332558}
> >> hits=0 status=0 QTime=228
> >>
> >> I can tell Solr is talking to the MCF authority service, because
> >> "Null+authority+connection+for+testing" is the description I used on
> >> the Manifold side.
> >>
> >> There are documents in the index that include fields like this:
> >>
> >> <doc> <arr name="allow_token_document"> <str>Null:Fred</str> </arr>
> >> <arr name="title"> <str/> </arr> <str
> >> name="id">http://rss.cnn.com/~r/rss/cnn_world/~3/
> iTYAcfUavzM/orig-burger-king-bullying.cnn</str>
> >> <arr name="deny_token_document"> <str>Null:DEAD_AUTHORITY</str> </arr>
> >> <str name="stream_content_type">text/html; charset=utf-8</str> <str
> >> name="keywords">world, Burger King stands up to bullying - CNN
> >> Video</str> <str name="description">Burger King creates a PSA that
> >> asks their customers to take a closer look at bullying. </str> <str
> >> name="stream_name">docname</str> <str name="dc_title">Burger King
> >> stands up to bullying - CNN Video</str> <arr name="content_type">
> >> <str>text/html; charset=UTF-8</str> </arr> <long
> >> name="stream_size">489145</long> <str
> >> name="x_parsed_by">org.apache.tika.parser.DefaultParser
> >> org.apache.tika.parser.html.HtmlParser</str> <str
> >> name="stream_source_info">docname</str> <str
> >> name="resourcename">docname</str> <str
> >> name="fb_app_id">80401312489</str> <arr name="deny_token_parent">
> >> <str>__no_security__</str> </arr> <arr name="allow_token_share">
> >> <str>__no_security__</str> </arr> <arr name="deny_token_share">
> >> <str>__no_security__</str> </arr> <arr name="allow_token_parent">
> >> <str>__no_security__</str> </arr>
> >> ...
> >> ...
> >> </doc>
> >>
> >>
> >> But nonetheless, no results are returned. I'm sure I'm missing
> >> something obvious here, but whatever it is is defeating me at the
> >> moment.
> >>
> >> The only thing I see that looks a little dodgy is this "Trying to
> >> match docs for user '[:Fred]'" given that the tokens look like
> >> "Null:Fred".
> >>
> >>
> >> Any ideas what the problem could be?
> >>
> >>
> >>
> >>
> >> Thanks,
> >>
> >>
> >> Phil
>
Re: Problem with Solr/ManifoldCF security filtering
Posted by Phillip Rhodes <mo...@gmail.com>.
FWIW, I tried adding an explicit "AuthenticatedUserDomain=Null" to my
initial query and I now see this kind of business in the Solr logs:
2017-10-29 01:02:27.991 INFO (qtp834133664-18) [ x:gettingstarted]
o.a.s.m.ManifoldCFSearchComponent Trying to match docs for user
'[Null:George]'
2017-10-29 01:02:27.997 INFO (qtp834133664-18) [ x:gettingstarted]
o.a.s.c.S.Request [gettingstarted] webapp=/solr path=/select pa
rams={q=*&AuthenticatedUserDomain=Null&AuthenticatedUserName=George&indent=on&wt=xml}
hits=0 status=0 QTime=5
2017-10-29 01:02:43.786 INFO (qtp834133664-14) [ x:gettingstarted]
o.a.s.m.ManifoldCFSearchComponent Trying to match docs for user
'[Null:George]'
2017-10-29 01:02:43.794 INFO (qtp834133664-14) [ x:gettingstarted]
o.a.s.c.S.Request [gettingstarted] webapp=/solr path=/select pa
rams={q=afghanistan&AuthenticatedUserDomain=Null&AuthenticatedUserName=George&indent=on&wt=xml}
hits=0 status=0 QTime=50
but still no results are returned. :-(
Phil
This message optimized for indexing by NSA PRISM
On Sat, Oct 28, 2017 at 8:39 PM, Phillip Rhodes
<mo...@gmail.com> wrote:
> Just to follow up on this: if I hand craft a query to the MCF
> authority service that looks like this:
>
> http://manifoldcf.aws:8345/mcf-authority-service/UserACLs?username=Fred
>
> I get back
>
> AUTHORIZED:Null+authority+connection+for+testing
> TOKEN:Null:Fred
>
> which looks right to me, given what I know about this so far.
>
> And "Null:Fred" matches what is getting put into the Solr documents.
>
>
> Thanks,
>
>
> Phil
>
>
> This message optimized for indexing by NSA PRISM
>
>
> On Sat, Oct 28, 2017 at 8:36 PM, Phillip Rhodes
> <mo...@gmail.com> wrote:
>> MCF Gang:
>>
>> I've followed the instructions in the "ManifoldCF in Action" docs to
>> setup security integration between ManifoldCF and Solr. I've added
>> the ManifoldCF SearchComponent to Solr, and I see that my indexed
>> documents are getting allow_token_share, allow_token_parent,
>> allow_token_share, etc. tokens added.
>>
>> But when I query with the MCF plugin added and the
>> AuthenticatedUserName parameter added, I never get any results.
>>
>> I tried just with with username "Fred" and I see this in the solr logs:
>>
>> 2017-10-29 00:18:51.527 INFO (qtp834133664-16) [ ]
>> o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for
>> 2147483647\
>> transient cores
>> 2017-10-29 00:18:52.742 INFO (qtp834133664-15) [ ]
>> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
>> params={indexInfo=fa\
>> lse&wt=json&_=1509236332203} status=0 QTime=6
>> 2017-10-29 00:18:53.009 INFO (qtp834133664-11) [ ]
>> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system
>> params={wt=jso\
>> n&_=1509236332206} status=0 QTime=201
>> 2017-10-29 00:19:14.349 INFO (qtp834133664-16) [ x:gettingstarted]
>> o.a.s.m.ManifoldCFSearchComponent Trying to match docs for user\
>> '[:Fred]'
>> 2017-10-29 00:19:14.476 INFO (qtp834133664-16) [ x:gettingstarted]
>> o.a.s.m.ManifoldCFSearchComponent Saw authority response AUTHOR\
>> IZED:Null+authority+connection+for+testing
>> 2017-10-29 00:19:14.529 INFO (qtp834133664-16) [ x:gettingstarted]
>> o.a.s.c.S.Request [gettingstarted] webapp=/solr path=/select p\
>> arams={q=*:*&AuthenticatedUserName=Fred&indent=on&wt=xml&_=1509236332558}
>> hits=0 status=0 QTime=228
>>
>> I can tell Solr is talking to the MCF authority service, because
>> "Null+authority+connection+for+testing" is the description I used on
>> the Manifold side.
>>
>> There are documents in the index that include fields like this:
>>
>> <doc> <arr name="allow_token_document"> <str>Null:Fred</str> </arr>
>> <arr name="title"> <str/> </arr> <str
>> name="id">http://rss.cnn.com/~r/rss/cnn_world/~3/iTYAcfUavzM/orig-burger-king-bullying.cnn</str>
>> <arr name="deny_token_document"> <str>Null:DEAD_AUTHORITY</str> </arr>
>> <str name="stream_content_type">text/html; charset=utf-8</str> <str
>> name="keywords">world, Burger King stands up to bullying - CNN
>> Video</str> <str name="description">Burger King creates a PSA that
>> asks their customers to take a closer look at bullying. </str> <str
>> name="stream_name">docname</str> <str name="dc_title">Burger King
>> stands up to bullying - CNN Video</str> <arr name="content_type">
>> <str>text/html; charset=UTF-8</str> </arr> <long
>> name="stream_size">489145</long> <str
>> name="x_parsed_by">org.apache.tika.parser.DefaultParser
>> org.apache.tika.parser.html.HtmlParser</str> <str
>> name="stream_source_info">docname</str> <str
>> name="resourcename">docname</str> <str
>> name="fb_app_id">80401312489</str> <arr name="deny_token_parent">
>> <str>__no_security__</str> </arr> <arr name="allow_token_share">
>> <str>__no_security__</str> </arr> <arr name="deny_token_share">
>> <str>__no_security__</str> </arr> <arr name="allow_token_parent">
>> <str>__no_security__</str> </arr>
>> ...
>> ...
>> </doc>
>>
>>
>> But nonetheless, no results are returned. I'm sure I'm missing
>> something obvious here, but whatever it is is defeating me at the
>> moment.
>>
>> The only thing I see that looks a little dodgy is this "Trying to
>> match docs for user '[:Fred]'" given that the tokens look like
>> "Null:Fred".
>>
>>
>> Any ideas what the problem could be?
>>
>>
>>
>>
>> Thanks,
>>
>>
>> Phil
Re: Problem with Solr/ManifoldCF security filtering
Posted by Phillip Rhodes <mo...@gmail.com>.
Just to follow up on this: if I hand craft a query to the MCF
authority service that looks like this:
http://manifoldcf.aws:8345/mcf-authority-service/UserACLs?username=Fred
I get back
AUTHORIZED:Null+authority+connection+for+testing
TOKEN:Null:Fred
which looks right to me, given what I know about this so far.
And "Null:Fred" matches what is getting put into the Solr documents.
Thanks,
Phil
This message optimized for indexing by NSA PRISM
On Sat, Oct 28, 2017 at 8:36 PM, Phillip Rhodes
<mo...@gmail.com> wrote:
> MCF Gang:
>
> I've followed the instructions in the "ManifoldCF in Action" docs to
> setup security integration between ManifoldCF and Solr. I've added
> the ManifoldCF SearchComponent to Solr, and I see that my indexed
> documents are getting allow_token_share, allow_token_parent,
> allow_token_share, etc. tokens added.
>
> But when I query with the MCF plugin added and the
> AuthenticatedUserName parameter added, I never get any results.
>
> I tried just with with username "Fred" and I see this in the solr logs:
>
> 2017-10-29 00:18:51.527 INFO (qtp834133664-16) [ ]
> o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for
> 2147483647\
> transient cores
> 2017-10-29 00:18:52.742 INFO (qtp834133664-15) [ ]
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
> params={indexInfo=fa\
> lse&wt=json&_=1509236332203} status=0 QTime=6
> 2017-10-29 00:18:53.009 INFO (qtp834133664-11) [ ]
> o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/info/system
> params={wt=jso\
> n&_=1509236332206} status=0 QTime=201
> 2017-10-29 00:19:14.349 INFO (qtp834133664-16) [ x:gettingstarted]
> o.a.s.m.ManifoldCFSearchComponent Trying to match docs for user\
> '[:Fred]'
> 2017-10-29 00:19:14.476 INFO (qtp834133664-16) [ x:gettingstarted]
> o.a.s.m.ManifoldCFSearchComponent Saw authority response AUTHOR\
> IZED:Null+authority+connection+for+testing
> 2017-10-29 00:19:14.529 INFO (qtp834133664-16) [ x:gettingstarted]
> o.a.s.c.S.Request [gettingstarted] webapp=/solr path=/select p\
> arams={q=*:*&AuthenticatedUserName=Fred&indent=on&wt=xml&_=1509236332558}
> hits=0 status=0 QTime=228
>
> I can tell Solr is talking to the MCF authority service, because
> "Null+authority+connection+for+testing" is the description I used on
> the Manifold side.
>
> There are documents in the index that include fields like this:
>
> <doc> <arr name="allow_token_document"> <str>Null:Fred</str> </arr>
> <arr name="title"> <str/> </arr> <str
> name="id">http://rss.cnn.com/~r/rss/cnn_world/~3/iTYAcfUavzM/orig-burger-king-bullying.cnn</str>
> <arr name="deny_token_document"> <str>Null:DEAD_AUTHORITY</str> </arr>
> <str name="stream_content_type">text/html; charset=utf-8</str> <str
> name="keywords">world, Burger King stands up to bullying - CNN
> Video</str> <str name="description">Burger King creates a PSA that
> asks their customers to take a closer look at bullying. </str> <str
> name="stream_name">docname</str> <str name="dc_title">Burger King
> stands up to bullying - CNN Video</str> <arr name="content_type">
> <str>text/html; charset=UTF-8</str> </arr> <long
> name="stream_size">489145</long> <str
> name="x_parsed_by">org.apache.tika.parser.DefaultParser
> org.apache.tika.parser.html.HtmlParser</str> <str
> name="stream_source_info">docname</str> <str
> name="resourcename">docname</str> <str
> name="fb_app_id">80401312489</str> <arr name="deny_token_parent">
> <str>__no_security__</str> </arr> <arr name="allow_token_share">
> <str>__no_security__</str> </arr> <arr name="deny_token_share">
> <str>__no_security__</str> </arr> <arr name="allow_token_parent">
> <str>__no_security__</str> </arr>
> ...
> ...
> </doc>
>
>
> But nonetheless, no results are returned. I'm sure I'm missing
> something obvious here, but whatever it is is defeating me at the
> moment.
>
> The only thing I see that looks a little dodgy is this "Trying to
> match docs for user '[:Fred]'" given that the tokens look like
> "Null:Fred".
>
>
> Any ideas what the problem could be?
>
>
>
>
> Thanks,
>
>
> Phil