You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Jordi Domingo Borràs <jo...@gmail.com> on 2017/04/05 12:45:58 UTC
Streaming Expression strange behaviour
Hi!
I'm new to Streaming Expressions (running solr cloud 6.5.0) and I'm facing
a weird behaviour that I don't understand. I'm able to make a join and
retrieve results:
select(
innerJoin(
select(
search(iocs, q=ip:"10.153.9.175", fl="ip,indicator, type", sort="ip asc"),
indicator AS t1_indicator,
type AS t1_type,
ip AS t1_ip
),
select(
search(iocs2, q=ip:"10.153.9.175", fl="ip,indicator, type", sort="ip asc"),
indicator AS t2_indicator,
type AS t2_type,
ip AS t2_ip
),
on="t1_ip=t2_ip"
),
t1_indicator as t1_indicator,
t2_indicator as t2_indicator,
t1_type as t1_type,
t2_type as t2_type
)
The problem comes when I try to make the same query against all data:
select(
innerJoin(
select(
search(iocs, *q=*:**, fl="ip,indicator, type", sort="ip asc"),
indicator AS t1_indicator,
type AS t1_type,
ip AS t1_ip
),
select(
search(iocs2, *q=*:**, fl="ip,indicator, type", sort="ip asc"),
indicator AS t2_indicator,
type AS t2_type,
ip AS t2_ip
),
on="t1_ip=t2_ip"
),
t1_indicator as t1_indicator,
t2_indicator as t2_indicator,
t1_type as t1_type,
t2_type as t2_type
)
This time the result is:
{ "result-set": { "docs": [ { "EOF": true, "RESPONSE_TIME": 7 } ] } }
When I try to execute any of the searches alone (i.e. search(iocs, *q=*:**,
fl="ip,indicator, type", sort="ip asc")), I get lots of results.
Any help is really appreciated. Thanks,
Jordi
Re: Streaming Expression strange behaviour
Posted by Joel Bernstein <jo...@gmail.com>.
Yeah, that's a really common mistake.
I think the plan is to make a separate "shuffle" expression that always
uses the /export handler. That way people aren't dealing with Solr request
handler decisions, they just choose the proper function for the task.Then
we can also make the search expression more user friendly, not require a
sort etc...
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Apr 5, 2017 at 9:12 AM, Jordi Domingo Borràs <
jordi.domingo@gmail.com> wrote:
> Hi there,
>
> The problem with my query is that it's using the select endpoint by default
> (instead of export). The select endpoint returns a limited number of rows
> so I don't get any results. The fastest option (not for production) is to
> include the number of desired rows in the search query:
>
> search(iocs, q=*:*, fl="ip,indicator, type", sort="ip asc", rows=100000)
>
> Thanks,
> Jordi
>
> On Wed, Apr 5, 2017 at 2:45 PM, Jordi Domingo Borràs <
> jordi.domingo@gmail.com> wrote:
>
> > Hi!
> >
> > I'm new to Streaming Expressions (running solr cloud 6.5.0) and I'm
> facing
> > a weird behaviour that I don't understand. I'm able to make a join and
> > retrieve results:
> >
> > select(
> > innerJoin(
> > select(
> > search(iocs, q=ip:"10.153.9.175", fl="ip,indicator, type", sort="ip
> asc"),
> > indicator AS t1_indicator,
> > type AS t1_type,
> > ip AS t1_ip
> > ),
> > select(
> > search(iocs2, q=ip:"10.153.9.175", fl="ip,indicator, type", sort="ip
> > asc"),
> > indicator AS t2_indicator,
> > type AS t2_type,
> > ip AS t2_ip
> > ),
> > on="t1_ip=t2_ip"
> > ),
> > t1_indicator as t1_indicator,
> > t2_indicator as t2_indicator,
> > t1_type as t1_type,
> > t2_type as t2_type
> > )
> >
> > The problem comes when I try to make the same query against all data:
> >
> > select(
> > innerJoin(
> > select(
> > search(iocs, *q=*:**, fl="ip,indicator, type", sort="ip asc"),
> > indicator AS t1_indicator,
> > type AS t1_type,
> > ip AS t1_ip
> > ),
> > select(
> > search(iocs2, *q=*:**, fl="ip,indicator, type", sort="ip asc"),
> > indicator AS t2_indicator,
> > type AS t2_type,
> > ip AS t2_ip
> > ),
> > on="t1_ip=t2_ip"
> > ),
> > t1_indicator as t1_indicator,
> > t2_indicator as t2_indicator,
> > t1_type as t1_type,
> > t2_type as t2_type
> > )
> >
> > This time the result is:
> >
> > { "result-set": { "docs": [ { "EOF": true, "RESPONSE_TIME": 7 } ] } }
> >
> > When I try to execute any of the searches alone (i.e. search(iocs,
> *q=*:**,
> > fl="ip,indicator, type", sort="ip asc")), I get lots of results.
> >
> > Any help is really appreciated. Thanks,
> >
> > Jordi
> >
>
Re: Streaming Expression strange behaviour
Posted by Jordi Domingo Borràs <jo...@gmail.com>.
Hi there,
The problem with my query is that it's using the select endpoint by default
(instead of export). The select endpoint returns a limited number of rows
so I don't get any results. The fastest option (not for production) is to
include the number of desired rows in the search query:
search(iocs, q=*:*, fl="ip,indicator, type", sort="ip asc", rows=100000)
Thanks,
Jordi
On Wed, Apr 5, 2017 at 2:45 PM, Jordi Domingo Borràs <
jordi.domingo@gmail.com> wrote:
> Hi!
>
> I'm new to Streaming Expressions (running solr cloud 6.5.0) and I'm facing
> a weird behaviour that I don't understand. I'm able to make a join and
> retrieve results:
>
> select(
> innerJoin(
> select(
> search(iocs, q=ip:"10.153.9.175", fl="ip,indicator, type", sort="ip asc"),
> indicator AS t1_indicator,
> type AS t1_type,
> ip AS t1_ip
> ),
> select(
> search(iocs2, q=ip:"10.153.9.175", fl="ip,indicator, type", sort="ip
> asc"),
> indicator AS t2_indicator,
> type AS t2_type,
> ip AS t2_ip
> ),
> on="t1_ip=t2_ip"
> ),
> t1_indicator as t1_indicator,
> t2_indicator as t2_indicator,
> t1_type as t1_type,
> t2_type as t2_type
> )
>
> The problem comes when I try to make the same query against all data:
>
> select(
> innerJoin(
> select(
> search(iocs, *q=*:**, fl="ip,indicator, type", sort="ip asc"),
> indicator AS t1_indicator,
> type AS t1_type,
> ip AS t1_ip
> ),
> select(
> search(iocs2, *q=*:**, fl="ip,indicator, type", sort="ip asc"),
> indicator AS t2_indicator,
> type AS t2_type,
> ip AS t2_ip
> ),
> on="t1_ip=t2_ip"
> ),
> t1_indicator as t1_indicator,
> t2_indicator as t2_indicator,
> t1_type as t1_type,
> t2_type as t2_type
> )
>
> This time the result is:
>
> { "result-set": { "docs": [ { "EOF": true, "RESPONSE_TIME": 7 } ] } }
>
> When I try to execute any of the searches alone (i.e. search(iocs, *q=*:**,
> fl="ip,indicator, type", sort="ip asc")), I get lots of results.
>
> Any help is really appreciated. Thanks,
>
> Jordi
>