solr for reporting purposes

[adeelmahmood <ad...@gmail.com>]

we are trying to use solr for somewhat of a reporting system too (along with
search) .. since it provides such amazing control over queries and basically
over the data that user wants .. they might as well be able to dump that
data in an excel file too if needed .. our data isnt too much close to 25K
docs with 15-20 fields in each doc .. and mostly these reports will be for
close to 500 - 4000 records .. i am thinking about setting up a simple
servlet that grabs all this data that submits the user query to solr over
http .. grabs all that results data and dumps it in an excel file .. i was
just hoping to get some idea of whether this is going to cause any
performance impact on solr search .. especially since its all on the same
server and some users will be doing reports while others will be searching
.. right now search is working GREAT .. its blazing fast .. i dont wanna
loose this but at the same time reporting is an important requirement as
well .. 

also i would appreciate any hints towards some creative ways of doing it ..
something like getting 500 some records in a single request and then using
some timer task repeat the process .. 

thanks for ur help
-- 
View this message in context: http://old.nabble.com/solr-for-reporting-purposes-tp27725967p27725967.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: solr for reporting purposes

[adeelmahmood <ad...@gmail.com>]

I just want to clarify if its not obvious .. that the reason I am concerned
about the performance of solr is becaues for reporting requests I will
probably have to request all result rows at the same time .. instead of 10
or 20


adeelmahmood wrote:
> 
> we are trying to use solr for somewhat of a reporting system too (along
> with search) .. since it provides such amazing control over queries and
> basically over the data that user wants .. they might as well be able to
> dump that data in an excel file too if needed .. our data isnt too much
> close to 25K docs with 15-20 fields in each doc .. and mostly these
> reports will be for close to 500 - 4000 records .. i am thinking about
> setting up a simple servlet that grabs all this data that submits the user
> query to solr over http .. grabs all that results data and dumps it in an
> excel file .. i was just hoping to get some idea of whether this is going
> to cause any performance impact on solr search .. especially since its all
> on the same server and some users will be doing reports while others will
> be searching .. right now search is working GREAT .. its blazing fast .. i
> dont wanna loose this but at the same time reporting is an important
> requirement as well .. 
> 
> also i would appreciate any hints towards some creative ways of doing it
> .. something like getting 500 some records in a single request and then
> using some timer task repeat the process .. 
> 
> thanks for ur help
> 

-- 
View this message in context: http://old.nabble.com/solr-for-reporting-purposes-tp27725967p27726016.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: solr for reporting purposes

[adeelmahmood <ad...@gmail.com>]

well thanks for ur reply .. as far as the load goes again I think most of the
reports will be for 1000-4000 records and we dont have that many users ..
its an internal system so we have about 400 users per day and we are opening
this up for only half of those people (a specific role of people) .. so
close to 200 people could potentially use it .. so practially speaking i
think we can have up to 50 requests at a given time .. but again since its
reports they are gonna be needed every day .. once you get a report you have
it for a while .. so overall i dont think its that much of user load that we
have .. what do you think

also i was thinking about handling requests in a 500 records limit fashion
.. so a request for 2000 records will be handled as 5 separate (refresh by a
5 sec timeout) requests .. do you think thats a good idea to ask solr to
return 500 rows at a time but make that request 5 times .. or its better to
just ask for 2000 rows alltogether 



Ron Chan wrote:
> 
> we've done it successfully for similar requirements 
> 
> the resource requirements depends on how many concurrent people will be
> running those types of reports 
> 
> up to 4000 records is not a problem at all, one report at a time, but if
> you had concurrent requests running into thousands as well then you may
> have a problem, although you will probably run into memory problems at the
> rendering end before you have problems with Solr, i.e. not a Solr problem
> as such, but a problem generally of unrestricted adhoc reporting 
> 
> 
> 
> 
> ----- Original Message ----- 
> From: "adeelmahmood" <ad...@gmail.com> 
> To: solr-user@lucene.apache.org 
> Sent: Saturday, 27 February, 2010 5:57:00 AM 
> Subject: Re: solr for reporting purposes 
> 
> 
> I just want to clarify if its not obvious .. that the reason I am
> concerned 
> about the performance of solr is becaues for reporting requests I will 
> probably have to request all result rows at the same time .. instead of 10 
> or 20 
> 
> 
> adeelmahmood wrote: 
>> 
>> we are trying to use solr for somewhat of a reporting system too (along 
>> with search) .. since it provides such amazing control over queries and 
>> basically over the data that user wants .. they might as well be able to 
>> dump that data in an excel file too if needed .. our data isnt too much 
>> close to 25K docs with 15-20 fields in each doc .. and mostly these 
>> reports will be for close to 500 - 4000 records .. i am thinking about 
>> setting up a simple servlet that grabs all this data that submits the
>> user 
>> query to solr over http .. grabs all that results data and dumps it in an 
>> excel file .. i was just hoping to get some idea of whether this is going 
>> to cause any performance impact on solr search .. especially since its
>> all 
>> on the same server and some users will be doing reports while others will 
>> be searching .. right now search is working GREAT .. its blazing fast ..
>> i 
>> dont wanna loose this but at the same time reporting is an important 
>> requirement as well .. 
>> 
>> also i would appreciate any hints towards some creative ways of doing it 
>> .. something like getting 500 some records in a single request and then 
>> using some timer task repeat the process .. 
>> 
>> thanks for ur help 
>> 
> 
> -- 
> View this message in context:
> http://old.nabble.com/solr-for-reporting-purposes-tp27725967p27726016.html 
> Sent from the Solr - User mailing list archive at Nabble.com. 
> 
> 
> 

-- 
View this message in context: http://old.nabble.com/solr-for-reporting-purposes-tp27725967p27743896.html
Sent from the Solr - User mailing list archive at Nabble.com.