DIH - URLDataSource import size

[Raheel Hasan <ra...@gmail.com>]

Hi,

I have an issue that is only coming on live environment. The DIH
with URLDataSource is not working when the file size imported is large
(i.e. 100kb above - which is not so large). If its large, it returns
nothing (as seen in the Debug section of DataImport at Solr Admin).

However, when working on local environment, this issue doesnt come at all.

(note that I am using it with URLDataSource with PlainTextEntityProcessor
in the entity field).

Please help me as I tried to get it done a lot, but cant !!

Thanks a lot.

-- 
Regards,
Raheel Hasan

Re: DIH - URLDataSource import size

[Chris Hostetter <ho...@fucit.org>]

: I have an issue that is only coming on live environment. The DIH
: with URLDataSource is not working when the file size imported is large
: (i.e. 100kb above - which is not so large). If its large, it returns
: nothing (as seen in the Debug section of DataImport at Solr Admin).

are you sure the problem is the *size* of the file you are trying to load, 
and not something related to network topology?

if a remote URL fetch works in dev, but not in production my first guess 
would be that the URL you are trying to fetch is not reachable by your 
production server due to some sort of firewall ... can you actaully 
confirm that a smaller file -- fetched from the same server -- works in 
production?


-Hoss

Re: DIH - URLDataSource import size

[Shalin Shekhar Mangar <sh...@gmail.com>]

Following up within 15 hours is not going to do any good -- it just
increases email traffic for everyone. Please understand that a lot of
people here are in different time zones and almost all of them are
volunteers answering questions in addition to their day jobs.

Are there any exceptions in the logs of your production environment?


On Wed, Oct 23, 2013 at 1:10 PM, Raheel Hasan <ra...@gmail.com>wrote:

> anyone?
>
>
> On Tue, Oct 22, 2013 at 9:50 PM, Raheel Hasan <raheelhasan.fsd@gmail.com
> >wrote:
>
> > Hi,
> >
> > I have an issue that is only coming on live environment. The DIH
> > with URLDataSource is not working when the file size imported is large
> > (i.e. 100kb above - which is not so large). If its large, it returns
> > nothing (as seen in the Debug section of DataImport at Solr Admin).
> >
> > However, when working on local environment, this issue doesnt come at
> all.
> >
> > (note that I am using it with URLDataSource with PlainTextEntityProcessor
> > in the entity field).
> >
> > Please help me as I tried to get it done a lot, but cant !!
> >
> > Thanks a lot.
> >
> > --
> > Regards,
> > Raheel Hasan
> >
>
>
>
> --
> Regards,
> Raheel Hasan
>



-- 
Regards,
Shalin Shekhar Mangar.

Re: DIH - URLDataSource import size

[Raheel Hasan <ra...@gmail.com>]

anyone?


On Tue, Oct 22, 2013 at 9:50 PM, Raheel Hasan <ra...@gmail.com>wrote:

> Hi,
>
> I have an issue that is only coming on live environment. The DIH
> with URLDataSource is not working when the file size imported is large
> (i.e. 100kb above - which is not so large). If its large, it returns
> nothing (as seen in the Debug section of DataImport at Solr Admin).
>
> However, when working on local environment, this issue doesnt come at all.
>
> (note that I am using it with URLDataSource with PlainTextEntityProcessor
> in the entity field).
>
> Please help me as I tried to get it done a lot, but cant !!
>
> Thanks a lot.
>
> --
> Regards,
> Raheel Hasan
>



-- 
Regards,
Raheel Hasan