You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by Weder Carlos Vieira <we...@gmail.com> on 2013/07/30 16:28:27 UTC
Help with 'read data'
Hello everyone,
I would like to say that Nutch 2.2.1 is working very well.
I spent the last few days testing this new version, I liked a lot,
congratulations.
Now I would like to receive some tips of you, I want to create a new
website interface to read the urls crawled, parsed and saved on the
database and show its contents on the pages.
My doubt is, what the best way that I can read this data?
What can I use for middle way, between database and my application, to
facilitate the selection of data and obtain the best results.
Could you share with me some stuff to read, some tips, some experience? how
can I design this structure?
Thanks.
Re: Help with 'read data'
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Great.
On Tue, Jul 30, 2013 at 9:16 AM, Weder Carlos Vieira <weder.vierra@gmail.com
> wrote:
> Hello Lewis,
>
> I changed the ivy.xml GORA 0.3 to 0.2.1 version.
>
>
> Weder
>
>
> On Tue, Jul 30, 2013 at 1:09 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
> > Hi,
> > AFAIK, MySQL is not compatible with Nutch 2.2.1... how did you get your
> > data in there?
> > Regarding your application... LAMP springs to mind like a shot in the
> arm.
> > Lewis
> >
> > On Tuesday, July 30, 2013, Weder Carlos Vieira <we...@gmail.com>
> > wrote:
> > > Thanks Rerngvit,
> > >
> > > I forget to say that my database is MySQL.
> > >
> > > The main doubt I think is, should I make direct selects on the
> databases
> > > through my application?
> > > Or there is a way to use solr on the middle, between my application and
> > > MySQL, to manage researchs?
> > >
> > > Sorry, I really have many doubts, I hope to clear my ideas.
> > >
> > >
> > > Thanks again...
> > > Weder
> > >
> > >
> > > On Tue, Jul 30, 2013 at 11:58 AM, Rerngvit Yanggratoke <
> > > rerngvit@n3infinity.com> wrote:
> > >
> > >> If you are using Nutch 2.2.1, the crawled data is already stored in a
> > >> Nosql database, e.g., Apache HBase. What you need to develop is a
> client
> > >> code that reads data out of this database. You probably would need to
> > >> understand how fields are stored. I recommend have a look in the
> mapping
> > >> configuration file, e.g., conf/gora-hbase-mapping.xml.
> > >>
> > >> On Jul 30, 2556 BE, at 4:28 PM, Weder Carlos Vieira wrote:
> > >>
> > >> > Hello everyone,
> > >> >
> > >> > I would like to say that Nutch 2.2.1 is working very well.
> > >> > I spent the last few days testing this new version, I liked a lot,
> > >> > congratulations.
> > >> >
> > >> > Now I would like to receive some tips of you, I want to create a new
> > >> > website interface to read the urls crawled, parsed and saved on the
> > >> > database and show its contents on the pages.
> > >> >
> > >> > My doubt is, what the best way that I can read this data?
> > >> > What can I use for middle way, between database and my application,
> to
> > >> > facilitate the selection of data and obtain the best results.
> > >> >
> > >> >
> > >> > Could you share with me some stuff to read, some tips, some
> > experience?
> > >> how
> > >> > can I design this structure?
> > >> >
> > >> >
> > >> > Thanks.
> > >>
> > >>
> > >
> >
> > --
> > *Lewis*
> >
>
--
*Lewis*
Re: Help with 'read data'
Posted by Weder Carlos Vieira <we...@gmail.com>.
Hello Lewis,
I changed the ivy.xml GORA 0.3 to 0.2.1 version.
Weder
On Tue, Jul 30, 2013 at 1:09 PM, Lewis John Mcgibbney <
lewis.mcgibbney@gmail.com> wrote:
> Hi,
> AFAIK, MySQL is not compatible with Nutch 2.2.1... how did you get your
> data in there?
> Regarding your application... LAMP springs to mind like a shot in the arm.
> Lewis
>
> On Tuesday, July 30, 2013, Weder Carlos Vieira <we...@gmail.com>
> wrote:
> > Thanks Rerngvit,
> >
> > I forget to say that my database is MySQL.
> >
> > The main doubt I think is, should I make direct selects on the databases
> > through my application?
> > Or there is a way to use solr on the middle, between my application and
> > MySQL, to manage researchs?
> >
> > Sorry, I really have many doubts, I hope to clear my ideas.
> >
> >
> > Thanks again...
> > Weder
> >
> >
> > On Tue, Jul 30, 2013 at 11:58 AM, Rerngvit Yanggratoke <
> > rerngvit@n3infinity.com> wrote:
> >
> >> If you are using Nutch 2.2.1, the crawled data is already stored in a
> >> Nosql database, e.g., Apache HBase. What you need to develop is a client
> >> code that reads data out of this database. You probably would need to
> >> understand how fields are stored. I recommend have a look in the mapping
> >> configuration file, e.g., conf/gora-hbase-mapping.xml.
> >>
> >> On Jul 30, 2556 BE, at 4:28 PM, Weder Carlos Vieira wrote:
> >>
> >> > Hello everyone,
> >> >
> >> > I would like to say that Nutch 2.2.1 is working very well.
> >> > I spent the last few days testing this new version, I liked a lot,
> >> > congratulations.
> >> >
> >> > Now I would like to receive some tips of you, I want to create a new
> >> > website interface to read the urls crawled, parsed and saved on the
> >> > database and show its contents on the pages.
> >> >
> >> > My doubt is, what the best way that I can read this data?
> >> > What can I use for middle way, between database and my application, to
> >> > facilitate the selection of data and obtain the best results.
> >> >
> >> >
> >> > Could you share with me some stuff to read, some tips, some
> experience?
> >> how
> >> > can I design this structure?
> >> >
> >> >
> >> > Thanks.
> >>
> >>
> >
>
> --
> *Lewis*
>
Re: Help with 'read data'
Posted by Lewis John Mcgibbney <le...@gmail.com>.
Hi,
AFAIK, MySQL is not compatible with Nutch 2.2.1... how did you get your
data in there?
Regarding your application... LAMP springs to mind like a shot in the arm.
Lewis
On Tuesday, July 30, 2013, Weder Carlos Vieira <we...@gmail.com>
wrote:
> Thanks Rerngvit,
>
> I forget to say that my database is MySQL.
>
> The main doubt I think is, should I make direct selects on the databases
> through my application?
> Or there is a way to use solr on the middle, between my application and
> MySQL, to manage researchs?
>
> Sorry, I really have many doubts, I hope to clear my ideas.
>
>
> Thanks again...
> Weder
>
>
> On Tue, Jul 30, 2013 at 11:58 AM, Rerngvit Yanggratoke <
> rerngvit@n3infinity.com> wrote:
>
>> If you are using Nutch 2.2.1, the crawled data is already stored in a
>> Nosql database, e.g., Apache HBase. What you need to develop is a client
>> code that reads data out of this database. You probably would need to
>> understand how fields are stored. I recommend have a look in the mapping
>> configuration file, e.g., conf/gora-hbase-mapping.xml.
>>
>> On Jul 30, 2556 BE, at 4:28 PM, Weder Carlos Vieira wrote:
>>
>> > Hello everyone,
>> >
>> > I would like to say that Nutch 2.2.1 is working very well.
>> > I spent the last few days testing this new version, I liked a lot,
>> > congratulations.
>> >
>> > Now I would like to receive some tips of you, I want to create a new
>> > website interface to read the urls crawled, parsed and saved on the
>> > database and show its contents on the pages.
>> >
>> > My doubt is, what the best way that I can read this data?
>> > What can I use for middle way, between database and my application, to
>> > facilitate the selection of data and obtain the best results.
>> >
>> >
>> > Could you share with me some stuff to read, some tips, some experience?
>> how
>> > can I design this structure?
>> >
>> >
>> > Thanks.
>>
>>
>
--
*Lewis*
Re: Help with 'read data'
Posted by Weder Carlos Vieira <we...@gmail.com>.
Thanks Rerngvit,
I forget to say that my database is MySQL.
The main doubt I think is, should I make direct selects on the databases
through my application?
Or there is a way to use solr on the middle, between my application and
MySQL, to manage researchs?
Sorry, I really have many doubts, I hope to clear my ideas.
Thanks again...
Weder
On Tue, Jul 30, 2013 at 11:58 AM, Rerngvit Yanggratoke <
rerngvit@n3infinity.com> wrote:
> If you are using Nutch 2.2.1, the crawled data is already stored in a
> Nosql database, e.g., Apache HBase. What you need to develop is a client
> code that reads data out of this database. You probably would need to
> understand how fields are stored. I recommend have a look in the mapping
> configuration file, e.g., conf/gora-hbase-mapping.xml.
>
> On Jul 30, 2556 BE, at 4:28 PM, Weder Carlos Vieira wrote:
>
> > Hello everyone,
> >
> > I would like to say that Nutch 2.2.1 is working very well.
> > I spent the last few days testing this new version, I liked a lot,
> > congratulations.
> >
> > Now I would like to receive some tips of you, I want to create a new
> > website interface to read the urls crawled, parsed and saved on the
> > database and show its contents on the pages.
> >
> > My doubt is, what the best way that I can read this data?
> > What can I use for middle way, between database and my application, to
> > facilitate the selection of data and obtain the best results.
> >
> >
> > Could you share with me some stuff to read, some tips, some experience?
> how
> > can I design this structure?
> >
> >
> > Thanks.
>
>
Re: Help with 'read data'
Posted by Rerngvit Yanggratoke <re...@n3infinity.com>.
If you are using Nutch 2.2.1, the crawled data is already stored in a Nosql database, e.g., Apache HBase. What you need to develop is a client code that reads data out of this database. You probably would need to understand how fields are stored. I recommend have a look in the mapping configuration file, e.g., conf/gora-hbase-mapping.xml.
On Jul 30, 2556 BE, at 4:28 PM, Weder Carlos Vieira wrote:
> Hello everyone,
>
> I would like to say that Nutch 2.2.1 is working very well.
> I spent the last few days testing this new version, I liked a lot,
> congratulations.
>
> Now I would like to receive some tips of you, I want to create a new
> website interface to read the urls crawled, parsed and saved on the
> database and show its contents on the pages.
>
> My doubt is, what the best way that I can read this data?
> What can I use for middle way, between database and my application, to
> facilitate the selection of data and obtain the best results.
>
>
> Could you share with me some stuff to read, some tips, some experience? how
> can I design this structure?
>
>
> Thanks.