number of block duplicated

[Stefan Groschupf <sg...@media-style.com>]

Hi,
I remember that I heard it is possible to change the number of block  
duplicates somehow, but didn't find any property in the nutch- 
default.xml.
Can someone give the lost link to this property?

Thanks.
Stefan 

Link index & page content obtained separately.

[Pashabhai <pa...@yahoo.com>]

Hi ,

Can anybody please  explain how indexes and page
content are linked together.

In a situation, where only indexes are sent by third
party but not the page content.

can i fetch the page content using Fetcher and then
link index & page together.

Presently i am able to show search results without
snippet & cached page.


Thanks 
P



 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: number of block duplicated

[Doug Cutting <cu...@nutch.org>]

Dominik Friedrich wrote:
> Patch attached.

Thanks!

I've added some of these to nutch-default.xml.  I didn't add them all, 
since putting them in nutch-default is like declaring something public: 
we'll need to keep supporting that feature in the future.  Some of these 
are things that users should not change, or are things which might soon 
be removed.

Cheers,

Doug

Re: number of block duplicated

[Dominik Friedrich <do...@wipe-records.org>]

Patch attached. I haven't added any properties from plugins or test classes.

best regards,
Dominik

Stefan Groschupf schrieb:
>>
>> to my nutch-site.xml. I saw in this file and other files also 
>> properties that are not in nutch-default.xml and some are not even 
>> read from NutchConf but set directly in the source code, e.g. in 
>> mapred/MRConstants.java. I could go through the current code and 
>> create an updated nutch-site.xml with all properties some day next week.
>
> That would be great! !!
>
> Thanks!
>
>
>

Re: number of block duplicated

[Stefan Groschupf <sg...@media-style.com>]

>
> to my nutch-site.xml. I saw in this file and other files also  
> properties that are not in nutch-default.xml and some are not even  
> read from NutchConf but set directly in the source code, e.g. in  
> mapred/MRConstants.java. I could go through the current code and  
> create an updated nutch-site.xml with all properties some day next  
> week.

That would be great! !!

Thanks!

Re: number of block duplicated

[Dominik Friedrich <do...@wipe-records.org>]

In ndfs/FSNamesystem.java is a property ndfs.replication so I added

<property>
  <name>ndfs.replication</name>
  <value>2</value>
  <description>Number of replications of fs blocks</description>
</property>

to my nutch-site.xml. I saw in this file and other files also properties 
that are not in nutch-default.xml and some are not even read from 
NutchConf but set directly in the source code, e.g. in 
mapred/MRConstants.java. I could go through the current code and create 
an updated nutch-site.xml with all properties some day next week.

regards,
Dominik


Stefan Groschupf schrieb:
> Hi,
> I remember that I heard it is possible to change the number of block 
> duplicates somehow, but didn't find any property in the 
> nutch-default.xml.
> Can someone give the lost link to this property?
>
> Thanks.
> Stefan
>