You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@nutch.apache.org by 高睿 <ga...@163.com> on 2012/12/19 13:51:27 UTC
What's the different between marker and metadata?
Hi,
What's the different between marker and metadata?
public class WebPage extends PersistentBase {
//...
public Utf8 getFromMarkers(Utf8 key) {
if (markers == null) { return null; }
return markers.get(key);
}
public void putToMarkers(Utf8 key, Utf8 value) {
getStateManager().setDirty(this, 20);
markers.put(key, value);
}
//...
public ByteBuffer getFromMetadata(Utf8 key) {
if (metadata == null) { return null; }
return metadata.get(key);
}
public void putToMetadata(Utf8 key, ByteBuffer value) {
getStateManager().setDirty(this, 21);
metadata.put(key, value);
}
//...
}
Re: What's the different between marker and metadata?
Posted by Ferdy Galema <fe...@kalooga.com>.
Late reply, so this one is for the record:
Markers are used for controlling the set of urls for processing:
Generator --> Sets fetch markers for the Fetcher.
Fetcher --> Reads the urls markers and checks if it needs to fetch them.
etc.
http://wiki.apache.org/nutch/Nutch2Crawling
On Wed, Dec 19, 2012 at 1:51 PM, 高睿 <ga...@163.com> wrote:
> Hi,
>
> What's the different between marker and metadata?
>
> public class WebPage extends PersistentBase {
> //...
> public Utf8 getFromMarkers(Utf8 key) {
> if (markers == null) { return null; }
> return markers.get(key);
> }
> public void putToMarkers(Utf8 key, Utf8 value) {
> getStateManager().setDirty(this, 20);
> markers.put(key, value);
> }
>
> //...
> public ByteBuffer getFromMetadata(Utf8 key) {
> if (metadata == null) { return null; }
> return metadata.get(key);
> }
> public void putToMetadata(Utf8 key, ByteBuffer value) {
> getStateManager().setDirty(this, 21);
> metadata.put(key, value);
> }
> //...
> }
>
>
--
*Ferdy Galema*
Kalooga Development
--
*Kalooga* | Visual RelevanceCheck out our Visual Gallery Layer now!<http://spitsnieuws.nl/archives/entertainment/2012/12/huis-amy-winehouse-levert-weinig-op>
Kalooga
Helperpark 288
9723 ZA Groningen
The Netherlands
+31 50 2103400
www.kalooga.com
info@kalooga.comKalooga EMEA
53 Davies Street
W1K 5JH London
United Kingdom
+44 20 7129 1430Kalooga Spain and LatAM
Maria de Sevilla Diago No 3
28022 Madrid - Madrid
Spain
+34 670 580 872