Re: Re: Error: No agents listed in 'http.agent.name' property.

[fayazvf <fa...@gmail.com>]

hi,
  I am facing the same problem as well.I tried with copying property tag
containing http.agent.name into conf/nutch-site.xml. But i get the same
problem. Can u help me out?


--
View this message in context: http://lucene.472066.n3.nabble.com/Error-No-agents-listed-in-http-agent-name-property-tp1808332p2892972.html
Sent from the Nutch - User mailing list archive at Nabble.com.

RE: Re: Error: No agents listed in 'http.agent.name' property.

["McGibbney, Lewis John" <Le...@gcu.ac.uk>]

Hi,

If you provide the log output then we can see exactly what the message is. My initial thoughts are that this is possibly a WARN and not an error. For some reason, on some occasions, even if we do add the aforementioned property to nutch-site.xml it still seems to display this warning!
________________________________________
From: fayazvf [faizvf@gmail.com]
Sent: 03 May 2011 06:54
To: nutch-user@lucene.apache.org
Subject: Re: Re: Error: No agents listed in 'http.agent.name' property.

hi,
  I am facing the same problem as well.I tried with copying property tag
containing http.agent.name into conf/nutch-site.xml. But i get the same
problem. Can u help me out?


--
View this message in context: http://lucene.472066.n3.nabble.com/Error-No-agents-listed-in-http-agent-name-property-tp1808332p2892972.html
Sent from the Nutch - User mailing list archive at Nabble.com.

Email has been scanned for viruses by Altman Technologies' email management service - www.altman.co.uk/emailsystems

Glasgow Caledonian University is a registered Scottish charity, number SC021474

Winner: Times Higher Education’s Widening Participation Initiative of the Year 2009 and Herald Society’s Education Initiative of the Year 2009.
http://www.gcu.ac.uk/newsevents/news/bycategory/theuniversity/1/name,6219,en.html

Winner: Times Higher Education’s Outstanding Support for Early Career Researchers of the Year 2010, GCU as a lead with Universities Scotland partners.
http://www.gcu.ac.uk/newsevents/news/bycategory/theuniversity/1/name,15691,en.html

Re: Re: Error: No agents listed in 'http.agent.name' property.

[Luis Cappa Banda <lu...@gmail.com>]

Hello.

I think that I had the same problem some weeks ago. Try to resolve it
including in *nutch-site.xml* the next properties:


<property>
  <name>*http.agent.name*</name>
  <value>*YourCrawlerName*</value>
  <description>HTTP 'User-Agent' request header. MUST NOT be empty -
  please set this to a single word uniquely related to your organization.
  </description>
</property>

<property>
  <name>*http.robots.agents*</name>
  <value>*YourCrawlerName,**</value>
  <description>The agent strings we'll look for in robots.txt files,
  comma-separated, in decreasing order of precedence. You should
  put the value of http.agent.name as the first agent name, and keep the
  default * at the end of the list. E.g.: BlurflDev,Blurfl,*
  </description>
</property>



It's mandatory to include your *http.agent.name* into
*http.robots.agents*property.


Good luck!

Re: Re: Error: No agents listed in 'http.agent.name' property.

[Gabriele Kahlout <ga...@mysimpatico.com>]

Try prefixing your script /crawling command with:

$ xmlstarlet edit -L -u "/configuration/property[name='']"/value -v 'test'
conf/nutch-default.xml

$ xmlstarlet sel -t -c "/configuration/property[name='http.agent.name']"/value
conf/nutch-default.xml

After the second you should see printed:
<value>test</value>


On Tue, May 3, 2011 at 7:54 AM, fayazvf <fa...@gmail.com> wrote:

> hi,
>  I am facing the same problem as well.I tried with copying property tag
> containing http.agent.name into conf/nutch-site.xml. But i get the same
> problem. Can u help me out?
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Error-No-agents-listed-in-http-agent-name-property-tp1808332p2892972.html
> Sent from the Nutch - User mailing list archive at Nabble.com.
>



-- 
Regards,
K. Gabriele

--- unchanged since 20/9/10 ---
P.S. If the subject contains "[LON]" or the addressee acknowledges the
receipt within 48 hours then I don't resend the email.
subject(this) ∈ L(LON*) ∨ ∃x. (x ∈ MyInbox ∧ Acknowledges(x, this) ∧ time(x)
< Now + 48h) ⇒ ¬resend(I, this).

If an email is sent by a sender that is not a trusted contact or the email
does not contain a valid code then the email is not received. A valid code
starts with a hyphen and ends with "X".
∀x. x ∈ MyInbox ⇒ from(x) ∈ MySafeSenderList ∨ (∃y. y ∈ subject(x) ∧ y ∈
L(-[a-z]+[0-9]X)).