Debugging Nutch in Netbeans

[Kevin MacDonald <ke...@hautesecure.com>]

Does anyone have an ant script that would allow debugging into Nutch running
on Netbeans?

Thanks

Kevin

Re: Debugging Nutch in Netbeans

[Andrzej Bialecki <ab...@getopt.org>]

Kevin MacDonald wrote:
> I figured out how to debug into Nutch on Netbeans. This is a really good
> how-to on the subject if anyone's interested.
> http://www.netbeans.org/kb/articles/freeform-import.html
> 
> Something I am unable to do however, is debug very far into it. I am running
> on Vista, and when the code executes at some point it attempts to invoke
> "bash" and fails. Everything works fine at the command line using Cygwin,
> but when debugging, Netbeans is not running in the Cygwin shell, so an
> exception happens. Before I resign myself to printf debugging I am hoping
> that there is a way of configuring Nutch (or possibly hadoop) to not do
> whatever it is currently doing that causes the failure. If anyone has
> experience getting Nutch working under Windows/Vista without using Cygwin I
> would very much appreciate any hints you can give me.

First, the bad news: currently there is NO WAY to run Hadoop (and 
consequently Nutch) under Windows without Cygwin. Hadoop code makes 
explicit calls to unix-like utilities, such as whoami, df, ls, chmod, 
chgrp, bash, etc, etc ... and if they are not found on the path then bad 
things will happen, as you witnessed yourself.

If there is enough pressure from Windows users, Hadoop developers might 
work out the support for native Windows equivalents of these utilities, 
and include them in libhadoop.dll (or provide workarounds, or return 
fake values, or ...). At the moment, there is not enough demand (in 
terms of dedicated users) to try to provide such support, so it's easier 
just to install Cygwin and be done with it. :)


  Below is the
> stacktrace I get when debugging.
> 
> javax.security.auth.login.LoginException: Login failed: Cannot run program
> "bash": CreateProcess error=2, The system cannot find the file specified

After you have installed the Cygwin, please set the PATH, which Netbeans 
uses, so that it includes the Cygwin .../bin and .../usr/bin. I believe 
the Cygwin setup already does this for you.

-- 
Best regards,
Andrzej Bialecki     <><
  ___. ___ ___ ___ _ _   __________________________________
[__ || __|__/|__||\/|  Information Retrieval, Semantic Web
___|||__||  \|  ||  |  Embedded Unix, System Integration
http://www.sigram.com  Contact: info at sigram dot com

Re: Debugging Nutch in Netbeans

[Kevin MacDonald <ke...@hautesecure.com>]

I figured out how to debug into Nutch on Netbeans. This is a really good
how-to on the subject if anyone's interested.
http://www.netbeans.org/kb/articles/freeform-import.html

Something I am unable to do however, is debug very far into it. I am running
on Vista, and when the code executes at some point it attempts to invoke
"bash" and fails. Everything works fine at the command line using Cygwin,
but when debugging, Netbeans is not running in the Cygwin shell, so an
exception happens. Before I resign myself to printf debugging I am hoping
that there is a way of configuring Nutch (or possibly hadoop) to not do
whatever it is currently doing that causes the failure. If anyone has
experience getting Nutch working under Windows/Vista without using Cygwin I
would very much appreciate any hints you can give me. Below is the
stacktrace I get when debugging.

javax.security.auth.login.LoginException: Login failed: Cannot run program
"bash": CreateProcess error=2, The system cannot find the file specified
        at
org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:250)
        at
org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:275)
        at
org.apache.hadoop.security.UnixUserGroupInformation.login(UnixUserGroupInformation.java:257)
        at
org.apache.hadoop.security.UserGroupInformation.login(UserGroupInformation.java:67)
        at
org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:1353)
        at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1289)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:203)
        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:108)
        at org.apache.nutch.crawl.Crawl.main(Crawl.java:89)


Thanks

Kevin

On Mon, Sep 8, 2008 at 10:12 AM, Kevin MacDonald <ke...@hautesecure.com>wrote:

> Does anyone have an ant script that would allow debugging into Nutch
> running on Netbeans?
>
> Thanks
>
> Kevin
>