giraph on YARN logging

[Eric Kimbrel <le...@gmail.com>]

I have used graph on mr-v1 successfully over the last year, but have recently updated to using YARN and I am having a lot of trouble.

My biggest problem is that I can’t figure out how to actually look at the logs from my job and figure out what has gone wrong and why it is failing.   I can view the application master logs while the application is running, but other than that everything feels totally invisible.

Does anyone have any resources or recommendations for how to get better visibility / logging to understand what is happening when running graph on YARN?

Thanks

Re: giraph on YARN logging

[Rafal Wojdyla <ra...@spotify.com>]

Do you have log aggregation enabled?
Take a look at:
http://hortonworks.com/blog/simplifying-user-logs-management-and-access-in-yarn/

Cheers - Rafal


On Sat, Jan 4, 2014 at 1:54 AM, Eric Kimbrel <le...@gmail.com> wrote:

> I have used graph on mr-v1 successfully over the last year, but have
> recently updated to using YARN and I am having a lot of trouble.
>
> My biggest problem is that I can’t figure out how to actually look at the
> logs from my job and figure out what has gone wrong and why it is failing.
>   I can view the application master logs while the application is running,
> but other than that everything feels totally invisible.
>
> Does anyone have any resources or recommendations for how to get better
> visibility / logging to understand what is happening when running graph on
> YARN?
>
> Thanks




-- 
Cheers - Rafal