You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Martin Neumann <mn...@spotify.com> on 2014/02/17 13:07:24 UTC
debugging graph-programs
Hej,
I find Giraph programs quite difficult to debug (all I have is log outputs).
Anyone has some hints for debugging tools or coding practices to make it
simpler?
I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce debugging
aids (at least I think so, I haven't tried any)
cheers
Martin
Re: debugging graph-programs
Posted by Lukas Nalezenec <lu...@firma.seznam.cz>.
Hi,
My two small hints are:
1. Dont use Giraph without memory profiler (Eclipse MAT).
2. Write unit tests.
Lukas
On 17.2.2014 13:07, Martin Neumann wrote:
> Hej,
>
> I find Giraph programs quite difficult to debug (all I have is log
> outputs).
> Anyone has some hints for debugging tools or coding practices to make
> it simpler?
>
> I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce
> debugging aids (at least I think so, I haven't tried any)
>
> cheers
> Martin
Re: debugging graph-programs
Posted by Sebastian Schelter <ss...@apache.org>.
Hi Martin,
There is no need to install ZooKeeper to use InternalVertexRunner. It
will also use the input and output formats specified.
InternalVertexRunner writes your input data to disk and starts ZooKeeper
and Giraph locally in a single VM.
It is very easy to use, I suggest to look at the following class for an
example:
org.apache.giraph.examples.ConnectedComponentsComputationTest#testToyData
Best,
Sebastian
On 02/19/2014 12:55 PM, Martin Neumann wrote:
> I saw in the documentation that InternalVertexRunner has a configuration
> field (LOCAL_ZOOKEEPER_PORT) does that mean I have to have Zookeper
> installed and configured? Also what else would I need to have installed to
> test that way?
>
> I couldn't find any good guides how to do this online so I'm thankful for
> every hint. So thanks everyone for the help so far.
>
>
>
> On Wed, Feb 19, 2014 at 10:44 AM, Lukas Nalezenec <
> lukas.nalezenec@firma.seznam.cz> wrote:
>
>> Hi,
>> Its simple, just pass the TestGraph to InternalVertexRunner.
>>
>> If i had lot of logic in input format, I would test as much as i could in
>> pure isolated InputFormat unit test without any framework. I would make
>> POJO mock for Record Reader and then add some Mockito mocks.
>> (InternalVertexRunner is little bit slow - it starts giraph in every unit
>> test). Then i would test the input format together with computations in
>> InternalVertexRunner in separate unit test classes.
>> I have never used InternalVertexRunner for testing input/output formats -
>> I am not sure if it possible.
>>
>> Lukas
>>
>>
>>
>>
>> On 18.2.2014 17:38, Martin Neumann wrote:
>>
>> @ Lukas: Do you have some links where I can read how to use it to debug
>> programs?
>>
>> My current program has a lot of logic in the EdgeInputFormat as well
>> that was why I was looking at MapReduce tools since loading the data uses
>> the same classes (as far as I can see). But I agree its a rather desperate
>> try.
>>
>> I need a simple way to test the input format, output format and the
>> vertex program.
>> For the vertex program I currently develop it on a white bord to get the
>> interactions right then some unit test to make sure it works on a micro
>> level. I have not found a good way to test the input format (e.g. if the
>> Graph was build correctly) or how to do a good end to end test.
>>
>> cheers Martin
>>
>> On Tue, Feb 18, 2014 at 5:08 PM, Lukas Nalezenec <
>> lukas.nalezenec@firma.seznam.cz> wrote:
>>
>>> Hi,
>>> How about org.apache.giraph.utils.TestGraph?
>>> I would not try using mrunit for testing Giraph.
>>>
>>> cheers
>>> Lukas
>>>
>>>
>>>
>>> On 18.2.2014 17:01, Mirko Kämpf wrote:
>>>
>>> Hi Martin,
>>>
>>> it depends on the details of your implementation. In principle, you do
>>> not write Mapper and Reducer
>>> classes, which are tested by MR-Unit. You usually have to implement the
>>> application / algorithm specific
>>> logic, in e.g. the Vertex class. MR-Unit might not help that much here.
>>>
>>> This leads to the question, if there is a comparable approach / or tool
>>> like MR Unit for Giraph?
>>>
>>> Best wishes
>>> Mirko
>>>
>>>
>>>
>>>
>>> On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mn...@spotify.com>wrote:
>>>
>>>> How much of the Giraph 1.1.0 version is still based on MapReduce?
>>>> Would something like mrunit (http://mrunit.apache.org/) work with it?
>>>>
>>>>
>>>> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann <mn...@spotify.com>wrote:
>>>>
>>>>> I'm using the Yarn version (1.1.0 trunk) and I was not able to get
>>>>> it to work that way. But I'm no expert on that, if anyone knows a way to
>>>>> get it done It would be great.
>>>>>
>>>>>
>>>>> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <roman@shaposhnik.org
>>>>>> wrote:
>>>>>
>>>>>> I find running Giraph apps via hadoop's local execution mode
>>>>>> quite useful for debugging.
>>>>>>
>>>>>> Thanks,
>>>>>> Roman.
>>>>>>
>>>>>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mn...@spotify.com>
>>>>>> wrote:
>>>>>>> Hej,
>>>>>>>
>>>>>>> I find Giraph programs quite difficult to debug (all I have is log
>>>>>> outputs).
>>>>>>> Anyone has some hints for debugging tools or coding practices to
>>>>>> make it
>>>>>>> simpler?
>>>>>>>
>>>>>>> I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce
>>>>>> debugging
>>>>>>> aids (at least I think so, I haven't tried any)
>>>>>>>
>>>>>>> cheers
>>>>>>> Martin
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>
Re: debugging graph-programs
Posted by Martin Neumann <mn...@spotify.com>.
I saw in the documentation that InternalVertexRunner has a configuration
field (LOCAL_ZOOKEEPER_PORT) does that mean I have to have Zookeper
installed and configured? Also what else would I need to have installed to
test that way?
I couldn't find any good guides how to do this online so I'm thankful for
every hint. So thanks everyone for the help so far.
On Wed, Feb 19, 2014 at 10:44 AM, Lukas Nalezenec <
lukas.nalezenec@firma.seznam.cz> wrote:
> Hi,
> Its simple, just pass the TestGraph to InternalVertexRunner.
>
> If i had lot of logic in input format, I would test as much as i could in
> pure isolated InputFormat unit test without any framework. I would make
> POJO mock for Record Reader and then add some Mockito mocks.
> (InternalVertexRunner is little bit slow - it starts giraph in every unit
> test). Then i would test the input format together with computations in
> InternalVertexRunner in separate unit test classes.
> I have never used InternalVertexRunner for testing input/output formats -
> I am not sure if it possible.
>
> Lukas
>
>
>
>
> On 18.2.2014 17:38, Martin Neumann wrote:
>
> @ Lukas: Do you have some links where I can read how to use it to debug
> programs?
>
> My current program has a lot of logic in the EdgeInputFormat as well
> that was why I was looking at MapReduce tools since loading the data uses
> the same classes (as far as I can see). But I agree its a rather desperate
> try.
>
> I need a simple way to test the input format, output format and the
> vertex program.
> For the vertex program I currently develop it on a white bord to get the
> interactions right then some unit test to make sure it works on a micro
> level. I have not found a good way to test the input format (e.g. if the
> Graph was build correctly) or how to do a good end to end test.
>
> cheers Martin
>
> On Tue, Feb 18, 2014 at 5:08 PM, Lukas Nalezenec <
> lukas.nalezenec@firma.seznam.cz> wrote:
>
>> Hi,
>> How about org.apache.giraph.utils.TestGraph?
>> I would not try using mrunit for testing Giraph.
>>
>> cheers
>> Lukas
>>
>>
>>
>> On 18.2.2014 17:01, Mirko Kämpf wrote:
>>
>> Hi Martin,
>>
>> it depends on the details of your implementation. In principle, you do
>> not write Mapper and Reducer
>> classes, which are tested by MR-Unit. You usually have to implement the
>> application / algorithm specific
>> logic, in e.g. the Vertex class. MR-Unit might not help that much here.
>>
>> This leads to the question, if there is a comparable approach / or tool
>> like MR Unit for Giraph?
>>
>> Best wishes
>> Mirko
>>
>>
>>
>>
>> On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mn...@spotify.com>wrote:
>>
>>> How much of the Giraph 1.1.0 version is still based on MapReduce?
>>> Would something like mrunit (http://mrunit.apache.org/) work with it?
>>>
>>>
>>> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann <mn...@spotify.com>wrote:
>>>
>>>> I'm using the Yarn version (1.1.0 trunk) and I was not able to get
>>>> it to work that way. But I'm no expert on that, if anyone knows a way to
>>>> get it done It would be great.
>>>>
>>>>
>>>> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <roman@shaposhnik.org
>>>> > wrote:
>>>>
>>>>> I find running Giraph apps via hadoop's local execution mode
>>>>> quite useful for debugging.
>>>>>
>>>>> Thanks,
>>>>> Roman.
>>>>>
>>>>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mn...@spotify.com>
>>>>> wrote:
>>>>> > Hej,
>>>>> >
>>>>> > I find Giraph programs quite difficult to debug (all I have is log
>>>>> outputs).
>>>>> > Anyone has some hints for debugging tools or coding practices to
>>>>> make it
>>>>> > simpler?
>>>>> >
>>>>> > I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce
>>>>> debugging
>>>>> > aids (at least I think so, I haven't tried any)
>>>>> >
>>>>> > cheers
>>>>> > Martin
>>>>>
>>>>
>>>>
>>>
>>
>>
>
>
Re: debugging graph-programs
Posted by Lukas Nalezenec <lu...@firma.seznam.cz>.
Hi,
Its simple, just pass the TestGraph to InternalVertexRunner.
If i had lot of logic in input format, I would test as much as i could
in pure isolated InputFormat unit test without any framework. I would
make POJO mock for Record Reader and then add some Mockito mocks.
(InternalVertexRunner is little bit slow - it starts giraph in every
unit test). Then i would test the input format together with
computations in InternalVertexRunner in separate unit test classes.
I have never used InternalVertexRunner for testing input/output formats
- I am not sure if it possible.
Lukas
On 18.2.2014 17:38, Martin Neumann wrote:
> @ Lukas: Do you have some links where I can read how to use it to
> debug programs?
>
> My current program has a lot of logic in the EdgeInputFormat as well
> that was why I was looking at MapReduce tools since loading the data
> uses the same classes (as far as I can see). But I agree its a rather
> desperate try.
>
> I need a simple way to test the input format, output format and the
> vertex program.
> For the vertex program I currently develop it on a white bord to get
> the interactions right then some unit test to make sure it works on a
> micro level. I have not found a good way to test the input format
> (e.g. if the Graph was build correctly) or how to do a good end to end
> test.
>
> cheers Martin
>
> On Tue, Feb 18, 2014 at 5:08 PM, Lukas Nalezenec
> <lukas.nalezenec@firma.seznam.cz
> <ma...@firma.seznam.cz>> wrote:
>
> Hi,
> How about org.apache.giraph.utils.TestGraph?
> I would not try using mrunit for testing Giraph.
>
> cheers
> Lukas
>
>
>
> On 18.2.2014 17:01, Mirko Kämpf wrote:
>> Hi Martin,
>>
>> it depends on the details of your implementation. In principle,
>> you do not write Mapper and Reducer
>> classes, which are tested by MR-Unit. You usually have to
>> implement the application / algorithm specific
>> logic, in e.g. the Vertex class. MR-Unit might not help that much
>> here.
>>
>> This leads to the question, if there is a comparable approach /
>> or tool like MR Unit for Giraph?
>>
>> Best wishes
>> Mirko
>>
>>
>>
>>
>> On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann
>> <mneumann@spotify.com <ma...@spotify.com>> wrote:
>>
>> How much of the Giraph 1.1.0 version is still based on
>> MapReduce?
>> Would something like mrunit (http://mrunit.apache.org/) work
>> with it?
>>
>>
>> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann
>> <mneumann@spotify.com <ma...@spotify.com>> wrote:
>>
>> I'm using the Yarn version (1.1.0 trunk) and I was not
>> able to get it to work that way. But I'm no expert on
>> that, if anyone knows a way to get it done It would be great.
>>
>>
>> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik
>> <roman@shaposhnik.org <ma...@shaposhnik.org>> wrote:
>>
>> I find running Giraph apps via hadoop's local
>> execution mode
>> quite useful for debugging.
>>
>> Thanks,
>> Roman.
>>
>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann
>> <mneumann@spotify.com <ma...@spotify.com>>
>> wrote:
>> > Hej,
>> >
>> > I find Giraph programs quite difficult to debug
>> (all I have is log outputs).
>> > Anyone has some hints for debugging tools or coding
>> practices to make it
>> > simpler?
>> >
>> > I'm using Giraph 1.1.0 trunk on YARN so I can't use
>> Map/Reduce debugging
>> > aids (at least I think so, I haven't tried any)
>> >
>> > cheers
>> > Martin
>>
>>
>>
>>
>
>
Re: debugging graph-programs
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
Well, then you should really use this approach:
https://git-wip-us.apache.org/repos/asf?p=giraph.git;a=blob;f=giraph-examples/src/test/java/org/apache/giraph/examples/SimpleShortestPathsComputationTest.java;h=dc6c84aeda01283c5d29d2787e89fa01fbe65241;hb=HEAD#l114
Thanks,
Roman.
On Tue, Feb 18, 2014 at 8:38 AM, Martin Neumann <mn...@spotify.com> wrote:
> @ Lukas: Do you have some links where I can read how to use it to debug
> programs?
>
> My current program has a lot of logic in the EdgeInputFormat as well that
> was why I was looking at MapReduce tools since loading the data uses the
> same classes (as far as I can see). But I agree its a rather desperate try.
>
> I need a simple way to test the input format, output format and the vertex
> program.
> For the vertex program I currently develop it on a white bord to get the
> interactions right then some unit test to make sure it works on a micro
> level. I have not found a good way to test the input format (e.g. if the
> Graph was build correctly) or how to do a good end to end test.
>
> cheers Martin
>
> On Tue, Feb 18, 2014 at 5:08 PM, Lukas Nalezenec
> <lu...@firma.seznam.cz> wrote:
>>
>> Hi,
>> How about org.apache.giraph.utils.TestGraph?
>> I would not try using mrunit for testing Giraph.
>>
>> cheers
>> Lukas
>>
>>
>>
>> On 18.2.2014 17:01, Mirko Kämpf wrote:
>>
>> Hi Martin,
>>
>> it depends on the details of your implementation. In principle, you do not
>> write Mapper and Reducer
>> classes, which are tested by MR-Unit. You usually have to implement the
>> application / algorithm specific
>> logic, in e.g. the Vertex class. MR-Unit might not help that much here.
>>
>> This leads to the question, if there is a comparable approach / or tool
>> like MR Unit for Giraph?
>>
>> Best wishes
>> Mirko
>>
>>
>>
>>
>> On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mn...@spotify.com>
>> wrote:
>>>
>>> How much of the Giraph 1.1.0 version is still based on MapReduce?
>>> Would something like mrunit (http://mrunit.apache.org/) work with it?
>>>
>>>
>>> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann <mn...@spotify.com>
>>> wrote:
>>>>
>>>> I'm using the Yarn version (1.1.0 trunk) and I was not able to get it
>>>> to work that way. But I'm no expert on that, if anyone knows a way to get it
>>>> done It would be great.
>>>>
>>>>
>>>> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <ro...@shaposhnik.org>
>>>> wrote:
>>>>>
>>>>> I find running Giraph apps via hadoop's local execution mode
>>>>> quite useful for debugging.
>>>>>
>>>>> Thanks,
>>>>> Roman.
>>>>>
>>>>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mn...@spotify.com>
>>>>> wrote:
>>>>> > Hej,
>>>>> >
>>>>> > I find Giraph programs quite difficult to debug (all I have is log
>>>>> > outputs).
>>>>> > Anyone has some hints for debugging tools or coding practices to make
>>>>> > it
>>>>> > simpler?
>>>>> >
>>>>> > I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce
>>>>> > debugging
>>>>> > aids (at least I think so, I haven't tried any)
>>>>> >
>>>>> > cheers
>>>>> > Martin
>>>>
>>>>
>>>
>>
>>
>
Re: debugging graph-programs
Posted by Martin Neumann <mn...@spotify.com>.
@ Lukas: Do you have some links where I can read how to use it to debug
programs?
My current program has a lot of logic in the EdgeInputFormat as well that
was why I was looking at MapReduce tools since loading the data uses the
same classes (as far as I can see). But I agree its a rather desperate try.
I need a simple way to test the input format, output format and the vertex
program.
For the vertex program I currently develop it on a white bord to get the
interactions right then some unit test to make sure it works on a micro
level. I have not found a good way to test the input format (e.g. if the
Graph was build correctly) or how to do a good end to end test.
cheers Martin
On Tue, Feb 18, 2014 at 5:08 PM, Lukas Nalezenec <
lukas.nalezenec@firma.seznam.cz> wrote:
> Hi,
> How about org.apache.giraph.utils.TestGraph?
> I would not try using mrunit for testing Giraph.
>
> cheers
> Lukas
>
>
>
> On 18.2.2014 17:01, Mirko Kämpf wrote:
>
> Hi Martin,
>
> it depends on the details of your implementation. In principle, you do
> not write Mapper and Reducer
> classes, which are tested by MR-Unit. You usually have to implement the
> application / algorithm specific
> logic, in e.g. the Vertex class. MR-Unit might not help that much here.
>
> This leads to the question, if there is a comparable approach / or tool
> like MR Unit for Giraph?
>
> Best wishes
> Mirko
>
>
>
>
> On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mn...@spotify.com>wrote:
>
>> How much of the Giraph 1.1.0 version is still based on MapReduce?
>> Would something like mrunit (http://mrunit.apache.org/) work with it?
>>
>>
>> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann <mn...@spotify.com>wrote:
>>
>>> I'm using the Yarn version (1.1.0 trunk) and I was not able to get it
>>> to work that way. But I'm no expert on that, if anyone knows a way to get
>>> it done It would be great.
>>>
>>>
>>> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <ro...@shaposhnik.org>wrote:
>>>
>>>> I find running Giraph apps via hadoop's local execution mode
>>>> quite useful for debugging.
>>>>
>>>> Thanks,
>>>> Roman.
>>>>
>>>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mn...@spotify.com>
>>>> wrote:
>>>> > Hej,
>>>> >
>>>> > I find Giraph programs quite difficult to debug (all I have is log
>>>> outputs).
>>>> > Anyone has some hints for debugging tools or coding practices to make
>>>> it
>>>> > simpler?
>>>> >
>>>> > I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce
>>>> debugging
>>>> > aids (at least I think so, I haven't tried any)
>>>> >
>>>> > cheers
>>>> > Martin
>>>>
>>>
>>>
>>
>
>
Re: debugging graph-programs
Posted by Lukas Nalezenec <lu...@firma.seznam.cz>.
Hi,
How about org.apache.giraph.utils.TestGraph?
I would not try using mrunit for testing Giraph.
cheers
Lukas
On 18.2.2014 17:01, Mirko Kämpf wrote:
> Hi Martin,
>
> it depends on the details of your implementation. In principle, you do
> not write Mapper and Reducer
> classes, which are tested by MR-Unit. You usually have to implement
> the application / algorithm specific
> logic, in e.g. the Vertex class. MR-Unit might not help that much here.
>
> This leads to the question, if there is a comparable approach / or
> tool like MR Unit for Giraph?
>
> Best wishes
> Mirko
>
>
>
>
> On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mneumann@spotify.com
> <ma...@spotify.com>> wrote:
>
> How much of the Giraph 1.1.0 version is still based on MapReduce?
> Would something like mrunit (http://mrunit.apache.org/) work with it?
>
>
> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann
> <mneumann@spotify.com <ma...@spotify.com>> wrote:
>
> I'm using the Yarn version (1.1.0 trunk) and I was not able to
> get it to work that way. But I'm no expert on that, if anyone
> knows a way to get it done It would be great.
>
>
> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik
> <roman@shaposhnik.org <ma...@shaposhnik.org>> wrote:
>
> I find running Giraph apps via hadoop's local execution mode
> quite useful for debugging.
>
> Thanks,
> Roman.
>
> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann
> <mneumann@spotify.com <ma...@spotify.com>> wrote:
> > Hej,
> >
> > I find Giraph programs quite difficult to debug (all I
> have is log outputs).
> > Anyone has some hints for debugging tools or coding
> practices to make it
> > simpler?
> >
> > I'm using Giraph 1.1.0 trunk on YARN so I can't use
> Map/Reduce debugging
> > aids (at least I think so, I haven't tried any)
> >
> > cheers
> > Martin
>
>
>
>
Re: debugging graph-programs
Posted by Mirko Kämpf <mi...@cloudera.com>.
Hi Martin,
it depends on the details of your implementation. In principle, you do not
write Mapper and Reducer
classes, which are tested by MR-Unit. You usually have to implement the
application / algorithm specific
logic, in e.g. the Vertex class. MR-Unit might not help that much here.
This leads to the question, if there is a comparable approach / or tool
like MR Unit for Giraph?
Best wishes
Mirko
On Tue, Feb 18, 2014 at 3:52 PM, Martin Neumann <mn...@spotify.com>wrote:
> How much of the Giraph 1.1.0 version is still based on MapReduce?
> Would something like mrunit (http://mrunit.apache.org/) work with it?
>
>
> On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann <mn...@spotify.com>wrote:
>
>> I'm using the Yarn version (1.1.0 trunk) and I was not able to get it to
>> work that way. But I'm no expert on that, if anyone knows a way to get it
>> done It would be great.
>>
>>
>> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <ro...@shaposhnik.org>wrote:
>>
>>> I find running Giraph apps via hadoop's local execution mode
>>> quite useful for debugging.
>>>
>>> Thanks,
>>> Roman.
>>>
>>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mn...@spotify.com>
>>> wrote:
>>> > Hej,
>>> >
>>> > I find Giraph programs quite difficult to debug (all I have is log
>>> outputs).
>>> > Anyone has some hints for debugging tools or coding practices to make
>>> it
>>> > simpler?
>>> >
>>> > I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce
>>> debugging
>>> > aids (at least I think so, I haven't tried any)
>>> >
>>> > cheers
>>> > Martin
>>>
>>
>>
>
Re: debugging graph-programs
Posted by Martin Neumann <mn...@spotify.com>.
How much of the Giraph 1.1.0 version is still based on MapReduce?
Would something like mrunit (http://mrunit.apache.org/) work with it?
On Tue, Feb 18, 2014 at 12:52 PM, Martin Neumann <mn...@spotify.com>wrote:
> I'm using the Yarn version (1.1.0 trunk) and I was not able to get it to
> work that way. But I'm no expert on that, if anyone knows a way to get it
> done It would be great.
>
>
> On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <ro...@shaposhnik.org>wrote:
>
>> I find running Giraph apps via hadoop's local execution mode
>> quite useful for debugging.
>>
>> Thanks,
>> Roman.
>>
>> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mn...@spotify.com>
>> wrote:
>> > Hej,
>> >
>> > I find Giraph programs quite difficult to debug (all I have is log
>> outputs).
>> > Anyone has some hints for debugging tools or coding practices to make it
>> > simpler?
>> >
>> > I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce debugging
>> > aids (at least I think so, I haven't tried any)
>> >
>> > cheers
>> > Martin
>>
>
>
Re: debugging graph-programs
Posted by Martin Neumann <mn...@spotify.com>.
I'm using the Yarn version (1.1.0 trunk) and I was not able to get it to
work that way. But I'm no expert on that, if anyone knows a way to get it
done It would be great.
On Tue, Feb 18, 2014 at 2:07 AM, Roman Shaposhnik <ro...@shaposhnik.org>wrote:
> I find running Giraph apps via hadoop's local execution mode
> quite useful for debugging.
>
> Thanks,
> Roman.
>
> On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mn...@spotify.com>
> wrote:
> > Hej,
> >
> > I find Giraph programs quite difficult to debug (all I have is log
> outputs).
> > Anyone has some hints for debugging tools or coding practices to make it
> > simpler?
> >
> > I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce debugging
> > aids (at least I think so, I haven't tried any)
> >
> > cheers
> > Martin
>
Re: debugging graph-programs
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
I find running Giraph apps via hadoop's local execution mode
quite useful for debugging.
Thanks,
Roman.
On Mon, Feb 17, 2014 at 4:07 AM, Martin Neumann <mn...@spotify.com> wrote:
> Hej,
>
> I find Giraph programs quite difficult to debug (all I have is log outputs).
> Anyone has some hints for debugging tools or coding practices to make it
> simpler?
>
> I'm using Giraph 1.1.0 trunk on YARN so I can't use Map/Reduce debugging
> aids (at least I think so, I haven't tried any)
>
> cheers
> Martin