You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Alexander Frolov <al...@gmail.com> on 2014/02/03 17:06:43 UTC
IntIntNullTextInputFormat problem
Hello,
I am trying to load data in from file in IntIntNullTextInputFormat. But I
get is the following message:
[hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
/user/hduser/input/test -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/output/amazon-shortestpaths -w 1
14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentException:
checkClassTypes: vertex index types not assignable, computation - class
org.apache.hadoop.io.LongWritable, VertexInputFormat - class
org.apache.hadoop.io.IntWritable
at
org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at
org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:226)
at
org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at
org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
The input file is here (just for testing):
[hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
/user/hduser/input/test
0 1
1 0
Could you please help with this issue?
Best,
Alex
Re: IntIntNullTextInputFormat problem
Posted by Alexander Frolov <al...@gmail.com>.
Sebastian,
thank you very much for your help!
On Tue, Feb 4, 2014 at 10:10 AM, Sebastian Schelter <ss...@apache.org> wrote:
> Hi Alexander,
>
> the only inputformat that we have which is compatible with
> SimpleShortestPathsComputation is the
>
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>
> which is used in
>
> org.apache.giraph.examples.SimpleShortestPathsComputationTest#testToyData
>
> It's not as nice to use as the other inputformats we have, so I opened
> https://issues.apache.org/jira/browse/GIRAPH-844 to ensure that we have a
> nice easy to use TextInputFormat for the shortest paths example.
>
> --sebastian
>
>
> On 02/03/2014 06:52 PM, Alexander Frolov wrote:
>
>> It seems that I understood the problem. PageRank and
>> SingleSourceShortestPath both need weighted graph to compute. I just
>> missed it somehow.
>> Thank you for help.
>>
>>
>> On Mon, Feb 3, 2014 at 9:25 PM, Alexander Frolov
>> <al...@gmail.com>wrote:
>>
>>
>>>
>>>
>>> On Mon, Feb 3, 2014 at 9:06 PM, Sebastian Schelter <ss...@apache.org>
>>> wrote:
>>>
>>> If you just wanna play with Giraph, I suggest you start with the
>>>> PageRank
>>>> example:
>>>>
>>>> https://giraph.apache.org/pagerank.html
>>>>
>>>
>>>
>>> I have seen it. I think there is no much difference in SSSP and PageRank
>>> examples. By the way, I have not found LongDoubleNullTextInputFormat in
>>> my
>>> sources (I am using release-1.0.0-RC2-152-g585511f) which is mentioned
>>> there.
>>> The same issue with SimplePageRankComputation.
>>>
>>> [hduser@localhost giraph]$ $HADOOP_HOME/bin/hadoop jar
>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>> org.apache.giraph.GiraphRunner
>>> org.apache.giraph.examples.SimplePageRankComputation -vif
>>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
>>> /user/hduser/input/test -vof
>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>> /user/hduser/output/amazon-shortestpaths -w 1
>>> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge input format
>>> specified. Ensure your InputFormat does not require one.
>>> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge output format
>>> specified. Ensure your OutputFormat does not require one.
>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>> checkClassTypes: vertex value types not assignable, computation - class
>>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
>>> org.apache.hadoop.io.LongWritable
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>> GiraphConfigurationValidator.java:381)
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>> java:228)
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>> validateConfiguration(GiraphConfigurationValidator.java:141)
>>> at
>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>> ConfigurationUtils.java:214)
>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>> NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>> DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>
>>>
>>>
>>>> --sebastian
>>>>
>>>>
>>>> On 02/03/2014 05:49 PM, Alexander Frolov wrote:
>>>>
>>>> I've tried .LongLongNullTextInputFormat and got this:
>>>>>
>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>> org.apache.giraph.GiraphRunner
>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
>>>>> /user/hduser/input/test -vof
>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format
>>>>> specified. Ensure your InputFormat does not require one.
>>>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format
>>>>> specified. Ensure your OutputFormat does not require one.
>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>> checkClassTypes: vertex value types not assignable, computation - class
>>>>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
>>>>> org.apache.hadoop.io.LongWritable
>>>>> at
>>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>>> GiraphConfigurationValidator.java:381)
>>>>> at
>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>>> java:228)
>>>>> at
>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>> validateConfiguration(GiraphConfigurationValidator.java:141)
>>>>>
>>>>> at
>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>> ConfigurationUtils.java:214)
>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>> NativeMethodAccessorImpl.java:57)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov
>>>>> <al...@gmail.com>wrote:
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <ss...@apache.org>
>>>>>> wrote
>>>>>>
>>>>>> What types does the vertex class that you use in your computation
>>>>>>
>>>>>>> require?
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> I see.
>>>>>>
>>>>>> public class SimpleShortestPathsComputation extends BasicComputation<
>>>>>> LongWritable, DoubleWritable, FloatWritable, DoubleWritable>
>>>>>> {..}
>>>>>>
>>>>>> So that is a mistake.
>>>>>> Thank you very much.
>>>>>>
>>>>>>
>>>>>> Is there any means to import data from file like this
>>>>> (amazon0601.txt
>>>>>
>>>>>> from
>>>>>> SNAP data collection):
>>>>>>
>>>>>> 0 1
>>>>>> 0 2
>>>>>> 0 3
>>>>>> 0 4
>>>>>> 0 5
>>>>>> 0 6
>>>>>> 0 7
>>>>>> 0 8
>>>>>> 0 9
>>>>>> 0 10
>>>>>> ...
>>>>>>
>>>>>> and to perform basic tests from giraph-examples? Most of the tests are
>>>>>> using LongWritable for vertex id.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 02/03/2014 05:28 PM, Alexander Frolov wrote:
>>>>>>>
>>>>>>> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ssc@apache.org
>>>>>>> >
>>>>>>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Alexander,
>>>>>>>>
>>>>>>>>
>>>>>>>>> what do you use as type for your vertex ids? It looks that you are
>>>>>>>>> trying
>>>>>>>>> to use longs, while IntIntNullTextInputFormat only provides ints,
>>>>>>>>> that
>>>>>>>>> could be the error.
>>>>>>>>>
>>>>>>>>> --sebastian
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I also thought so, but I could not understand why it regard it
>>>>>>>>> as
>>>>>>>>> long.
>>>>>>>>>
>>>>>>>>> I just use -vif org.apache.giraph.io.formats.
>>>>>>>> IntIntNullTextInputFormat
>>>>>>>> -vip /user/hduser/input/test -vof
>>>>>>>>
>>>>>>>> and file consists only of two lines:
>>>>>>>>
>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>>>> /user/hduser/input/test
>>>>>>>> 0 1
>>>>>>>> 1 0
>>>>>>>>
>>>>>>>> I dont know any other means to specify type of vertex ids.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I am trying to load data in from file in
>>>>>>>>>> IntIntNullTextInputFormat.
>>>>>>>>>> But I
>>>>>>>>>> get is the following message:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>>>>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>>>>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>>>>>>> org.apache.giraph.GiraphRunner
>>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>>>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>>>>>>>> /user/hduser/input/test -vof
>>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>>>>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>>>>>>>
>>>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input
>>>>>>>>>> format
>>>>>>>>>> specified. Ensure your InputFormat does not require one.
>>>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output
>>>>>>>>>> format
>>>>>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>>>>>>> checkClassTypes: vertex index types not assignable, computation -
>>>>>>>>>> class
>>>>>>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>>>>>>>>> org.apache.hadoop.io.IntWritable
>>>>>>>>>> at
>>>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>>>>> checkAssignable(
>>>>>>>>>> GiraphConfigurationValidator.java:381)
>>>>>>>>>> at
>>>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>>>>>>>> java:226)
>>>>>>>>>> at
>>>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>>>>> validateConfiguration(
>>>>>>>>>> GiraphConfigurationValidator.java:141)
>>>>>>>>>> at
>>>>>>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>>>>>>> ConfigurationUtils.java:214)
>>>>>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>>>> at
>>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>>>>>>> NativeMethodAccessorImpl.java:57)
>>>>>>>>>> at
>>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>>>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The input file is here (just for testing):
>>>>>>>>>>
>>>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>>>>>> /user/hduser/input/test
>>>>>>>>>> 0 1
>>>>>>>>>> 1 0
>>>>>>>>>>
>>>>>>>>>> Could you please help with this issue?
>>>>>>>>>>
>>>>>>>>>> Best,
>>>>>>>>>> Alex
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: IntIntNullTextInputFormat problem
Posted by Sebastian Schelter <ss...@apache.org>.
Hi Alexander,
the only inputformat that we have which is compatible with
SimpleShortestPathsComputation is the
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
which is used in
org.apache.giraph.examples.SimpleShortestPathsComputationTest#testToyData
It's not as nice to use as the other inputformats we have, so I opened
https://issues.apache.org/jira/browse/GIRAPH-844 to ensure that we have
a nice easy to use TextInputFormat for the shortest paths example.
--sebastian
On 02/03/2014 06:52 PM, Alexander Frolov wrote:
> It seems that I understood the problem. PageRank and
> SingleSourceShortestPath both need weighted graph to compute. I just
> missed it somehow.
> Thank you for help.
>
>
> On Mon, Feb 3, 2014 at 9:25 PM, Alexander Frolov
> <al...@gmail.com>wrote:
>
>>
>>
>>
>> On Mon, Feb 3, 2014 at 9:06 PM, Sebastian Schelter <ss...@apache.org> wrote:
>>
>>> If you just wanna play with Giraph, I suggest you start with the PageRank
>>> example:
>>>
>>> https://giraph.apache.org/pagerank.html
>>
>>
>> I have seen it. I think there is no much difference in SSSP and PageRank
>> examples. By the way, I have not found LongDoubleNullTextInputFormat in my
>> sources (I am using release-1.0.0-RC2-152-g585511f) which is mentioned
>> there.
>> The same issue with SimplePageRankComputation.
>>
>> [hduser@localhost giraph]$ $HADOOP_HOME/bin/hadoop jar
>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimplePageRankComputation -vif
>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
>> /user/hduser/input/test -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/hduser/output/amazon-shortestpaths -w 1
>> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge output format
>> specified. Ensure your OutputFormat does not require one.
>> Exception in thread "main" java.lang.IllegalArgumentException:
>> checkClassTypes: vertex value types not assignable, computation - class
>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
>> org.apache.hadoop.io.LongWritable
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
>> at
>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>>>
>>> --sebastian
>>>
>>>
>>> On 02/03/2014 05:49 PM, Alexander Frolov wrote:
>>>
>>>> I've tried .LongLongNullTextInputFormat and got this:
>>>>
>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>> org.apache.giraph.GiraphRunner
>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
>>>> /user/hduser/input/test -vof
>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format
>>>> specified. Ensure your InputFormat does not require one.
>>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format
>>>> specified. Ensure your OutputFormat does not require one.
>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>> checkClassTypes: vertex value types not assignable, computation - class
>>>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
>>>> org.apache.hadoop.io.LongWritable
>>>> at
>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>> GiraphConfigurationValidator.java:381)
>>>> at
>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>> java:228)
>>>> at
>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>> validateConfiguration(GiraphConfigurationValidator.java:141)
>>>> at
>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>> ConfigurationUtils.java:214)
>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>> NativeMethodAccessorImpl.java:57)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>> DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov
>>>> <al...@gmail.com>wrote:
>>>>
>>>>
>>>>>
>>>>>
>>>>> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <ss...@apache.org>
>>>>> wrote
>>>>>
>>>>> What types does the vertex class that you use in your computation
>>>>>> require?
>>>>>>
>>>>>
>>>>>
>>>>> I see.
>>>>>
>>>>> public class SimpleShortestPathsComputation extends BasicComputation<
>>>>> LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..}
>>>>>
>>>>> So that is a mistake.
>>>>> Thank you very much.
>>>>>
>>>>>
>>>> Is there any means to import data from file like this (amazon0601.txt
>>>>> from
>>>>> SNAP data collection):
>>>>>
>>>>> 0 1
>>>>> 0 2
>>>>> 0 3
>>>>> 0 4
>>>>> 0 5
>>>>> 0 6
>>>>> 0 7
>>>>> 0 8
>>>>> 0 9
>>>>> 0 10
>>>>> ...
>>>>>
>>>>> and to perform basic tests from giraph-examples? Most of the tests are
>>>>> using LongWritable for vertex id.
>>>>>
>>>>>
>>>>>
>>>>>> On 02/03/2014 05:28 PM, Alexander Frolov wrote:
>>>>>>
>>>>>> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ss...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi Alexander,
>>>>>>>
>>>>>>>>
>>>>>>>> what do you use as type for your vertex ids? It looks that you are
>>>>>>>> trying
>>>>>>>> to use longs, while IntIntNullTextInputFormat only provides ints,
>>>>>>>> that
>>>>>>>> could be the error.
>>>>>>>>
>>>>>>>> --sebastian
>>>>>>>>
>>>>>>>>
>>>>>>>> I also thought so, but I could not understand why it regard it as
>>>>>>>> long.
>>>>>>>>
>>>>>>> I just use -vif org.apache.giraph.io.formats.
>>>>>>> IntIntNullTextInputFormat
>>>>>>> -vip /user/hduser/input/test -vof
>>>>>>>
>>>>>>> and file consists only of two lines:
>>>>>>>
>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>>> /user/hduser/input/test
>>>>>>> 0 1
>>>>>>> 1 0
>>>>>>>
>>>>>>> I dont know any other means to specify type of vertex ids.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>>>
>>>>>>>>> I am trying to load data in from file in IntIntNullTextInputFormat.
>>>>>>>>> But I
>>>>>>>>> get is the following message:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>>>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>>>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>>>>>> org.apache.giraph.GiraphRunner
>>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>>>>>>> /user/hduser/input/test -vof
>>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>>>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>>>>>>
>>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input
>>>>>>>>> format
>>>>>>>>> specified. Ensure your InputFormat does not require one.
>>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output
>>>>>>>>> format
>>>>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>>>>>> checkClassTypes: vertex index types not assignable, computation -
>>>>>>>>> class
>>>>>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>>>>>>>> org.apache.hadoop.io.IntWritable
>>>>>>>>> at
>>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>>>>>>> GiraphConfigurationValidator.java:381)
>>>>>>>>> at
>>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>>>>>>> java:226)
>>>>>>>>> at
>>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>>>> validateConfiguration(
>>>>>>>>> GiraphConfigurationValidator.java:141)
>>>>>>>>> at
>>>>>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>>>>>> ConfigurationUtils.java:214)
>>>>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>>> at
>>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>>>>>> NativeMethodAccessorImpl.java:57)
>>>>>>>>> at
>>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The input file is here (just for testing):
>>>>>>>>>
>>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>>>>> /user/hduser/input/test
>>>>>>>>> 0 1
>>>>>>>>> 1 0
>>>>>>>>>
>>>>>>>>> Could you please help with this issue?
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Alex
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: IntIntNullTextInputFormat problem
Posted by Alexander Frolov <al...@gmail.com>.
It seems that I understood the problem. PageRank and
SingleSourceShortestPath both need weighted graph to compute. I just
missed it somehow.
Thank you for help.
On Mon, Feb 3, 2014 at 9:25 PM, Alexander Frolov
<al...@gmail.com>wrote:
>
>
>
> On Mon, Feb 3, 2014 at 9:06 PM, Sebastian Schelter <ss...@apache.org> wrote:
>
>> If you just wanna play with Giraph, I suggest you start with the PageRank
>> example:
>>
>> https://giraph.apache.org/pagerank.html
>
>
> I have seen it. I think there is no much difference in SSSP and PageRank
> examples. By the way, I have not found LongDoubleNullTextInputFormat in my
> sources (I am using release-1.0.0-RC2-152-g585511f) which is mentioned
> there.
> The same issue with SimplePageRankComputation.
>
> [hduser@localhost giraph]$ $HADOOP_HOME/bin/hadoop jar
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimplePageRankComputation -vif
> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
> /user/hduser/input/test -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/hduser/output/amazon-shortestpaths -w 1
> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge output format
> specified. Ensure your OutputFormat does not require one.
> Exception in thread "main" java.lang.IllegalArgumentException:
> checkClassTypes: vertex value types not assignable, computation - class
> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
> org.apache.hadoop.io.LongWritable
> at
> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
> at
> org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
> at
> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
> at
> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
>>
>> --sebastian
>>
>>
>> On 02/03/2014 05:49 PM, Alexander Frolov wrote:
>>
>>> I've tried .LongLongNullTextInputFormat and got this:
>>>
>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>> org.apache.giraph.GiraphRunner
>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
>>> /user/hduser/input/test -vof
>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>> /user/hduser/output/amazon-shortestpaths -w 1
>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format
>>> specified. Ensure your InputFormat does not require one.
>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format
>>> specified. Ensure your OutputFormat does not require one.
>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>> checkClassTypes: vertex value types not assignable, computation - class
>>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
>>> org.apache.hadoop.io.LongWritable
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>> GiraphConfigurationValidator.java:381)
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>> java:228)
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>> validateConfiguration(GiraphConfigurationValidator.java:141)
>>> at
>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>> ConfigurationUtils.java:214)
>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>> NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>> DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>
>>>
>>>
>>>
>>> On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov
>>> <al...@gmail.com>wrote:
>>>
>>>
>>>>
>>>>
>>>> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <ss...@apache.org>
>>>> wrote
>>>>
>>>> What types does the vertex class that you use in your computation
>>>>> require?
>>>>>
>>>>
>>>>
>>>> I see.
>>>>
>>>> public class SimpleShortestPathsComputation extends BasicComputation<
>>>> LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..}
>>>>
>>>> So that is a mistake.
>>>> Thank you very much.
>>>>
>>>>
>>> Is there any means to import data from file like this (amazon0601.txt
>>>> from
>>>> SNAP data collection):
>>>>
>>>> 0 1
>>>> 0 2
>>>> 0 3
>>>> 0 4
>>>> 0 5
>>>> 0 6
>>>> 0 7
>>>> 0 8
>>>> 0 9
>>>> 0 10
>>>> ...
>>>>
>>>> and to perform basic tests from giraph-examples? Most of the tests are
>>>> using LongWritable for vertex id.
>>>>
>>>>
>>>>
>>>>> On 02/03/2014 05:28 PM, Alexander Frolov wrote:
>>>>>
>>>>> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ss...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> Hi Alexander,
>>>>>>
>>>>>>>
>>>>>>> what do you use as type for your vertex ids? It looks that you are
>>>>>>> trying
>>>>>>> to use longs, while IntIntNullTextInputFormat only provides ints,
>>>>>>> that
>>>>>>> could be the error.
>>>>>>>
>>>>>>> --sebastian
>>>>>>>
>>>>>>>
>>>>>>> I also thought so, but I could not understand why it regard it as
>>>>>>> long.
>>>>>>>
>>>>>> I just use -vif org.apache.giraph.io.formats.
>>>>>> IntIntNullTextInputFormat
>>>>>> -vip /user/hduser/input/test -vof
>>>>>>
>>>>>> and file consists only of two lines:
>>>>>>
>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>> /user/hduser/input/test
>>>>>> 0 1
>>>>>> 1 0
>>>>>>
>>>>>> I dont know any other means to specify type of vertex ids.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>>>
>>>>>>>> I am trying to load data in from file in IntIntNullTextInputFormat.
>>>>>>>> But I
>>>>>>>> get is the following message:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>>>>> org.apache.giraph.GiraphRunner
>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>>>>>> /user/hduser/input/test -vof
>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>>>>>
>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input
>>>>>>>> format
>>>>>>>> specified. Ensure your InputFormat does not require one.
>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output
>>>>>>>> format
>>>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>>>>> checkClassTypes: vertex index types not assignable, computation -
>>>>>>>> class
>>>>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>>>>>>> org.apache.hadoop.io.IntWritable
>>>>>>>> at
>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>>>>>> GiraphConfigurationValidator.java:381)
>>>>>>>> at
>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>>>>>> java:226)
>>>>>>>> at
>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>>> validateConfiguration(
>>>>>>>> GiraphConfigurationValidator.java:141)
>>>>>>>> at
>>>>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>>>>> ConfigurationUtils.java:214)
>>>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>>>>> NativeMethodAccessorImpl.java:57)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> The input file is here (just for testing):
>>>>>>>>
>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>>>> /user/hduser/input/test
>>>>>>>> 0 1
>>>>>>>> 1 0
>>>>>>>>
>>>>>>>> Could you please help with this issue?
>>>>>>>>
>>>>>>>> Best,
>>>>>>>> Alex
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: IntIntNullTextInputFormat problem
Posted by Alexander Frolov <al...@gmail.com>.
On Mon, Feb 3, 2014 at 9:06 PM, Sebastian Schelter <ss...@apache.org> wrote:
> If you just wanna play with Giraph, I suggest you start with the PageRank
> example:
>
> https://giraph.apache.org/pagerank.html
I have seen it. I think there is no much difference in SSSP and PageRank
examples. By the way, I have not found LongDoubleNullTextInputFormat in my
sources (I am using release-1.0.0-RC2-152-g585511f) which is mentioned
there.
The same issue with SimplePageRankComputation.
[hduser@localhost giraph]$ $HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimplePageRankComputation -vif
org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
/user/hduser/input/test -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/output/amazon-shortestpaths -w 1
14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentException:
checkClassTypes: vertex value types not assignable, computation - class
org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
org.apache.hadoop.io.LongWritable
at
org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at
org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
at
org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at
org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> --sebastian
>
>
> On 02/03/2014 05:49 PM, Alexander Frolov wrote:
>
>> I've tried .LongLongNullTextInputFormat and got this:
>>
>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
>> /user/hduser/input/test -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/hduser/output/amazon-shortestpaths -w 1
>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format
>> specified. Ensure your OutputFormat does not require one.
>> Exception in thread "main" java.lang.IllegalArgumentException:
>> checkClassTypes: vertex value types not assignable, computation - class
>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
>> org.apache.hadoop.io.LongWritable
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>> GiraphConfigurationValidator.java:381)
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.
>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>> java:228)
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(
>> GiraphConfigurationValidator.java:141)
>> at
>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>> ConfigurationUtils.java:214)
>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>>
>>
>> On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov
>> <al...@gmail.com>wrote:
>>
>>
>>>
>>>
>>> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <ss...@apache.org>
>>> wrote
>>>
>>> What types does the vertex class that you use in your computation
>>>> require?
>>>>
>>>
>>>
>>> I see.
>>>
>>> public class SimpleShortestPathsComputation extends BasicComputation<
>>> LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..}
>>>
>>> So that is a mistake.
>>> Thank you very much.
>>>
>>>
>> Is there any means to import data from file like this (amazon0601.txt
>>> from
>>> SNAP data collection):
>>>
>>> 0 1
>>> 0 2
>>> 0 3
>>> 0 4
>>> 0 5
>>> 0 6
>>> 0 7
>>> 0 8
>>> 0 9
>>> 0 10
>>> ...
>>>
>>> and to perform basic tests from giraph-examples? Most of the tests are
>>> using LongWritable for vertex id.
>>>
>>>
>>>
>>>> On 02/03/2014 05:28 PM, Alexander Frolov wrote:
>>>>
>>>> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ss...@apache.org>
>>>>> wrote:
>>>>>
>>>>> Hi Alexander,
>>>>>
>>>>>>
>>>>>> what do you use as type for your vertex ids? It looks that you are
>>>>>> trying
>>>>>> to use longs, while IntIntNullTextInputFormat only provides ints, that
>>>>>> could be the error.
>>>>>>
>>>>>> --sebastian
>>>>>>
>>>>>>
>>>>>> I also thought so, but I could not understand why it regard it as
>>>>>> long.
>>>>>>
>>>>> I just use -vif org.apache.giraph.io.formats.
>>>>> IntIntNullTextInputFormat
>>>>> -vip /user/hduser/input/test -vof
>>>>>
>>>>> and file consists only of two lines:
>>>>>
>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>> /user/hduser/input/test
>>>>> 0 1
>>>>> 1 0
>>>>>
>>>>> I dont know any other means to specify type of vertex ids.
>>>>>
>>>>>
>>>>>
>>>>> On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>>>
>>>>>>> I am trying to load data in from file in IntIntNullTextInputFormat.
>>>>>>> But I
>>>>>>> get is the following message:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>>>> org.apache.giraph.GiraphRunner
>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>>>>> /user/hduser/input/test -vof
>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>>>>
>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input format
>>>>>>> specified. Ensure your InputFormat does not require one.
>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output
>>>>>>> format
>>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>>>> checkClassTypes: vertex index types not assignable, computation -
>>>>>>> class
>>>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>>>>>> org.apache.hadoop.io.IntWritable
>>>>>>> at
>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>>>>> GiraphConfigurationValidator.java:381)
>>>>>>> at
>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>>>>> java:226)
>>>>>>> at
>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>> validateConfiguration(
>>>>>>> GiraphConfigurationValidator.java:141)
>>>>>>> at
>>>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>>>> ConfigurationUtils.java:214)
>>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>>>> NativeMethodAccessorImpl.java:57)
>>>>>>> at
>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> The input file is here (just for testing):
>>>>>>>
>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>>> /user/hduser/input/test
>>>>>>> 0 1
>>>>>>> 1 0
>>>>>>>
>>>>>>> Could you please help with this issue?
>>>>>>>
>>>>>>> Best,
>>>>>>> Alex
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: IntIntNullTextInputFormat problem
Posted by Sebastian Schelter <ss...@apache.org>.
If you just wanna play with Giraph, I suggest you start with the
PageRank example:
https://giraph.apache.org/pagerank.html
--sebastian
On 02/03/2014 05:49 PM, Alexander Frolov wrote:
> I've tried .LongLongNullTextInputFormat and got this:
>
> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
> /user/hduser/input/test -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/hduser/output/amazon-shortestpaths -w 1
> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format
> specified. Ensure your OutputFormat does not require one.
> Exception in thread "main" java.lang.IllegalArgumentException:
> checkClassTypes: vertex value types not assignable, computation - class
> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
> org.apache.hadoop.io.LongWritable
> at
> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
> at
> org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
> at
> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
> at
> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
>
>
> On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov
> <al...@gmail.com>wrote:
>
>>
>>
>>
>> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <ss...@apache.org> wrote
>>
>>> What types does the vertex class that you use in your computation require?
>>
>>
>> I see.
>>
>> public class SimpleShortestPathsComputation extends BasicComputation<
>> LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..}
>>
>> So that is a mistake.
>> Thank you very much.
>>
>
>> Is there any means to import data from file like this (amazon0601.txt from
>> SNAP data collection):
>>
>> 0 1
>> 0 2
>> 0 3
>> 0 4
>> 0 5
>> 0 6
>> 0 7
>> 0 8
>> 0 9
>> 0 10
>> ...
>>
>> and to perform basic tests from giraph-examples? Most of the tests are
>> using LongWritable for vertex id.
>>
>>
>>>
>>> On 02/03/2014 05:28 PM, Alexander Frolov wrote:
>>>
>>>> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ss...@apache.org>
>>>> wrote:
>>>>
>>>> Hi Alexander,
>>>>>
>>>>> what do you use as type for your vertex ids? It looks that you are
>>>>> trying
>>>>> to use longs, while IntIntNullTextInputFormat only provides ints, that
>>>>> could be the error.
>>>>>
>>>>> --sebastian
>>>>>
>>>>>
>>>>> I also thought so, but I could not understand why it regard it as long.
>>>> I just use -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat
>>>> -vip /user/hduser/input/test -vof
>>>>
>>>> and file consists only of two lines:
>>>>
>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>> /user/hduser/input/test
>>>> 0 1
>>>> 1 0
>>>>
>>>> I dont know any other means to specify type of vertex ids.
>>>>
>>>>
>>>>
>>>>> On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>>>>
>>>>> Hello,
>>>>>>
>>>>>> I am trying to load data in from file in IntIntNullTextInputFormat.
>>>>>> But I
>>>>>> get is the following message:
>>>>>>
>>>>>>
>>>>>>
>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>>> org.apache.giraph.GiraphRunner
>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>>>> /user/hduser/input/test -vof
>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>>>
>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input format
>>>>>> specified. Ensure your InputFormat does not require one.
>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output format
>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>>> checkClassTypes: vertex index types not assignable, computation - class
>>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>>>>> org.apache.hadoop.io.IntWritable
>>>>>> at
>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>>>> GiraphConfigurationValidator.java:381)
>>>>>> at
>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>>>> java:226)
>>>>>> at
>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>> validateConfiguration(
>>>>>> GiraphConfigurationValidator.java:141)
>>>>>> at
>>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>>> ConfigurationUtils.java:214)
>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>> at
>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>>> NativeMethodAccessorImpl.java:57)
>>>>>> at
>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> The input file is here (just for testing):
>>>>>>
>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>> /user/hduser/input/test
>>>>>> 0 1
>>>>>> 1 0
>>>>>>
>>>>>> Could you please help with this issue?
>>>>>>
>>>>>> Best,
>>>>>> Alex
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: IntIntNullTextInputFormat problem
Posted by Alexander Frolov <al...@gmail.com>.
I've tried .LongLongNullTextInputFormat and got this:
[hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
/user/hduser/input/test -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/output/amazon-shortestpaths -w 1
14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentException:
checkClassTypes: vertex value types not assignable, computation - class
org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
org.apache.hadoop.io.LongWritable
at
org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at
org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
at
org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at
org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov
<al...@gmail.com>wrote:
>
>
>
> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <ss...@apache.org> wrote
>
>> What types does the vertex class that you use in your computation require?
>
>
> I see.
>
> public class SimpleShortestPathsComputation extends BasicComputation<
> LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..}
>
> So that is a mistake.
> Thank you very much.
>
> Is there any means to import data from file like this (amazon0601.txt from
> SNAP data collection):
>
> 0 1
> 0 2
> 0 3
> 0 4
> 0 5
> 0 6
> 0 7
> 0 8
> 0 9
> 0 10
> ...
>
> and to perform basic tests from giraph-examples? Most of the tests are
> using LongWritable for vertex id.
>
>
>>
>> On 02/03/2014 05:28 PM, Alexander Frolov wrote:
>>
>>> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ss...@apache.org>
>>> wrote:
>>>
>>> Hi Alexander,
>>>>
>>>> what do you use as type for your vertex ids? It looks that you are
>>>> trying
>>>> to use longs, while IntIntNullTextInputFormat only provides ints, that
>>>> could be the error.
>>>>
>>>> --sebastian
>>>>
>>>>
>>>> I also thought so, but I could not understand why it regard it as long.
>>> I just use -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat
>>> -vip /user/hduser/input/test -vof
>>>
>>> and file consists only of two lines:
>>>
>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>> /user/hduser/input/test
>>> 0 1
>>> 1 0
>>>
>>> I dont know any other means to specify type of vertex ids.
>>>
>>>
>>>
>>>> On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>>>
>>>> Hello,
>>>>>
>>>>> I am trying to load data in from file in IntIntNullTextInputFormat.
>>>>> But I
>>>>> get is the following message:
>>>>>
>>>>>
>>>>>
>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>> org.apache.giraph.GiraphRunner
>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>>> /user/hduser/input/test -vof
>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>>
>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input format
>>>>> specified. Ensure your InputFormat does not require one.
>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output format
>>>>> specified. Ensure your OutputFormat does not require one.
>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>> checkClassTypes: vertex index types not assignable, computation - class
>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>>>> org.apache.hadoop.io.IntWritable
>>>>> at
>>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>>> GiraphConfigurationValidator.java:381)
>>>>> at
>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>>> java:226)
>>>>> at
>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>> validateConfiguration(
>>>>> GiraphConfigurationValidator.java:141)
>>>>> at
>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>> ConfigurationUtils.java:214)
>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>> at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>> NativeMethodAccessorImpl.java:57)
>>>>> at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> The input file is here (just for testing):
>>>>>
>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>> /user/hduser/input/test
>>>>> 0 1
>>>>> 1 0
>>>>>
>>>>> Could you please help with this issue?
>>>>>
>>>>> Best,
>>>>> Alex
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: IntIntNullTextInputFormat problem
Posted by Alexander Frolov <al...@gmail.com>.
On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <ss...@apache.org> wrote
> What types does the vertex class that you use in your computation require?
I see.
public class SimpleShortestPathsComputation extends BasicComputation<
LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..}
So that is a mistake.
Thank you very much.
Is there any means to import data from file like this (amazon0601.txt from
SNAP data collection):
0 1
0 2
0 3
0 4
0 5
0 6
0 7
0 8
0 9
0 10
...
and to perform basic tests from giraph-examples? Most of the tests are
using LongWritable for vertex id.
>
> On 02/03/2014 05:28 PM, Alexander Frolov wrote:
>
>> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ss...@apache.org>
>> wrote:
>>
>> Hi Alexander,
>>>
>>> what do you use as type for your vertex ids? It looks that you are trying
>>> to use longs, while IntIntNullTextInputFormat only provides ints, that
>>> could be the error.
>>>
>>> --sebastian
>>>
>>>
>>> I also thought so, but I could not understand why it regard it as long.
>> I just use -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat
>> -vip /user/hduser/input/test -vof
>>
>> and file consists only of two lines:
>>
>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>> /user/hduser/input/test
>> 0 1
>> 1 0
>>
>> I dont know any other means to specify type of vertex ids.
>>
>>
>>
>>> On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>>
>>> Hello,
>>>>
>>>> I am trying to load data in from file in IntIntNullTextInputFormat. But
>>>> I
>>>> get is the following message:
>>>>
>>>>
>>>>
>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>> org.apache.giraph.GiraphRunner
>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>> /user/hduser/input/test -vof
>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>
>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input format
>>>> specified. Ensure your InputFormat does not require one.
>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output format
>>>> specified. Ensure your OutputFormat does not require one.
>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>> checkClassTypes: vertex index types not assignable, computation - class
>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>>> org.apache.hadoop.io.IntWritable
>>>> at
>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>> GiraphConfigurationValidator.java:381)
>>>> at
>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>> java:226)
>>>> at
>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>> validateConfiguration(
>>>> GiraphConfigurationValidator.java:141)
>>>> at
>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>> ConfigurationUtils.java:214)
>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>> NativeMethodAccessorImpl.java:57)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>> DelegatingMethodAccessorImpl.java:43)
>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>
>>>>
>>>>
>>>>
>>>> The input file is here (just for testing):
>>>>
>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>> /user/hduser/input/test
>>>> 0 1
>>>> 1 0
>>>>
>>>> Could you please help with this issue?
>>>>
>>>> Best,
>>>> Alex
>>>>
>>>>
>>>>
>>>
>>
>
Re: IntIntNullTextInputFormat problem
Posted by Sebastian Schelter <ss...@apache.org>.
What types does the vertex class that you use in your computation require?
On 02/03/2014 05:28 PM, Alexander Frolov wrote:
> On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ss...@apache.org> wrote:
>
>> Hi Alexander,
>>
>> what do you use as type for your vertex ids? It looks that you are trying
>> to use longs, while IntIntNullTextInputFormat only provides ints, that
>> could be the error.
>>
>> --sebastian
>>
>>
> I also thought so, but I could not understand why it regard it as long.
> I just use -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat
> -vip /user/hduser/input/test -vof
>
> and file consists only of two lines:
>
> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
> /user/hduser/input/test
> 0 1
> 1 0
>
> I dont know any other means to specify type of vertex ids.
>
>
>>
>> On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>
>>> Hello,
>>>
>>> I am trying to load data in from file in IntIntNullTextInputFormat. But I
>>> get is the following message:
>>>
>>>
>>>
>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>> org.apache.giraph.GiraphRunner
>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>> /user/hduser/input/test -vof
>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>
>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input format
>>> specified. Ensure your InputFormat does not require one.
>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output format
>>> specified. Ensure your OutputFormat does not require one.
>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>> checkClassTypes: vertex index types not assignable, computation - class
>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>> org.apache.hadoop.io.IntWritable
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>> GiraphConfigurationValidator.java:381)
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>> java:226)
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(
>>> GiraphConfigurationValidator.java:141)
>>> at
>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>> ConfigurationUtils.java:214)
>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>> NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>> DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>
>>>
>>>
>>>
>>> The input file is here (just for testing):
>>>
>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>> /user/hduser/input/test
>>> 0 1
>>> 1 0
>>>
>>> Could you please help with this issue?
>>>
>>> Best,
>>> Alex
>>>
>>>
>>
>
Re: IntIntNullTextInputFormat problem
Posted by Alexander Frolov <al...@gmail.com>.
On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ss...@apache.org> wrote:
> Hi Alexander,
>
> what do you use as type for your vertex ids? It looks that you are trying
> to use longs, while IntIntNullTextInputFormat only provides ints, that
> could be the error.
>
> --sebastian
>
>
I also thought so, but I could not understand why it regard it as long.
I just use -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat
-vip /user/hduser/input/test -vof
and file consists only of two lines:
[hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
/user/hduser/input/test
0 1
1 0
I dont know any other means to specify type of vertex ids.
>
> On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>
>> Hello,
>>
>> I am trying to load data in from file in IntIntNullTextInputFormat. But I
>> get is the following message:
>>
>>
>>
>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>> /user/hduser/input/test -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/hduser/output/amazon-shortestpaths -w 1
>>
>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output format
>> specified. Ensure your OutputFormat does not require one.
>> Exception in thread "main" java.lang.IllegalArgumentException:
>> checkClassTypes: vertex index types not assignable, computation - class
>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>> org.apache.hadoop.io.IntWritable
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>> GiraphConfigurationValidator.java:381)
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.
>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>> java:226)
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(
>> GiraphConfigurationValidator.java:141)
>> at
>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>> ConfigurationUtils.java:214)
>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>>
>>
>> The input file is here (just for testing):
>>
>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>> /user/hduser/input/test
>> 0 1
>> 1 0
>>
>> Could you please help with this issue?
>>
>> Best,
>> Alex
>>
>>
>
Re: IntIntNullTextInputFormat problem
Posted by Sebastian Schelter <ss...@apache.org>.
Hi Alexander,
what do you use as type for your vertex ids? It looks that you are
trying to use longs, while IntIntNullTextInputFormat only provides ints,
that could be the error.
--sebastian
On 02/03/2014 05:06 PM, Alexander Frolov wrote:
> Hello,
>
> I am trying to load data in from file in IntIntNullTextInputFormat. But I
> get is the following message:
>
>
>
> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
> /user/hduser/input/test -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/hduser/output/amazon-shortestpaths -w 1
>
> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output format
> specified. Ensure your OutputFormat does not require one.
> Exception in thread "main" java.lang.IllegalArgumentException:
> checkClassTypes: vertex index types not assignable, computation - class
> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
> org.apache.hadoop.io.IntWritable
> at
> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
> at
> org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:226)
> at
> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
> at
> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
>
>
> The input file is here (just for testing):
>
> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
> /user/hduser/input/test
> 0 1
> 1 0
>
> Could you please help with this issue?
>
> Best,
> Alex
>