You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by "Ramani, Arun" <ar...@paypal.com> on 2013/08/29 23:05:50 UTC
Passing Custom Arguments for giraph.zkList
Hi,
I am trying to pass a zookeeper quorum to my giraph job and it throws the following exception:
13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format specified. Ensure your OutputFormat does not require one.
13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument [giraph.zkList] to zk1 in GiraphConfiguration
Exception in thread "main" java.lang.IllegalArgumentException: Unable to parse custom argument: zk2:port
at org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288)
at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
at com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
I pass the zklist like this:
Hadoop jar GRAPH.jar <CLASSNAME> -vip <CLASS NAME> -vif <CLASS NAME> -wc <CLASS NAME> -w <worker number> -ca giraph.zkList="zk1:<port>,zk2:<port>,zk3:<port>,zk4:<port>,zk5:<port>"
Please suggest what is wrong with this invocation.
Thanks
Arun Ramani
Re: Passing Custom Arguments for giraph.zkList
Posted by Claudio Martella <cl...@gmail.com>.
the problem is not the format of the string, but the way you're passing it.
Try passing it as -D giraph.zkList="..." before the giraphrunner options.
that should work.
On Thu, Aug 29, 2013 at 11:47 PM, Ramani, Arun <ar...@paypal.com> wrote:
> Hi Claudio,
>
> Yes zk1, zk2, zk3, zk4 and zk5 are all zookeeper hostnames. These 5
> hosts make a zookeeper quorum. Please let me know how to pass this.
>
> Thanks
> Arun Ramani
>
> From: Claudio Martella <cl...@gmail.com>
> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Date: Thursday, August 29, 2013 2:18 PM
> To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Subject: Re: Passing Custom Arguments for giraph.zkList
>
> zk1 is supposed to be a hostname.
>
>
> On Thu, Aug 29, 2013 at 11:05 PM, Ramani, Arun <ar...@paypal.com> wrote:
>
>> Hi,
>>
>> I am trying to pass a zookeeper quorum to my giraph job and it throws
>> the following exception:
>>
>> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format
>> specified. Ensure your OutputFormat does not require one.
>> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument
>> [giraph.zkList] to zk1 in GiraphConfiguration
>> Exception in thread "main" java.lang.IllegalArgumentException: Unable to
>> parse custom argument: zk2:port
>> at
>> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288)
>> at
>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
>> at
>> com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at
>> com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
>>
>> I pass the zklist like this:
>>
>> Hadoop jar GRAPH.jar <CLASSNAME> -vip <CLASS NAME> -vif <CLASS NAME>
>> -wc <CLASS NAME> -w <worker number> -ca
>> giraph.zkList="zk1:<port>,zk2:<port>,zk3:<port>,zk4:<port>,zk5:<port>"
>>
>> Please suggest what is wrong with this invocation.
>>
>> Thanks
>> Arun Ramani
>>
>
>
>
> --
> Claudio Martella
> claudio.martella@gmail.com
>
--
Claudio Martella
claudio.martella@gmail.com
Re: Passing Custom Arguments for giraph.zkList
Posted by "Ramani, Arun" <ar...@paypal.com>.
Hi Claudio,
Yes zk1, zk2, zk3, zk4 and zk5 are all zookeeper hostnames. These 5 hosts make a zookeeper quorum. Please let me know how to pass this.
Thanks
Arun Ramani
From: Claudio Martella <cl...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Thursday, August 29, 2013 2:18 PM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: Passing Custom Arguments for giraph.zkList
zk1 is supposed to be a hostname.
On Thu, Aug 29, 2013 at 11:05 PM, Ramani, Arun <ar...@paypal.com>> wrote:
Hi,
I am trying to pass a zookeeper quorum to my giraph job and it throws the following exception:
13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format specified. Ensure your OutputFormat does not require one.
13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument [giraph.zkList] to zk1 in GiraphConfiguration
Exception in thread "main" java.lang.IllegalArgumentException: Unable to parse custom argument: zk2:port
at org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288)
at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
at com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
I pass the zklist like this:
Hadoop jar GRAPH.jar <CLASSNAME> -vip <CLASS NAME> -vif <CLASS NAME> -wc <CLASS NAME> -w <worker number> -ca giraph.zkList="zk1:<port>,zk2:<port>,zk3:<port>,zk4:<port>,zk5:<port>"
Please suggest what is wrong with this invocation.
Thanks
Arun Ramani
--
Claudio Martella
claudio.martella@gmail.com<ma...@gmail.com>
Re: Passing Custom Arguments for giraph.zkList
Posted by Claudio Martella <cl...@gmail.com>.
zk1 is supposed to be a hostname.
On Thu, Aug 29, 2013 at 11:05 PM, Ramani, Arun <ar...@paypal.com> wrote:
> Hi,
>
> I am trying to pass a zookeeper quorum to my giraph job and it throws
> the following exception:
>
> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format
> specified. Ensure your OutputFormat does not require one.
> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument
> [giraph.zkList] to zk1 in GiraphConfiguration
> Exception in thread "main" java.lang.IllegalArgumentException: Unable to
> parse custom argument: zk2:port
> at
> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288)
> at
> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
> at
> com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at
> com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
>
> I pass the zklist like this:
>
> Hadoop jar GRAPH.jar <CLASSNAME> -vip <CLASS NAME> -vif <CLASS NAME> -wc
> <CLASS NAME> -w <worker number> -ca
> giraph.zkList="zk1:<port>,zk2:<port>,zk3:<port>,zk4:<port>,zk5:<port>"
>
> Please suggest what is wrong with this invocation.
>
> Thanks
> Arun Ramani
>
--
Claudio Martella
claudio.martella@gmail.com