You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by "Ramani, Arun" <ar...@paypal.com> on 2013/08/29 23:05:50 UTC

Passing Custom Arguments for giraph.zkList

Hi,

I am trying to pass a zookeeper quorum to my giraph job and it throws the following exception:

13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format specified. Ensure your OutputFormat does not require one.
13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument [giraph.zkList] to zk1 in GiraphConfiguration
Exception in thread "main" java.lang.IllegalArgumentException: Unable to parse custom  argument: zk2:port
        at org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288)
        at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
        at com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:197)

I pass the zklist like this:

Hadoop jar GRAPH.jar <CLASSNAME> -vip <CLASS NAME> -vif <CLASS NAME> -wc <CLASS NAME> -w <worker number> -ca giraph.zkList="zk1:<port>,zk2:<port>,zk3:<port>,zk4:<port>,zk5:<port>"

Please suggest what is wrong with this invocation.

Thanks
Arun Ramani

Re: Passing Custom Arguments for giraph.zkList

Posted by Claudio Martella <cl...@gmail.com>.
the problem is not the format of the string, but the way you're passing it.
Try passing it as -D giraph.zkList="..." before the giraphrunner options.
that should work.


On Thu, Aug 29, 2013 at 11:47 PM, Ramani, Arun <ar...@paypal.com> wrote:

>  Hi Claudio,
>
>  Yes zk1, zk2, zk3, zk4 and zk5 are all zookeeper hostnames. These 5
> hosts make a zookeeper quorum. Please let me know how to pass this.
>
>  Thanks
> Arun Ramani
>
>   From: Claudio Martella <cl...@gmail.com>
> Reply-To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Date: Thursday, August 29, 2013 2:18 PM
> To: "user@giraph.apache.org" <us...@giraph.apache.org>
> Subject: Re: Passing Custom Arguments for giraph.zkList
>
>   zk1 is supposed to be a hostname.
>
>
> On Thu, Aug 29, 2013 at 11:05 PM, Ramani, Arun <ar...@paypal.com> wrote:
>
>>  Hi,
>>
>>  I am trying to pass a zookeeper quorum to my giraph job and it throws
>> the following exception:
>>
>>  13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format
>> specified. Ensure your OutputFormat does not require one.
>> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument
>> [giraph.zkList] to zk1 in GiraphConfiguration
>> Exception in thread "main" java.lang.IllegalArgumentException: Unable to
>> parse custom  argument: zk2:port
>>         at
>> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288)
>>         at
>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
>>         at
>> com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>         at
>> com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
>>
>>  I pass the zklist like this:
>>
>>  Hadoop jar GRAPH.jar <CLASSNAME> -vip <CLASS NAME> -vif <CLASS NAME>
>> -wc <CLASS NAME> -w <worker number> -ca
>> giraph.zkList="zk1:<port>,zk2:<port>,zk3:<port>,zk4:<port>,zk5:<port>"
>>
>>  Please suggest what is wrong with this invocation.
>>
>>  Thanks
>>  Arun Ramani
>>
>
>
>
>  --
>    Claudio Martella
>    claudio.martella@gmail.com
>



-- 
   Claudio Martella
   claudio.martella@gmail.com

Re: Passing Custom Arguments for giraph.zkList

Posted by "Ramani, Arun" <ar...@paypal.com>.
Hi Claudio,

Yes zk1, zk2, zk3, zk4 and zk5 are all zookeeper hostnames. These 5 hosts make a zookeeper quorum. Please let me know how to pass this.

Thanks
Arun Ramani

From: Claudio Martella <cl...@gmail.com>>
Reply-To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Date: Thursday, August 29, 2013 2:18 PM
To: "user@giraph.apache.org<ma...@giraph.apache.org>" <us...@giraph.apache.org>>
Subject: Re: Passing Custom Arguments for giraph.zkList

zk1 is supposed to be a hostname.


On Thu, Aug 29, 2013 at 11:05 PM, Ramani, Arun <ar...@paypal.com>> wrote:
Hi,

I am trying to pass a zookeeper quorum to my giraph job and it throws the following exception:

13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format specified. Ensure your OutputFormat does not require one.
13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument [giraph.zkList] to zk1 in GiraphConfiguration
Exception in thread "main" java.lang.IllegalArgumentException: Unable to parse custom  argument: zk2:port
        at org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288)
        at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
        at com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
        at com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:197)

I pass the zklist like this:

Hadoop jar GRAPH.jar <CLASSNAME> -vip <CLASS NAME> -vif <CLASS NAME> -wc <CLASS NAME> -w <worker number> -ca giraph.zkList="zk1:<port>,zk2:<port>,zk3:<port>,zk4:<port>,zk5:<port>"

Please suggest what is wrong with this invocation.

Thanks
Arun Ramani



--
   Claudio Martella
   claudio.martella@gmail.com<ma...@gmail.com>

Re: Passing Custom Arguments for giraph.zkList

Posted by Claudio Martella <cl...@gmail.com>.
zk1 is supposed to be a hostname.


On Thu, Aug 29, 2013 at 11:05 PM, Ramani, Arun <ar...@paypal.com> wrote:

>  Hi,
>
>  I am trying to pass a zookeeper quorum to my giraph job and it throws
> the following exception:
>
>  13/08/29 13:14:38 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: No output format
> specified. Ensure your OutputFormat does not require one.
> 13/08/29 13:14:38 INFO utils.ConfigurationUtils: Setting custom argument
> [giraph.zkList] to zk1 in GiraphConfiguration
> Exception in thread "main" java.lang.IllegalArgumentException: Unable to
> parse custom  argument: zk2:port
>         at
> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:288)
>         at
> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:147)
>         at
> com.paypal.risk.rd.giraph.AccountPropagation.run(AccountPropagation.java:46)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>         at
> com.paypal.risk.rd.giraph.AccountPropagation.main(AccountPropagation.java:98)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
>
>  I pass the zklist like this:
>
>  Hadoop jar GRAPH.jar <CLASSNAME> -vip <CLASS NAME> -vif <CLASS NAME> -wc
> <CLASS NAME> -w <worker number> -ca
> giraph.zkList="zk1:<port>,zk2:<port>,zk3:<port>,zk4:<port>,zk5:<port>"
>
>  Please suggest what is wrong with this invocation.
>
>  Thanks
> Arun Ramani
>



-- 
   Claudio Martella
   claudio.martella@gmail.com