You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by jeremy p <at...@gmail.com> on 2015/04/21 20:29:55 UTC
How to pass arguments to a Samza job
Hello all,
Is there a way to pass arguments to a Samza job from the command line? If
not, is there any way to pass arguments to a Samza job besides the
.properties file? Also, is there a way to pass application-specific
properties using the .properties file? (such as my.random.config.value=foo)
Thanks for taking a look!
--Jeremy
Re: How to pass arguments to a Samza job
Posted by jeremy p <at...@gmail.com>.
All good. Thank you both for the responses and verification!
On Tue, Apr 21, 2015 at 5:33 PM, Yan Fang <ya...@gmail.com> wrote:
> Hi Jeremy,
>
> Naveen is correct. You can also pass the argument through "--config". I
> overlooked it. Sorry for the confusion.
>
> Cheers,
>
> Fang, Yan
> yanfang724@gmail.com
>
> On Tue, Apr 21, 2015 at 2:10 PM, Naveen S <na...@gmail.com> wrote:
>
> > Hey Jeremy,
> > Sorry, it's probably because it's not documented, but
> > anything with "--config" will be passed to the property overrides. I just
> > tested it out with hello-samza and it does work.
> >
> > deploy/samza/bin/run-job.sh
> >
> --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory
> > --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties
> > --config test=3 --config test2=6
> >
> > On Tue, Apr 21, 2015 at 1:26 PM, jeremy p <
> athomewithagroovebox@gmail.com>
> > wrote:
> >
> > > I'm a little confused. Yan is saying that you cannot pass in command
> > line
> > > parameters, but Nav is saying that it is possible. I guess I'll just
> > build
> > > a small project and test it out to see which one is correct.
> > >
> > > Thank you both.
> > >
> > > On Tue, Apr 21, 2015 at 2:56 PM, Nav s <na...@gmail.com> wrote:
> > >
> > > > Hey Jeremy,
> > > > You can implement InitableTask
> > > > <
> > > >
> > >
> >
> http://samza.apache.org/learn/documentation/0.9/api/javadocs/org/apache/samza/task/InitableTask.html
> > > > >,
> > > > which should give you a handle to the config (You should be able to
> > > access
> > > > your configs from this class). You can pass commandline arguments
> with
> > > the
> > > > flag "--config key=value" in your run-job script.
> > > >
> > > >
> > > > On Tue, Apr 21, 2015 at 11:29 AM, jeremy p <
> > > athomewithagroovebox@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hello all,
> > > > >
> > > > > Is there a way to pass arguments to a Samza job from the command
> > line?
> > > > If
> > > > > not, is there any way to pass arguments to a Samza job besides the
> > > > > .properties file? Also, is there a way to pass
> application-specific
> > > > > properties using the .properties file? (such as
> > > > > my.random.config.value=foo)
> > > > >
> > > > > Thanks for taking a look!
> > > > >
> > > > > --Jeremy
> > > > >
> > > >
> > >
> >
>
Re: How to pass arguments to a Samza job
Posted by Yan Fang <ya...@gmail.com>.
Hi Jeremy,
Naveen is correct. You can also pass the argument through "--config". I
overlooked it. Sorry for the confusion.
Cheers,
Fang, Yan
yanfang724@gmail.com
On Tue, Apr 21, 2015 at 2:10 PM, Naveen S <na...@gmail.com> wrote:
> Hey Jeremy,
> Sorry, it's probably because it's not documented, but
> anything with "--config" will be passed to the property overrides. I just
> tested it out with hello-samza and it does work.
>
> deploy/samza/bin/run-job.sh
> --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory
> --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties
> --config test=3 --config test2=6
>
> On Tue, Apr 21, 2015 at 1:26 PM, jeremy p <at...@gmail.com>
> wrote:
>
> > I'm a little confused. Yan is saying that you cannot pass in command
> line
> > parameters, but Nav is saying that it is possible. I guess I'll just
> build
> > a small project and test it out to see which one is correct.
> >
> > Thank you both.
> >
> > On Tue, Apr 21, 2015 at 2:56 PM, Nav s <na...@gmail.com> wrote:
> >
> > > Hey Jeremy,
> > > You can implement InitableTask
> > > <
> > >
> >
> http://samza.apache.org/learn/documentation/0.9/api/javadocs/org/apache/samza/task/InitableTask.html
> > > >,
> > > which should give you a handle to the config (You should be able to
> > access
> > > your configs from this class). You can pass commandline arguments with
> > the
> > > flag "--config key=value" in your run-job script.
> > >
> > >
> > > On Tue, Apr 21, 2015 at 11:29 AM, jeremy p <
> > athomewithagroovebox@gmail.com
> > > >
> > > wrote:
> > >
> > > > Hello all,
> > > >
> > > > Is there a way to pass arguments to a Samza job from the command
> line?
> > > If
> > > > not, is there any way to pass arguments to a Samza job besides the
> > > > .properties file? Also, is there a way to pass application-specific
> > > > properties using the .properties file? (such as
> > > > my.random.config.value=foo)
> > > >
> > > > Thanks for taking a look!
> > > >
> > > > --Jeremy
> > > >
> > >
> >
>
Re: How to pass arguments to a Samza job
Posted by Naveen S <na...@gmail.com>.
Hey Jeremy,
Sorry, it's probably because it's not documented, but
anything with "--config" will be passed to the property overrides. I just
tested it out with hello-samza and it does work.
deploy/samza/bin/run-job.sh
--config-factory=org.apache.samza.config.factories.PropertiesConfigFactory
--config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties
--config test=3 --config test2=6
On Tue, Apr 21, 2015 at 1:26 PM, jeremy p <at...@gmail.com>
wrote:
> I'm a little confused. Yan is saying that you cannot pass in command line
> parameters, but Nav is saying that it is possible. I guess I'll just build
> a small project and test it out to see which one is correct.
>
> Thank you both.
>
> On Tue, Apr 21, 2015 at 2:56 PM, Nav s <na...@gmail.com> wrote:
>
> > Hey Jeremy,
> > You can implement InitableTask
> > <
> >
> http://samza.apache.org/learn/documentation/0.9/api/javadocs/org/apache/samza/task/InitableTask.html
> > >,
> > which should give you a handle to the config (You should be able to
> access
> > your configs from this class). You can pass commandline arguments with
> the
> > flag "--config key=value" in your run-job script.
> >
> >
> > On Tue, Apr 21, 2015 at 11:29 AM, jeremy p <
> athomewithagroovebox@gmail.com
> > >
> > wrote:
> >
> > > Hello all,
> > >
> > > Is there a way to pass arguments to a Samza job from the command line?
> > If
> > > not, is there any way to pass arguments to a Samza job besides the
> > > .properties file? Also, is there a way to pass application-specific
> > > properties using the .properties file? (such as
> > > my.random.config.value=foo)
> > >
> > > Thanks for taking a look!
> > >
> > > --Jeremy
> > >
> >
>
Re: How to pass arguments to a Samza job
Posted by jeremy p <at...@gmail.com>.
I'm a little confused. Yan is saying that you cannot pass in command line
parameters, but Nav is saying that it is possible. I guess I'll just build
a small project and test it out to see which one is correct.
Thank you both.
On Tue, Apr 21, 2015 at 2:56 PM, Nav s <na...@gmail.com> wrote:
> Hey Jeremy,
> You can implement InitableTask
> <
> http://samza.apache.org/learn/documentation/0.9/api/javadocs/org/apache/samza/task/InitableTask.html
> >,
> which should give you a handle to the config (You should be able to access
> your configs from this class). You can pass commandline arguments with the
> flag "--config key=value" in your run-job script.
>
>
> On Tue, Apr 21, 2015 at 11:29 AM, jeremy p <athomewithagroovebox@gmail.com
> >
> wrote:
>
> > Hello all,
> >
> > Is there a way to pass arguments to a Samza job from the command line?
> If
> > not, is there any way to pass arguments to a Samza job besides the
> > .properties file? Also, is there a way to pass application-specific
> > properties using the .properties file? (such as
> > my.random.config.value=foo)
> >
> > Thanks for taking a look!
> >
> > --Jeremy
> >
>
Re: How to pass arguments to a Samza job
Posted by Nav s <na...@gmail.com>.
Hey Jeremy,
You can implement InitableTask
<http://samza.apache.org/learn/documentation/0.9/api/javadocs/org/apache/samza/task/InitableTask.html>,
which should give you a handle to the config (You should be able to access
your configs from this class). You can pass commandline arguments with the
flag "--config key=value" in your run-job script.
On Tue, Apr 21, 2015 at 11:29 AM, jeremy p <at...@gmail.com>
wrote:
> Hello all,
>
> Is there a way to pass arguments to a Samza job from the command line? If
> not, is there any way to pass arguments to a Samza job besides the
> .properties file? Also, is there a way to pass application-specific
> properties using the .properties file? (such as
> my.random.config.value=foo)
>
> Thanks for taking a look!
>
> --Jeremy
>
Re: How to pass arguments to a Samza job
Posted by Yan Fang <ya...@gmail.com>.
Hi Jeremy,
Currently Samza does not support accepting from command. After SAMZA-655
<https://issues.apache.org/jira/browse/SAMZA-655> and SAMZA-348
<https://issues.apache.org/jira/browse/SAMZA-348> , you will have other two
options - environment variable and stream.
*Then How to pass argument from the properties file?*
1. specify the my.random.config.value=foo in the .properties file
2. in the task init() method, context.getSamzaContainerContext().config.get(
"my.random.config.value")
Hope this helps.
Thanks,
Fang, Yan
yanfang724@gmail.com
On Tue, Apr 21, 2015 at 11:29 AM, jeremy p <at...@gmail.com>
wrote:
> Hello all,
>
> Is there a way to pass arguments to a Samza job from the command line? If
> not, is there any way to pass arguments to a Samza job besides the
> .properties file? Also, is there a way to pass application-specific
> properties using the .properties file? (such as
> my.random.config.value=foo)
>
> Thanks for taking a look!
>
> --Jeremy
>