Access config properties inside Job

["Balusamy, Elangovan" <El...@altisource.com>]

Folks,

We are using an external component in one of our tasks, the external component persists some info to MySQL DB, the component reads database connection info from a specific property file that should be available in classpath.   Is there a way I can specify custom property files as part of Job runner so that the  property file is available in classpath for the samza task.

Based on the dev list I am aware of moving this to samza properties or accessing using relative path, but I would prefer something like below to job runner.

deploy/samza/bin/run-job.sh
--config-factory=org.apache.samza.config.factories.PropertiesConfigFactory
--config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties
--classpath-config-file=file://$PWD/deploy/samza/config/mycustom-peristence.properties


This will make mycustom-peristence.properties to be available in classpath.


Thanks
Elango
***********************************************************************************************************************

This email message and any attachments are intended solely for the use of the addressee. If you are not the intended recipient, you are prohibited from reading, disclosing, reproducing, distributing, disseminating or otherwise using this transmission. If you have received this message in error, please promptly notify the sender by reply email and immediately delete this message from your system. This message and any attachments may contain information that is confidential, privileged or exempt from disclosure. Delivery of this message to any person other than the intended recipient is not intended to waive any right or privilege. Message transmission is not guaranteed to be secure or free of software viruses.
***********************************************************************************************************************

Re: Access config properties inside Job

[Yan Fang <ya...@gmail.com>]

Hi Elango,

Currently Samza does not support having two config files, like
--classpath-config-file.
But there are a few workarounds:

1. you already mentioned putting the properties in the samza properties file

2. use key=value pair. so it would look like deploy/samza/bin/run-job.sh
--config-factory=org.apache.samza.config.factories.PropertiesConfigFactory
--config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties
*--your.config.property=your.config proerty.value*

3. if you are using master's code, we provide a
org.apache.samza.config.EnvironmentConfigRewriter
, which reads the Environment variables which start with SAMZA_. See
SAMZA-655 <https://issues.apache.org/jira/browse/SAMZA-655>

Thanks,

Fang, Yan
yanfang724@gmail.com

On Thu, May 14, 2015 at 4:35 AM, Balusamy, Elangovan <
Elangovan.Balusamy@altisource.com> wrote:

> Folks,
>
> We are using an external component in one of our tasks, the external
> component persists some info to MySQL DB, the component reads database
> connection info from a specific property file that should be available in
> classpath.   Is there a way I can specify custom property files as part of
> Job runner so that the  property file is available in classpath for the
> samza task.
>
> Based on the dev list I am aware of moving this to samza properties or
> accessing using relative path, but I would prefer something like below to
> job runner.
>
> deploy/samza/bin/run-job.sh
> --config-factory=org.apache.samza.config.factories.PropertiesConfigFactory
> --config-path=file://$PWD/deploy/samza/config/wikipedia-feed.properties
>
> --classpath-config-file=file://$PWD/deploy/samza/config/mycustom-peristence.properties
>
>
> This will make mycustom-peristence.properties to be available in classpath.
>
>
> Thanks
> Elango
>
> ***********************************************************************************************************************
>
> This email message and any attachments are intended solely for the use of
> the addressee. If you are not the intended recipient, you are prohibited
> from reading, disclosing, reproducing, distributing, disseminating or
> otherwise using this transmission. If you have received this message in
> error, please promptly notify the sender by reply email and immediately
> delete this message from your system. This message and any attachments may
> contain information that is confidential, privileged or exempt from
> disclosure. Delivery of this message to any person other than the intended
> recipient is not intended to waive any right or privilege. Message
> transmission is not guaranteed to be secure or free of software viruses.
>
> ***********************************************************************************************************************
>