You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Fred Ji (JIRA)" <ji...@apache.org> on 2017/03/16 19:30:41 UTC
[jira] [Updated] (SAMZA-1143) Universal config support for
localized resource
[ https://issues.apache.org/jira/browse/SAMZA-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Fred Ji updated SAMZA-1143:
---------------------------
Description:
Currently, every time we add a new resource for localization, we need to modify the Samza code, as the preparation for the localResource is coded for each specific localizer, such as "yarn.package.path" for localizing the package.
But sometimes, we need to get different resources for localization, one use case is to get the certificate for the samza app to communicate with other services. We do not know details until we use them, so it is better to keep the context for that in the configuration so that the users who are using them can control these resource localization.
A common configuration for localizer resource would be like the following:
key: yarn.localizer.resource.<resourceVisibility>.<resourceType>.<resourceName>
e.g. localizer.resource.APPLICATION.FILE.identity
where the valid resourceVisibility is a string format of LocalResourceVisibility(enum):
PUBLIC, PRIVATE, or APPLICATION;
and the valid resourceType is a string format of LocalResourceType (enum):
ARCHIVE, FILE, PATTERN;
e.g. localizer.resource.APPLICATION.FILE.identity
the valid resourceName could be any non empty String.
value: <resourcePath>
where resourcePath is a string format of the URL, e.g. http://hostname.com/test, hdfs://host.com/readme, file:///temp/readme
was:
Currently, every time we add a new resource for localization, we need to modify the Samza code, as the preparation for the localResource is coded for each specific localizer, such as "yarn.package.path" for localizing the package.
But sometimes, we need to get different resources for localization, one use case is to get the certificate for the samza app to communicate with other services. We do not know details until we use them, so it is better to keep the context for that in the configuration so that the users who are using them can control these resource localization.
A common configuration for localizer resource would be like the following:
key: yarn.localizer.resource.<resourceVisibility>.<resourceType>.<resourceName>
value: <resourcePath>
> Universal config support for localized resource
> -----------------------------------------------
>
> Key: SAMZA-1143
> URL: https://issues.apache.org/jira/browse/SAMZA-1143
> Project: Samza
> Issue Type: New Feature
> Reporter: Fred Ji
>
> Currently, every time we add a new resource for localization, we need to modify the Samza code, as the preparation for the localResource is coded for each specific localizer, such as "yarn.package.path" for localizing the package.
> But sometimes, we need to get different resources for localization, one use case is to get the certificate for the samza app to communicate with other services. We do not know details until we use them, so it is better to keep the context for that in the configuration so that the users who are using them can control these resource localization.
> A common configuration for localizer resource would be like the following:
> key: yarn.localizer.resource.<resourceVisibility>.<resourceType>.<resourceName>
> e.g. localizer.resource.APPLICATION.FILE.identity
> where the valid resourceVisibility is a string format of LocalResourceVisibility(enum):
> PUBLIC, PRIVATE, or APPLICATION;
> and the valid resourceType is a string format of LocalResourceType (enum):
> ARCHIVE, FILE, PATTERN;
> e.g. localizer.resource.APPLICATION.FILE.identity
> the valid resourceName could be any non empty String.
> value: <resourcePath>
> where resourcePath is a string format of the URL, e.g. http://hostname.com/test, hdfs://host.com/readme, file:///temp/readme
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)