You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Jon Bringhurst (Jira)" <ji...@apache.org> on 2023/01/31 19:08:00 UTC
[jira] [Updated] (SAMZA-2774) Fix incorrect symlinks used to generate classpath in run-class.sh
[ https://issues.apache.org/jira/browse/SAMZA-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jon Bringhurst updated SAMZA-2774:
----------------------------------
Description:
Manifest generation (application-level) is pointing to container-level symlinks instead of application-level files.
This will cause issues when multiple containers for an application land on a single host and the container the manifest is pointing to gets cleaned up.
{noformat}
[yyy@zzz filecache]$ cat /export/content/data/samsa-yarn/usercache/vapp5251/appcache/application_1668550466222_0989/filecache/10/xxx-0.0.672.tgz/classpath_workspace/manifest.txt|tail -n 2
/export/content/data/samsa-yarn/usercache/yyy/appcache/application_1668550466222_0989/container_e114_1668550466222_0989_01_000003/__package/lib/zstd-jni-1.5.2-3.jar
[yyy@zzz filecache]$
{noformat}
Something like "realpath" (but not realpath, since it doesn't work with osx) should be used in "run-class.sh" to resolve the container-level symlinks to application-level files.
> Fix incorrect symlinks used to generate classpath in run-class.sh
> -----------------------------------------------------------------
>
> Key: SAMZA-2774
> URL: https://issues.apache.org/jira/browse/SAMZA-2774
> Project: Samza
> Issue Type: Improvement
> Reporter: Jon Bringhurst
> Priority: Major
>
> Manifest generation (application-level) is pointing to container-level symlinks instead of application-level files.
> This will cause issues when multiple containers for an application land on a single host and the container the manifest is pointing to gets cleaned up.
> {noformat}
> [yyy@zzz filecache]$ cat /export/content/data/samsa-yarn/usercache/vapp5251/appcache/application_1668550466222_0989/filecache/10/xxx-0.0.672.tgz/classpath_workspace/manifest.txt|tail -n 2
> /export/content/data/samsa-yarn/usercache/yyy/appcache/application_1668550466222_0989/container_e114_1668550466222_0989_01_000003/__package/lib/zstd-jni-1.5.2-3.jar
> [yyy@zzz filecache]$
> {noformat}
> Something like "realpath" (but not realpath, since it doesn't work with osx) should be used in "run-class.sh" to resolve the container-level symlinks to application-level files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)