(spark) branch master updated: [SPARK-49190][K8S][FOLLOW-UP] Add how to populate executor log url vars with env vars

[do...@apache.org]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new 47c0b55e06d1 [SPARK-49190][K8S][FOLLOW-UP] Add how to populate executor log url vars with env vars
47c0b55e06d1 is described below

commit 47c0b55e06d1dd8ec3a8dcee6d1e962c1cf0b942
Author: Enrico Minack <gi...@enrico.minack.dev>
AuthorDate: Tue Aug 20 08:00:22 2024 -0700

    [SPARK-49190][K8S][FOLLOW-UP] Add how to populate executor log url vars with env vars
    
    ### What changes were proposed in this pull request?
    Mentions in the docs how to populate custom executor log url vars with environment variable's values.
    
    ### Why are the changes needed?
    This feature exists since 3.0.0 but is not documented elsewhere.
    
    ### Does this PR introduce _any_ user-facing change?
    Improves documentation.
    
    ### How was this patch tested?
    Not tested.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    No.
    
    Closes #47801 from EnricoMi/docs-kubernetes-executor-log.
    
    Authored-by: Enrico Minack <gi...@enrico.minack.dev>
    Signed-off-by: Dongjoon Hyun <do...@apache.org>
---
 docs/running-on-kubernetes.md | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md
index 043c08bee064..a5fa7e20c495 100644
--- a/docs/running-on-kubernetes.md
+++ b/docs/running-on-kubernetes.md
@@ -436,6 +436,13 @@ When there exists a log collection system, you can expose it at Spark Driver `Ex
 spark.ui.custom.executor.log.url='https://log-server/log?appId={{APP_ID}}&execId={{EXECUTOR_ID}}'
 ```
 
+You can add additional custom variables to this url template, populated with the values of existing executor environment variables like
+
+```
+spark.executorEnv.SPARK_EXECUTOR_ATTRIBUTE_YOUR_VAR='$(EXISTING_EXECUTOR_ENV_VAR)'
+spark.ui.custom.executor.log.url='https://log-server/log?appId={{APP_ID}}&execId={{EXECUTOR_ID}}&your_var={{YOUR_VAR}}'
+```
+
 ### Accessing Driver UI
 
 The UI associated with any application can be accessed locally using


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org