You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@uniffle.apache.org by "maobaolong (via GitHub)" <gi...@apache.org> on 2024/08/09 09:42:29 UTC

[PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

maobaolong opened a new pull request, #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028

   
   <!--
   1. Title: [#<issue>] <type>(<scope>): <subject>
      Examples:
        - "[#123] feat(operator): support xxx"
        - "[#233] fix: check null before access result in xxx"
        - "[MINOR] refactor: fix typo in variable name"
        - "[MINOR] docs: fix typo in README"
        - "[#255] test: fix flaky test NameOfTheTest"
      Reference: https://www.conventionalcommits.org/en/v1.0.0/
   2. Contributor guidelines:
      https://github.com/apache/incubator-uniffle/blob/master/CONTRIBUTING.md
   3. If the PR is unfinished, please mark this PR as draft.
   -->
   
   ### What changes were proposed in this pull request?
   
   Fix log4j2.xml script and intellij launch conf to support output audit logs
   
   ### Why are the changes needed?
   
   Without this PR, there are some issues
   
   - Empty logs file generated.
   - The audit logs cannot generated in logs folder when launch Uniffle servers from intellij idea.
   
   ### Does this PR introduce _any_ user-facing change?
   
   No.
   
   ### How was this patch tested?
   
   Start servers and check the logs.
   
   ```
   ll incubator-uniffle/logs
   total 44K
   -rw-r--r-- 1 mbl staff 5.7K  8  9 17:36 coordinator.log
   -rw-r--r-- 1 mbl staff  625  8  9 17:36 coordinator_rpc_audit.log
   -rw-r--r-- 1 mbl staff 1.1K  8  9 17:34 dashboard.log
   -rw-r--r-- 1 mbl staff  15K  8  9 17:36 shuffle_server.log
   -rw-r--r-- 1 mbl staff  12K  8  9 17:36 shuffle_server_rpc_audit.log
   ```
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2287972444

   Could you elaborate a bit more on the problem? There's no audit log when running in prod or only under IntelliJ local run.
   
   I don't see significant difference between this and previous log4j2.xml and I tend to keep the log4j2.xml configuration compact and lean.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2290578909

   @advancedxy Changed follow your suggestion, but a little pain is that we have to config many path for shuffle-server launchConfiguration.
   
   
   Compare these two
   
   ```xml
   <option name="VM_PARAMETERS" value="-Dlog4j2.configurationFile=file:$PROJECT_DIR$/.idea/local_dev/log4j2.xml -Dlog.dir=$PROJECT_DIR$/logs/ -Dlogger.type=ShuffleServer" />
   ```
   
   ```xml
       <option name="VM_PARAMETERS" value="-Dlog4j2.configurationFile=file:$PROJECT_DIR$/.idea/local_dev/log4j2.xml -Dlog.path=$PROJECT_DIR$/logs/shuffle_server.log -Drpc.audit.log.path=$PROJECT_DIR$/logs/shuffle_server_rpc_audit.log -Dstorage.audit.log.path=$PROJECT_DIR$/logs/shuffle_server_storage_audit.log " />
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2288737932

   @advancedxy yes, in the case `1`,  the coordinator.log is correct, because the Process is `Coordinator`.
   
   In the case `2`, we start anything without through `start-xxx.sh` but through `launchConfiguration` of `intellij idea`, so we have to specify a bundle of jvm arguments about logs.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy merged PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong closed pull request #2028: [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs
URL: https://github.com/apache/incubator-uniffle/pull/2028


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2285280638

   @advancedxy Would you like to take a look at this PR? Thanks!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2277606585

   ## Test Results
    2 747 files  +31   2 747 suites  +31   5h 49m 2s :stopwatch: + 13m 30s
      981 tests ± 0     980 :white_check_mark: + 1   1 :zzz: ±0  0 :x:  - 1 
   12 298 runs  +60  12 283 :white_check_mark: +61  15 :zzz: ±0  0 :x:  - 1 
   
   Results for commit ca80b41f. ± Comparison against base commit 4b23c691.
   
   [test-results]:data:application/gzip;base64,H4sIADnqtWYC/03MTQ7CIBCG4as0rF0wFAt4GQNDSYj9MRRWxrsLDViW3zOT90OcX+aDPAYmuLgN5Eg+9tumoKPftyJUcZYp32P5UBLaeh4J8SR60cu/M10/Tvslw/Uxh7CHKiFtpQmMKVlni2aT499q9d6gq567j+K+rj7mQVBLajg4g45zMDAKBLSOUW0nK9FNFIQZmSbfH75xwEgSAQAA
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2292572910

   @advancedxy Thanks for your review and suggestion, PTAL.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2296350846

   Merged, thanks @maobaolong.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong closed pull request #2028: [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs
URL: https://github.com/apache/incubator-uniffle/pull/2028


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2288030501

   @advancedxy Thanks for your review!
   
   There are two main changes in this PR.
   
   1. I found there are empty audit log file generated with environment variable which is not belong to current process.
   
   For example, I found the following logs generated in coordinator node.
   
   # docker exec -it  rss-coordinator bash
   [rss]$ ll
   -rw-r--r-- 1 rssadmin rssadmin    0 Jul 24 17:29 '${sys:shuffle.server.storage.audit.log.path}'
   -rw-r--r-- 1 rssadmin rssadmin    0 Jul 24 17:29 '${sys:shuffle.server.rpc.audit.log.path}'
   
   So I added `createOnDemand="true"` to the appender.
   
   2. I start uniffle cluster by intellij idea and found the similar problem too, so I have to define the `shuffle.server.storage.audit.log.path` `shuffle.server.rpc.audit.log.path` `log.path` `coordinator.rpc.audit.path`, it make me unhappy for these mess thing, in this time, I remember the suggestion from your side is that we should define the log name in the inner script, we should left  necessary variables to user, `log.dir` and `logger.type` is enough.
   
   3. Fix a typo issue in `dev/intellij/install-runconfig.sh` which introduced by accident.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2288962326

   > yes, in the case 1, the coordinator.log is correct, because the Process is Coordinator.
   
   I think I finally understand the problem in your use case: the process of coordinator will produce **extra empty log files without system property substitution**. That's indeed a problem and should be fixed, and the `createOnDemand` could be used.  However, I don't think the approach you took in this PR is sufficiently elegant: the log4j2.xml has a lot of duplication code. I would prefer something like below:
   ```xml
   <Configuration status="WARN" monitorInterval="30">
     <Appenders>
       <Console name="console" target="SYSTEM_ERR">
         <PatternLayout pattern="[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%t] [%p] %c{1} - %m%n%ex"/>
       </Console>
       <RollingFile name="RollingAppender" fileName="${sys:log.path}" filePattern="${sys:log.path}.%i">
         <PatternLayout pattern="[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%t] [%p] %c{1} - %m%n%ex"/>
         <Policies>
           <SizeBasedTriggeringPolicy size="2GB"/>
         </Policies>
         <DefaultRolloverStrategy max="10"/>
       </RollingFile>
       <RollingFile name="StorageAuditAppender" fileName="${sys:storage.audit.log.path}" filePattern="${sys:storage.audit.log.path}.%i"
                    createOnDemand="true">
         <PatternLayout pattern="%m%n"/>
         <Policies>
           <SizeBasedTriggeringPolicy size="2GB"/>
         </Policies>
         <DefaultRolloverStrategy max="10"/>
       </RollingFile>
       <RollingFile name="RPCAuditAppender" fileName="${sys:rpc.audit.log.path}" filePattern="${sys:rpc.audit.log.path}.%i"
                    createOnDemand="true">
         <PatternLayout pattern="[%d{yyyy-MM-dd HH:mm:ss.SSS}] %m%n"/>
         <Policies>
           <SizeBasedTriggeringPolicy size="2GB"/>
         </Policies>
         <DefaultRolloverStrategy max="10"/>
       </RollingFile>
     </Appenders>
     <Loggers>
       <Root level="info">
         <AppenderRef ref="console"/>
         <AppenderRef ref="RollingAppender"/>
       </Root>
       <Logger name="io.grpc.netty.shaded.io.grpc.netty" level="info" additivity="false">
         <AppenderRef ref="console"/>
         <AppenderRef ref="RollingAppender"/>
       </Logger>
       <Logger name="org.apache.hadoop" level="info" additivity="false">
         <AppenderRef ref="console"/>
         <AppenderRef ref="RollingAppender"/>
       </Logger>
       <Logger name="org.eclipse.jetty" level="info" additivity="false">
         <AppenderRef ref="console"/>
         <AppenderRef ref="RollingAppender"/>
       </Logger>
       <Logger name="SHUFFLE_SERVER_STORAGE_AUDIT_LOG" level="INFO" additivity="false">
         <AppenderRef ref="StorageAuditAppender"/>
       </Logger>
       <Logger name="SHUFFLE_SERVER_RPC_AUDIT_LOG" level="INFO" additivity="false">
         <AppenderRef ref="RPCAuditAppender"/>
       </Logger>
       <Logger name="COORDINATOR_RPC_AUDIT_LOG" level="INFO" additivity="false">
         <AppenderRef ref="RPCAuditAppender"/>
       </Logger>
     </Loggers>
   </Configuration>
   ```
   which has less changes to the previous config file. The start script should be updated accordingly.
   
   >In the case 2, we start anything without through start-xxx.sh but through launchConfiguration of intellij idea, so we have to specify a bundle of jvm arguments about logs.
   
   As for the IntelliJ run case, I think we should just update the launchConfiguration accordingly. It doesn't need to refactor all the log4j2.xml to just make the IntelliJ run configuration file convenient.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2296381314

   @advancedxy Thanks for your review and suggestion!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong closed pull request #2028: [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs
URL: https://github.com/apache/incubator-uniffle/pull/2028


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#discussion_r1718052183


##########
dev/intellij/runConfs/Dashboard.run:
##########
@@ -16,19 +16,12 @@
 -->
 <component name="ProjectRunConfigurationManager">
   <configuration default="false" name="Dashboard" type="Application" factoryName="Application" nameIsGenerated="true">
+    <option name="ALTERNATIVE_JRE_PATH" value="1.8" />
+    <option name="ALTERNATIVE_JRE_PATH_ENABLED" value="true" />

Review Comment:
   ditto.



##########
dev/intellij/runConfs/CoordinatorServer.run:
##########
@@ -16,17 +16,13 @@
 -->
 <component name="ProjectRunConfigurationManager">
   <configuration default="false" name="CoordinatorServer" type="Application" factoryName="Application" nameIsGenerated="true">
+    <option name="ALTERNATIVE_JRE_PATH" value="1.8" />
+    <option name="ALTERNATIVE_JRE_PATH_ENABLED" value="true" />
     <option name="INCLUDE_PROVIDED_SCOPE" value="true" />

Review Comment:
   is this related or necessary?



##########
dev/intellij/runConfs/CoordinatorServer.run:
##########
@@ -16,17 +16,13 @@
 -->
 <component name="ProjectRunConfigurationManager">
   <configuration default="false" name="CoordinatorServer" type="Application" factoryName="Application" nameIsGenerated="true">
+    <option name="ALTERNATIVE_JRE_PATH" value="1.8" />
+    <option name="ALTERNATIVE_JRE_PATH_ENABLED" value="true" />
     <option name="INCLUDE_PROVIDED_SCOPE" value="true" />
     <option name="MAIN_CLASS_NAME" value="org.apache.uniffle.coordinator.CoordinatorServer" />
     <module name="coordinator" />
     <option name="PROGRAM_PARAMETERS" value="--conf $PROJECT_DIR$/.idea/local_dev/coordinator.conf" />
-    <option name="VM_PARAMETERS" value="-Dlog4j2.configurationFile=file:$PROJECT_DIR$/.idea/local_dev/log4j2.xml -Dlog.path=$PROJECT_DIR$/logs/coordinator.log" />
-    <extension name="coverage">
-      <pattern>
-        <option name="PATTERN" value="org.apache.uniffle.coordinator.*" />
-        <option name="ENABLED" value="true" />
-      </pattern>
-    </extension>

Review Comment:
   why this options are removed?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2290802710

   > Changed follow your suggestion, but a little pain is that we have to config many path for shuffle-server launchConfiguration.
   
   This is just one time thing? I think we can simply modify it and call it a day. 
   
   > If we add new more other type log file in the future, we have to add the log define in the start-xxx.sh, log4j2.xml and intellij launch configuraiton, if we configure a log.dir, than we only need update log4j2.xml, any thoughts?
   
   Will we add more audit types in the near feature? If so, I think we should reconsider how to construct the log4j configurations more properly. However, I think the audit types should be finite and limited to 1-3.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2296214483

   @advancedxy Thanks for your remind, the ci all passed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on code in PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#discussion_r1718314188


##########
dev/intellij/runConfs/ShuffleServer.run:
##########
@@ -20,10 +20,10 @@
     <option name="MAIN_CLASS_NAME" value="org.apache.uniffle.server.ShuffleServer" />
     <module name="shuffle-server" />
     <option name="PROGRAM_PARAMETERS" value="--conf $PROJECT_DIR$/.idea/local_dev/server.conf" />
-    <option name="VM_PARAMETERS" value="-Dlog4j2.configurationFile=file:$PROJECT_DIR$/.idea/local_dev/log4j2.xml -Dlog.path=$PROJECT_DIR$/logs/shuffle_server.log" />
+    <option name="VM_PARAMETERS" value="-Dlog4j2.configurationFile=file:$PROJECT_DIR$/.idea/local_dev/log4j2.xml -Dlog.path=$PROJECT_DIR$/logs/shuffle_server.log -Drpc.audit.log.path=$PROJECT_DIR$/logs/shuffle_server_rpc_audit.log -Dstorage.audit.log.path=$PROJECT_DIR$/logs/shuffle_server_storage_audit.log " />
     <extension name="coverage">
       <pattern>
-        <option name="PATTERN" value="org.apache.uniffle.server.*" />
+        <option name="PATTERN" value="org.apache.uniffle.dashboard.web.*" />

Review Comment:
   I think this should be server rather than `dashboard.web`?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2292584848

   ## [Codecov](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2028?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   All modified and coverable lines are covered by tests :white_check_mark:
   > Project coverage is 52.99%. Comparing base [(`5ddcc28`)](https://app.codecov.io/gh/apache/incubator-uniffle/commit/5ddcc28fa0bc1254dfd5805297ea90670304704e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`96429ff`)](https://app.codecov.io/gh/apache/incubator-uniffle/commit/96429ff9156a8326a21f44131b29c31514634f52?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   > Report is 47 commits behind head on master.
   
   <details><summary>Additional details and impacted files</summary>
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master    #2028      +/-   ##
   ============================================
   + Coverage     52.77%   52.99%   +0.21%     
   - Complexity     2498     2678     +180     
   ============================================
     Files           398      428      +30     
     Lines         18135    19862    +1727     
     Branches       1660     1804     +144     
   ============================================
   + Hits           9570    10525     +955     
   - Misses         7981     8664     +683     
   - Partials        584      673      +89     
   ```
   
   </details>
   
   [:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/apache/incubator-uniffle/pull/2028?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).   
   :loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong commented on code in PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#discussion_r1719186173


##########
dev/intellij/runConfs/ShuffleServer.run:
##########
@@ -20,10 +20,10 @@
     <option name="MAIN_CLASS_NAME" value="org.apache.uniffle.server.ShuffleServer" />
     <module name="shuffle-server" />
     <option name="PROGRAM_PARAMETERS" value="--conf $PROJECT_DIR$/.idea/local_dev/server.conf" />
-    <option name="VM_PARAMETERS" value="-Dlog4j2.configurationFile=file:$PROJECT_DIR$/.idea/local_dev/log4j2.xml -Dlog.path=$PROJECT_DIR$/logs/shuffle_server.log" />
+    <option name="VM_PARAMETERS" value="-Dlog4j2.configurationFile=file:$PROJECT_DIR$/.idea/local_dev/log4j2.xml -Dlog.path=$PROJECT_DIR$/logs/shuffle_server.log -Drpc.audit.log.path=$PROJECT_DIR$/logs/shuffle_server_rpc_audit.log -Dstorage.audit.log.path=$PROJECT_DIR$/logs/shuffle_server_storage_audit.log " />
     <extension name="coverage">
       <pattern>
-        <option name="PATTERN" value="org.apache.uniffle.server.*" />
+        <option name="PATTERN" value="org.apache.uniffle.dashboard.web.*" />

Review Comment:
   Yep



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "advancedxy (via GitHub)" <gi...@apache.org>.
advancedxy commented on PR #2028:
URL: https://github.com/apache/incubator-uniffle/pull/2028#issuecomment-2288709687

   > -rw-r--r-- 1 rssadmin rssadmin 0 Jul 24 17:29 '${sys:shuffle.server.storage.audit.log.path}'
   -rw-r--r-- 1 rssadmin rssadmin 0 Jul 24 17:29 '${sys:shuffle.server.rpc.audit.log.path}'
   > 
   > So I added createOnDemand="true" to the appender.
   
   So is `coordinator.log` generated correctly? I didn't see the difference between the audit log and coordinator.log.
   When started by `bin/start-coordinator.sh`, these system properties should be both set?
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org


Re: [PR] [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs [incubator-uniffle]

Posted by "maobaolong (via GitHub)" <gi...@apache.org>.
maobaolong closed pull request #2028: [MINOR] fix(script): Fix log4j2.xml script and intellij launch conf to support output audit logs
URL: https://github.com/apache/incubator-uniffle/pull/2028


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@uniffle.apache.org
For additional commands, e-mail: issues-help@uniffle.apache.org