You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@seatunnel.apache.org by "kim-up (via GitHub)" <gi...@apache.org> on 2023/07/18 11:34:13 UTC

[GitHub] [seatunnel] kim-up opened a new pull request, #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

kim-up opened a new pull request, #5112:
URL: https://github.com/apache/seatunnel/pull/5112

   PR for  #5070 
   The final effect : https://github.com/kim-up/incubator-seatunnel/blob/telemetry/docs/en/seatunnel-engine/telemetry.md
   
   <!--
   
   Thank you for contributing to SeaTunnel! Please make sure that your code changes
   are covered with tests. And in case of new features or big changes
   remember to adjust the documentation.
   
   Feel free to ping committers for the review!
   
   ## Contribution Checklist
   
     - Make sure that the pull request corresponds to a [GITHUB issue](https://github.com/apache/seatunnel/issues).
   
     - Name the pull request in the form "[Feature] [component] Title of the pull request", where *Feature* can be replaced by `Hotfix`, `Bug`, etc.
   
     - Minor fixes should be named following this pattern: `[hotfix] [docs] Fix typo in README.md doc`.
   
   -->
   
   ## Purpose of this pull request
   
   <!-- Describe the purpose of this pull request. For example: This pull request adds checkstyle plugin.-->
   
   ## Check list
   
   * [x] Code changed are covered with tests, or it does not need tests for reason:
   * [ ] If any new Jar binary package adding in your PR, please add License Notice according
     [New License Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/contribution/new-license.md)
   * [x] If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
   * [ ] If you are contributing the connector code, please check that the following files are updated:
     1. Update change log that in connector document. For more details you can refer to [connector-v2](https://github.com/apache/seatunnel/tree/dev/docs/en/connector-v2)
     2. Update [plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties) and add new connector information in it
     3. Update the pom file of [seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
   * [x] Update the [`release-note`](https://github.com/apache/seatunnel/blob/dev/release-note.md).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] kim-up commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "kim-up (via GitHub)" <gi...@apache.org>.
kim-up commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1663377422

   > Can we add the [slowoperation](https://docs.hazelcast.com/imdg/4.2/performance/slowoperationdetector) metric? It's quite valuable for troubleshooting. You can refer to the Hazelcast Management Center for guidance.
   
   Good idea! I will research it later. After the current PR is merged, create a new PR to improve it, because the current PR is relatively large.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070) [seatunnel]

Posted by "corgy-w (via GitHub)" <gi...@apache.org>.
corgy-w commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-2311495664

   @kim-up I intend to continue this feature of yours, as it has been too long since the last update. I will be creating a new branch and porting your code to implement the necessary changes. Please be aware of this and looking forward to cooperating with you,TKS! :>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] liugddx commented on a diff in pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on code in PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#discussion_r1266654489


##########
seatunnel-examples/seatunnel-engine-examples/src/main/java/org/apache/seatunnel/example/engine/SeaTunnelEngineExample.java:
##########
@@ -23,23 +23,25 @@
 import org.apache.seatunnel.core.starter.seatunnel.args.ClientCommandArgs;
 
 import java.io.FileNotFoundException;
+import java.io.IOException;
 import java.net.URISyntaxException;
 import java.net.URL;
 import java.nio.file.Paths;
 
 public class SeaTunnelEngineExample {
 
     public static void main(String[] args)
-            throws FileNotFoundException, URISyntaxException, CommandException {
-        String configurePath = args.length > 0 ? args[0] : "/examples/fake_to_console.conf";
+            throws IOException, URISyntaxException, CommandException {
+        String configurePath =
+                args.length > 0 ? args[0] : "/examples/stream_fakesource_to_file.conf";

Review Comment:
   revert.



##########
seatunnel-examples/seatunnel-engine-examples/src/main/java/org/apache/seatunnel/example/engine/SeaTunnelEngineExample.java:
##########
@@ -23,23 +23,25 @@
 import org.apache.seatunnel.core.starter.seatunnel.args.ClientCommandArgs;
 
 import java.io.FileNotFoundException;
+import java.io.IOException;
 import java.net.URISyntaxException;
 import java.net.URL;
 import java.nio.file.Paths;
 
 public class SeaTunnelEngineExample {
 
     public static void main(String[] args)
-            throws FileNotFoundException, URISyntaxException, CommandException {
-        String configurePath = args.length > 0 ? args[0] : "/examples/fake_to_console.conf";
+            throws IOException, URISyntaxException, CommandException {
+        String configurePath =
+                args.length > 0 ? args[0] : "/examples/stream_fakesource_to_file.conf";
         String configFile = getTestConfigFile(configurePath);
         ClientCommandArgs clientCommandArgs = new ClientCommandArgs();
         clientCommandArgs.setConfigFile(configFile);
         clientCommandArgs.setCheckConfig(false);
         clientCommandArgs.setJobName(Paths.get(configFile).getFileName().toString());
         // Change Execution Mode to CLUSTER to use client mode, before do this, you should start
         // SeaTunnelEngineServerExample
-        clientCommandArgs.setMasterType(MasterType.LOCAL);
+        clientCommandArgs.setMasterType(MasterType.CLUSTER);

Review Comment:
   Same as above.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070) [seatunnel]

Posted by "appleshcn (via GitHub)" <gi...@apache.org>.
appleshcn commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1823969884

   @TyrantLucifer @kim-up @hailin0 
   we need this feature too, does it planned to be merged?
   Thanks


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] liugddx commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1640062972

   Good job.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] hailin0 commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1643739988

   > > @ic4y @hailin0 @Hisoka-X @EricJoy2048
   > > This feature is not small, may need to discuss further development.
   > 
   > Maybe we can mark it unstable. So we can change it after release. This just is an option I offered, not final decided.
   
   +1 
   Good for quick experiments to improve it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] liugddx commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1640073614

   > @ic4y @hailin0 @Hisoka-X @EricJoy2048
   > 
   > This feature is not small, may need to discuss further development.
   
   @TyrantLucifer 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070) [seatunnel]

Posted by "emmanuelsosa (via GitHub)" <gi...@apache.org>.
emmanuelsosa commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-2189717454

   @kim-up we’re excited to see this merged. Thank you for your contribution.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070) [seatunnel]

Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1823937848

   Please merge the dev branch and wait for ci to run successfully


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] liugddx commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "liugddx (via GitHub)" <gi...@apache.org>.
liugddx commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1640073420

   @ic4y @hailin0 @Hisoka-X @EricJoy2048 
   
   This feature is not small, may need to discuss further development.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070) [seatunnel]

Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1778836009

   @kim-up 
   good pr, please fix the conflict


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] Hisoka-X commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1643672930

   > @ic4y @hailin0 @Hisoka-X @EricJoy2048
   > 
   > This feature is not small, may need to discuss further development.
   
   Maybe we can mark it unstable. So we can change it after release. This just is an option I offered, not final decided.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] kim-up commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "kim-up (via GitHub)" <gi...@apache.org>.
kim-up commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1653383952

   @liugddx @TyrantLucifer @Hisoka-X @hailin0 @EricJoy2048 PTAL


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] Hisoka-X commented on a diff in pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "Hisoka-X (via GitHub)" <gi...@apache.org>.
Hisoka-X commented on code in PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#discussion_r1269280607


##########
seatunnel-e2e/seatunnel-engine-e2e/connector-seatunnel-e2e-base/src/test/resources/hazelcast.yaml:
##########
@@ -15,6 +15,7 @@
 # limitations under the License.
 #
 
+

Review Comment:
   Please revert this.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070) [seatunnel]

Posted by "HoganEdwardChu (via GitHub)" <gi...@apache.org>.
HoganEdwardChu commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1756734011

   @TyrantLucifer @kim-up  
   we need this feature does it planned to be merged? 
   Thanks
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070) [seatunnel]

Posted by "hailin0 (via GitHub)" <gi...@apache.org>.
hailin0 closed pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)
URL: https://github.com/apache/seatunnel/pull/5112


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] HoganEdwardChu commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "HoganEdwardChu (via GitHub)" <gi...@apache.org>.
HoganEdwardChu commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1723110920

   We need this feature when does it merged?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] ic4y commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "ic4y (via GitHub)" <gi...@apache.org>.
ic4y commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1661926829

   Can we add the [slowoperation](https://docs.hazelcast.com/imdg/4.2/performance/slowoperationdetector) metric? It's quite valuable for troubleshooting. You can refer to the Hazelcast Management Center for guidance.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


[GitHub] [seatunnel] TyrantLucifer commented on pull request #5112: [Feature][zeta][monitoring] Support exposing monitoring metrics by prometheus exporter protocol (#5070)

Posted by "TyrantLucifer (via GitHub)" <gi...@apache.org>.
TyrantLucifer commented on PR #5112:
URL: https://github.com/apache/seatunnel/pull/5112#issuecomment-1640068054

   document has some dead links, please check.
   
   <img width="521" alt="image" src="https://github.com/apache/seatunnel/assets/51053924/1ee24312-13a4-4c51-af17-5893249c7377">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@seatunnel.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org