You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "yswdqz (via GitHub)" <gi...@apache.org> on 2023/02/20 14:21:35 UTC

[GitHub] [skywalking] yswdqz opened a new pull request, #10418: Support Monitoring DynamoDB.

yswdqz opened a new pull request, #10418:
URL: https://github.com/apache/skywalking/pull/10418

   ### <Feature description>
   - [ ] If this is non-trivial feature, paste the links/URLs to the design doc.
   - [x] Update the documentation to include this new feature.
   - [x] Tests(including UT, IT, E2E) are added to verify the new feature.
   - [x] If it's UI related, attach the screenshots below.
   
   - [ ] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #<issue number>.
   - [x] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md).
   
   ![image](https://user-images.githubusercontent.com/74546965/220131013-e5ed01db-b507-44db-995b-9a12969c405f.png)
   ![image](https://user-images.githubusercontent.com/74546965/220131121-0a20b51c-9c4e-439a-8643-14e38dea26f7.png)
   ![image](https://user-images.githubusercontent.com/74546965/220131701-5e38f0a3-f107-4b59-bf17-4096fa2c8e1b.png)
   ![image](https://user-images.githubusercontent.com/74546965/220132023-c2321836-1d7b-42bd-ae22-f66cb8a432e3.png)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] yswdqz commented on pull request #10418: Support Monitoring DynamoDB.

Posted by "yswdqz (via GitHub)" <gi...@apache.org>.
yswdqz commented on PR #10418:
URL: https://github.com/apache/skywalking/pull/10418#issuecomment-1437192209

   Now it is ok.
   ![image](https://user-images.githubusercontent.com/74546965/220145970-6ffe1302-1e9c-4e30-b300-66d270749327.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on pull request #10418: Support Monitoring DynamoDB.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on PR #10418:
URL: https://github.com/apache/skywalking/pull/10418#issuecomment-1437167853

   Should we put `inspections` as the tab0? Usually we want to see general metrics first, then to check for table by table.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on a diff in pull request #10418: Support Monitoring DynamoDB.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #10418:
URL: https://github.com/apache/skywalking/pull/10418#discussion_r1112064286


##########
docs/en/setup/backend/backend-aws-dynamodb-monitoring.md:
##########
@@ -0,0 +1,39 @@
+# AWS DynamoDb monitoring
+SkyWalking leverages Amazon Kinesis Data Filehose with [Amazon CloudWatch](https://aws.amazon.com/cn/cloudwatch/) to transfer the metrics into the [Meter System](./../../concepts-and-designs/meter.md).
+
+### Data flow
+1. Amazon CloudWatch fetches metrics from DynamoDB and pushes metrics to SkyWalking OAP Server via Amazon Kinesis data firehose.
+2. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+
+### Set up
+1. Enable [AWS CloudWatch](https://aws.amazon.com/cn/cloudwatch/)
+2. Create [Amazon Kinesis Data Filehose](https://aws.amazon.com/cn/kinesis/data-firehose/), set source to `Direct PUT`, set destination to  `HTTP Endpoint`, and set `HTTP EndPoint url` to `aws-firehose-receiver`'s port (refer to [aws-firehose-receiver](aws-firehose-receiver.md))
+
+   Note that AWS requires that the `HTTP Endpoint URL` must be through HTTPS listening at 443, therefore need to load the certificate in [aws-firehose-receiver](aws-firehose-receiver.md) and set the port to 443.
+   Or, you can use another gateway to accept the requests and route them to `aws-filehose-receiver`.

Review Comment:
   I think we need a proxy as always. And we don't need to mention this part again, as `firehose receiver` doc includes this before.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng merged pull request #10418: Support Monitoring DynamoDB.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng merged PR #10418:
URL: https://github.com/apache/skywalking/pull/10418


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on a diff in pull request #10418: Support Monitoring DynamoDB.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #10418:
URL: https://github.com/apache/skywalking/pull/10418#discussion_r1112066215


##########
docs/en/setup/backend/backend-aws-dynamodb-monitoring.md:
##########
@@ -0,0 +1,39 @@
+# AWS DynamoDb monitoring
+SkyWalking leverages Amazon Kinesis Data Filehose with [Amazon CloudWatch](https://aws.amazon.com/cn/cloudwatch/) to transfer the metrics into the [Meter System](./../../concepts-and-designs/meter.md).
+
+### Data flow
+1. Amazon CloudWatch fetches metrics from DynamoDB and pushes metrics to SkyWalking OAP Server via Amazon Kinesis data firehose.
+2. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
+
+### Set up
+1. Enable [AWS CloudWatch](https://aws.amazon.com/cn/cloudwatch/)
+2. Create [Amazon Kinesis Data Filehose](https://aws.amazon.com/cn/kinesis/data-firehose/), set source to `Direct PUT`, set destination to  `HTTP Endpoint`, and set `HTTP EndPoint url` to `aws-firehose-receiver`'s port (refer to [aws-firehose-receiver](aws-firehose-receiver.md))
+
+   Note that AWS requires that the `HTTP Endpoint URL` must be through HTTPS listening at 443, therefore need to load the certificate in [aws-firehose-receiver](aws-firehose-receiver.md) and set the port to 443.
+   Or, you can use another gateway to accept the requests and route them to `aws-filehose-receiver`.
+3. Create a [metric stream](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Metric-Streams.html), set namespace to DynanoDB, and set `Kinesis Data Firehose` to the firehose you just created.
+4. Config [aws-firehose-receiver](aws-firehose-receiver.md) to receive data.

Review Comment:
   You could refer s3 document, the most important part of this doc is for pointing to AWS docs about how to setup firehose for DynamoDB, and forward to OAP.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] yswdqz commented on pull request #10418: Support Monitoring DynamoDB.

Posted by "yswdqz (via GitHub)" <gi...@apache.org>.
yswdqz commented on PR #10418:
URL: https://github.com/apache/skywalking/pull/10418#issuecomment-1437169710

   Got it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org