You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@apisix.apache.org by GitBox <gi...@apache.org> on 2022/01/26 12:20:45 UTC

[GitHub] [apisix] bisakhmondal opened a new pull request #6212: feat(loggly): support http/s bulk sending for batch logs

bisakhmondal opened a new pull request #6212:
URL: https://github.com/apache/apisix/pull/6212


   ### What this PR does / why we need it:
   <!--- Why is this change required? What problem does it solve? -->
   <!--- If it fixes an open issue, please link to the issue here. -->
   
   #6112  [Phase -2 Part 1]
   ### Pre-submission checklist:
   
   Todo: tests, docs.
   
   <!--
   Please follow the PR manners:
   1. Use Draft if the PR is not ready to be reviewed
   2. Test is required for the feat/fix PR, unless you have a good reason
   3. Doc is required for the feat PR
   4. Use a new commit to resolve review instead of `push -f`
   5. If you need to resolve merge conflicts after the PR is reviewed, please merge master but do not rebase
   6. Use "request review" to notify the reviewer once you have resolved the review
   7. Only reviewer can click "Resolve conversation" to mark the reviewer's review resolved
   -->
   
   * [x] Did you explain what problem does this PR solve? Or what new features have been added?
   * [ ] Have you added corresponding test cases?
   * [ ] Have you modified the corresponding document?
   * [x] Is this PR backward compatible? **If it is not backward compatible, please discuss on the [mailing list](https://github.com/apache/apisix/tree/master#community) first**
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix] spacewander commented on a change in pull request #6212: feat(loggly): support http/s bulk sending for batch logs

Posted by GitBox <gi...@apache.org>.
spacewander commented on a change in pull request #6212:
URL: https://github.com/apache/apisix/pull/6212#discussion_r793221388



##########
File path: apisix/plugins/loggly.lua
##########
@@ -232,13 +233,71 @@ local function send_data_over_udp(message)
 end
 
 
+local function send_bulk_over_http(message, metadata, conf)
+    local endpoint = path.join(metadata.value.host, "bulk", conf.customer_token, "tag", "bulk")
+    local has_prefix = metadata.value.host:find("http")

Review comment:
       Better to use https://github.com/apache/apisix/blob/615ee41312282ca2375abddae1d074637a901d89/apisix/core/string.lua#L46




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix] bisakhmondal commented on a change in pull request #6212: feat(loggly): support http/s bulk sending for batch logs

Posted by GitBox <gi...@apache.org>.
bisakhmondal commented on a change in pull request #6212:
URL: https://github.com/apache/apisix/pull/6212#discussion_r793463129



##########
File path: apisix/plugins/loggly.lua
##########
@@ -232,13 +233,71 @@ local function send_data_over_udp(message)
 end
 
 
+local function send_bulk_over_http(message, metadata, conf)
+    local endpoint = path.join(metadata.value.host, "bulk", conf.customer_token, "tag", "bulk")
+    local has_prefix = core.string.has_prefix(metadata.value.host, "http")
+    if not has_prefix then
+        if metadata.value.protocol == "http" then
+            endpoint = "http://" .. endpoint
+        else
+            endpoint = "https://" .. endpoint
+        end
+    end
+
+    local httpc = http.new()
+    httpc:set_timeout(metadata.value.timeout)
+    local res, err = httpc:request_uri(endpoint, {
+        ssl_verify = conf.ssl_verify,
+        method = "POST",
+        body = message,
+        headers = {
+            ["Content-Type"] = "application/json",
+            ["X-LOGGLY-TAG"] = conf.tags
+        },
+    })
+
+    if not res then
+        return false, "failed to write log to loggly, " .. err
+    end
+
+    if res.status ~= 200 then
+        local body = core.json.decode(res.body)
+        if not body then
+            return false, "failed to send log to loggly, http status code: " .. res.status
+        else
+            return false, "failed to send log to loggly, " .. res.body

Review comment:
       Yep. Sounds good to me. Thanks!




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix] tokers commented on a change in pull request #6212: feat(loggly): support http/s bulk sending for batch logs

Posted by GitBox <gi...@apache.org>.
tokers commented on a change in pull request #6212:
URL: https://github.com/apache/apisix/pull/6212#discussion_r793450401



##########
File path: apisix/plugins/loggly.lua
##########
@@ -232,13 +233,71 @@ local function send_data_over_udp(message)
 end
 
 
+local function send_bulk_over_http(message, metadata, conf)
+    local endpoint = path.join(metadata.value.host, "bulk", conf.customer_token, "tag", "bulk")
+    local has_prefix = core.string.has_prefix(metadata.value.host, "http")
+    if not has_prefix then
+        if metadata.value.protocol == "http" then
+            endpoint = "http://" .. endpoint
+        else
+            endpoint = "https://" .. endpoint
+        end
+    end
+
+    local httpc = http.new()
+    httpc:set_timeout(metadata.value.timeout)
+    local res, err = httpc:request_uri(endpoint, {
+        ssl_verify = conf.ssl_verify,
+        method = "POST",
+        body = message,
+        headers = {
+            ["Content-Type"] = "application/json",
+            ["X-LOGGLY-TAG"] = conf.tags
+        },
+    })
+
+    if not res then
+        return false, "failed to write log to loggly, " .. err
+    end
+
+    if res.status ~= 200 then
+        local body = core.json.decode(res.body)
+        if not body then
+            return false, "failed to send log to loggly, http status code: " .. res.status
+        else
+            return false, "failed to send log to loggly, " .. res.body

Review comment:
       Should we also print the status code here?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix] bisakhmondal commented on a change in pull request #6212: feat(loggly): support http/s bulk sending for batch logs

Posted by GitBox <gi...@apache.org>.
bisakhmondal commented on a change in pull request #6212:
URL: https://github.com/apache/apisix/pull/6212#discussion_r793267767



##########
File path: apisix/plugins/loggly.lua
##########
@@ -232,13 +233,71 @@ local function send_data_over_udp(message)
 end
 
 
+local function send_bulk_over_http(message, metadata, conf)
+    local endpoint = path.join(metadata.value.host, "bulk", conf.customer_token, "tag", "bulk")
+    local has_prefix = metadata.value.host:find("http")

Review comment:
       Nice. Thanks for the suggestion.
   Updated.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix] bisakhmondal commented on pull request #6212: feat(loggly): support http/s bulk sending for batch logs

Posted by GitBox <gi...@apache.org>.
bisakhmondal commented on pull request #6212:
URL: https://github.com/apache/apisix/pull/6212#issuecomment-1022148630


   Sample: for batch size 3 
   ![image](https://user-images.githubusercontent.com/41498427/151162000-faf94db1-57b5-4d29-9023-587c85ab7d91.png)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [apisix] spacewander merged pull request #6212: feat(loggly): support http/s bulk sending for batch logs

Posted by GitBox <gi...@apache.org>.
spacewander merged pull request #6212:
URL: https://github.com/apache/apisix/pull/6212


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@apisix.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org