You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@skywalking.apache.org by "weixiang1862 (via GitHub)" <gi...@apache.org> on 2023/08/02 06:53:16 UTC

[GitHub] [skywalking] weixiang1862 opened a new pull request, #11168: Support LogQL http query api.

weixiang1862 opened a new pull request, #11168:
URL: https://github.com/apache/skywalking/pull/11168

   <!--
       ⚠️ Please make sure to read this template first, pull requests that don't accord with this template
       maybe closed without notice.
       Texts surrounded by `<` and `>` are meant to be replaced by you, e.g. <framework name>, <issue number>.
       Put an `x` in the `[ ]` to mark the item as CHECKED. `[x]`
   -->
   
   <!-- ==== πŸ› Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist πŸ‘‡ ====
   ### Fix <bug description or the bug issue number or bug issue link>
   - [ ] Add a unit test to verify that the fix works.
   - [ ] Explain briefly why the bug exists and how to fix it.
        ==== πŸ› Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist πŸ‘† ==== -->
   
   <!-- ==== πŸ“ˆ Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist πŸ‘‡ ====
   ### Improve the performance of <class or module or ...>
   - [ ] Add a benchmark for the improvement, refer to [the existing ones](https://github.com/apache/skywalking/blob/master/apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java)
   - [ ] The benchmark result.
   ```text
   <Paste the benchmark results here>
   ```
   - [ ] Links/URLs to the theory proof or discussion articles/blogs. <links/URLs here>
        ==== πŸ“ˆ Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist πŸ‘† ==== -->
   
   ### LogQL Service
   - [ ] If this is non-trivial feature, paste the links/URLs to the design doc.
   - [x] Update the documentation to include this new feature.
   - [x] Tests(including UT, IT, E2E) are added to verify the new feature.
   - [x] If it's UI related, attach the screenshots below.
   ![d0d453718453e01fa1d6ab0f1a50bed](https://github.com/apache/skywalking/assets/16346644/883d0e29-088a-425a-8c29-49b1501c40ab)
   
   - [x] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #11130.
   - [x] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md).
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281624249


##########
test/e2e-v2/cases/logql/logql-cases.yaml:
##########
@@ -0,0 +1,28 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+cases:
+  # log tag query
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/labels -d 'start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))
+    expected: expected/logql-log-tags.yml
+  # log tag value query
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/label/level/values -d 'start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))
+    expected: expected/logql-log-tag-values.yml
+  # log range query FORWARD
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/query_range -d 'query={service="e2e-service-provider"}&start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))'&limit=100&direction=FORWARD'
+    expected: expected/logql-logs.yml
+  # log range query BACKWARD
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/query_range -d 'query={service="e2e-service-provider"}&start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))'&limit=100&direction=BACKWARD'

Review Comment:
   Ok, I will add more cases.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281533845


##########
oap-server/server-query-plugin/logql-plugin/src/main/java/org/apache/skywalking/oap/query/logql/handler/LogQLApiHandler.java:
##########
@@ -0,0 +1,233 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.query.logql.handler;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.linecorp.armeria.common.HttpData;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.ResponseHeaders;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Path;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import org.antlr.v4.runtime.CharStreams;
+import org.antlr.v4.runtime.CommonTokenStream;
+import org.antlr.v4.runtime.misc.ParseCancellationException;
+import org.antlr.v4.runtime.tree.ParseTree;
+import org.apache.skywalking.logql.rt.grammar.LogQLLexer;
+import org.apache.skywalking.logql.rt.grammar.LogQLParser;
+import org.apache.skywalking.oap.query.logql.entity.LabelName;
+import org.apache.skywalking.oap.query.logql.entity.LogDirection;
+import org.apache.skywalking.oap.query.logql.entity.ResultStatus;
+import org.apache.skywalking.oap.query.logql.entity.response.LabelValuesQueryRsp;
+import org.apache.skywalking.oap.query.logql.entity.response.LabelsQueryRsp;
+import org.apache.skywalking.oap.query.logql.entity.response.LogRangeQueryRsp;
+import org.apache.skywalking.oap.query.logql.entity.response.QueryResponse;
+import org.apache.skywalking.oap.query.logql.entity.response.ResultType;
+import org.apache.skywalking.oap.query.logql.entity.response.StreamLog;
+import org.apache.skywalking.oap.query.logql.entity.response.TimeValuePair;
+import org.apache.skywalking.oap.query.logql.rt.LogQLExprVisitor;
+import org.apache.skywalking.oap.query.logql.rt.exception.ParseErrorListener;
+import org.apache.skywalking.oap.query.logql.rt.result.LogQLParseResult;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.TagType;
+import org.apache.skywalking.oap.server.core.query.DurationUtils;
+import org.apache.skywalking.oap.server.core.query.LogQueryService;
+import org.apache.skywalking.oap.server.core.query.TagAutoCompleteQueryService;
+import org.apache.skywalking.oap.server.core.query.input.Duration;
+import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
+import org.apache.skywalking.oap.server.core.query.type.Log;
+import org.apache.skywalking.oap.server.core.query.type.Logs;
+import org.apache.skywalking.oap.server.core.query.type.Pagination;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+
+public class LogQLApiHandler {
+
+    private final LogQueryService logQueryService;
+    private final TagAutoCompleteQueryService tagAutoCompleteQueryService;
+    private static final ObjectMapper MAPPER = new ObjectMapper();
+
+    public LogQLApiHandler(ModuleManager moduleManager) {
+        this.logQueryService = moduleManager.find(CoreModule.NAME)
+                                            .provider()
+                                            .getService(LogQueryService.class);
+        this.tagAutoCompleteQueryService = moduleManager.find(CoreModule.NAME)
+                                                        .provider()
+                                                        .getService(TagAutoCompleteQueryService.class);
+    }
+
+    @Get
+    @Path("/loki/api/v1/labels")
+    public HttpResponse labels(
+        @Param("start") Long start,
+        @Param("end") Long end) throws IOException {
+        LabelsQueryRsp labelsQueryRsp = new LabelsQueryRsp();
+        labelsQueryRsp.setStatus(ResultStatus.SUCCESS);
+
+        Duration duration = DurationUtils.timestamp2Duration(nano2Millis(start), nano2Millis(end));
+        tagAutoCompleteQueryService.queryTagAutocompleteKeys(TagType.LOG, duration)
+                                   .forEach(tag -> labelsQueryRsp.getData().add(tag));
+
+        return successResponse(labelsQueryRsp);
+    }
+
+    @Get
+    @Path("/loki/api/v1/label/{label_name}/values")
+    public HttpResponse labelValues(
+        @Param("label_name") String labelName,
+        @Param("start") Long start,
+        @Param("end") Long end) throws IOException {
+        LabelValuesQueryRsp response = new LabelValuesQueryRsp();
+        response.setStatus(ResultStatus.SUCCESS);
+
+        Duration duration = DurationUtils.timestamp2Duration(nano2Millis(start), nano2Millis(end));
+        tagAutoCompleteQueryService.queryTagAutocompleteValues(TagType.LOG, labelName, duration)
+                                   .forEach(value -> response.getData().add(value));
+
+        return successResponse(response);
+    }
+
+    @Get
+    @Path("/loki/api/v1/query_range")
+    public HttpResponse rangeQuery(
+        @Param("start") Long start,
+        @Param("end") Long end,
+        @Param("query") String query,
+        @Param("limit") Integer limit,
+        @Param("direction") LogDirection direction) throws IOException {
+        LogRangeQueryRsp logRangeQueryRsp = new LogRangeQueryRsp();
+        logRangeQueryRsp.setStatus(ResultStatus.SUCCESS);
+
+        LogQLLexer lexer = new LogQLLexer(CharStreams.fromString(query));
+        lexer.addErrorListener(new ParseErrorListener());
+        LogQLParser parser = new LogQLParser(new CommonTokenStream(lexer));
+        parser.addErrorListener(new ParseErrorListener());
+        ParseTree tree;
+        try {
+            tree = parser.root();
+        } catch (ParseCancellationException e) {
+            return badResponse(e.getMessage());
+        }
+
+        LogQLExprVisitor visitor = new LogQLExprVisitor();
+        LogQLParseResult parseResult = visitor.visit(tree);
+        Map<String, String> labelMap = parseResult.getLabelMap();
+
+        String serviceId = labelMap.containsKey(LabelName.SERVICE.getLabel()) ?
+            IDManager.ServiceID.buildId(labelMap.get(LabelName.SERVICE.getLabel()), true) : null;
+        String serviceInstanceId = labelMap.containsKey(LabelName.SERVICE_INSTANCE.getLabel()) ?
+            IDManager.ServiceInstanceID.buildId(serviceId, labelMap.get(LabelName.SERVICE_INSTANCE.getLabel())) : null;
+        String endpointId = labelMap.containsKey(LabelName.ENDPOINT.getLabel()) ?
+            IDManager.EndpointID.buildId(serviceId, labelMap.get(LabelName.ENDPOINT.getLabel())) : null;

Review Comment:
   Ok, thanks. I will add a check.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wankai123 commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "wankai123 (via GitHub)" <gi...@apache.org>.
wankai123 commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281522704


##########
oap-server/server-query-plugin/logql-plugin/src/main/java/org/apache/skywalking/oap/query/logql/handler/LogQLApiHandler.java:
##########
@@ -0,0 +1,233 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.query.logql.handler;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.linecorp.armeria.common.HttpData;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.ResponseHeaders;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Path;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import org.antlr.v4.runtime.CharStreams;
+import org.antlr.v4.runtime.CommonTokenStream;
+import org.antlr.v4.runtime.misc.ParseCancellationException;
+import org.antlr.v4.runtime.tree.ParseTree;
+import org.apache.skywalking.logql.rt.grammar.LogQLLexer;
+import org.apache.skywalking.logql.rt.grammar.LogQLParser;
+import org.apache.skywalking.oap.query.logql.entity.LabelName;
+import org.apache.skywalking.oap.query.logql.entity.LogDirection;
+import org.apache.skywalking.oap.query.logql.entity.ResultStatus;
+import org.apache.skywalking.oap.query.logql.entity.response.LabelValuesQueryRsp;
+import org.apache.skywalking.oap.query.logql.entity.response.LabelsQueryRsp;
+import org.apache.skywalking.oap.query.logql.entity.response.LogRangeQueryRsp;
+import org.apache.skywalking.oap.query.logql.entity.response.QueryResponse;
+import org.apache.skywalking.oap.query.logql.entity.response.ResultType;
+import org.apache.skywalking.oap.query.logql.entity.response.StreamLog;
+import org.apache.skywalking.oap.query.logql.entity.response.TimeValuePair;
+import org.apache.skywalking.oap.query.logql.rt.LogQLExprVisitor;
+import org.apache.skywalking.oap.query.logql.rt.exception.ParseErrorListener;
+import org.apache.skywalking.oap.query.logql.rt.result.LogQLParseResult;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.TagType;
+import org.apache.skywalking.oap.server.core.query.DurationUtils;
+import org.apache.skywalking.oap.server.core.query.LogQueryService;
+import org.apache.skywalking.oap.server.core.query.TagAutoCompleteQueryService;
+import org.apache.skywalking.oap.server.core.query.input.Duration;
+import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
+import org.apache.skywalking.oap.server.core.query.type.Log;
+import org.apache.skywalking.oap.server.core.query.type.Logs;
+import org.apache.skywalking.oap.server.core.query.type.Pagination;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+
+public class LogQLApiHandler {
+
+    private final LogQueryService logQueryService;
+    private final TagAutoCompleteQueryService tagAutoCompleteQueryService;
+    private static final ObjectMapper MAPPER = new ObjectMapper();
+
+    public LogQLApiHandler(ModuleManager moduleManager) {
+        this.logQueryService = moduleManager.find(CoreModule.NAME)
+                                            .provider()
+                                            .getService(LogQueryService.class);
+        this.tagAutoCompleteQueryService = moduleManager.find(CoreModule.NAME)
+                                                        .provider()
+                                                        .getService(TagAutoCompleteQueryService.class);
+    }
+
+    @Get
+    @Path("/loki/api/v1/labels")
+    public HttpResponse labels(
+        @Param("start") Long start,
+        @Param("end") Long end) throws IOException {
+        LabelsQueryRsp labelsQueryRsp = new LabelsQueryRsp();
+        labelsQueryRsp.setStatus(ResultStatus.SUCCESS);
+
+        Duration duration = DurationUtils.timestamp2Duration(nano2Millis(start), nano2Millis(end));
+        tagAutoCompleteQueryService.queryTagAutocompleteKeys(TagType.LOG, duration)
+                                   .forEach(tag -> labelsQueryRsp.getData().add(tag));
+
+        return successResponse(labelsQueryRsp);
+    }
+
+    @Get
+    @Path("/loki/api/v1/label/{label_name}/values")
+    public HttpResponse labelValues(
+        @Param("label_name") String labelName,
+        @Param("start") Long start,
+        @Param("end") Long end) throws IOException {
+        LabelValuesQueryRsp response = new LabelValuesQueryRsp();
+        response.setStatus(ResultStatus.SUCCESS);
+
+        Duration duration = DurationUtils.timestamp2Duration(nano2Millis(start), nano2Millis(end));
+        tagAutoCompleteQueryService.queryTagAutocompleteValues(TagType.LOG, labelName, duration)
+                                   .forEach(value -> response.getData().add(value));
+
+        return successResponse(response);
+    }
+
+    @Get
+    @Path("/loki/api/v1/query_range")
+    public HttpResponse rangeQuery(
+        @Param("start") Long start,
+        @Param("end") Long end,
+        @Param("query") String query,
+        @Param("limit") Integer limit,
+        @Param("direction") LogDirection direction) throws IOException {
+        LogRangeQueryRsp logRangeQueryRsp = new LogRangeQueryRsp();
+        logRangeQueryRsp.setStatus(ResultStatus.SUCCESS);
+
+        LogQLLexer lexer = new LogQLLexer(CharStreams.fromString(query));
+        lexer.addErrorListener(new ParseErrorListener());
+        LogQLParser parser = new LogQLParser(new CommonTokenStream(lexer));
+        parser.addErrorListener(new ParseErrorListener());
+        ParseTree tree;
+        try {
+            tree = parser.root();
+        } catch (ParseCancellationException e) {
+            return badResponse(e.getMessage());
+        }
+
+        LogQLExprVisitor visitor = new LogQLExprVisitor();
+        LogQLParseResult parseResult = visitor.visit(tree);
+        Map<String, String> labelMap = parseResult.getLabelMap();
+
+        String serviceId = labelMap.containsKey(LabelName.SERVICE.getLabel()) ?
+            IDManager.ServiceID.buildId(labelMap.get(LabelName.SERVICE.getLabel()), true) : null;
+        String serviceInstanceId = labelMap.containsKey(LabelName.SERVICE_INSTANCE.getLabel()) ?
+            IDManager.ServiceInstanceID.buildId(serviceId, labelMap.get(LabelName.SERVICE_INSTANCE.getLabel())) : null;
+        String endpointId = labelMap.containsKey(LabelName.ENDPOINT.getLabel()) ?
+            IDManager.EndpointID.buildId(serviceId, labelMap.get(LabelName.ENDPOINT.getLabel())) : null;

Review Comment:
   If `serviceId` is null, build instanceId and endpoint id will get unexpected id `_blank`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on pull request #11168: Support LogQL http query api.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1661642880

   Please post the picture by commenting, then we know what they are.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on pull request #11168: Support LogQL HTTP query APIs.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1664088348

   > @weixiang1862 I have merged this, you could work on showcase, then we would be able to show this feature on preview demo.
   
   Ok.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] sonatype-lift[bot] commented on pull request #11168: Support LogQL http query api.

Posted by "sonatype-lift[bot] (via GitHub)" <gi...@apache.org>.
sonatype-lift[bot] commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1661602627

   ## Sonatype Lift is retiring
   Sonatype Lift will be retiring on <b>Sep 12, 2023</b>, with its analysis stopping on Aug 12, 2023. We understand that this news may come as a disappointment, and Sonatype is committed to helping you transition off it seamlessly. If you’d like to retain your data, please export your issues from the web console.
   We are extremely grateful and thank you for your support over the years.
   
   <b>[πŸ“– Read about the impacts and timeline](https://www.sonatype.com/lift-retirement)</b>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on pull request #11168: Support LogQL http query api.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1661607613

   Some pictures refer in docs now contains in this commit, once the docs reviewed, I will open another pr in `skywalking-website`and migrate them to there.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on pull request #11168: Support LogQL http query api.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1661654366

   > Please post the picture by commenting, then we know what they are.
   
   They are pictures in [docs/en/setup/backend/ui-grafana.md](https://github.com/apache/skywalking/pull/11168/files/e1d8fcb9e3a825197e5994c33e23d7b1429d1d0c#diff-46c8605fc482404cbe57a479d7b3f11ccb38d691b1f1100cb8d485fdd333f173).
   
   ![grafana-loki-datasource](https://github.com/apache/skywalking/assets/16346644/860889ed-204a-4211-a371-be2b8459bb4d)
   
   ![grafana-loki-variables1](https://github.com/apache/skywalking/assets/16346644/c458b453-400b-4058-bc39-291dac911a8b)
   
   ![grafana-loki-variables2](https://github.com/apache/skywalking/assets/16346644/52d2917a-62ba-49e2-90bd-54d05894d67f)
   
   ![grafana-loki-variables3](https://github.com/apache/skywalking/assets/16346644/1ee01939-6399-45f8-8df2-2eee0db13f23)
   
   ![grafana-log-panel](https://github.com/apache/skywalking/assets/16346644/86a12fc4-3bce-40dd-86fe-61046009db8c)
   
   ![grafana-loki-variables4](https://github.com/apache/skywalking/assets/16346644/77e5b7c9-fc4b-49cb-8164-1d5d18e79632)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wankai123 commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "wankai123 (via GitHub)" <gi...@apache.org>.
wankai123 commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281598697


##########
test/e2e-v2/cases/logql/logql-cases.yaml:
##########
@@ -0,0 +1,28 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+cases:
+  # log tag query
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/labels -d 'start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))
+    expected: expected/logql-log-tags.yml
+  # log tag value query
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/label/level/values -d 'start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))
+    expected: expected/logql-log-tag-values.yml
+  # log range query FORWARD
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/query_range -d 'query={service="e2e-service-provider"}&start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))'&limit=100&direction=FORWARD'
+    expected: expected/logql-logs.yml
+  # log range query BACKWARD
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/query_range -d 'query={service="e2e-service-provider"}&start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))'&limit=100&direction=BACKWARD'

Review Comment:
   Could the query expr cover more cases such as include serviceInstance / traceId / tags



##########
oap-server/server-query-plugin/logql-plugin/src/main/java/org/apache/skywalking/oap/query/logql/entity/LabelName.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.query.logql.entity;
+
+import com.fasterxml.jackson.annotation.JsonValue;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.Map;
+
+@JsonSerialize()

Review Comment:
   ```suggestion
   
   ```
   I think don't need this.



##########
oap-server/server-query-plugin/logql-plugin/src/main/java/org/apache/skywalking/oap/query/logql/entity/LabelName.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.query.logql.entity;
+
+import com.fasterxml.jackson.annotation.JsonValue;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.Map;
+
+@JsonSerialize()
+public enum LabelName {
+    SERVICE("service"),
+    SERVICE_INSTANCE("service_instance"),
+    ENDPOINT("endpoint"),
+    TRACE_ID("trace_id");
+
+    final String label;
+
+    private static final Map<String, LabelName> DICTIONARY = new HashMap<>();
+
+    static {
+        Arrays.stream(LabelName.values()).forEach(l -> {
+            DICTIONARY.put(l.label, l);
+        });
+    }
+
+    LabelName(final String label) {
+        this.label = label;
+    }
+
+    public static boolean containsLabel(String label) {
+        return DICTIONARY.containsKey(label);
+    }
+
+    @JsonValue

Review Comment:
   ```suggestion
   
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "kezhenxu94 (via GitHub)" <gi...@apache.org>.
kezhenxu94 commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281539647


##########
oap-server/server-starter/src/main/resources/application.yml:
##########
@@ -431,6 +431,18 @@ promql:
     restMaxThreads: ${SW_PROMQL_REST_MAX_THREADS:200}
     restIdleTimeOut: ${SW_PROMQL_REST_IDLE_TIMEOUT:30000}
     restAcceptQueueSize: ${SW_PROMQL_REST_QUEUE_SIZE:0}
+    
+#This module is for LogQL API.
+logql:
+  selector: ${SW_LOGQL:default}
+  default:
+    # For HTTP server
+    restHost: ${SW_LOGQL_REST_HOST:0.0.0.0}
+    restPort: ${SW_LOGQL_REST_PORT:3100}

Review Comment:
   I suggest we don't enable logql, and promql which I missed before, they are occupying too many ports and are not common cases where users would use, just let the users enable these when they want



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on pull request #11168: Support LogQL http query api.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1663136638

   Meanwhile, I think you should be able to send the latest screenshots to the website repo.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on pull request #11168: Support LogQL http query api.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1663104586

   I think for `accurate match` in the query, blank text is fine. `*` is usually used in fuzzy query.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on pull request #11168: Support LogQL HTTP query APIs.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1664048112

   @weixiang1862 I have merged this, you could work on showcase, then we would be able to show this feature on preview demo.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on pull request #11168: Support LogQL http query api.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1663099864

   > > > > > I feel good about these screenshots on the Grafana, @wankai123 WDTY? If you agree, I would ask these to be hosted on website repo.
   > > 
   > > 
   > > > >
   > > 
   > > 
   > > > >
   > > 
   > > 
   > > > > ![image](https://user-images.githubusercontent.com/16773043/257841287-9d105d3d-39ba-4f12-bb6b-d6bb0dcdb4ed.png)
   > > 
   > > 
   > > > > If the ServiceInstance or Endpoint is `All`, will serviceInstance and endpoint label pass to the API handler？ Then backend will use `All` as the name to build a not existing Id.
   > > 
   > > 
   > > > 
   > > 
   > > 
   > > > I think `All` in the Grafana UI usually means this field has no query condition, instead of sending an `All` as a query condition.
   > > 
   > > 
   > > The all option in grafana submit all options sperate by `,` and it is not we wanted. So I custom the all value to `blank` and backend filter the blank label value(service, endpoint):
   > > ![image](https://user-images.githubusercontent.com/16346644/257847817-a115a1ba-9d49-433e-b969-23ea2fd6447a.png)
   > 
   > Can we customize the all value to `*`? That's much more understandable
   
   If we all think this is better, I feel good with this too.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1283206428


##########
docs/en/changes/changes.md:
##########
@@ -59,6 +59,7 @@
 * Bump up Armeria to 1.24.3.
 * Apply MQE on APISIX, AWS_EKS, AWS_GATEWAY and AWS_S3 layer UI templates.
 * Fix BooleanMatch and BooleanNotEqualMatch doing Boolean comparison.
+* Support LogQL http query api.

Review Comment:
   ```suggestion
   * Support LogQL HTTP query APIs.
   ```



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng merged pull request #11168: Support LogQL HTTP query APIs.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng merged PR #11168:
URL: https://github.com/apache/skywalking/pull/11168


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281569906


##########
oap-server/server-starter/src/main/resources/application.yml:
##########
@@ -431,6 +431,18 @@ promql:
     restMaxThreads: ${SW_PROMQL_REST_MAX_THREADS:200}
     restIdleTimeOut: ${SW_PROMQL_REST_IDLE_TIMEOUT:30000}
     restAcceptQueueSize: ${SW_PROMQL_REST_QUEUE_SIZE:0}
+    
+#This module is for LogQL API.
+logql:
+  selector: ${SW_LOGQL:default}
+  default:
+    # For HTTP server
+    restHost: ${SW_LOGQL_REST_HOST:0.0.0.0}
+    restPort: ${SW_LOGQL_REST_PORT:3100}

Review Comment:
   I am fine about enabling or not by default. It seems not much different. Are you concern about network security impact?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281680571


##########
test/e2e-v2/cases/logql/logql-cases.yaml:
##########
@@ -0,0 +1,28 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+cases:
+  # log tag query
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/labels -d 'start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))
+    expected: expected/logql-log-tags.yml
+  # log tag value query
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/label/level/values -d 'start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))
+    expected: expected/logql-log-tag-values.yml
+  # log range query FORWARD
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/query_range -d 'query={service="e2e-service-provider"}&start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))'&limit=100&direction=FORWARD'
+    expected: expected/logql-logs.yml
+  # log range query BACKWARD
+  - query: curl -X GET http://${oap_host}:${oap_3100}/loki/api/v1/query_range -d 'query={service="e2e-service-provider"}&start='$(($(($(date +%s)-1800))*1000000000))'&end='$(($(date +%s)*1000000000))'&limit=100&direction=BACKWARD'

Review Comment:
   Added.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wankai123 commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "wankai123 (via GitHub)" <gi...@apache.org>.
wankai123 commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281522704


##########
oap-server/server-query-plugin/logql-plugin/src/main/java/org/apache/skywalking/oap/query/logql/handler/LogQLApiHandler.java:
##########
@@ -0,0 +1,233 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.query.logql.handler;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.linecorp.armeria.common.HttpData;
+import com.linecorp.armeria.common.HttpResponse;
+import com.linecorp.armeria.common.HttpStatus;
+import com.linecorp.armeria.common.MediaType;
+import com.linecorp.armeria.common.ResponseHeaders;
+import com.linecorp.armeria.server.annotation.Get;
+import com.linecorp.armeria.server.annotation.Param;
+import com.linecorp.armeria.server.annotation.Path;
+import java.io.IOException;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+import org.antlr.v4.runtime.CharStreams;
+import org.antlr.v4.runtime.CommonTokenStream;
+import org.antlr.v4.runtime.misc.ParseCancellationException;
+import org.antlr.v4.runtime.tree.ParseTree;
+import org.apache.skywalking.logql.rt.grammar.LogQLLexer;
+import org.apache.skywalking.logql.rt.grammar.LogQLParser;
+import org.apache.skywalking.oap.query.logql.entity.LabelName;
+import org.apache.skywalking.oap.query.logql.entity.LogDirection;
+import org.apache.skywalking.oap.query.logql.entity.ResultStatus;
+import org.apache.skywalking.oap.query.logql.entity.response.LabelValuesQueryRsp;
+import org.apache.skywalking.oap.query.logql.entity.response.LabelsQueryRsp;
+import org.apache.skywalking.oap.query.logql.entity.response.LogRangeQueryRsp;
+import org.apache.skywalking.oap.query.logql.entity.response.QueryResponse;
+import org.apache.skywalking.oap.query.logql.entity.response.ResultType;
+import org.apache.skywalking.oap.query.logql.entity.response.StreamLog;
+import org.apache.skywalking.oap.query.logql.entity.response.TimeValuePair;
+import org.apache.skywalking.oap.query.logql.rt.LogQLExprVisitor;
+import org.apache.skywalking.oap.query.logql.rt.exception.ParseErrorListener;
+import org.apache.skywalking.oap.query.logql.rt.result.LogQLParseResult;
+import org.apache.skywalking.oap.server.core.CoreModule;
+import org.apache.skywalking.oap.server.core.analysis.IDManager;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.Tag;
+import org.apache.skywalking.oap.server.core.analysis.manual.searchtag.TagType;
+import org.apache.skywalking.oap.server.core.query.DurationUtils;
+import org.apache.skywalking.oap.server.core.query.LogQueryService;
+import org.apache.skywalking.oap.server.core.query.TagAutoCompleteQueryService;
+import org.apache.skywalking.oap.server.core.query.input.Duration;
+import org.apache.skywalking.oap.server.core.query.input.TraceScopeCondition;
+import org.apache.skywalking.oap.server.core.query.type.Log;
+import org.apache.skywalking.oap.server.core.query.type.Logs;
+import org.apache.skywalking.oap.server.core.query.type.Pagination;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+
+public class LogQLApiHandler {
+
+    private final LogQueryService logQueryService;
+    private final TagAutoCompleteQueryService tagAutoCompleteQueryService;
+    private static final ObjectMapper MAPPER = new ObjectMapper();
+
+    public LogQLApiHandler(ModuleManager moduleManager) {
+        this.logQueryService = moduleManager.find(CoreModule.NAME)
+                                            .provider()
+                                            .getService(LogQueryService.class);
+        this.tagAutoCompleteQueryService = moduleManager.find(CoreModule.NAME)
+                                                        .provider()
+                                                        .getService(TagAutoCompleteQueryService.class);
+    }
+
+    @Get
+    @Path("/loki/api/v1/labels")
+    public HttpResponse labels(
+        @Param("start") Long start,
+        @Param("end") Long end) throws IOException {
+        LabelsQueryRsp labelsQueryRsp = new LabelsQueryRsp();
+        labelsQueryRsp.setStatus(ResultStatus.SUCCESS);
+
+        Duration duration = DurationUtils.timestamp2Duration(nano2Millis(start), nano2Millis(end));
+        tagAutoCompleteQueryService.queryTagAutocompleteKeys(TagType.LOG, duration)
+                                   .forEach(tag -> labelsQueryRsp.getData().add(tag));
+
+        return successResponse(labelsQueryRsp);
+    }
+
+    @Get
+    @Path("/loki/api/v1/label/{label_name}/values")
+    public HttpResponse labelValues(
+        @Param("label_name") String labelName,
+        @Param("start") Long start,
+        @Param("end") Long end) throws IOException {
+        LabelValuesQueryRsp response = new LabelValuesQueryRsp();
+        response.setStatus(ResultStatus.SUCCESS);
+
+        Duration duration = DurationUtils.timestamp2Duration(nano2Millis(start), nano2Millis(end));
+        tagAutoCompleteQueryService.queryTagAutocompleteValues(TagType.LOG, labelName, duration)
+                                   .forEach(value -> response.getData().add(value));
+
+        return successResponse(response);
+    }
+
+    @Get
+    @Path("/loki/api/v1/query_range")
+    public HttpResponse rangeQuery(
+        @Param("start") Long start,
+        @Param("end") Long end,
+        @Param("query") String query,
+        @Param("limit") Integer limit,
+        @Param("direction") LogDirection direction) throws IOException {
+        LogRangeQueryRsp logRangeQueryRsp = new LogRangeQueryRsp();
+        logRangeQueryRsp.setStatus(ResultStatus.SUCCESS);
+
+        LogQLLexer lexer = new LogQLLexer(CharStreams.fromString(query));
+        lexer.addErrorListener(new ParseErrorListener());
+        LogQLParser parser = new LogQLParser(new CommonTokenStream(lexer));
+        parser.addErrorListener(new ParseErrorListener());
+        ParseTree tree;
+        try {
+            tree = parser.root();
+        } catch (ParseCancellationException e) {
+            return badResponse(e.getMessage());
+        }
+
+        LogQLExprVisitor visitor = new LogQLExprVisitor();
+        LogQLParseResult parseResult = visitor.visit(tree);
+        Map<String, String> labelMap = parseResult.getLabelMap();
+
+        String serviceId = labelMap.containsKey(LabelName.SERVICE.getLabel()) ?
+            IDManager.ServiceID.buildId(labelMap.get(LabelName.SERVICE.getLabel()), true) : null;
+        String serviceInstanceId = labelMap.containsKey(LabelName.SERVICE_INSTANCE.getLabel()) ?
+            IDManager.ServiceInstanceID.buildId(serviceId, labelMap.get(LabelName.SERVICE_INSTANCE.getLabel())) : null;
+        String endpointId = labelMap.containsKey(LabelName.ENDPOINT.getLabel()) ?
+            IDManager.EndpointID.buildId(serviceId, labelMap.get(LabelName.ENDPOINT.getLabel())) : null;

Review Comment:
   If `serviceId` is null, build instanceId and endpointId will get unexpected id with `_blank`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on pull request #11168: Support LogQL http query api.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1663128132

   > I think for `accurate match` in the query, blank text is fine. `*` is usually used in fuzzy query.
   > 
   > As @wankai123 and @kezhenxu94 most concern is about readability, you could hard code `*` at the backend to indicate the `all` option. @weixiang1862 You could make the `empty value` works from protocol perspective, as it seems very clear. And supporting `*` to be empty as well to increase the Grafana UI setup experience. We don't have to pick one. :)
   
   Ok, I will  adjust like this.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281498997


##########
docs/en/setup/backend/ui-grafana.md:
##########
@@ -55,6 +60,27 @@ General settings:
 #### Sampled Records
 Same as the Sort Metrics.
 
+## Configure Log Dashboard
+### Dashboards Settings
+The following steps are the example of config a log dashboard:
+1. Create a dashboard named `Log`.
+2. Configure variables for the dashboard:
+<img src="./grafana-loki-variables1.jpg"/>
+3. Please make sure `service_instance` and `endpoint` variable enabled `Include All` option and set `Custom all value` to blank (typed by space button on the keyboard):
+<img src="./grafana-loki-variables2.jpg"/>
+4. `Tags` variable is a little different from others, for more details, please refer [Ad hoc filters](https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#add-ad-hoc-filters):
+<img src="./grafana-loki-variables3.jpg"/>
+6. After configure, you can select log query variables on the top of the dashboard:
+<img src="./grafana-loki-variables4.jpg"/>
+
+### Add Log Panel
+The following steps show how to add a log panel.
+1. Choose `Logs` chart.
+2. Set the `Line limit` value (The max number of logs to return in a query) and `Order` value (Determines the sort order of logs).
+3. Add LogQL expressions, use the variables configured above for the labels and searching keyword.
+4. Test query and save the panel.
+<img src="./grafana-log-panel.jpg"/>
+
 ## Preview on demo.skywalking.a.o

Review Comment:
   Preview section is also expected later.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281506101


##########
oap-server/server-starter/src/main/resources/application.yml:
##########
@@ -431,6 +431,18 @@ promql:
     restMaxThreads: ${SW_PROMQL_REST_MAX_THREADS:200}
     restIdleTimeOut: ${SW_PROMQL_REST_IDLE_TIMEOUT:30000}
     restAcceptQueueSize: ${SW_PROMQL_REST_QUEUE_SIZE:0}
+    
+#This module is for LogQL API.
+logql:
+  selector: ${SW_LOGQL:default}
+  default:
+    # For HTTP server
+    restHost: ${SW_LOGQL_REST_HOST:0.0.0.0}
+    restPort: ${SW_LOGQL_REST_PORT:3100}

Review Comment:
   With this, `showcase` and `helm` codes need to be updated to expose the new port, right? @kezhenxu94 @wankai123 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on pull request #11168: Support LogQL http query api.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1663142019

   > Meanwhile, I think you should be able to send the latest screenshots to the website repo.
   
   Ok.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] weixiang1862 commented on a diff in pull request #11168: Support LogQL http query api.

Posted by "weixiang1862 (via GitHub)" <gi...@apache.org>.
weixiang1862 commented on code in PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#discussion_r1281625073


##########
oap-server/server-query-plugin/logql-plugin/src/main/java/org/apache/skywalking/oap/query/logql/entity/LabelName.java:
##########
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.query.logql.entity;
+
+import com.fasterxml.jackson.annotation.JsonValue;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.Map;
+
+@JsonSerialize()

Review Comment:
   Yes, you all right. It is useless in logql.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on pull request #11168: Support LogQL http query api.

Posted by "wu-sheng (via GitHub)" <gi...@apache.org>.
wu-sheng commented on PR #11168:
URL: https://github.com/apache/skywalking/pull/11168#issuecomment-1663510928

   @kezhenxu94 Is the new way good for you?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org