You are viewing a plain text version of this content. The canonical link for it is here.

Posted to notifications@skywalking.apache.org by GitBox <gi...@apache.org> on 2022/11/16 03:56:23 UTC

[GitHub] [skywalking] wu-sheng opened a new pull request, #9964: Add `LongText` to support longer logs persistent as a text type

wu-sheng opened a new pull request, #9964:
URL: https://github.com/apache/skywalking/pull/9964

   <!--
       ⚠️ Please make sure to read this template first, pull requests that don't accord with this template
       maybe closed without notice.
       Texts surrounded by `<` and `>` are meant to be replaced by you, e.g. <framework name>, <issue number>.
       Put an `x` in the `[ ]` to mark the item as CHECKED. `[x]`
   -->
   
   <!-- ==== 🐛 Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist 👇 ====
   ### Fix <bug description or the bug issue number or bug issue link>
   - [ ] Add a unit test to verify that the fix works.
   - [ ] Explain briefly why the bug exists and how to fix it.
        ==== 🐛 Remove this line WHEN AND ONLY WHEN you're fixing a bug, follow the checklist 👆 ==== -->
   
   <!-- ==== 📈 Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist 👇 ====
   ### Improve the performance of <class or module or ...>
   - [ ] Add a benchmark for the improvement, refer to [the existing ones](https://github.com/apache/skywalking/blob/master/apm-commons/apm-datacarrier/src/test/java/org/apache/skywalking/apm/commons/datacarrier/LinkedArrayBenchmark.java)
   - [ ] The benchmark result.
   ```text
   <Paste the benchmark results here>
   ```
   - [ ] Links/URLs to the theory proof or discussion articles/blogs. <links/URLs here>
        ==== 📈 Remove this line WHEN AND ONLY WHEN you're improving the performance, follow the checklist 👆 ==== -->
   
   <!-- ==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👇 ====
   ### <Feature description>
   - [ ] If this is non-trivial feature, paste the links/URLs to the design doc.
   - [ ] Update the documentation to include this new feature.
   - [ ] Tests(including UT, IT, E2E) are added to verify the new feature.
   - [ ] If it's UI related, attach the screenshots below.
        ==== 🆕 Remove this line WHEN AND ONLY WHEN you're adding a new feature, follow the checklist 👆 ==== -->
   
   - [x] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Closes #9961.
   - [x] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md).
   
   
   Add `LongText` to support longer logs persistent as a text type in ElasticSearch, instead of a keyword, to avoid length limitation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng commented on a diff in pull request #9964: Add `LongText` to support longer logs persistent as a text type

Posted by GitBox <gi...@apache.org>.

wu-sheng commented on code in PR #9964:
URL: https://github.com/apache/skywalking/pull/9964#discussion_r1023480978


##########
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/record/LongText.java:
##########
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.core.analysis.record;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.Setter;
+import lombok.ToString;
+import org.apache.skywalking.oap.server.core.storage.type.StorageDataComplexObject;
+
+/**
+ * LongText represents a string field, but the length is not predictable and could be longer than 1000.
+ * This is a wrapper of Java String only, which provides an indicator of long text field.
+ */
+@Getter
+@Setter
+@ToString
+@EqualsAndHashCode
+public class LongText implements StorageDataComplexObject<LongText> {

Review Comment:
   I am adding this to adopt this
   
   https://github.com/apache/skywalking/blob/07bfde6e254e364b6ab31324c9e342a5f7e64653/oap-server/server-storage-plugin/storage-elasticsearch-plugin/src/main/java/org/apache/skywalking/oap/server/storage/plugin/elasticsearch/base/ColumnTypeEsMapping.java#L39-L42
   
   Move from `keyword` to `text`.
   
   @kezhenxu94 Could you confirm this way is good to keep the log text is still the raw text, rather than JSON wrapped? I am not sure about `@JsonSerialize(using = StorageDataComplexObject.Serializer.class)` in its parent class.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9964: Add `LongText` to support longer logs persistent as a text type

Posted by GitBox <gi...@apache.org>.

kezhenxu94 commented on code in PR #9964:
URL: https://github.com/apache/skywalking/pull/9964#discussion_r1023481687


##########
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/record/LongText.java:
##########
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.core.analysis.record;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.Setter;
+import lombok.ToString;
+import org.apache.skywalking.oap.server.core.storage.type.StorageDataComplexObject;
+
+/**
+ * LongText represents a string field, but the length is not predictable and could be longer than 1000.
+ * This is a wrapper of Java String only, which provides an indicator of long text field.
+ */
+@Getter
+@Setter
+@ToString
+@EqualsAndHashCode
+public class LongText implements StorageDataComplexObject<LongText> {

Review Comment:
   Yes. The serializer uses `toStorageData` as the sterilized string and you just return the raw text in that method so it’s not wrapped to JSON. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] kezhenxu94 commented on a diff in pull request #9964: Add `LongText` to support longer logs persistent as a text type

Posted by GitBox <gi...@apache.org>.

kezhenxu94 commented on code in PR #9964:
URL: https://github.com/apache/skywalking/pull/9964#discussion_r1023481687


##########
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/record/LongText.java:
##########
@@ -0,0 +1,56 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.core.analysis.record;
+
+import lombok.EqualsAndHashCode;
+import lombok.Getter;
+import lombok.Setter;
+import lombok.ToString;
+import org.apache.skywalking.oap.server.core.storage.type.StorageDataComplexObject;
+
+/**
+ * LongText represents a string field, but the length is not predictable and could be longer than 1000.
+ * This is a wrapper of Java String only, which provides an indicator of long text field.
+ */
+@Getter
+@Setter
+@ToString
+@EqualsAndHashCode
+public class LongText implements StorageDataComplexObject<LongText> {

Review Comment:
   Yes. The serializer uses `toStorageData` as the serialized string and you just return the raw text in that method so it’s not wrapped to JSON. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [skywalking] wu-sheng merged pull request #9964: Add `LongText` to support longer logs persistent as a text type

Posted by GitBox <gi...@apache.org>.

wu-sheng merged PR #9964:
URL: https://github.com/apache/skywalking/pull/9964


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@skywalking.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org