You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by GitBox <gi...@apache.org> on 2022/12/08 11:34:31 UTC
[GitHub] [pinot] kirkrodrigues opened a new pull request, #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion
kirkrodrigues opened a new pull request, #9942:
URL: https://github.com/apache/pinot/pull/9942
At a high-level, the plugin takes two inputs: a JSON record and a list of fields (unstructured log messages) to encode with [CLP](https://github.com/y-scope/clp). The plugin will extract and encode the user-specified fields into CLP's three-column format and store the output in a Pinot `GenericRow` object.
This is part of the change requested in #9819 and described in this [design doc](https://docs.google.com/document/d/10H1j5Ev3KoT_TScafOMO0BKU_h3NijuGmb13x7Cy8Ag).
# Release notes
* New plugin added: `pinot-clp-log` to encode user-specified fields with CLP during ingestion.
* Users can use the plugin by specifying these configuration options in their `tableIndexConfig.streamConfigs`:
```json
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.inputformat.clplog.CLPLogMessageDecoder",
"stream.kafka.decoder.prop.fieldsForClpEncoding": "<field-names>"
```
where `<field-names>` is a comma-separated list of fields you wish to encode with CLP.
# Testing performed
* Validated fields are encoded correctly using the added unit test.
* Created a table with the following settings:
```json
"tableIndexConfig": {
...,
"streamConfigs": {
...,
"stream.kafka.decoder.class.name": "org.apache.pinot.plugin.inputformat.clplog.CLPLogMessageDecoder",
"stream.kafka.decoder.prop.fieldsForClpEncoding": "message",
}
}
```
* Ingested logs through Kafka and ensured log events containing the message field were transformed such that the `message` field was replaced with CLP's three fields: `message_logtype`, `message_dictionaryVars`, and `message_encodedVars`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] kirkrodrigues commented on pull request #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion
Posted by GitBox <gi...@apache.org>.
kirkrodrigues commented on PR #9942:
URL: https://github.com/apache/pinot/pull/9942#issuecomment-1375623141
Yep, will do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] chenboat commented on a diff in pull request #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion
Posted by GitBox <gi...@apache.org>.
chenboat commented on code in PR #9942:
URL: https://github.com/apache/pinot/pull/9942#discussion_r1044877771
##########
pinot-distribution/pinot-assembly.xml:
##########
@@ -146,6 +146,12 @@
</source>
<destName>plugins/pinot-input-format/pinot-avro/pinot-avro-${project.version}-shaded.jar</destName>
</file>
+ <file>
+ <source>
+ ${pinot.root}/pinot-plugins/pinot-input-format/pinot-clp-log/target/pinot-clp-log-${project.version}-shaded.jar
Review Comment:
why shaded jar is used? Are you seeing version conflict?
##########
pinot-plugins/pinot-input-format/pinot-clp-log/src/test/java/org/apache/pinot/plugin/inputformat/clplog/CLPLogRecordExtractorTest.java:
##########
@@ -0,0 +1,110 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.inputformat.clplog;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.yscope.clp.compressorfrontend.MessageDecoder;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.testng.annotations.Test;
+
+import static org.testng.Assert.assertEquals;
+import static org.testng.Assert.assertNotEquals;
+import static org.testng.Assert.assertNull;
+import static org.testng.Assert.fail;
+
+
+public class CLPLogRecordExtractorTest {
Review Comment:
why is this an extractor test when it tests and decoder actually? Can we test the extractor directly?
##########
pinot-plugins/pinot-input-format/pinot-clp-log/src/main/java/org/apache/pinot/plugin/inputformat/clplog/CLPLogRecordExtractor.java:
##########
@@ -0,0 +1,157 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.inputformat.clplog;
+
+import com.yscope.clp.compressorfrontend.EncodedMessage;
+import com.yscope.clp.compressorfrontend.MessageEncoder;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import javax.annotation.Nullable;
+import org.apache.pinot.spi.data.readers.BaseRecordExtractor;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.data.readers.RecordExtractorConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * A record extractor for log events. For configuration options, see {@link CLPLogRecordExtractorConfig}. This is an
+ * experimental feature.
+ * <p></p>
+ * The goal of this record extractor is to allow us to encode the fields specified in
+ * {@link CLPLogRecordExtractorConfig} using CLP. CLP is a compressor designed to encode unstructured log messages in a
+ * way that makes them more compressible. It does this by decomposing a message into three fields:
Review Comment:
Make the comments complete by stating what to do with the fields NOT in the config.
##########
pinot-plugins/pinot-input-format/pinot-clp-log/src/main/java/org/apache/pinot/plugin/inputformat/clplog/CLPLogRecordExtractor.java:
##########
@@ -0,0 +1,157 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.inputformat.clplog;
+
+import com.yscope.clp.compressorfrontend.EncodedMessage;
+import com.yscope.clp.compressorfrontend.MessageEncoder;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import javax.annotation.Nullable;
+import org.apache.pinot.spi.data.readers.BaseRecordExtractor;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.data.readers.RecordExtractorConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * A record extractor for log events. For configuration options, see {@link CLPLogRecordExtractorConfig}. This is an
+ * experimental feature.
+ * <p></p>
+ * The goal of this record extractor is to allow us to encode the fields specified in
+ * {@link CLPLogRecordExtractorConfig} using CLP. CLP is a compressor designed to encode unstructured log messages in a
+ * way that makes them more compressible. It does this by decomposing a message into three fields:
+ * <ul>
+ * <li>the message's static text, called a log type;</li>
+ * <li>repetitive variable values, called dictionary variables; and</li>
+ * <li>non-repetitive variable values (called encoded variables since we encode them specially if possible).</li>
+ * </ul>
+ * For instance, if the field "message" is encoded, then the extractor will output:
+ * <ul>
+ * <li>message_logtype</li>
+ * <li>message_dictionaryVars</li>
+ * <li>message_encodedVars</li>
+ * </ul>
+ * This class' implementation is based on {@link org.apache.pinot.plugin.inputformat.json.JSONRecordExtractor}.
+ */
+public class CLPLogRecordExtractor extends BaseRecordExtractor<Map<String, Object>> {
+ private static final Logger LOGGER = LoggerFactory.getLogger(CLPLogRecordExtractor.class);
+
+ private static final String LOGTYPE_COLUMN_SUFFIX = "_logtype";
+ private static final String DICTIONARY_VARS_COLUMN_SUFFIX = "_dictionaryVars";
+ private static final String ENCODED_VARS_COLUMN_SUFFIX = "_encodedVars";
+
+ private Set<String> _fields;
+ private boolean _extractAll = false;
Review Comment:
add comments. What does extractAll mean?
##########
pinot-plugins/pinot-input-format/pinot-clp-log/src/test/java/org/apache/pinot/plugin/inputformat/clplog/CLPLogRecordExtractorTest.java:
##########
@@ -0,0 +1,110 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.inputformat.clplog;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.yscope.clp.compressorfrontend.MessageDecoder;
+import java.io.IOException;
+import java.util.Arrays;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.testng.annotations.Test;
+
+import static org.testng.Assert.assertEquals;
+import static org.testng.Assert.assertNotEquals;
+import static org.testng.Assert.assertNull;
+import static org.testng.Assert.fail;
+
+
+public class CLPLogRecordExtractorTest {
+ @Test
+ void testCLPEncoding() {
+ // Setup decoder
+ CLPLogMessageDecoder messageDecoder = new CLPLogMessageDecoder();
+ Map<String, String> props = new HashMap<>();
+ Set<String> fieldsToRead = new HashSet<>();
+ // Add two fields for CLP encoding
+ props.put("fieldsForClpEncoding", "message1,message2");
+ fieldsToRead.add("message1_logtype");
+ fieldsToRead.add("message1_encodedVars");
+ fieldsToRead.add("message1_dictionaryVars");
+ fieldsToRead.add("message2_logtype");
+ fieldsToRead.add("message2_encodedVars");
+ fieldsToRead.add("message2_dictionaryVars");
+ // Add an unencoded field
+ fieldsToRead.add("timestamp");
+ try {
+ messageDecoder.init(props, fieldsToRead, null);
Review Comment:
We also need to test different scenarios based on props's value.
##########
pinot-plugins/pinot-input-format/pinot-clp-log/src/main/java/org/apache/pinot/plugin/inputformat/clplog/CLPLogRecordExtractor.java:
##########
@@ -0,0 +1,157 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pinot.plugin.inputformat.clplog;
+
+import com.yscope.clp.compressorfrontend.EncodedMessage;
+import com.yscope.clp.compressorfrontend.MessageEncoder;
+import java.io.IOException;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import javax.annotation.Nullable;
+import org.apache.pinot.spi.data.readers.BaseRecordExtractor;
+import org.apache.pinot.spi.data.readers.GenericRow;
+import org.apache.pinot.spi.data.readers.RecordExtractorConfig;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+
+/**
+ * A record extractor for log events. For configuration options, see {@link CLPLogRecordExtractorConfig}. This is an
+ * experimental feature.
+ * <p></p>
+ * The goal of this record extractor is to allow us to encode the fields specified in
+ * {@link CLPLogRecordExtractorConfig} using CLP. CLP is a compressor designed to encode unstructured log messages in a
+ * way that makes them more compressible. It does this by decomposing a message into three fields:
+ * <ul>
+ * <li>the message's static text, called a log type;</li>
+ * <li>repetitive variable values, called dictionary variables; and</li>
+ * <li>non-repetitive variable values (called encoded variables since we encode them specially if possible).</li>
+ * </ul>
+ * For instance, if the field "message" is encoded, then the extractor will output:
+ * <ul>
+ * <li>message_logtype</li>
+ * <li>message_dictionaryVars</li>
+ * <li>message_encodedVars</li>
+ * </ul>
+ * This class' implementation is based on {@link org.apache.pinot.plugin.inputformat.json.JSONRecordExtractor}.
+ */
+public class CLPLogRecordExtractor extends BaseRecordExtractor<Map<String, Object>> {
+ private static final Logger LOGGER = LoggerFactory.getLogger(CLPLogRecordExtractor.class);
+
+ private static final String LOGTYPE_COLUMN_SUFFIX = "_logtype";
+ private static final String DICTIONARY_VARS_COLUMN_SUFFIX = "_dictionaryVars";
+ private static final String ENCODED_VARS_COLUMN_SUFFIX = "_encodedVars";
+
+ private Set<String> _fields;
+ private boolean _extractAll = false;
+ private CLPLogRecordExtractorConfig _config;
+
+ private EncodedMessage _clpEncodedMessage;
+ private MessageEncoder _clpMessageEncoder;
+
+ @Override
+ public void init(Set<String> fields, @Nullable RecordExtractorConfig recordExtractorConfig) {
+ _config = (CLPLogRecordExtractorConfig) recordExtractorConfig;
+ if (fields == null || fields.isEmpty()) {
+ _extractAll = true;
+ _fields = Collections.emptySet();
+ } else {
+ _fields = new HashSet<>(fields);
+ // Remove the fields to be CLP-encoded to make it easier to work with them
+ // in `extract`
+ _fields.removeAll(_config.getFieldsForClpEncoding());
+ }
+
+ _clpEncodedMessage = new EncodedMessage();
+ _clpMessageEncoder = new MessageEncoder();
+ }
+
+ @Override
+ public GenericRow extract(Map<String, Object> from, GenericRow to) {
+ Set<String> clpEncodedFieldNames = _config.getFieldsForClpEncoding();
+
+ if (_extractAll) {
+ for (Map.Entry<String, Object> recordEntry : from.entrySet()) {
+ String recordKey = recordEntry.getKey();
+ Object recordValue = recordEntry.getValue();
+ if (clpEncodedFieldNames.contains(recordKey)) {
+ encodeFieldWithClp(recordKey, recordValue, to);
+ } else {
+ if (null != recordValue) {
+ recordValue = convert(recordValue);
+ }
+ to.putValue(recordKey, recordValue);
+ }
+ }
Review Comment:
Directly "return to;" and remove the else block.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] kirkrodrigues commented on a diff in pull request #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion
Posted by GitBox <gi...@apache.org>.
kirkrodrigues commented on code in PR #9942:
URL: https://github.com/apache/pinot/pull/9942#discussion_r1051272869
##########
pinot-distribution/pinot-assembly.xml:
##########
@@ -146,6 +146,12 @@
</source>
<destName>plugins/pinot-input-format/pinot-avro/pinot-avro-${project.version}-shaded.jar</destName>
</file>
+ <file>
+ <source>
+ ${pinot.root}/pinot-plugins/pinot-input-format/pinot-clp-log/target/pinot-clp-log-${project.version}-shaded.jar
Review Comment:
No, I'm not seeing a version conflict, but aren't all plugins shaded? If you're asking why I had shading in the plugin's pom.xml, that was an error (thanks for catching it).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on pull request #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on PR #9942:
URL: https://github.com/apache/pinot/pull/9942#issuecomment-1343502944
This is a great feature! Can you please give public access to the design doc?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] codecov-commenter commented on pull request #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion
Posted by GitBox <gi...@apache.org>.
codecov-commenter commented on PR #9942:
URL: https://github.com/apache/pinot/pull/9942#issuecomment-1343569787
# [Codecov](https://codecov.io/gh/apache/pinot/pull/9942?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#9942](https://codecov.io/gh/apache/pinot/pull/9942?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8a8f73e) into [master](https://codecov.io/gh/apache/pinot/commit/8a2fbf9caff8cb1bc7ec4bda94827fc556fc3866?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8a2fbf9) will **decrease** coverage by `12.15%`.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## master #9942 +/- ##
=============================================
- Coverage 28.00% 15.84% -12.16%
- Complexity 53 176 +123
=============================================
Files 1970 1928 -42
Lines 106216 104139 -2077
Branches 16115 15864 -251
=============================================
- Hits 29744 16500 -13244
- Misses 73522 86424 +12902
+ Partials 2950 1215 -1735
```
| Flag | Coverage Δ | |
|---|---|---|
| integration1 | `?` | |
| integration2 | `?` | |
| unittests2 | `15.84% <ø> (?)` | |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/pinot/pull/9942?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [...src/main/java/org/apache/pinot/sql/FilterKind.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvRmlsdGVyS2luZC5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ain/java/org/apache/pinot/core/data/table/Key.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL3RhYmxlL0tleS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...n/java/org/apache/pinot/core/data/table/Table.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL3RhYmxlL1RhYmxlLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../java/org/apache/pinot/core/data/table/Record.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9kYXRhL3RhYmxlL1JlY29yZC5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../java/org/apache/pinot/core/util/GroupByUtils.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS91dGlsL0dyb3VwQnlVdGlscy5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [.../java/org/apache/pinot/common/auth/AuthConfig.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vYXV0aC9BdXRoQ29uZmlnLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...va/org/apache/pinot/core/routing/RoutingTable.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9yb3V0aW5nL1JvdXRpbmdUYWJsZS5qYXZh) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...ava/org/apache/pinot/sql/parsers/PinotSqlType.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9zcWwvcGFyc2Vycy9QaW5vdFNxbFR5cGUuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...a/org/apache/pinot/core/operator/BaseOperator.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29yZS9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcGlub3QvY29yZS9vcGVyYXRvci9CYXNlT3BlcmF0b3IuamF2YQ==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [...va/org/apache/pinot/common/config/NettyConfig.java](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-cGlub3QtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9waW5vdC9jb21tb24vY29uZmlnL05ldHR5Q29uZmlnLmphdmE=) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| ... and [1013 more](https://codecov.io/gh/apache/pinot/pull/9942/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] Jackie-Jiang commented on pull request #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion
Posted by GitBox <gi...@apache.org>.
Jackie-Jiang commented on PR #9942:
URL: https://github.com/apache/pinot/pull/9942#issuecomment-1374641444
Thanks for adding this powerful feature. Can you help add a documentation page in the [pinot doc](https://github.com/pinot-contrib/pinot-docs) describing when and how to use the feature?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] chenboat merged pull request #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion
Posted by GitBox <gi...@apache.org>.
chenboat merged PR #9942:
URL: https://github.com/apache/pinot/pull/9942
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
[GitHub] [pinot] kirkrodrigues commented on pull request #9942: [feature] Add pinot-clp-log plugin to encode user-specified fields with CLP during ingestion
Posted by GitBox <gi...@apache.org>.
kirkrodrigues commented on PR #9942:
URL: https://github.com/apache/pinot/pull/9942#issuecomment-1344771898
> This is a great feature! Can you please give public access to the design doc?
Apologies for the delay, the previous doc couldn't be shared due to some security settings. I've updated the link to a publicly accessible doc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org