You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@logging.apache.org by GitBox <gi...@apache.org> on 2021/07/20 07:17:06 UTC
[GitHub] [logging-log4j2] rocketraman opened a new pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
rocketraman opened a new pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r666196134
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
@vy While you research the additional fields stuff, just a reminder on this -^
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#issuecomment-882885332
> @rocketraman, I have pushed my changes to #551. I had already started working on the ticket before you had submitted your PR, hence I wanted to stick to that. #551 has the following differences:
I'm a little confused as to why you would have started work on the PR when you specifically asked me to do so on the dev mailing list when I proposed this modification, and I did that less than 3 hours later: https://lists.apache.org/thread.html/rdc25435870e47284564575b53d3b0dbf69e33ae7736f388878b89bf0%40%3Cdev.logging.apache.org%3E.
> Nevertheless, credit still goes to you in `changes.xml`
I do appreciate that, though it would have been nice to get the "commit credit" as well. Nevertheless, not a big deal.
> #551 has the following differences:
I'll take a look shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#issuecomment-882905913
@rocketraman, sorry for the mess. I very well see your point for _commit credit_. I will base #551 on top of #543.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r665938957
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
@rocketraman, would you give me a couple of days to check out how others are incorporating these fields. I don't want to add custom fields only because we just feel like they might be useful. I want to have a little bit more educated guess.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r672627108
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import com.google.common.base.Charsets;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.apache.logging.log4j.core.impl.ThrowableProxy;
+import org.apache.logging.log4j.util.SortedArrayStringMap;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.util.*;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ Assertions.assertThat(Instant.parse(jsonTemplateLayoutMap.get("timestamp").toString()).toEpochMilli())
+ .isEqualTo(logEvent.getTimeMillis());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("severity")).isEqualTo(logEvent.getLevel().toString());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_logger")).isEqualTo(logEvent.getLoggerName());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_thread").toString())
+ .isEqualTo(logEvent.getThreadName());
+
+ if (logEvent.getContextData().size() == 0) {
+ Assertions.assertThat(jsonTemplateLayoutMap.get("logging.googleapis.com/labels"))
+ .isNull();
+ } else {
+ Assertions.assertThat(logEvent.getContextData().toMap().keySet())
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).keySet());
+ logEvent.getContextData().toMap().keySet().forEach(k -> {
+ Assertions.assertThat((Object) logEvent.getContextData().getValue(k))
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).get(k));
+ });
+ }
+
+ if (logEvent.getSource() == null) {
+ // not quite sure why the output has "?." in sourceLocation.function
Review comment:
This feels like a bug (feature?) in `PatternLayout`. I am checking this out...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r665463979
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
> I have a question regarding these extra fields: Why do we have them?
I thought that having these additional fields would be useful, because one can use Google Cloud Logging to filter on `_thread` and `_logger`, as well as the structured `_exception` field.
> Why do we deviate from the standard?
We're not really -- the Google Cloud logging format specifies certain field names that it explicitly understands, but it also allows for adding additional fields as may be useful.
> Additionally, why do we choose for these fields names?
I prefixed the additional fields with `_` to differentiate them from the "standard" names understood by Google. I actually got this idea from the `GelfLayout.json`. However, I'm also fine with removing the `_`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy merged pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy merged pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman closed pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman closed pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#issuecomment-883433350
> @rocketraman, sorry for the mess. I very well see your point for _commit credit_. I will base #551 on top of #543.
I appreciate that @vy. I see you've merged this in already, awesome!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r670640279
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
@vy To help concretize the end-effect of this template, here are some examples of cloud console log display.
Non-error log: note the `logger` and `thread` in the payload. Personally I like this data in the payload rather than as labels, though one could make an argument for putting logger in `labels` instead.
And here is an example with an exception -- the destructured exception is in the payload. This is useful because the individual exception fields, especially `exception_class` and `exception_message`, can be filtered on.
I thought this would help concretize the end-effect of this template. Looking forward to your reply.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r664090889
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
These seem like a plugin mishap to me. How do you run the tests? I guess you try to run them from IDE, IntelliJ? Would you mind trying the following, please?
```bash
./mvnw clean install -DskipTests && ./mvnw -pl log4j-layout-template-json test -Dtest=GcpLayoutTest
```
If this succeeds, after `clean install`, try clicking on the _Reload All Maven Projects_ button in IntelliJ and run the test class again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#issuecomment-882880077
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman closed pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman closed pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r672627108
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import com.google.common.base.Charsets;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.apache.logging.log4j.core.impl.ThrowableProxy;
+import org.apache.logging.log4j.util.SortedArrayStringMap;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.util.*;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ Assertions.assertThat(Instant.parse(jsonTemplateLayoutMap.get("timestamp").toString()).toEpochMilli())
+ .isEqualTo(logEvent.getTimeMillis());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("severity")).isEqualTo(logEvent.getLevel().toString());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_logger")).isEqualTo(logEvent.getLoggerName());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_thread").toString())
+ .isEqualTo(logEvent.getThreadName());
+
+ if (logEvent.getContextData().size() == 0) {
+ Assertions.assertThat(jsonTemplateLayoutMap.get("logging.googleapis.com/labels"))
+ .isNull();
+ } else {
+ Assertions.assertThat(logEvent.getContextData().toMap().keySet())
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).keySet());
+ logEvent.getContextData().toMap().keySet().forEach(k -> {
+ Assertions.assertThat((Object) logEvent.getContextData().getValue(k))
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).get(k));
+ });
+ }
+
+ if (logEvent.getSource() == null) {
+ // not quite sure why the output has "?." in sourceLocation.function
Review comment:
This feels like a bug (feature?) in `PatternLayout`. I am checking this out...
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import com.google.common.base.Charsets;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.apache.logging.log4j.core.impl.ThrowableProxy;
+import org.apache.logging.log4j.util.SortedArrayStringMap;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.util.*;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ Assertions.assertThat(Instant.parse(jsonTemplateLayoutMap.get("timestamp").toString()).toEpochMilli())
+ .isEqualTo(logEvent.getTimeMillis());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("severity")).isEqualTo(logEvent.getLevel().toString());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_logger")).isEqualTo(logEvent.getLoggerName());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_thread").toString())
+ .isEqualTo(logEvent.getThreadName());
+
+ if (logEvent.getContextData().size() == 0) {
+ Assertions.assertThat(jsonTemplateLayoutMap.get("logging.googleapis.com/labels"))
+ .isNull();
+ } else {
+ Assertions.assertThat(logEvent.getContextData().toMap().keySet())
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).keySet());
+ logEvent.getContextData().toMap().keySet().forEach(k -> {
+ Assertions.assertThat((Object) logEvent.getContextData().getValue(k))
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).get(k));
+ });
+ }
+
+ if (logEvent.getSource() == null) {
+ // not quite sure why the output has "?." in sourceLocation.function
Review comment:
What is going on is this: `ClassNamePatternConverter` emits `ClassNamePatternConverter.NA` (which is `?`) due to the missing source location, next `PatternLayout` emits the `.` literal, finally `MethodLocationPatternConverter` emits nothing. I guess we need to implement `ConcatenationResolver` addressed in [LOG4J2-3120](https://issues.apache.org/jira/browse/LOG4J2-3120) first. I will also check `PatternLayout` documentation if we can replace `?.` outputs with `null`.
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import com.google.common.base.Charsets;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.apache.logging.log4j.core.impl.ThrowableProxy;
+import org.apache.logging.log4j.util.SortedArrayStringMap;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.util.*;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ Assertions.assertThat(Instant.parse(jsonTemplateLayoutMap.get("timestamp").toString()).toEpochMilli())
+ .isEqualTo(logEvent.getTimeMillis());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("severity")).isEqualTo(logEvent.getLevel().toString());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_logger")).isEqualTo(logEvent.getLoggerName());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_thread").toString())
+ .isEqualTo(logEvent.getThreadName());
+
+ if (logEvent.getContextData().size() == 0) {
+ Assertions.assertThat(jsonTemplateLayoutMap.get("logging.googleapis.com/labels"))
+ .isNull();
+ } else {
+ Assertions.assertThat(logEvent.getContextData().toMap().keySet())
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).keySet());
+ logEvent.getContextData().toMap().keySet().forEach(k -> {
+ Assertions.assertThat((Object) logEvent.getContextData().getValue(k))
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).get(k));
+ });
+ }
+
+ if (logEvent.getSource() == null) {
+ // not quite sure why the output has "?." in sourceLocation.function
Review comment:
`%replace{%C.%M}{^\?\.$}{}` does the trick, sort of, for now.
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
Checked these out and addressed them in #551.
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
Thanks so much for the great work and investigation, I have really appreciated it! I have incorporated your suggestions into #551. Looking forward to your upcoming `log4j-layout-template-json` contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#issuecomment-882885332
> @rocketraman, I have pushed my changes to #551. I had already started working on the ticket before you had submitted your PR, hence I wanted to stick to that. #551 has the following differences:
I'm a little confused as to why you would have started work on the PR when you specifically asked me to do so on the dev mailing list when I proposed this modification, and I did that less than 3 hours later: https://lists.apache.org/thread.html/rdc25435870e47284564575b53d3b0dbf69e33ae7736f388878b89bf0%40%3Cdev.logging.apache.org%3E.
> Nevertheless, credit still goes to you in `changes.xml`
I do appreciate that, though it would have been nice to get the "commit credit" as well. Nevertheless, not a big deal.
> #551 has the following differences:
I'll take a look shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#issuecomment-882880077
@rocketraman, I have pushed my changes to #551. I had already started working on the ticket before you had submitted your PR, hence I wanted to stick to that. Nevertheless, credit still goes to you in `changes.xml`. #551 has the following differences:
- `exception_class`, `exception_message`, and `exception_stacktrace` are respectively renamed to `class`, `message`, and `stackTrace`.
- Uses `CounterResolver` for `logging.googleapis.com/insertId`.
- Contains more versatile tests.
I am happy with the final state of #551. If you approve, please go ahead and squash+rebase it onto `release-2.x`. I would also appreciate it if you can `cherry-pick` it onto `master`, otherwise I can deal with it myself too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r665452441
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
@rocketraman, I was exactly working on this PR for the last 1 hour or so. :facepalm: :sweat_smile:
I have a question regarding these extra fields: Why do we have them? Why do we deviate from the standard? Additionally, why do we choose for these fields names?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r663937566
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
For some reason the delegated pattern resolver doesn't actually seem to be applying in the test case. For example, here is an example output:
```
{timestamp=2021-07-05T13:30:58.150Z, severity=%level, message=%m%xEx, logging.googleapis.com/sourceLocation={function=%C.%M}, logging.googleapis.com/insertId=%sn, _thread=ForkJoinPool-1-worker-18, _logger=a.B0}
```
The severity, message, and sourceLocation fields which all use the pattern resolver haven't been resolved. Any ideas?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman closed pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman closed pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#issuecomment-882885332
> @rocketraman, I have pushed my changes to #551. I had already started working on the ticket before you had submitted your PR, hence I wanted to stick to that. #551 has the following differences:
I'm a little confused as to why you would have started work on the PR when you specifically asked me to do so on the dev mailing list when I proposed this modification, and I did that less than 3 hours later: https://lists.apache.org/thread.html/rdc25435870e47284564575b53d3b0dbf69e33ae7736f388878b89bf0%40%3Cdev.logging.apache.org%3E.
> Nevertheless, credit still goes to you in `changes.xml`
I do appreciate that, though it would have been nice to get the "commit credit" as well. Nevertheless, not a big deal.
> #551 has the following differences:
I'll take a look shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r669797036
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
@vy Any updates?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r664023128
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
```
ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [logger]
ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [msg]
ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [n]
ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r672652484
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
Checked these out and addressed them in #551.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r672627108
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import com.google.common.base.Charsets;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.apache.logging.log4j.core.impl.ThrowableProxy;
+import org.apache.logging.log4j.util.SortedArrayStringMap;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.util.*;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ Assertions.assertThat(Instant.parse(jsonTemplateLayoutMap.get("timestamp").toString()).toEpochMilli())
+ .isEqualTo(logEvent.getTimeMillis());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("severity")).isEqualTo(logEvent.getLevel().toString());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_logger")).isEqualTo(logEvent.getLoggerName());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_thread").toString())
+ .isEqualTo(logEvent.getThreadName());
+
+ if (logEvent.getContextData().size() == 0) {
+ Assertions.assertThat(jsonTemplateLayoutMap.get("logging.googleapis.com/labels"))
+ .isNull();
+ } else {
+ Assertions.assertThat(logEvent.getContextData().toMap().keySet())
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).keySet());
+ logEvent.getContextData().toMap().keySet().forEach(k -> {
+ Assertions.assertThat((Object) logEvent.getContextData().getValue(k))
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).get(k));
+ });
+ }
+
+ if (logEvent.getSource() == null) {
+ // not quite sure why the output has "?." in sourceLocation.function
Review comment:
This feels like a bug (feature?) in `PatternLayout`. I am checking this out...
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import com.google.common.base.Charsets;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.apache.logging.log4j.core.impl.ThrowableProxy;
+import org.apache.logging.log4j.util.SortedArrayStringMap;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.util.*;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ Assertions.assertThat(Instant.parse(jsonTemplateLayoutMap.get("timestamp").toString()).toEpochMilli())
+ .isEqualTo(logEvent.getTimeMillis());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("severity")).isEqualTo(logEvent.getLevel().toString());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_logger")).isEqualTo(logEvent.getLoggerName());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_thread").toString())
+ .isEqualTo(logEvent.getThreadName());
+
+ if (logEvent.getContextData().size() == 0) {
+ Assertions.assertThat(jsonTemplateLayoutMap.get("logging.googleapis.com/labels"))
+ .isNull();
+ } else {
+ Assertions.assertThat(logEvent.getContextData().toMap().keySet())
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).keySet());
+ logEvent.getContextData().toMap().keySet().forEach(k -> {
+ Assertions.assertThat((Object) logEvent.getContextData().getValue(k))
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).get(k));
+ });
+ }
+
+ if (logEvent.getSource() == null) {
+ // not quite sure why the output has "?." in sourceLocation.function
Review comment:
What is going on is this: `ClassNamePatternConverter` emits `ClassNamePatternConverter.NA` (which is `?`) due to the missing source location, next `PatternLayout` emits the `.` literal, finally `MethodLocationPatternConverter` emits nothing. I guess we need to implement `ConcatenationResolver` addressed in [LOG4J2-3120](https://issues.apache.org/jira/browse/LOG4J2-3120) first. I will also check `PatternLayout` documentation if we can replace `?.` outputs with `null`.
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import com.google.common.base.Charsets;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.apache.logging.log4j.core.impl.ThrowableProxy;
+import org.apache.logging.log4j.util.SortedArrayStringMap;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.util.*;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ Assertions.assertThat(Instant.parse(jsonTemplateLayoutMap.get("timestamp").toString()).toEpochMilli())
+ .isEqualTo(logEvent.getTimeMillis());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("severity")).isEqualTo(logEvent.getLevel().toString());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_logger")).isEqualTo(logEvent.getLoggerName());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_thread").toString())
+ .isEqualTo(logEvent.getThreadName());
+
+ if (logEvent.getContextData().size() == 0) {
+ Assertions.assertThat(jsonTemplateLayoutMap.get("logging.googleapis.com/labels"))
+ .isNull();
+ } else {
+ Assertions.assertThat(logEvent.getContextData().toMap().keySet())
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).keySet());
+ logEvent.getContextData().toMap().keySet().forEach(k -> {
+ Assertions.assertThat((Object) logEvent.getContextData().getValue(k))
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).get(k));
+ });
+ }
+
+ if (logEvent.getSource() == null) {
+ // not quite sure why the output has "?." in sourceLocation.function
Review comment:
`%replace{%C.%M}{^\?\.$}{}` does the trick, sort of, for now.
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
Checked these out and addressed them in #551.
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
Thanks so much for the great work and investigation, I have really appreciated it! I have incorporated your suggestions into #551. Looking forward to your upcoming `log4j-layout-template-json` contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy merged pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy merged pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r672632995
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import com.google.common.base.Charsets;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.apache.logging.log4j.core.impl.ThrowableProxy;
+import org.apache.logging.log4j.util.SortedArrayStringMap;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.util.*;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ Assertions.assertThat(Instant.parse(jsonTemplateLayoutMap.get("timestamp").toString()).toEpochMilli())
+ .isEqualTo(logEvent.getTimeMillis());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("severity")).isEqualTo(logEvent.getLevel().toString());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_logger")).isEqualTo(logEvent.getLoggerName());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_thread").toString())
+ .isEqualTo(logEvent.getThreadName());
+
+ if (logEvent.getContextData().size() == 0) {
+ Assertions.assertThat(jsonTemplateLayoutMap.get("logging.googleapis.com/labels"))
+ .isNull();
+ } else {
+ Assertions.assertThat(logEvent.getContextData().toMap().keySet())
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).keySet());
+ logEvent.getContextData().toMap().keySet().forEach(k -> {
+ Assertions.assertThat((Object) logEvent.getContextData().getValue(k))
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).get(k));
+ });
+ }
+
+ if (logEvent.getSource() == null) {
+ // not quite sure why the output has "?." in sourceLocation.function
Review comment:
What is going on is this: `ClassNamePatternConverter` emits `ClassNamePatternConverter.NA` (which is `?`) due to the missing source location, next `PatternLayout` emits the `.` literal, finally `MethodLocationPatternConverter` emits nothing. I guess we need to implement `ConcatenationResolver` addressed in [LOG4J2-3120](https://issues.apache.org/jira/browse/LOG4J2-3120) first. I will also check `PatternLayout` documentation if we can replace `?.` outputs with `null`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#issuecomment-882880077
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r672653256
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
Thanks so much for the great work and investigation, I have really appreciated it! I have incorporated your suggestions into #551. Looking forward to your upcoming `log4j-layout-template-json` contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r665463979
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
> I have a question regarding these extra fields: Why do we have them?
Having these additional fields is useful because one can use Google Cloud Logging to filter and search on them.
> Why do we deviate from the standard?
We're not really -- the Google Cloud logging format specifies certain field names that it explicitly understands, but it also allows for adding additional fields as may be useful.
> Additionally, why do we choose for these fields names?
I prefixed the additional fields with `_` to differentiate them from the "standard" names understood by Google. I actually got this idea from the `GelfLayout.json`. However, I'm also fine with removing the `_`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r672651484
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,163 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import com.google.common.base.Charsets;
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.apache.logging.log4j.core.impl.ThrowableProxy;
+import org.apache.logging.log4j.util.SortedArrayStringMap;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayOutputStream;
+import java.io.PrintStream;
+import java.io.UnsupportedEncodingException;
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.time.Instant;
+import java.util.*;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ @SuppressWarnings("rawtypes")
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ Assertions.assertThat(Instant.parse(jsonTemplateLayoutMap.get("timestamp").toString()).toEpochMilli())
+ .isEqualTo(logEvent.getTimeMillis());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("severity")).isEqualTo(logEvent.getLevel().toString());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_logger")).isEqualTo(logEvent.getLoggerName());
+ Assertions.assertThat(jsonTemplateLayoutMap.get("_thread").toString())
+ .isEqualTo(logEvent.getThreadName());
+
+ if (logEvent.getContextData().size() == 0) {
+ Assertions.assertThat(jsonTemplateLayoutMap.get("logging.googleapis.com/labels"))
+ .isNull();
+ } else {
+ Assertions.assertThat(logEvent.getContextData().toMap().keySet())
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).keySet());
+ logEvent.getContextData().toMap().keySet().forEach(k -> {
+ Assertions.assertThat((Object) logEvent.getContextData().getValue(k))
+ .isEqualTo(((Map)jsonTemplateLayoutMap.get("logging.googleapis.com/labels")).get(k));
+ });
+ }
+
+ if (logEvent.getSource() == null) {
+ // not quite sure why the output has "?." in sourceLocation.function
Review comment:
`%replace{%C.%M}{^\?\.$}{}` does the trick, sort of, for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r663706396
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform Structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
+ is written to the `_exception` field as well as the `message` field --
+ the former is useful for explicitly searching/analyzing structured exception
+ information, while the latter is Google's expected place for the exception,
+ and integrates with Google Error Reporting.
Review comment:
```suggestion
and integrates with https://cloud.google.com/error-reporting[Google Error Reporting].
```
##########
File path: src/changes/changes.xml
##########
@@ -31,6 +31,9 @@
-->
<release version="2.15.0" date="2021-MM-DD" description="GA Release 2.15.0">
<!-- ADDS -->
+ <action issue="LOG4J2-3116" dev="vy" type="add" due-to="Raman Gupta">
Review comment:
```suggestion
<action issue="LOG4J2-3116" dev="rgupta" type="add">
```
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform Structured logging] with additional
Review comment:
```suggestion
Cloud Platform structured logging] with additional
```
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
@rocketraman, can we do something more meaningful here, e.g., checking a couple of fields against the `logEvent`?
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform Structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
+ is written to the `_exception` field as well as the `message` field --
Review comment:
```suggestion
is written to the `_exception` field as well as the `message` field –
```
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
These seem like a plugin mishap to me. How do you run the tests? I guess you try to run them from IDE, IntelliJ? Would you mind trying the following, please?
```bash
./mvnw clean install -DskipTests && ./mvnw -pl log4j-layout-template-json test -Dtest=GcpLayoutTest
```
If this succeeds, after `clean install`, try clicking on the _Reload All Maven Projects_ button in IntelliJ and run the test class again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r663937566
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
For some reason the delegated pattern resolver doesn't actually seem to be applying in the test case. For example, here is an example output:
```
{timestamp=2021-07-05T13:30:58.150Z, severity=%level, message=%m%xEx, logging.googleapis.com/sourceLocation={function=%C.%M}, logging.googleapis.com/insertId=%sn, _thread=ForkJoinPool-1-worker-18, _logger=a.B0}
```
The severity, message, and sourceLocation fields which all use the pattern resolver haven't been resolved. Any ideas?
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
```
ERROR StatusLogger Unrecognized format specifier [d]
ERROR StatusLogger Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [level]
ERROR StatusLogger Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [logger]
ERROR StatusLogger Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [msg]
ERROR StatusLogger Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
ERROR StatusLogger Unrecognized format specifier [n]
ERROR StatusLogger Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy merged pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy merged pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r665428126
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
@vy That worked, thank you. I've updated the test case (and fixed some problems the test revealed) in 81aa8dbce.
Your thoughts on these lines in the test would be useful:
1. https://github.com/apache/logging-log4j2/pull/543/commits/81aa8dbcef3b3999496dffd6ae605c6624e46fd1#diff-3498fad3fa3b9848532a5406e402db89f1ec033493bf2770f94eb0c47d5b192cR94-R96
2. https://github.com/apache/logging-log4j2/pull/543/commits/81aa8dbcef3b3999496dffd6ae605c6624e46fd1#diff-3498fad3fa3b9848532a5406e402db89f1ec033493bf2770f94eb0c47d5b192cR123-R127
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] vy commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
vy commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r663706396
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform Structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
+ is written to the `_exception` field as well as the `message` field --
+ the former is useful for explicitly searching/analyzing structured exception
+ information, while the latter is Google's expected place for the exception,
+ and integrates with Google Error Reporting.
Review comment:
```suggestion
and integrates with https://cloud.google.com/error-reporting[Google Error Reporting].
```
##########
File path: src/changes/changes.xml
##########
@@ -31,6 +31,9 @@
-->
<release version="2.15.0" date="2021-MM-DD" description="GA Release 2.15.0">
<!-- ADDS -->
+ <action issue="LOG4J2-3116" dev="vy" type="add" due-to="Raman Gupta">
Review comment:
```suggestion
<action issue="LOG4J2-3116" dev="rgupta" type="add">
```
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform Structured logging] with additional
Review comment:
```suggestion
Cloud Platform structured logging] with additional
```
##########
File path: log4j-layout-template-json/src/test/java/org/apache/logging/log4j/layout/template/json/GcpLayoutTest.java
##########
@@ -0,0 +1,76 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache license, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the license for the specific language governing permissions and
+ * limitations under the license.
+ */
+package org.apache.logging.log4j.layout.template.json;
+
+import org.apache.logging.log4j.core.LogEvent;
+import org.apache.logging.log4j.core.config.Configuration;
+import org.apache.logging.log4j.core.config.DefaultConfiguration;
+import org.assertj.core.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.Charset;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.List;
+import java.util.Map;
+
+import static org.apache.logging.log4j.layout.template.json.TestHelpers.serializeUsingLayout;
+
+class GcpLayoutTest {
+
+ private static final Configuration CONFIGURATION = new DefaultConfiguration();
+
+ private static final Charset CHARSET = StandardCharsets.UTF_8;
+
+ private static final String SERVICE_NAME = "test";
+
+ private static final JsonTemplateLayout JSON_TEMPLATE_LAYOUT = JsonTemplateLayout
+ .newBuilder()
+ .setConfiguration(CONFIGURATION)
+ .setCharset(CHARSET)
+ .setEventTemplateUri("classpath:GcpLayout.json")
+ .build();
+
+ @Test
+ void test_lite_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createLiteLogEvents(1_000);
+ test(logEvents);
+ }
+
+ @Test
+ void test_full_log_events() {
+ final List<LogEvent> logEvents = LogEventFixture.createFullLogEvents(1_000);
+ test(logEvents);
+ }
+
+ private static void test(final Collection<LogEvent> logEvents) {
+ for (final LogEvent logEvent : logEvents) {
+ test(logEvent);
+ }
+ }
+
+ private static void test(final LogEvent logEvent) {
+ final Map<String, Object> jsonTemplateLayoutMap = renderUsingJsonTemplateLayout(logEvent);
+ // for now, we just validate there are no errors
Review comment:
@rocketraman, can we do something more meaningful here, e.g., checking a couple of fields against the `logEvent`?
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform Structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
+ is written to the `_exception` field as well as the `message` field --
Review comment:
```suggestion
is written to the `_exception` field as well as the `message` field –
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [logging-log4j2] rocketraman commented on a change in pull request #543: LOG4J2-3116 Google Cloud structured logging via JsonTemplate
Posted by GitBox <gi...@apache.org>.
rocketraman commented on a change in pull request #543:
URL: https://github.com/apache/logging-log4j2/pull/543#discussion_r666194573
##########
File path: src/site/asciidoc/manual/json-template-layout.adoc.vm
##########
@@ -410,6 +410,15 @@ artifact, which contains the following predefined event templates:
xref:additional-event-template-fields[additional event template fields]
to avoid `hostName` property lookup at runtime, which incurs an extra cost.)
+- https://github.com/apache/logging-log4j2/tree/master/log4j-layout-template-json/src/main/resources/GcpLayout.json[`GcpLayout.json`]
+ described by https://cloud.google.com/logging/docs/structured-logging[Google
+ Cloud Platform structured logging] with additional
+ `_thread`, `_logger` and `_exception` fields. The exception trace, if any,
Review comment:
Sure take your time to research this. The code for Google's Logback adapter is here:
https://github.com/googleapis/java-logging-logback/blob/master/src/main/java/com/google/cloud/logging/logback/LoggingAppender.java#L296-L327
The use Google's cloud logging API directly ([LogEntry.Builder](https://googleapis.dev/java/google-cloud-logging/latest/com/google/cloud/logging/LogEntry.Builder.html) to create a [LogEntry](https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry)), and log:
* timestamp
* message (with stack)
* severity
* labels via the MDC enhancer
* they add the level name, numeric value, and logger name as additional labels
* they have the ability for users to define "enhancers" which can add additional fields
Note that enhancers can write to the payload field, which is arbitrary JSON. When an application writes JSON to standard output as this code would do, the fields that are not standard fields go the payload, along with the standard `message` field.
One approach would be to log just the "standard" fields by default and let users use `EventTemplateAdditionalFields` to add any additional fields they want. However, my personal opinion is that at least the `logger` and `thread` fields are low cost and very useful to include by default.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@logging.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org