You are viewing a plain text version of this content. The canonical link for it is here.
Posted to log4j-user@logging.apache.org by Meadowlark Bradsher <mb...@guidewire.com> on 2016/08/05 18:03:47 UTC
Kafka appender expecting String but getting Byte?
Hello,
I am attempting to use a kafka log appender. The topic is written to Kafka but the appender breaks during the messaging.
org.apache.kafka.common.errors.SerializationException: Can't convert value of class [B to class org.apache.kafka.common.serialization.StringSerializer specified in value.serializer
I had seen only one SO post about this error which seems to point to Spring as the culprit but I am not using Spring.
http://stackoverflow.com/questions/32368372/spring-integration-kafka-sending-a-basic-string
I am running this in IntelliJ without any Spring configuration. The log4j2.xml file is as follows (with modification to the Kafka host).
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"/>
</Console>
<Kafka name="Kafka" topic="log-test">
<PatternLayout pattern="%date %message"/>
<Property name="bootstrap.servers">localhost:9092</Property>
<Property name="key.class.type">java.lang.String</Property>
<Property name="value.class.type">java.lang.String</Property>
<Property name="key.serializer">org.apache.kafka.common.serialization.StringSerializer</Property>
<Property name="value.serializer">org.apache.kafka.common.serialization.StringSerializer</Property>
</Kafka>
</Appenders>
<Loggers>
<Root level="DEBUG">
<AppenderRef ref="Kafka"/>
</Root>
<Logger name="org.apache.kafka" level="INFO" >
<AppenderRef ref="Kafka"/>
</Logger>
<!-- avoid recursive logging/ haven't tried OFF yet -->
</Loggers>
</Configuration>
The test I am doing is
private static final Logger logger = LogManager.getRootLogger();
logger.info("{\"f1\": \"value1\"}");
I just downloaded the source to log4j2 to see if this will help me understand what is happening but perhaps this obvious to someone in this community?
Any pointers would be very helpful and appreciated.
Thanks
Meadowlark Bradsher
Re: Kafka appender expecting String but getting Byte?
Posted by Gary Gregory <ga...@gmail.com>.
Also, also, you can try building our Git master branch which uses Kafka
0.10.0.0 instead of 0.9.0.1
Gary
On Fri, Aug 5, 2016 at 6:04 PM, Gary Gregory <ga...@gmail.com> wrote:
> Also, try omitting these properties:
>
> <Property name="key.serializer">org.apache.kafka.common.
> serialization.StringSerializer</Property>
> <Property name="value.serializer">org.apache.kafka.common.
> serialization.StringSerializer</Property>
>
> Gary
>
> On Fri, Aug 5, 2016 at 6:02 PM, Gary Gregory <ga...@gmail.com>
> wrote:
>
>> Do you get a stack trace with the exception?
>>
>> Gary
>>
>> On Fri, Aug 5, 2016 at 11:03 AM, Meadowlark Bradsher <
>> mbradsher@guidewire.com> wrote:
>>
>>> Hello,
>>>
>>> I am attempting to use a kafka log appender. The topic is written to
>>> Kafka but the appender breaks during the messaging.
>>>
>>> org.apache.kafka.common.errors.SerializationException: Can't convert
>>> value of class [B to class org.apache.kafka.common.serialization.StringSerializer
>>> specified in value.serializer
>>>
>>> I had seen only one SO post about this error which seems to point to
>>> Spring as the culprit but I am not using Spring.
>>>
>>> http://stackoverflow.com/questions/32368372/spring-integrati
>>> on-kafka-sending-a-basic-string
>>>
>>> I am running this in IntelliJ without any Spring configuration. The
>>> log4j2.xml file is as follows (with modification to the Kafka host).
>>>
>>> <?xml version="1.0" encoding="UTF-8"?>
>>> <Configuration status="WARN">
>>> <Appenders>
>>> <Console name="Console" target="SYSTEM_OUT">
>>> <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36}
>>> - %msg%n"/>
>>> </Console>
>>> <Kafka name="Kafka" topic="log-test">
>>> <PatternLayout pattern="%date %message"/>
>>> <Property name="bootstrap.servers">localhost:9092</Property>
>>> <Property name="key.class.type">java.lang.String</Property>
>>> <Property name="value.class.type">java.lang.String</Property>
>>> <Property name="key.serializer">org.apac
>>> he.kafka.common.serialization.StringSerializer</Property>
>>> <Property name="value.serializer">org.ap
>>> ache.kafka.common.serialization.StringSerializer</Property>
>>> </Kafka>
>>> </Appenders>
>>> <Loggers>
>>> <Root level="DEBUG">
>>> <AppenderRef ref="Kafka"/>
>>> </Root>
>>> <Logger name="org.apache.kafka" level="INFO" >
>>> <AppenderRef ref="Kafka"/>
>>> </Logger>
>>> <!-- avoid recursive logging/ haven't tried OFF yet -->
>>> </Loggers>
>>> </Configuration>
>>>
>>>
>>> The test I am doing is
>>>
>>>
>>>
>>> private static final Logger logger = LogManager.getRootLogger();
>>>
>>> logger.info("{\"f1\": \"value1\"}");
>>>
>>>
>>>
>>> I just downloaded the source to log4j2 to see if this will help me
>>> understand what is happening but perhaps this obvious to someone in this
>>> community?
>>>
>>>
>>>
>>> Any pointers would be very helpful and appreciated.
>>>
>>>
>>>
>>>
>>>
>>> Thanks
>>>
>>> Meadowlark Bradsher
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> --
>> E-Mail: garydgregory@gmail.com | ggregory@apache.org
>> Java Persistence with Hibernate, Second Edition
>> <http://www.manning.com/bauer3/>
>> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
>> Spring Batch in Action <http://www.manning.com/templier/>
>> Blog: http://garygregory.wordpress.com
>> Home: http://garygregory.com/
>> Tweet! http://twitter.com/GaryGregory
>>
>
>
>
> --
> E-Mail: garydgregory@gmail.com | ggregory@apache.org
> Java Persistence with Hibernate, Second Edition
> <http://www.manning.com/bauer3/>
> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
> Spring Batch in Action <http://www.manning.com/templier/>
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>
--
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory
Re: Kafka appender expecting String but getting Byte?
Posted by Gary Gregory <ga...@gmail.com>.
Also, try omitting these properties:
<Property name="key.serializer">org.apache.kafka.common.
serialization.StringSerializer</Property>
<Property name="value.serializer">org.apache.kafka.common.
serialization.StringSerializer</Property>
Gary
On Fri, Aug 5, 2016 at 6:02 PM, Gary Gregory <ga...@gmail.com> wrote:
> Do you get a stack trace with the exception?
>
> Gary
>
> On Fri, Aug 5, 2016 at 11:03 AM, Meadowlark Bradsher <
> mbradsher@guidewire.com> wrote:
>
>> Hello,
>>
>> I am attempting to use a kafka log appender. The topic is written to
>> Kafka but the appender breaks during the messaging.
>>
>> org.apache.kafka.common.errors.SerializationException: Can't convert
>> value of class [B to class org.apache.kafka.common.serialization.StringSerializer
>> specified in value.serializer
>>
>> I had seen only one SO post about this error which seems to point to
>> Spring as the culprit but I am not using Spring.
>>
>> http://stackoverflow.com/questions/32368372/spring-integrati
>> on-kafka-sending-a-basic-string
>>
>> I am running this in IntelliJ without any Spring configuration. The
>> log4j2.xml file is as follows (with modification to the Kafka host).
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <Configuration status="WARN">
>> <Appenders>
>> <Console name="Console" target="SYSTEM_OUT">
>> <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36}
>> - %msg%n"/>
>> </Console>
>> <Kafka name="Kafka" topic="log-test">
>> <PatternLayout pattern="%date %message"/>
>> <Property name="bootstrap.servers">localhost:9092</Property>
>> <Property name="key.class.type">java.lang.String</Property>
>> <Property name="value.class.type">java.lang.String</Property>
>> <Property name="key.serializer">org.apac
>> he.kafka.common.serialization.StringSerializer</Property>
>> <Property name="value.serializer">org.ap
>> ache.kafka.common.serialization.StringSerializer</Property>
>> </Kafka>
>> </Appenders>
>> <Loggers>
>> <Root level="DEBUG">
>> <AppenderRef ref="Kafka"/>
>> </Root>
>> <Logger name="org.apache.kafka" level="INFO" >
>> <AppenderRef ref="Kafka"/>
>> </Logger>
>> <!-- avoid recursive logging/ haven't tried OFF yet -->
>> </Loggers>
>> </Configuration>
>>
>>
>> The test I am doing is
>>
>>
>>
>> private static final Logger logger = LogManager.getRootLogger();
>>
>> logger.info("{\"f1\": \"value1\"}");
>>
>>
>>
>> I just downloaded the source to log4j2 to see if this will help me
>> understand what is happening but perhaps this obvious to someone in this
>> community?
>>
>>
>>
>> Any pointers would be very helpful and appreciated.
>>
>>
>>
>>
>>
>> Thanks
>>
>> Meadowlark Bradsher
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> --
> E-Mail: garydgregory@gmail.com | ggregory@apache.org
> Java Persistence with Hibernate, Second Edition
> <http://www.manning.com/bauer3/>
> JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
> Spring Batch in Action <http://www.manning.com/templier/>
> Blog: http://garygregory.wordpress.com
> Home: http://garygregory.com/
> Tweet! http://twitter.com/GaryGregory
>
--
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory
Re: Kafka appender expecting String but getting Byte?
Posted by Gary Gregory <ga...@gmail.com>.
Do you get a stack trace with the exception?
Gary
On Fri, Aug 5, 2016 at 11:03 AM, Meadowlark Bradsher <
mbradsher@guidewire.com> wrote:
> Hello,
>
> I am attempting to use a kafka log appender. The topic is written to Kafka
> but the appender breaks during the messaging.
>
> org.apache.kafka.common.errors.SerializationException: Can't convert
> value of class [B to class org.apache.kafka.common.serialization.StringSerializer
> specified in value.serializer
>
> I had seen only one SO post about this error which seems to point to
> Spring as the culprit but I am not using Spring.
>
> http://stackoverflow.com/questions/32368372/spring-
> integration-kafka-sending-a-basic-string
>
> I am running this in IntelliJ without any Spring configuration. The
> log4j2.xml file is as follows (with modification to the Kafka host).
>
> <?xml version="1.0" encoding="UTF-8"?>
> <Configuration status="WARN">
> <Appenders>
> <Console name="Console" target="SYSTEM_OUT">
> <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} -
> %msg%n"/>
> </Console>
> <Kafka name="Kafka" topic="log-test">
> <PatternLayout pattern="%date %message"/>
> <Property name="bootstrap.servers">localhost:9092</Property>
> <Property name="key.class.type">java.lang.String</Property>
> <Property name="value.class.type">java.lang.String</Property>
> <Property name="key.serializer">org.apache.kafka.common.
> serialization.StringSerializer</Property>
> <Property name="value.serializer">org.apache.kafka.common.
> serialization.StringSerializer</Property>
> </Kafka>
> </Appenders>
> <Loggers>
> <Root level="DEBUG">
> <AppenderRef ref="Kafka"/>
> </Root>
> <Logger name="org.apache.kafka" level="INFO" >
> <AppenderRef ref="Kafka"/>
> </Logger>
> <!-- avoid recursive logging/ haven't tried OFF yet -->
> </Loggers>
> </Configuration>
>
>
> The test I am doing is
>
>
>
> private static final Logger logger = LogManager.getRootLogger();
>
> logger.info("{\"f1\": \"value1\"}");
>
>
>
> I just downloaded the source to log4j2 to see if this will help me
> understand what is happening but perhaps this obvious to someone in this
> community?
>
>
>
> Any pointers would be very helpful and appreciated.
>
>
>
>
>
> Thanks
>
> Meadowlark Bradsher
>
>
>
>
>
>
>
>
--
E-Mail: garydgregory@gmail.com | ggregory@apache.org
Java Persistence with Hibernate, Second Edition
<http://www.manning.com/bauer3/>
JUnit in Action, Second Edition <http://www.manning.com/tahchiev/>
Spring Batch in Action <http://www.manning.com/templier/>
Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory
Re: Kafka appender expecting String but getting Byte?
Posted by Mikael Ståldal <mi...@magine.com>.
Log4j always passes the LogEvent as a byte[] to Kafka, so using the
Kafka StringSerializer
will not work.
You should not specify *key.serializer* or *value.serializer* in the Kafka
appender config, they are automatically set to ByteArraySerializer by
Log4j. I will improve the documentation on that.
I think that *key.class.type* and *value.class.type *are Spring specific
and thus not relevant here.
On Fri, Aug 5, 2016 at 8:03 PM, Meadowlark Bradsher <mbradsher@guidewire.com
> wrote:
> Hello,
>
> I am attempting to use a kafka log appender. The topic is written to Kafka
> but the appender breaks during the messaging.
>
> org.apache.kafka.common.errors.SerializationException: Can't convert
> value of class [B to class org.apache.kafka.common.serialization.StringSerializer
> specified in value.serializer
>
> I had seen only one SO post about this error which seems to point to
> Spring as the culprit but I am not using Spring.
>
> http://stackoverflow.com/questions/32368372/spring-
> integration-kafka-sending-a-basic-string
>
> I am running this in IntelliJ without any Spring configuration. The
> log4j2.xml file is as follows (with modification to the Kafka host).
>
> <?xml version="1.0" encoding="UTF-8"?>
> <Configuration status="WARN">
> <Appenders>
> <Console name="Console" target="SYSTEM_OUT">
> <PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} -
> %msg%n"/>
> </Console>
> <Kafka name="Kafka" topic="log-test">
> <PatternLayout pattern="%date %message"/>
> <Property name="bootstrap.servers">localhost:9092</Property>
> <Property name="key.class.type">java.lang.String</Property>
> <Property name="value.class.type">java.lang.String</Property>
> <Property name="key.serializer">org.apache.kafka.common.
> serialization.StringSerializer</Property>
> <Property name="value.serializer">org.apache.kafka.common.
> serialization.StringSerializer</Property>
> </Kafka>
> </Appenders>
> <Loggers>
> <Root level="DEBUG">
> <AppenderRef ref="Kafka"/>
> </Root>
> <Logger name="org.apache.kafka" level="INFO" >
> <AppenderRef ref="Kafka"/>
> </Logger>
> <!-- avoid recursive logging/ haven't tried OFF yet -->
> </Loggers>
> </Configuration>
>
>
> The test I am doing is
>
>
>
> private static final Logger logger = LogManager.getRootLogger();
>
> logger.info("{\"f1\": \"value1\"}");
>
>
>
> I just downloaded the source to log4j2 to see if this will help me
> understand what is happening but perhaps this obvious to someone in this
> community?
>
>
>
> Any pointers would be very helpful and appreciated.
>
>
>
>
>
> Thanks
>
> Meadowlark Bradsher
>
>
>
>
>
>
>
>
--
[image: MagineTV]
*Mikael Ståldal*
Senior software developer
*Magine TV*
mikael.staldal@magine.com
Grev Turegatan 3 | 114 46 Stockholm, Sweden | www.magine.com
Privileged and/or Confidential Information may be contained in this
message. If you are not the addressee indicated in this message
(or responsible for delivery of the message to such a person), you may not
copy or deliver this message to anyone. In such case,
you should destroy this message and kindly notify the sender by reply
email.