[jira] [Commented] (EAGLE-286) Failed to deserialize json

["Edward Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/EAGLE-286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261349#comment-15261349 ] 

Edward Zhang commented on EAGLE-286:
------------------------------------

In PartitionManager, seems it directly acks this message if no tuples are generated from this message


 public EmitState next(SpoutOutputCollector collector) {
        if (_waitingToEmit.isEmpty()) {
            fill();
        }
        while (true) {
            MessageAndRealOffset toEmit = _waitingToEmit.pollFirst();
            if (toEmit == null) {
                return EmitState.NO_EMITTED;
            }
            Iterable<List<Object>> tups = KafkaUtils.generateTuples(_spoutConfig, toEmit.msg);
            if (tups != null) {
                for (List<Object> tup : tups) {
                    collector.emit(tup, new KafkaMessageId(_partition, toEmit.offset));
                }
                break;
            } else {
                ack(toEmit.offset);
            }
        }
        if (!_waitingToEmit.isEmpty()) {
            return EmitState.EMITTED_MORE_LEFT;
        } else {
            return EmitState.EMITTED_END;
        }
    }

> Failed to deserialize json
> --------------------------
>
>                 Key: EAGLE-286
>                 URL: https://issues.apache.org/jira/browse/EAGLE-286
>             Project: Eagle
>          Issue Type: Bug
>    Affects Versions: v0.3.0
>         Environment: Hortonworks Sandbox 2.2.4
>            Reporter: Huizhi Lu
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> When Eagle detects one failed message such as invalid json, it may not ack the message back to Kafka. So Kafka offset may not be updated to the latest one. This would become a big problem when more and more message are coming.
> Please see the following invalid json message. It is invalid because there is a special character (space) before "timestamp".
> After the special character is deleted, it is good.
> NOP [ERROR] Failed to deserialize json from: {"host": "/192.168.6.227","source": "/192.168.6.227","user": "root", "timestamp": 1455574202866}
> om.fasterxml.jackson.core.JsonParseException: Unexpected character ('Â' (code 194)): was expecting double-quote to start field name
>  at [Source: [B@49a298b0; line: 1, column: 70]
>         at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1369) ~[stormjar.jar:na]
>         at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:532) ~[stormjar.jar:na]
>         at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:453) ~[stormjar.jar:na]
>         at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnusualFieldName(UTF8StreamJsonParser.java:1720) ~[stormjar.jar:na]
>         at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._parseFieldName(UTF8StreamJsonParser.java:1410) ~[stormjar.jar:na]
>         at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:668) ~[stormjar.jar:na]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)