[jira] [Commented] (KAFKA-3544) Missing topics on startup

["Mykola Polonskyi (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KAFKA-3544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15552065#comment-15552065 ] 

Mykola Polonskyi commented on KAFKA-3544:
-----------------------------------------

[~guozhang] greetings!
My comment a bit late but I catch error with similar stack-trace (like Greg), with using of KGroupedTable#aggregation and the intermediate topic.
kafka streams 0.10.0.1
{code:kotlin}
private val streamId = "STREAM_USER_WITH_SKICARD"
connectionProperties.put(StreamsConfig.APPLICATION_ID_CONFIG, streamId)
val skicardByOwnerIdTopicName = "skicards_by_owner_id"
userTable.join(
            skicardsTable.groupBy {
                key, value -> KeyValue<String, SkicardInfo>(value.skicardInfo.ownerId, value.skicardInfo)
            }.aggregate(
                    { mutableSetOf<SkicardInfo>() }, // accumulator definition
                    { ownerIdString, skicardInfoObject, accumulator -> accumulator.put(skicardInfo) },
                    { ownerIdString, skicardInfoObject, accumulator -> accumulator }, //XXX fix this one
                    skicardByOwnerIdSerde,
                    skicardByOwnerIdTopicName
            ),
            { userCreatedOrUpdated, skicardInfoSet -> UserWithSkicardsCreatedOrUpdated(userCreatedOrUpdated.user, skicardInfoSet) }
    ).to(
            Serdes.StringSerde(),
            userWithSkicardsSerde,
            USER_WITH_SKICARDS_TOPIC.name
    )
{code}

And inside kafka got appeared topic "STREAM_USER_WITH_SKICARD-skicards_by_owner_id-repartition"

And then incide the debug I see that code try to retrieve meta-info (StreamPartitionAssignor.java:446) for topic skicards_by_owner_id-repartition instead "STREAM_USER_WITH_SKICARD-skicards_by_owner_id-repartition".

Is that bug that you spoke about? 
p.s. I`m not sure should I report it as new ticket that`s why provide the additional comment here. 

> Missing topics on startup
> -------------------------
>
>                 Key: KAFKA-3544
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3544
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 0.10.0.0
>            Reporter: Greg Fodor
>            Assignee: Guozhang Wang
>              Labels: semantics
>
> When running a relatively complex job with multiple tasks and state stores, on the first run I get errors due to some of the intermediate topics not existing. Subsequent runs work OK. My assumption is streams may be creating topics lazily, so if downstream tasks are initializing before their parents have had a chance to create their necessary topics then the children will attempt to start consuming from topics that do not exist yet.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)