You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "madRobot10 (via GitHub)" <gi...@apache.org> on 2024/12/25 08:11:20 UTC

[I] creating a table configured with Apache Kafka [pinot]

madRobot10 opened a new issue, #14715:
URL: https://github.com/apache/pinot/issues/14715

   I am trying to follow the doc. but I face this problem.
   
   I created the topic in Kafka and inserted some data on it. also, the schema is created, but the table is not.
   
   my schema:
   {
     "schemaName": "events",
     "enableColumnBasedNullHandling": false,
     "dimensionFieldSpecs": [
       {
         "name": "uuid",
         "dataType": "STRING",
         "notNull": false
       }
     ],
     "metricFieldSpecs": [
       {
         "name": "count",
         "dataType": "INT",
         "notNull": false
       }
     ],
     "dateTimeFieldSpecs": [
       {
         "name": "ts",
         "dataType": "TIMESTAMP",
         "notNull": false,
         "format": "TIMESTAMP",
         "granularity": "1:MILLISECONDS"
       }
     ]
   }
   
   Table conf.:
   {
     "tableName": "events",
     "tableType": "REALTIME",
     "segmentsConfig": {
       "timeColumnName": "ts",
       "schemaName": "events",
       "replicasPerPartition": "1"
     },
     "tenants": {
       "broker": "DefaultTenant",
       "server": "DefaultTenant"
     },
     "tableIndexConfig": {
       "loadMode": "MMAP",
       "streamConfigs": {
           "streamType": "kafka",
           "stream.kafka.broker.list": "kafka:9092",
           "stream.kafka.topic.name": "events",
           "stream.kafka.consumer.type": "simple",
           "stream.kafka.decoder.class.name": "org.apache.pinot.plugin.stream.kafka.KafkaJSONMessageDecoder",
           "realtime.segment.flush.threshold.rows": "10000"
         }
     },
     "metadata": {
       "customConfigs": {}
     }
   }
    the error message:
    {
     "code": 500,
     "error": "org.apache.pinot.spi.stream.TransientConsumerException: org.apache.pinot.shaded.org.apache.kafka.common.errors.TimeoutException: Failed to get offsets by times in 5000ms"
   }
   
   Kafka topic:
   madhhar@madhharus:~$ docker exec -it kafka kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic events --from-beginning
   {"ts": 1735112369985, "uuid": "c4456f4e-611b-48b6-b6da-550b4052c9d7", "count": 318}
   {"ts": 1735112370987, "uuid": "90727af9-0914-40b3-8af6-65f1c678d676", "count": 911}
   {"ts": 1735112371988, "uuid": "a280bbee-3e86-482f-839e-1872c0028995", "count": 768}
   {"ts": 1735112372990, "uuid": "3ee49eb9-b1b3-40b0-8173-938f32887e3e", "count": 78}
   {"ts": 1735112373991, "uuid": "065c78d7-c05d-4e9f-9d25-c5960b9d38cb", "count": 947}
   {"ts": 1735112374992, "uuid": "3d811d72-050f-4b10-a5aa-e332f6acb411", "count": 450}
   {"ts": 1735112375993, "uuid": "733eece5-2a57-48ef-a35c-0c3cf365f665", "count": 715}
   {"ts": 1735112376995, "uuid": "6be53521-969c-401c-a27a-8c2ef19bc0a3", "count": 759}
   {"ts": 1735112377995, "uuid": "b6075c33-50d8-4b5e-854a-45b14e0b9119", "count": 212}
   {"ts": 1735112378996, "uuid": "86734094-8e15-4376-9c3e-b56618e2bb09", "count": 875}
   {"ts": 1735112379998, "uuid": "fdfb2cce-bcce-4022-a3ee-827cb5f2170b", "count": 747}
   {"ts": 1735112380999, "uuid": "0c45bcec-161a-412d-bf38-e1c92f716a80", "count": 547}
   {"ts": 1735112382001, "uuid": "6fd0a307-d5d9-49fd-96af-67a5ce3f7d98", "count": 608}
   {"ts": 1735112383002, "uuid": "04e4a8dc-1de0-4f1a-88a5-af7a343f59a4", "count": 46}
   {"ts": 1735112384003, "uuid": "640944e6-abdf-49c5-8ef3-dd97e00dd676", "count": 224}
   {"ts": 1735112385005, "uuid": "7d41bd43-145b-4606-891f-fe1d04c5ab78", "count": 238}
   {"ts": 1735112386006, "uuid": "5a522e1c-cf89-47e4-9a51-07f9a800d3e0", "count": 244}
   {"ts": 1735112387007, "uuid": "0048f77f-38f4-4765-9c83-7d447b421095", "count": 779}
   {"ts": 1735112388008, "uuid": "31842f55-dd3a-44f8-beac-94fa7c7048a5", "count": 241}
   {"ts": 1735112389010, "uuid": "bbc6b245-d964-45ac-a106-13e2702130ab", "count": 994}
   {"ts": 1735112390011, "uuid": "1ad177f9-88ae-4559-8b70-bf37905d504e", "count": 122}
   {"ts": 1735112391013, "uuid": "bf2b002e-130e-42f8-be62-459a520727e3", "count": 394}
   {"ts": 1735112392014, "uuid": "d11d30f0-eeab-4efd-b94a-8a3258635178", "count": 600}
   {"ts": 1735112393015, "uuid": "9dd78e7a-96c0-4050-add0-4cceb5bc6bb6", "count": 114}
   {"ts": 1735112394016, "uuid": "6d3e629f-193a-400f-bd66-6d3fabeb1971", "count": 371}
   {"ts": 1735112395018, "uuid": "14f78ecb-fa2a-4417-84dc-23e408cd6705", "count": 272}
   {"ts": 1735112396019, "uuid": "fec5ca47-aa27-4a9b-851f-bcb275e1f3e7", "count": 323}
   {"ts": 1735112397021, "uuid": "56979da0-a18e-4178-81b2-6fbc6b41b861", "count": 888}
   {"ts": 1735112398022, "uuid": "1409767f-7745-4a70-a0e0-b23ef55888c8", "count": 658}
   {"ts": 1735112399024, "uuid": "1905899b-406b-499b-8abd-58a24f553b1e", "count": 151}
   {"ts": 1735112400025, "uuid": "45d6f861-5e94-4324-b6b9-d00e77cb60d9", "count": 433}
   {"ts": 1735112401026, "uuid": "7aae3d5b-399f-4a90-9fad-9e4bab167d75", "count": 860}
   {"ts": 1735112402028, "uuid": "19add349-d31c-44b3-ab8b-0cc1013bb037", "count": 300}
   {"ts": 1735112403029, "uuid": "1a878f82-302a-4895-bced-ae4b7ba7f48d", "count": 992}
   {"ts": 1735112404031, "uuid": "3dc66dad-e9e8-4ee6-95d9-d9a2e89c482d", "count": 352}
   ^CProcessed a total of 35 messages
   madhhar@madhharus:~$
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] creating a table configured with Apache Kafka [pinot]

Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #14715:
URL: https://github.com/apache/pinot/issues/14715#issuecomment-2588812190

   Seems like a connection issue. Let me know if the problem is solved now.
   You may get more help from the Slack community.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] creating a table configured with Apache Kafka [pinot]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #14715:
URL: https://github.com/apache/pinot/issues/14715#issuecomment-4747334483

   This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] creating a table configured with Apache Kafka [pinot]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #14715: creating a table configured with Apache Kafka
URL: https://github.com/apache/pinot/issues/14715


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] creating a table configured with Apache Kafka [pinot]

Posted by "JESUSR88 (via GitHub)" <gi...@apache.org>.
JESUSR88 commented on issue #14715:
URL: https://github.com/apache/pinot/issues/14715#issuecomment-3677379266

   Hola, en mi caso me sucede algo muy similar se supone que ya tengo casi todo creado con exito pero al tratar de registrar la tabla (en formato Realtime) simplemente no me lo permite. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org


Re: [I] creating a table configured with Apache Kafka [pinot]

Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #14715:
URL: https://github.com/apache/pinot/issues/14715#issuecomment-4493463244

   This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org