Apache Pinot Daily Email Digest (2022-03-21)

[Pinot Slack Email Digest <ap...@gmail.com>]

### _#general_

  
 **@itay:** @itay has joined the channel  
 **@yoav:** @yoav has joined the channel  
 **@saumya2700:** Hi All, is there any way that we can dump whole message from
topic in one column, my use case is that we have very complex avro-schema ,
and to create schema and map every field is very tedious work. I have done
some transformations using *tranformationConfigs* but for every topic and
schema it is going to be very complex.  
**@mayanks:** There is avro to Pinot schema utility in tools you can use to
get a starting point, and then customize it. You can also programmatically
generate it for your customizations  
**@g.kishore:** You can also store complex fields as Json in Pinot and use
json index  
**@mayanks:** There is avro to Pinot schema utility in tools you can use to
get a starting point, and then customize it. You can also programmatically
generate it for your customizations  

###  _#random_

  
 **@itay:** @itay has joined the channel  
 **@yoav:** @yoav has joined the channel  

###  _#troubleshooting_

  
 **@ryantle1028:** hi now i try to create real time table consume kafka. it's
created but status BAD. Please Help recommend this. error log is --> Mar 21
12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR
[MessageGenerationPhase] [HelixController-pipeline-default-pinot-
uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state
for resource: user_stream_tykb_REALTIME partition:
user_stream_tykb__2__0__20220321T0506Z from stateModelDefinitionclass
org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21
12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR
[MessageGenerationPhase] [HelixController-pipeline-default-pinot-
uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state
for resource: user_stream_tykb_REALTIME partition:
user_stream_tykb__3__0__20220321T0506Z from stateModelDefinitionclass
org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21
12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR
[MessageGenerationPhase] [HelixController-pipeline-default-pinot-
uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state
for resource: user_stream_tykb_REALTIME partition:
user_stream_tykb__4__0__20220321T0506Z from stateModelDefinitionclass
org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21
12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR
[MessageGenerationPhase] [HelixController-pipeline-default-pinot-
uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state
for resource: user_stream_tykb_REALTIME partition:
user_stream_tykb__5__0__20220321T0506Z from stateModelDefinitionclass
org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21
12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR
[MessageGenerationPhase] [HelixController-pipeline-default-pinot-
uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state
for resource: user_stream_tykb_REALTIME partition:
user_stream_tykb__6__0__20220321T0506Z from stateModelDefinitionclass
org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21
12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR
[MessageGenerationPhase] [HelixController-pipeline-default-pinot-
uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state
for resource: user_stream_tykb_REALTIME partition:
user_stream_tykb__7__0__20220321T0506Z from stateModelDefinitionclass
org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21
12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.937 ERROR
[MessageGenerationPhase] [HelixController-pipeline-default-pinot-
uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state
for resource: user_stream_tykb_REALTIME partition:
user_stream_tykb__8__0__20220321T0506Z from stateModelDefinitionclass
org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21
12:06:40 poc-pinot01 pinot-admin.sh: 2022/03/21 12:06:40.938 ERROR
[MessageGenerationPhase] [HelixController-pipeline-default-pinot-
uat-(1ad8bfc9_DEFAULT)] Event 1ad8bfc9_DEFAULT : Unable to find a next state
for resource: user_stream_tykb_REALTIME partition:
user_stream_tykb__9__0__20220321T0506Z from stateModelDefinitionclass
org.apache.helix.model.StateModelDefinition from:ERROR to:CONSUMING Mar 21
12:07:07 poc-pinot01 systemd-logind: Removed session 590.  
**@walterddr:** can you share your table config and settings for your kafka
topic?  
 **@alihaydar.atil:** Hi everyone, Do indexes also work for multi-valued
columns?  
**@walterddr:** which index method do you wan to use for mv columns?  
**@mayanks:** Inv index certainly is supported for MV columns.  
 **@itay:** @itay has joined the channel  
 **@yoav:** @yoav has joined the channel  
 **@luisfernandez:** looking for some help here as the dead node hasn't been
able to be deleted and the UI is still blocked on the realtime table, also I
have questions about the new server and the rebalance operation as I don't see
the data that I would expect there  

###  _#pinot-dev_

  
 **@ashish:** Thanks for taking a look. Will attach the profile/stacktrace
soon. The retrieval of dictId is ok - it's the call to getInternal that does
not seem to go through the cache and hence we see the same latency when the
same query is repeated. Usually, second run of the query runs faster but not
in the case we observed  
**@ashish:** Will try it out. Not sure why retrieving dict values through
getInternal shouldn't go through the data fetcher and cached - it would have
done that if the same column was present in projection only. No?  
 **@ashish:** Yes, that makes sense.  

###  _#getting-started_

  
 **@itay:** @itay has joined the channel  
 **@yoav:** @yoav has joined the channel  
\--------------------------------------------------------------------- To
unsubscribe, e-mail: dev-unsubscribe@pinot.apache.org For additional commands,
e-mail: dev-help@pinot.apache.org