Apache Pinot Daily Email Digest (2022-05-08)

[Pinot Slack Email Digest <ap...@gmail.com>]

### _#general_

  
 **@krishnaraoveera1294:** @krishnaraoveera1294 has joined the channel  
 **@vanduc.dn:** @vanduc.dn has joined the channel  

###  _#random_

  
 **@krishnaraoveera1294:** @krishnaraoveera1294 has joined the channel  
 **@vanduc.dn:** @vanduc.dn has joined the channel  

###  _#troubleshooting_

  
 **@diogo.baeder:** Hi folks! How can I use the `/ingestFromURI` endpoint from
the Controller API to ingest a file as a segment, but defining the segment
name myself? I tried passing `segment.name` to the `batchConfigMapStr`
parameter JSON, but it didn't work, the Controller ends up creating the
segment name by itself. I'd like to have more control over this, because I
want to be able to more easily replace segments.  
**@diogo.baeder:** Just for reference, this is my full URI:
```/ingestFromURI?tableNameWithType=cases_OFFLINE&batchConfigMapStr={\"inputFormat\":\"json\",\"input.fs.className\":\"org.apache.pinot.spi.filesystem.LocalPinotFS\",\"segment.name\":\"cases_br_2015_07\"}&sourceURIStr=file:///sensitive-
data/outputs/cases/br/2015_07.json```  
**@mayanks:** You can use batch ingestion that allows you full control:  
**@mayanks:** There’s segment name generator spec:  
**@diogo.baeder:** Ah, it's `segmentNameGeneratorSpec` then... I don't want to
use the usual batch ingestion approach because I want to be able to manually
upload segments, to be able to replace them later on demand. Batch ingestion
doesn't ingest files that have already been marked as processed, right?  
**@diogo.baeder:** Worked now! I just used `segmentNameGenerator.type` and
`segmentNameGenerator.configs.segment.name` (with the literal dots) as keys
for the `batchConfigMapStr` JSON and it's now generating segments as I want
:slightly_smiling_face:  
**@diogo.baeder:** Aaaand I could also replace the segment by just ingesting
the same segment again, with changed data! Yay!  
**@mark.needham:**  
**@mark.needham:** It sounds like you did something similar to what I did in
this one  
**@mark.needham:** maybe I can update it to show how to use the HTTP API  
**@diogo.baeder:** Yeah, I used the API just because I didn't want to depend
on the admin CLI on the server that will be managing the loading of data into
Pinot. In any case, it worked quite well for me, after I finally understood
how to properly configure the segment name.  
 **@krishnaraoveera1294:** @krishnaraoveera1294 has joined the channel  
 **@vanduc.dn:** @vanduc.dn has joined the channel  

###  _#getting-started_

  
 **@krishnaraoveera1294:** @krishnaraoveera1294 has joined the channel  
 **@vanduc.dn:** @vanduc.dn has joined the channel  

###  _#introductions_

  
 **@krishnaraoveera1294:** @krishnaraoveera1294 has joined the channel  
 **@vanduc.dn:** @vanduc.dn has joined the channel  
\--------------------------------------------------------------------- To
unsubscribe, e-mail: dev-unsubscribe@pinot.apache.org For additional commands,
e-mail: dev-help@pinot.apache.org