You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pinot.apache.org by Pinot Slack Email Digest <sn...@apache.org> on 2021/06/18 02:00:24 UTC
Apache Pinot Daily Email Digest (2021-06-17)
### _#general_
**@patidar.rahul8392:** I have one table in pinot I was using same realtime
table since last 15 days and it was working fine and loading realtime data
from Kafka at a time I was able to see one segment in consuming state and
others are in online state.but today suddenly it's stopped consuming the data
when I checked the segment status all are showing as online and no segment is
consuming the data.what could be the issue here. @xiangfu0 @mayanks
**@xiangfu0:** do you have Pinot server logs?
**@patidar.rahul8392:** Looks like some issue my data @xiangfu0 . That records
are not showing in topic also .I'm checking the issue will update you in case
of any issue again
**@nithinbharadwajd661:** @nithinbharadwajd661 has joined the channel
**@mark.needham:** I'm trying to query Pinot using the Python 'pinotdb'
library: ```version: '3.7' services: pinot: image: apachepinot/pinot:0.7.1
command: "QuickStart -type batch" container_name: "pinot-quickstart" volumes:
\- ./data:/data ports: \- "9000:9000" \- "8000:8000"``` ```from pinotdb import
connect conn = connect(host='localhost', port=9000, path='/query/sql',
scheme='http') curs = conn.cursor() curs.execute(""" SELECT * from cases LIMIT
10 """) for row in curs: print(row)``` I get this error when I run the query:
```Traceback (most recent call last): File "query.py", line 5, in <module>
curs.execute(""" File "/home/markhneedham/.local/share/virtualenvs/pinot-
playground-V0PLiJ36/lib/python3.8/site-packages/pinotdb/db.py", line 44, in g
return f(self, *args, **kwargs) File
"/home/markhneedham/.local/share/virtualenvs/pinot-
playground-V0PLiJ36/lib/python3.8/site-packages/pinotdb/db.py", line 289, in
execute self.check_sufficient_responded( File
"/home/markhneedham/.local/share/virtualenvs/pinot-
playground-V0PLiJ36/lib/python3.8/site-packages/pinotdb/db.py", line 253, in
check_sufficient_responded raise exceptions.DatabaseError(
pinotdb.exceptions.DatabaseError: Query SELECT * from cases LIMIT 10 timed
out: Out of -1, only -1 responded, while needed was -1``` Am I querying on the
right port? In the examples port 8009 is used, but I tried that and got a
different error!
**@mayanks:** Are you able to query Pinot using the query console? If so, may
be some issue in the python script?
**@mayanks:** Also, note that query console runs on the controller. However,
if you are using a client, then you can directly query broler port.
**@mark.needham:** query console meaning the web app at localhost:9000? If so
yep that works fine
**@kulbir.nijjer:** try using broker port usually it's 8000
**@mark.needham:** if I change to port 8000: ```Traceback (most recent call
last): File "query.py", line 5, in <module> curs.execute(""" File
"/home/markhneedham/.local/share/virtualenvs/pinot-
playground-V0PLiJ36/lib/python3.8/site-packages/pinotdb/db.py", line 44, in g
return f(self, *args, **kwargs) File
"/home/markhneedham/.local/share/virtualenvs/pinot-
playground-V0PLiJ36/lib/python3.8/site-packages/pinotdb/db.py", line 303, in
execute raise exceptions.DatabaseError(msg) pinotdb.exceptions.DatabaseError:
{'errorCode': 410, 'message': 'BrokerResourceMissingError'}```
**@mayanks:** This means no broker found for the table.
**@mark.needham:** if I change path to `/sql`: ```Traceback (most recent call
last): File "/home/markhneedham/.local/share/virtualenvs/pinot-
playground-V0PLiJ36/lib/python3.8/site-packages/pinotdb/db.py", line 275, in
execute payload = r.json() File
"/home/markhneedham/.local/share/virtualenvs/pinot-
playground-V0PLiJ36/lib/python3.8/site-packages/requests/models.py", line 900,
in json return complexjson.loads(self.text, **kwargs) File
"/usr/lib/python3.8/json/__init__.py", line 357, in loads return
_default_decoder.decode(s) File "/usr/lib/python3.8/json/decoder.py", line
337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File
"/usr/lib/python3.8/json/decoder.py", line 355, in raw_decode raise
JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) The
above exception was the direct cause of the following exception: Traceback
(most recent call last): File "query.py", line 5, in <module> curs.execute("""
File "/home/markhneedham/.local/share/virtualenvs/pinot-
playground-V0PLiJ36/lib/python3.8/site-packages/pinotdb/db.py", line 44, in g
return f(self, *args, **kwargs) File
"/home/markhneedham/.local/share/virtualenvs/pinot-
playground-V0PLiJ36/lib/python3.8/site-packages/pinotdb/db.py", line 277, in
execute raise exceptions.DatabaseError( pinotdb.exceptions.DatabaseError:
Error when querying SELECT * from cases LIMIT 10 from , raw response is:```
**@mayanks:** ``````
**@mayanks:** Lets first try outside of the python client to eliminate it from
the equation
**@mark.needham:** > curl: (56) Recv failure: Connection reset by peer My
Docker Compose script is like this now: ```version: '3.7' services: pinot:
image: apachepinot/pinot:0.7.1 command: "QuickStart -type batch"
container_name: "pinot-quickstart" volumes: \- ./data:/data ports: \-
"9000:9000" \- "8099:8099" \- "8000:8000"```
**@mark.needham:** ```$ sudo lsof -i :8099 COMMAND PID USER FD TYPE DEVICE
SIZE/OFF NODE NAME docker-pr 105198 root 4u IPv4 612824 0t0 TCP *:8099
(LISTEN) docker-pr 105206 root 4u IPv6 607186 0t0 TCP *:8099 (LISTEN)```
**@mark.needham:** it works on port 9000: ```from pinotdb import connect conn
= connect(host='localhost', port=9000, path='/sql', scheme='http') curs =
conn.cursor() curs.execute(""" SELECT * from baseballStats LIMIT 10 """) for
row in curs: print(row)```
**@mark.needham:** I had been getting a `BrokerResourceMissingError` because I
was querying a non existent table. It only just occurred to me what that error
message meant!
**@mayanks:** :+1:
**@mark.needham:** is that the port you'd expect to be available?
**@mark.needham:** I got it from looking at the request being made by the
query console
**@mayanks:** That port is the controller port. You'd want to query the broker
port. The console sits on top of controller because it needs to access other
things like ZK. However, for query path, you can/should directly hit the
broker rest api.
**@pedro.cls93:** Hello, I've had to downgrade pinot from a 0.8.0 snapshot
version to 0.7.1 (I needed some features from 0.8.0, but due to shifting needs
was forced to 0.7.1). I deleted my old table and am currently re-ingesting
into the 0.7.1 equivalent. However, I note that the UI is extremely slow with
the following error when I try to query the table: ```[ { "errorCode": 410,
"message": "BrokerResourceMissingError" } ]``` The broker logs show this
exception: ```2021/06/17 16:47:13.084 WARN [BaseInstanceSelector]
[ClusterChangeHandlingThread] Failed to find servers hosting segment:
HitExecutionView__10__19__20210616T1108Z for table: HitExecutionView_REALTIME
(all ONLINE/CONSUMING instances: [] and OFFLINE instances: [] are disabled,
counting segment as unavailable) 2021/06/17 17:05:18.183 ERROR
[BrokerResourceOnlineOfflineStateModelFactory] [HelixTaskExecutor-
message_handle_thread] Caught exception while processing transition from
OFFLINE to ONLINE for table: hitexecutionview_REALTIME
java.lang.IllegalStateException: Failed to find ideal state for table:
hitexecutionview_REALTIME at
shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:518)
~[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.pinot.broker.routing.RoutingManager.buildRouting(RoutingManager.java:309)
~[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.pinot.broker.broker.helix.BrokerResourceOnlineOfflineStateModelFactory$BrokerResourceOnlineOfflineStateModel.onBecomeOnlineFromOffline(BrokerResourceOnlineOfflineStateModelFactory.java:80)
[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_282] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_282] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_282] at java.lang.reflect.Method.invoke(Method.java:498)
~[?:1.8.0_282] at
org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:404)
[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:331)
[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97) [pinot-
all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49) [pinot-
all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_282] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_282] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_282] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
2021/06/17 17:05:18.283 ERROR [HelixStateTransitionHandler]
[HelixTaskExecutor-message_handle_thread] Exception while executing a state
transition task hitexecutionview_REALTIME
java.lang.reflect.InvocationTargetException: null at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_282] at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
~[?:1.8.0_282] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[?:1.8.0_282] at java.lang.reflect.Method.invoke(Method.java:498)
~[?:1.8.0_282] at
org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:404)
~[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:331)
[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97) [pinot-
all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49) [pinot-
all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_282] at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_282] at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_282] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282] Caused
by: java.lang.IllegalStateException: Failed to find ideal state for table:
hitexecutionview_REALTIME at
shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:518)
~[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.pinot.broker.routing.RoutingManager.buildRouting(RoutingManager.java:309)
~[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] at
org.apache.pinot.broker.broker.helix.BrokerResourceOnlineOfflineStateModelFactory$BrokerResourceOnlineOfflineStateModel.onBecomeOnlineFromOffline(BrokerResourceOnlineOfflineStateModelFactory.java:80)
~[pinot-all-0.7.1-jar-with-
dependencies.jar:0.7.1-afa4b252ab1c424ddd6c859bb305b2aa342b66ed] ... 12 more
2021/06/17 17:05:18.301 ERROR [StateModel] [HelixTaskExecutor-
message_handle_thread] Default rollback method invoked on error. Error Code:
ERROR 2021/06/17 17:05:18.383 ERROR [HelixTask] [HelixTaskExecutor-
message_handle_thread] Message execution failed. msgId:
366af265-24b4-4b59-a28f-1e6387e5d2aa, errorMsg:
java.lang.reflect.InvocationTargetException 2021/06/17 17:05:18.392 ERROR
[HelixStateTransitionHandler] [HelixTaskExecutor-message_handle_thread] Skip
internal error. errCode: ERROR, errMsg: null 2021/06/17 17:06:50.697 WARN
[RoutingManager] [HelixTaskExecutor-message_handle_thread] Routing does not
exist for table: hitexecutionview_REALTIME, skipping refreshing segment
2021/06/17 17:06:51.179 WARN [RoutingManager] [HelixTaskExecutor-
message_handle_thread] Routing does not exist for table:
hitexecutionview_REALTIME, skipping refreshing segment 2021/06/17 17:07:22.699
WARN [RoutingManager] [HelixTaskExecutor-message_handle_thread] Routing does
not exist for table: hitexecutionview_REALTIME, skipping refreshing segment
2021/06/17 17:07:23.997 WARN [RoutingManager] [HelixTaskExecutor-
message_handle_thread] Routing does not exist for table:
hitexecutionview_REALTIME, skipping refreshing segment 2021/06/17 17:07:25.800
WARN [RoutingManager] [HelixTaskExecutor-message_handle_thread] Routing does
not exist for table: hitexecutionview_REALTIME, skipping refreshing segment```
Any ideas?
**@mayanks:** The log suggests there isn't an ideal-state for the table. Can
you confirm? If so, likely means table was not created
**@pedro.cls93:** Where can I confirm? Zookeeper browser?
**@mayanks:** yes
**@pedro.cls93:** I see an ideal state for the table
**@pedro.cls93:** This new table has the same name as the older table but
lowercased. Perhaps that is the issue?
**@mayanks:** and it has all segments ONLINE/CONSUMING?
**@pedro.cls93:** yes
**@mayanks:** Yes, that explains the `BrokerResourceMissing`
**@mayanks:** as in broker did not find the table
**@pedro.cls93:** How can I fix this? It seems data was not correctly or
completely deleted when I deleted the old table.
**@mayanks:** I think when you delete a RT table, you have to wait for its
ideal-state and external view to be completely gone from ZK browers before
recreating. Can you try again?
**@mayanks:** It should be documented in docs, if we haven't already (i
remember seeing it though)
**@pedro.cls93:** of course
**@pedro.cls93:** Does not seem to work, this is starting to look like a
degraded filesystem.
**@pedro.cls93:** In the zookeeper browser in the server instances I see the
following errors per partition:
**@pedro.cls93:** ```{ "id": "103988a1c17001e__HitExecutionView_REALTIME",
"simpleFields": {}, "mapFields": { "HELIX_ERROR 20210617-164711.000407
STATE_TRANSITION 86863a54-a4a6-4ea3-ace3-49b20fdb44e1": { "AdditionalInfo":
"Exception while executing a state transition task
HitExecutionView__0__16__20210615T1105Zjava.lang.reflect.InvocationTargetException\n\tat
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
java.lang.reflect.Method.invoke(Method.java:498)\n\tat
org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:404)\n\tat
org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:331)\n\tat
org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97)\n\tat
org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49)\n\tat
java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat
java.lang.Thread.run(Thread.java:748)\nCaused by:
java.lang.IllegalStateException: Unsupported json index version: 2\n\tat
shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:493)\n\tat
org.apache.pinot.core.segment.index.readers.json.ImmutableJsonIndexReader.<init>(ImmutableJsonIndexReader.java:56)\n\tat
org.apache.pinot.core.segment.index.column.PhysicalColumnIndexContainer.<init>(PhysicalColumnIndexContainer.java:114)\n\tat
org.apache.pinot.core.indexsegment.immutable.ImmutableSegmentLoader.load(ImmutableSegmentLoader.java:115)\n\tat
org.apache.pinot.core.data.manager.realtime.RealtimeTableDataManager.addSegment(RealtimeTableDataManager.java:283)\n\tat
org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addRealtimeSegment(HelixInstanceDataManager.java:138)\n\tat
org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeOnlineFromOffline(SegmentOnlineOfflineStateModelFactory.java:164)\n\t...
12 more\n", "Class": "class
org.apache.helix.messaging.handling.HelixStateTransitionHandler", "MSG_ID":
"f73f44ab-62fe-4f24-86af-8e44ae873a33", "Message state": "READ" },
"HELIX_ERROR 20210617-164711.000434 STATE_TRANSITION
254788e1-6826-4321-a3ae-3ecb978ca3ae": { "AdditionalInfo": "Message execution
failed. msgId: f73f44ab-62fe-4f24-86af-8e44ae873a33, errorMsg:
java.lang.reflect.InvocationTargetException", "Class": "class
org.apache.helix.messaging.handling.HelixStateTransitionHandler", "MSG_ID":
"f73f44ab-62fe-4f24-86af-8e44ae873a33", "Message state": "READ" } },
"listFields": {} }```
**@mayanks:** ```Unsupported json index version: 2\n\tat
shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:493)\n\tat
org.apache.pinot.core.segment.index.readers.json.ImmutableJsonIndexReader.<init>```
**@pedro.cls93:** Where does that come from?
**@mayanks:** In your log above
**@pedro.cls93:** I know, I mean what causes it in the pinot code path?
**@mayanks:** Seems like going back from 0.8.0 to 0.7.1 might be causing this.
**@mayanks:** This is when the code is trying to add a new realtime segment.
**@pedro.cls93:** That is strange, I tested this downgrade change in lower
environments without issue.
**@pedro.cls93:** Perhaps segments where not online and consuming...
:thinking_face:
**@mayanks:** I think the delete + recreate and downgrade was not cleanly down
and cluster went into wierd state
**@pedro.cls93:** Is there a growing list of 0.8.0 features I can take a look
at?
**@canerbalci:** Is table storage quota enforced for Realtime tables? If so,
are consuming segments counted towards the quota or only on disk flushed
segments are considered?
**@aaronlevin:** @aaronlevin has joined the channel
**@jadami:** @jadami has joined the channel
**@dang:** @dang has joined the channel
**@ojasmulay:** @ojasmulay has joined the channel
**@mail765:** @mail765 has joined the channel
**@mail765:** Hey All, Glad to be part of the community
:slightly_smiling_face: I work as a Tableau developer and recently was
requested to load data from Pinot into Tableau and while searching the net, I
found that Pinot has JDBC connector t use with Tableau but was confused around
how to actually do it. Landed on the community slack and was wondering if
anyone has had any luck connecting Pinot with Tableau using the JDBC
connector? Link: P.S: I also understand that this question might be too silly
or "non-technical" for folks out here, but any kind of help will be
appreciated! Coffee is on me!
**@mayanks:** Hello, welcome to the community. Pinot does have a JDBC driver,
but since Pinot does not support full sql (eg complex joins, nested queries
etc), it is not yet in a shape to be directly plugged into Tableau
**@mayanks:** Something we hope to add in near future cc @xiangfu0
**@mail765:** @mayanks Thanks mayank for the response. What if I just have ONE
table, not joining or nested queries. Do you think I can make Pinot work in
Tableau?
**@mayanks:** We should be able to make it work, will need to check on custom
functions
**@mail765:** @mayanks awesome, if possible, can you point me in the right
direction or documentation links?
**@mayanks:** @kharekartik
**@baluvignesh:** @baluvignesh has joined the channel
### _#random_
**@nithinbharadwajd661:** @nithinbharadwajd661 has joined the channel
**@aaronlevin:** @aaronlevin has joined the channel
**@jadami:** @jadami has joined the channel
**@dang:** @dang has joined the channel
**@ojasmulay:** @ojasmulay has joined the channel
**@mail765:** @mail765 has joined the channel
**@baluvignesh:** @baluvignesh has joined the channel
### _#troubleshooting_
**@dlavoie:** Pinot is a rest API that can support high QPS, what benefits
are you looking into a connection pool? Connection pooling would likely be a
threading configuration from your java service http client.
**@chxing:** Thx for your replay, I think connection pool can support high qps
with little threads, but pinot don’t support , we can just use thread pool in
service code right?
**@dlavoie:** Connection pool make sense for a stateful protocol like jdbc or
raw tcp. HTTP is stateless. What you have on the server is HTTP listeners
thread pool that answer client commands. You can scale those by adding more
pinot brokers. That’s nothing else than regular a Jetty web server. Pinot is
designed to serve high QPS with ( > 100k / sec) stateless http commands. If
you introduce a thread pool on the client side, you will only create a
potentially bottleneck in your client code. You can create as many concurrent
http client threads against Pinot. The only benefit of pooling on the client
end is that you will control potential resource exhaustion from your client.
You’ll be reusing threads, not tcp connections.
**@prasanna.gsl:** • Hi All, Can someone confirm if Apache Pinot has support
for special characters? For text based indexing, does it take care of these
localized characters? Is localized search supported?
**@nithinbharadwajd661:** @nithinbharadwajd661 has joined the channel
**@aaronlevin:** @aaronlevin has joined the channel
**@jadami:** @jadami has joined the channel
**@dang:** @dang has joined the channel
**@ojasmulay:** @ojasmulay has joined the channel
**@mail765:** @mail765 has joined the channel
**@baluvignesh:** @baluvignesh has joined the channel
### _#pinot-k8s-operator_
**@aaronlevin:** @aaronlevin has joined the channel
### _#pinot-dev_
**@ken:** In `SegmentDeletionManager.removeAgedDeletedSegments()`, there’s
this bit of code: ``` try { // Check that the directory for deleted segments
exists. if (!pinotFS.isDirectory(deletedDirURI)) { LOGGER.warn("Deleted
segment directory {} does not exist or it is not directory.",
deletedDirURI.toString()); return; }``` But if we’ve never deleted any
segments, then this directory won’t exist yet. So shouldn’t this check be “if
it exists && it’s not a directory”? Otherwise we get regular warnings in our
log files, which trigger an alert, etc, etc.
**@ken:** Also the `HadoopPinotFS.isDirectory()` call throws a
`FileNotFoundException` in this case, which isn’t caught in this method,
bubbles, up, and generates a `Could not get file status for
hdfs:///user/hadoop/pinot-beta/pinot-segments//Deleted_Segments` error.
**@ken:** Which is what really causes our alerting system to go off…
**@mayanks:** Here's what I'd think: ```1. What is the expected behavior wrt
Deleted Segments directory? Is it that if directory does not exist then we
don't save, or is it that we always save, in which case we should create this
directory in ControllerStarter. 2\. Depending on what's the consensus on 1, we
can either assert or make this a debug message after adding Ken's suggested
check, instead of warn.```
**@mayanks:** IMHO, we should not force DeletedSegments directory to be
created (unless user specifies in a conf), and make this a debug message,
along with exists() check.
**@mayanks:** Thoughts welcome.
**@ken:** I don’t know enough about how segment retention is designed to work,
so hard for me to weigh in with anything useful, sorry.
**@ken:** What would be the “user specified in conf” approach? Some new flag?
**@mayanks:** Similar to dataDir
**@mayanks:** For now, may be just file a PR with your suggested fix, we can
discuss there.
**@ken:** OK
**@aaronlevin:** @aaronlevin has joined the channel
**@jadami:** @jadami has joined the channel
### _#community_
**@aaronlevin:** @aaronlevin has joined the channel
### _#announcements_
**@vaibhav.mital:** @vaibhav.mital has joined the channel
**@aaronlevin:** @aaronlevin has joined the channel
**@jadami:** @jadami has joined the channel
### _#getting-started_
**@vaibhav.mital:** @vaibhav.mital has joined the channel
**@nithinbharadwajd661:** @nithinbharadwajd661 has joined the channel
**@nishanth:** @nishanth has joined the channel
**@jadami:** @jadami has joined the channel
### _#releases_
**@jadami:** @jadami has joined the channel
### _#pinot-docsrus_
**@mark.needham:** @mark.needham has joined the channel
\--------------------------------------------------------------------- To
unsubscribe, e-mail: dev-unsubscribe@pinot.apache.org For additional commands,
e-mail: dev-help@pinot.apache.org