You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "xiangfu0 (via GitHub)" <gi...@apache.org> on 2025/05/21 07:35:30 UTC
[I] Ensure data dir is accessible from controller/server/minion [pinot]
xiangfu0 opened a new issue, #15860:
URL: https://github.com/apache/pinot/issues/15860
During the component start, create and delete a tmp file into the data dir then delete it.
This is to ensure the permission is set correctly and fail earlier with a more explicit error.
Ref: https://apache-pinot.slack.com/archives/C011C9JHN7R/p1747663133588709
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add sanity check into Pinot Components start() method to ensure data dir is accessible [pinot]
Posted by "NihalJain (via GitHub)" <gi...@apache.org>.
NihalJain commented on issue #15860:
URL: https://github.com/apache/pinot/issues/15860#issuecomment-2897060333
Can you please assign this to me, I can try this one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add sanity check into Pinot Components start() method to ensure data dir is accessible [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #15860: Add sanity check into Pinot Components start() method to ensure data dir is accessible
URL: https://github.com/apache/pinot/issues/15860
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add sanity check into Pinot Components start() method to ensure data dir is accessible [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #15860:
URL: https://github.com/apache/pinot/issues/15860#issuecomment-5769483094
This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add sanity check into Pinot Components start() method to ensure data dir is accessible [pinot]
Posted by "xiangfu0 (via GitHub)" <gi...@apache.org>.
xiangfu0 commented on issue #15860:
URL: https://github.com/apache/pinot/issues/15860#issuecomment-2898636566
> Can you please assign this to me, I can try this one?
Done, thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add sanity check into Pinot Components start() method to ensure data dir is accessible [pinot]
Posted by "NihalJain (via GitHub)" <gi...@apache.org>.
NihalJain commented on issue #15860:
URL: https://github.com/apache/pinot/issues/15860#issuecomment-2902171612
So I was trying to reproduce this issue.
- Created a data directory with 700 permission
- Created data/index with 400 permission
- Created data/segmentTar with 400 permission
Started server with above directories as data directory, server starts as expected but I get the BAD status in UI, after I tried to load any data to a table with errors in server logs:
```
2025/05/22 23:45:20.081 ERROR [77_8098 - SegmentOnlineOfflineStateModel] [HelixTaskExecutor-message_handle_thread_30] Caught exception while processing SegmentOnlineOfflineStateModel.onBecomeOnlineFromOffline() for table: airlineStats_OFFLINE, segment: airlineStats_OFFLINE_16085_16085_0
java.lang.IllegalStateException: Unable to create index directory at data/index/airlineStats_OFFLINE. Please check for available space and write-permissions for this directory.
at org.apache.pinot.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:602) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.pinot.core.data.manager.BaseTableDataManager.init(BaseTableDataManager.java:176) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.pinot.core.data.manager.provider.DefaultTableDataManagerProvider.getTableDataManager(DefaultTableDataManagerProvider.java:95) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.createTableDataManager(HelixInstanceDataManager.java:304) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708) ~[?:?]
at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addOnlineSegment(HelixInstanceDataManager.java:268) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeOnlineFromOffline(SegmentOnlineOfflineStateModelFactory.java:167) [pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at jdk.internal.reflect.GeneratedMethodAccessor71.invoke(Unknown Source) ~[?:?]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:350) [pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:278) [pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97) [pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49) [pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.base/java.lang.Thread.run(Thread.java:842) [?:?]
2025/05/22 23:45:20.083 ERROR [HelixStateTransitionHandler] [HelixTaskExecutor-message_handle_thread_30] Exception while executing a state transition task airlineStats_OFFLINE_16085_16085_0
java.lang.reflect.InvocationTargetException: null
at jdk.internal.reflect.GeneratedMethodAccessor71.invoke(Unknown Source) ~[?:?]
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.invoke(HelixStateTransitionHandler.java:350) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.helix.messaging.handling.HelixStateTransitionHandler.handleMessage(HelixStateTransitionHandler.java:278) [pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:97) [pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.helix.messaging.handling.HelixTask.call(HelixTask.java:49) [pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.base/java.lang.Thread.run(Thread.java:842) [?:?]
Caused by: java.lang.IllegalStateException: Unable to create index directory at data/index/airlineStats_OFFLINE. Please check for available space and write-permissions for this directory.
at org.apache.pinot.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:602) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.pinot.core.data.manager.BaseTableDataManager.init(BaseTableDataManager.java:176) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.pinot.core.data.manager.provider.DefaultTableDataManagerProvider.getTableDataManager(DefaultTableDataManagerProvider.java:95) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.createTableDataManager(HelixInstanceDataManager.java:304) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708) ~[?:?]
at org.apache.pinot.server.starter.helix.HelixInstanceDataManager.addOnlineSegment(HelixInstanceDataManager.java:268) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
at org.apache.pinot.server.starter.helix.SegmentOnlineOfflineStateModelFactory$SegmentOnlineOfflineStateModel.onBecomeOnlineFromOffline(SegmentOnlineOfflineStateModelFactory.java:167) ~[pinot-all-1.4.0-SNAPSHOT-jar-with-dependencies.jar:1.4.0-SNAPSHOT-5ede5782d132c1b3dbe1ca81e94a69614e49af6e]
... 11 more
2025/05/22 23:45:20.085 ERROR [StateModel] [HelixTaskExecutor-message_handle_thread_30] Default rollback method invoked on error. Error Code: ERROR
2025/05/22 23:45:20.091 ERROR [HelixTask] [HelixTaskExecutor-message_handle_thread_30] Message execution failed. msgId: 123aed45-d46c-48fe-aa9f-38a43591f4d4, errorMsg: java.lang.reflect.InvocationTargetException
2025/05/22 23:45:20.096 ERROR [HelixStateTransitionHandler] [HelixTaskExecutor-message_handle_thread_30] Skip internal error. errCode: ERROR, errMsg: null
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] Add sanity check into Pinot Components start() method to ensure data dir is accessible [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #15860:
URL: https://github.com/apache/pinot/issues/15860#issuecomment-5335834301
This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org