You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pinot.apache.org by "dang-stripe (via GitHub)" <gi...@apache.org> on 2025/04/02 22:06:19 UTC
[I] `Attempt to re-create an existing index for key` exception when adding derived column to startree index [pinot]
dang-stripe opened a new issue, #15440:
URL: https://github.com/apache/pinot/issues/15440
We ran into this error while restarting servers to reload segments. This caused the servers to fail to startup.
```
[2025-04-01 20:24:22.723960] ERROR [example_table_v1_OFFLINE-OfflineTableDataManager] [HelixTaskExecutor-message_handle_thread_4:155] Failed to load existing segment: example_table_v1_1989 with CRC: 3623618827 on tier: default
[2025-04-01 20:24:22.724033] java.lang.RuntimeException: Attempt to re-create an existing index for key: reporting_week.forward_index, for segmentDirectory: /pinot/data/index/example_table_v1_OFFLINE/example_table_v1_1989/v3
[2025-04-01 20:24:22.724062] at org.apache.pinot.segment.local.segment.store.SingleFileIndexDirectory.checkKeyNotPresent(SingleFileIndexDirectory.java:197) ~[pinot-all-1.2.0-2025-02-12-8fe105e2b7-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2025-02-12-8fe105e2b7-SNAPSHOT-8fe105e2b707ef1a6ca00f9fd546259ad88114c4]
[2025-04-01 20:24:22.724106] at org.apache.pinot.segment.local.segment.store.SingleFileIndexDirectory.allocNewBufferInternal(SingleFileIndexDirectory.java:170) ~[pinot-all-1.2.0-2025-02-12-8fe105e2b7-SNAPSHOT-jar-with-dependencies.jar:1.2.0-2025-02-12-8fe105e2b7-SNAPSHOT-8fe105e2b707ef1a6ca00f9fd546259ad88114c4]
[2025-04-01 20:24:22.724134] at org.apache.pinot.segment.local.segment.store.SingleFileIndexDirectory.newBuffer(SingleFileIndexDirectory.java:138)
```
This happened after applying a table change that adding new derived columns and adding the derived column to the `dimensionsSplitOrder` of the startree index.
table config transform
```
{
"columnName": "reporting_week",
"transformFunction": "dateTrunc('WEEK', reporting_date)"
},
```
schema
```
{
"name": "reporting_week",
"dataType": "TIMESTAMP",
"notNull": false,
"format": "TIMESTAMP",
"granularity": "1:DAYS"
},
```
We found that setting `enableDynamicStarTreeCreation` to true bypassed this error and let the servers recover.
cc @Jackie-Jiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] `Attempt to re-create an existing index for key` exception when adding derived column to startree index [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed issue #15440: `Attempt to re-create an existing index for key` exception when adding derived column to startree index
URL: https://github.com/apache/pinot/issues/15440
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] `Attempt to re-create an existing index for key` exception when adding derived column to startree index [pinot]
Posted by "Jackie-Jiang (via GitHub)" <gi...@apache.org>.
Jackie-Jiang commented on issue #15440:
URL: https://github.com/apache/pinot/issues/15440#issuecomment-2864776827
Created #15745 to mimic the flow, which passed without problem. Do you have a deeper stack trace?
Any chance you added TIMESTAMP index to this column?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] `Attempt to re-create an existing index for key` exception when adding derived column to startree index [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #15440:
URL: https://github.com/apache/pinot/issues/15440#issuecomment-4676146663
This issue has been closed because it has been inactive for 180 days. If you believe this issue is still relevant, please feel free to reopen it or create a new issue. Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org
Re: [I] `Attempt to re-create an existing index for key` exception when adding derived column to startree index [pinot]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #15440:
URL: https://github.com/apache/pinot/issues/15440#issuecomment-4340081574
This issue has been automatically marked as stale because it has not had any activity in the last 150 days. It will be closed in 30 days if no further activity occurs. If this issue is still relevant, please leave a comment to keep it open. Thank you for your contributions!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org