You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Radhika Kundam (Jira)" <ji...@apache.org> on 2023/03/27 21:54:00 UTC
[jira] [Updated] (ATLAS-4738) Dynamic Index Recovery issues and improvements
[ https://issues.apache.org/jira/browse/ATLAS-4738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Radhika Kundam updated ATLAS-4738:
----------------------------------
Description:
# Though there is no issues with SOLR, on Atlas startup Index recovery is getting started as it's considering 1970-01-01T00:00:00Z as default recovery start time. When no index recovery data is available by default it'll consider recovery start time based on TTL i.e., (current time - tx.log TTL).
By default SOLR tx.log default ttl is configured as 10days.
2. Custom start time configuration atlas.graph.index.recovery.start.time is not working as it conflicts with janusgraph internal configurations. This config should be updated to *atlas.index.recovery.start.time*
3. When SOLR went down with existing architecture we note down start time with a buffer of solr health checkup frequency which will be (current time - retrytime of solr health). To avoid any possible corner case scenarios updating the buffer two times of retry time so next recovery start time should be (current time - 2 * retry time)
4. Added REST support for index recovery which can give more flexibility to the user to start index recovery on-demand and will be valuable feature.
a. Get index recovery timing details
API: GET api/atlas/v2/indexrecovery/
Response:
\{ "Latest start time": "2023-03-25T20:34:59.704Z", "On-demand start time": "2023-03-24T06:17:23.656Z", "Previous start time": "2023-03-25T20:24:51.583Z" }
b. Start index recovery at a specific time
API: POST api/atlas/v2/indexrecovery/start?startTime=2023-03-24T06:17:23.656Z
Response:
On success => 200 OK
On empty start Time => 400 Bad request
On Index recovery failure => 500 Internal Server Error
was:
1. Though there is no issues with SOLR, on Atlas startup Index recovery is getting started as it's considering 1970-01-01T00:00:00Z as default recovery start time. When no index recovery data is available by default it'll consider recovery start time based on TTL i.e., (current time - tx.log TTL).
By default SOLR tx.log default ttl is configured as 10days.
2. Custom start time configuration atlas.graph.index.recovery.start.time is not working as it conflicts with janusgraph internal configurations. This config should be updated to *atlas.index.recovery.start.time*
3. When SOLR went down with existing architecture we note down start time with a buffer of solr health checkup frequency which will be (current time - retrytime of solr health). To avoid any possible corner case scenarios updating the buffer two times of retry time so next recovery start time should be (current time - 2 * retry time)
4. Added REST support for index recovery which can give more flexibility to the user to start index recovery on-demand and will be valuable feature.
a. Get index recovery timing details
API: GET api/atlas/v2/indexrecovery/
Response:
{
"Latest start time": "2023-03-25T20:34:59.704Z",
"On-demand start time": "2023-03-24T06:17:23.656Z",
"Previous start time": "2023-03-25T20:24:51.583Z"
}
b. Start index recovery at a specific time
API: POST api/atlas/v2/indexrecovery/start?startTime=2023-03-24T06:17:23.656Z
Response:
On success => 200 OK
On empty start Time => 400 Bad request
On Index recovery failure => 500 Internal Server Error
> Dynamic Index Recovery issues and improvements
> ----------------------------------------------
>
> Key: ATLAS-4738
> URL: https://issues.apache.org/jira/browse/ATLAS-4738
> Project: Atlas
> Issue Type: Improvement
> Components: atlas-core
> Reporter: Radhika Kundam
> Assignee: Radhika Kundam
> Priority: Major
>
> # Though there is no issues with SOLR, on Atlas startup Index recovery is getting started as it's considering 1970-01-01T00:00:00Z as default recovery start time. When no index recovery data is available by default it'll consider recovery start time based on TTL i.e., (current time - tx.log TTL).
> By default SOLR tx.log default ttl is configured as 10days.
> 2. Custom start time configuration atlas.graph.index.recovery.start.time is not working as it conflicts with janusgraph internal configurations. This config should be updated to *atlas.index.recovery.start.time*
> 3. When SOLR went down with existing architecture we note down start time with a buffer of solr health checkup frequency which will be (current time - retrytime of solr health). To avoid any possible corner case scenarios updating the buffer two times of retry time so next recovery start time should be (current time - 2 * retry time)
> 4. Added REST support for index recovery which can give more flexibility to the user to start index recovery on-demand and will be valuable feature.
> a. Get index recovery timing details
> API: GET api/atlas/v2/indexrecovery/
> Response:
> \{ "Latest start time": "2023-03-25T20:34:59.704Z", "On-demand start time": "2023-03-24T06:17:23.656Z", "Previous start time": "2023-03-25T20:24:51.583Z" }
> b. Start index recovery at a specific time
> API: POST api/atlas/v2/indexrecovery/start?startTime=2023-03-24T06:17:23.656Z
> Response:
> On success => 200 OK
> On empty start Time => 400 Bad request
> On Index recovery failure => 500 Internal Server Error
--
This message was sent by Atlassian Jira
(v8.20.10#820010)