You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Arpit Agarwal (Jira)" <ji...@apache.org> on 2023/08/03 17:11:00 UTC
[jira] [Resolved] (HDDS-9113) [snapshot] Snapshot path contents listing is failing intermittently
[ https://issues.apache.org/jira/browse/HDDS-9113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arpit Agarwal resolved HDDS-9113.
---------------------------------
Resolution: Duplicate
> [snapshot] Snapshot path contents listing is failing intermittently
> -------------------------------------------------------------------
>
> Key: HDDS-9113
> URL: https://issues.apache.org/jira/browse/HDDS-9113
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Manager
> Reporter: Arun Sarin
> Priority: Major
>
> Snapshot path contents listing is failing intermittently
> {code:java}
> 2023-08-01 19:36:50,541|INFO|MainThread|machine.py:188 - run()||GUID=e574a938-a16e-4a1e-9c42-5786f286e727|RUNNING: /opt/cloudera/parcels/CDH/bin/ozone sh snapshot create o3://ozone1/vol-fiisp/buck-6j174 snap-2yg7s
> 2023-08-01 19:36:54,676|INFO|MainThread|machine.py:230 - run()||GUID=e574a938-a16e-4a1e-9c42-5786f286e727|Exit Code: 0
> 2023-08-01 19:36:54,679|INFO|MainThread|machine.py:188 - run()||GUID=7dd9a1cf-a3ca-4260-9eb2-13cdafeb2519|RUNNING: klist -k -t /home/hrt_qa/hadoopqa/keytabs/hrt_qa.headless.keytab | grep -v HTTP
> 2023-08-01 19:36:54,698|INFO|MainThread|machine.py:230 - run()||GUID=7dd9a1cf-a3ca-4260-9eb2-13cdafeb2519|Exit Code: 0
> 2023-08-01 19:36:54,699|INFO|MainThread|machine.py:2131 - get_principal_from_user()|--- user principal is hrt_qa@ROOT.HWX.SITE
> 2023-08-01 19:36:54,700|INFO|MainThread|machine.py:188 - run()||GUID=d0dd3259-5244-416c-afc0-adf6185698ad|RUNNING: /opt/cloudera/parcels/CDH/bin/ozone fs -ls -R ofs://ozone1/vol-fiisp/buck-6j174/.snapshot/snap-2yg7s
> 2023-08-01 19:37:02,974|INFO|MainThread|machine.py:203 - run()||GUID=d0dd3259-5244-416c-afc0-adf6185698ad|ls: `ofs://ozone1/vol-fiisp/buck-6j174/.snapshot/snap-2yg7s': No such file or directory
> 2023-08-01 19:37:03,023|INFO|MainThread|machine.py:232 - run()||GUID=d0dd3259-5244-416c-afc0-adf6185698ad|Exit Code: 1
> 2023-08-01 19:37:03,023|INFO|MainThread|machine.py:238 - run()|Command /opt/cloudera/parcels/CDH/bin/ozone fs -ls -R ofs://ozone1/vol-fiisp/buck-6j174/.snapshot/snap-2yg7s failed after 0 retries
> 2023-08-01 19:37:03,037|INFO|MainThread|conftest.py:272 - pytest_runtest_makereport()|call: <CallInfo when='call' exception: Snapshot path contents list failed
> assert 1 == 0>{code}
> Snapshot creation was successful [validated in om-audit.log]
> {noformat}
> 2023-08-01 19:36:54,627 | INFO | OMAudit | user=hdfs@ROOT.HWX.SITE | ip=10.64.62.46 | op=CREATE_SNAPSHOT {volume=vol-fiisp, bucket=buck-6j174, snapshotName=snap-2yg7s} | ret=SUCCESS | {noformat}
> The volume and bucket has space and namespace quota set -
> {code:java}
> Volume quota for current test SpaceQuota: 8053063680,NameSpace quota: 19
> Bucket quota for current test SpaceQuota: 8053063680,NameSpace quota: 19 {code}
> ozone-om.log error stack-trace :
> {code:java}
> 2023-08-01 19:36:50,236 INFO [SstFilteringService#0]-org.apache.hadoop.ozone.om.snapshot.SnapshotCache: Loading snapshot. Table key: /vol-dsjsj/buck-h5ed5/snap-4w2ww
> 2023-08-01 19:36:50,237 ERROR [SstFilteringService#0]-org.apache.hadoop.ozone.om.SstFilteringService: Error during Snapshot sst filtering
> FILE_NOT_FOUND org.apache.hadoop.ozone.om.exceptions.OMException: Unable to load snapshot. Snapshot with table key '/vol-dsjsj/buck-h5ed5/snap-4w2ww' is no longer active
> at org.apache.hadoop.ozone.om.snapshot.SnapshotCache.get(SnapshotCache.java:205)
> at org.apache.hadoop.ozone.om.snapshot.SnapshotCache.get(SnapshotCache.java:151)
> at org.apache.hadoop.ozone.om.SstFilteringService$SstFilteringTask.call(SstFilteringService.java:178)
> at org.apache.hadoop.hdds.utils.BackgroundService$PeriodicalTask.lambda$run$0(BackgroundService.java:121)
> at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
> at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
> at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at java.base/java.lang.Thread.run(Thread.java:834) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org