You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/06/15 23:07:26 UTC
[GitHub] [hudi] tooptoop4 opened a new issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
tooptoop4 opened a new issue #1738:
URL: https://github.com/apache/hudi/issues/1738
```
2020-06-15 20:02:11,059 [main] INFO com.uber.hoodie.common.table.log.HoodieLogFormat$WriterBuilder - Building HoodieLogFormat Writer
2020-06-15 20:02:11,060 [main] INFO com.uber.hoodie.common.table.log.HoodieLogFormat$WriterBuilder - Computing the next log version for commits in s3a://REDACT/.hoodie/archived
Exception in thread "main" com.uber.hoodie.exception.HoodieException: Unable to initialize HoodieLogFormat writer
at com.uber.hoodie.io.HoodieCommitArchiveLog.openWriter(HoodieCommitArchiveLog.java:93)
at com.uber.hoodie.io.HoodieCommitArchiveLog.archiveIfRequired(HoodieCommitArchiveLog.java:115)
at com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:542)
at com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:489)
at com.uber.hoodie.HoodieWriteClient.commit(HoodieWriteClient.java:480)
...
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:890)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:192)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:217)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.io.FileNotFoundException: No such file or directory: s3a://REDACT/.hoodie/archived
at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:1616)
at org.apache.hadoop.fs.s3a.S3AFileSystem.innerListStatus(S3AFileSystem.java:1377)
at org.apache.hadoop.fs.s3a.S3AFileSystem.listStatus(S3AFileSystem.java:1353)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1537)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1580)
at com.uber.hoodie.common.util.FSUtils.getAllLogFiles(FSUtils.java:302)
at com.uber.hoodie.common.util.FSUtils.getLatestLogVersion(FSUtils.java:316)
at com.uber.hoodie.common.util.FSUtils.getCurrentLogVersion(FSUtils.java:327)
at com.uber.hoodie.common.table.log.HoodieLogFormat$WriterBuilder.build(HoodieLogFormat.java:184)
at com.uber.hoodie.io.HoodieCommitArchiveLog.openWriter(HoodieCommitArchiveLog.java:88)
... 27 more
2020-06-15 20:02:11,098 [pool-1-thread-1] INFO org.apache.spark.SparkContext - Invoking stop() from shutdown hook
current s3 listing:
bash-4.2$ ~/bin/aws s3 ls s3://REDACT/ --recursive
2020-06-01 17:15:15 0 PREREDACT/.hoodie/.aux/
2020-06-01 17:20:13 0 PREREDACT/.hoodie/.temp/
2020-06-01 16:35:18 1091 PREREDACT/.hoodie/20200529234618.clean
2020-06-01 16:35:18 2022 PREREDACT/.hoodie/20200529234618.commit
2020-06-01 16:35:18 1091 PREREDACT/.hoodie/20200529234910.clean
2020-06-01 16:35:18 6150 PREREDACT/.hoodie/20200529234910.commit
2020-06-01 16:35:18 1091 PREREDACT/.hoodie/20200529235222.clean
2020-06-01 16:35:18 3846 PREREDACT/.hoodie/20200529235222.commit
2020-06-01 16:35:18 1091 PREREDACT/.hoodie/20200529235443.clean
2020-06-01 16:35:18 3851 PREREDACT/.hoodie/20200529235443.commit
2020-06-01 16:35:18 1091 PREREDACT/.hoodie/20200529235719.clean
2020-06-01 16:35:18 3850 PREREDACT/.hoodie/20200529235719.commit
2020-06-01 16:35:18 1091 PREREDACT/.hoodie/20200529235933.clean
2020-06-01 16:35:18 3853 PREREDACT/.hoodie/20200529235933.commit
2020-06-01 16:35:18 1091 PREREDACT/.hoodie/20200530000232.clean
2020-06-01 16:35:18 3850 PREREDACT/.hoodie/20200530000232.commit
2020-06-01 16:35:19 1091 PREREDACT/.hoodie/20200530000535.clean
2020-06-01 16:35:19 3854 PREREDACT/.hoodie/20200530000535.commit
2020-06-01 16:35:19 1091 PREREDACT/.hoodie/20200530000831.clean
2020-06-01 16:35:19 3851 PREREDACT/.hoodie/20200530000831.commit
2020-06-01 16:35:19 1091 PREREDACT/.hoodie/20200530001136.clean
2020-06-01 16:35:19 3854 PREREDACT/.hoodie/20200530001136.commit
2020-06-01 16:35:19 1104 PREREDACT/.hoodie/20200530001432.clean
2020-06-01 16:35:19 3851 PREREDACT/.hoodie/20200530001432.commit
2020-06-01 16:35:19 1710 PREREDACT/.hoodie/20200530001729.clean
2020-06-01 16:35:19 3856 PREREDACT/.hoodie/20200530001729.commit
2020-06-04 14:22:24 1104 PREREDACT/.hoodie/20200604132158.clean
2020-06-04 14:22:21 3854 PREREDACT/.hoodie/20200604132158.commit
2020-06-04 14:23:47 1104 PREREDACT/.hoodie/20200604132316.clean
2020-06-04 14:23:43 6205 PREREDACT/.hoodie/20200604132316.commit
2020-06-04 15:31:36 1104 PREREDACT/.hoodie/20200604143105.clean
2020-06-04 15:31:33 3877 PREREDACT/.hoodie/20200604143105.commit
2020-06-04 15:33:02 1104 PREREDACT/.hoodie/20200604143235.clean
2020-06-04 15:32:57 3875 PREREDACT/.hoodie/20200604143235.commit
2020-06-04 15:34:31 1104 PREREDACT/.hoodie/20200604143400.clean
2020-06-04 15:34:27 3875 PREREDACT/.hoodie/20200604143400.commit
2020-06-07 03:05:37 1104 PREREDACT/.hoodie/20200607020507.clean
2020-06-07 03:05:34 3879 PREREDACT/.hoodie/20200607020507.commit
2020-06-07 03:07:17 1104 PREREDACT/.hoodie/20200607020642.clean
2020-06-07 03:07:14 3877 PREREDACT/.hoodie/20200607020642.commit
2020-06-07 03:08:47 1104 PREREDACT/.hoodie/20200607020820.clean
2020-06-07 03:08:44 3877 PREREDACT/.hoodie/20200607020820.commit
2020-06-07 03:12:13 1104 PREREDACT/.hoodie/20200607021138.clean
2020-06-07 03:12:11 6203 PREREDACT/.hoodie/20200607021138.commit
2020-06-07 03:13:36 1104 PREREDACT/.hoodie/20200607021313.clean
2020-06-07 03:13:33 3874 PREREDACT/.hoodie/20200607021313.commit
2020-06-07 03:15:10 1740 PREREDACT/.hoodie/20200607021437.clean
2020-06-07 03:15:07 3879 PREREDACT/.hoodie/20200607021437.commit
2020-06-07 03:16:39 3639 PREREDACT/.hoodie/20200607021611.clean
2020-06-07 03:16:36 3877 PREREDACT/.hoodie/20200607021611.commit
2020-06-07 03:18:13 1740 PREREDACT/.hoodie/20200607021742.clean
2020-06-07 03:18:10 3879 PREREDACT/.hoodie/20200607021742.commit
2020-06-07 03:19:40 1740 PREREDACT/.hoodie/20200607021914.clean
2020-06-07 03:19:38 3876 PREREDACT/.hoodie/20200607021914.commit
2020-06-07 03:21:13 1740 PREREDACT/.hoodie/20200607022043.clean
2020-06-07 03:21:11 3879 PREREDACT/.hoodie/20200607022043.commit
2020-06-07 03:22:52 1740 PREREDACT/.hoodie/20200607022224.clean
2020-06-07 03:22:50 3879 PREREDACT/.hoodie/20200607022224.commit
2020-06-07 03:24:18 1740 PREREDACT/.hoodie/20200607022350.clean
2020-06-07 03:24:15 3876 PREREDACT/.hoodie/20200607022350.commit
2020-06-09 17:04:10 1740 PREREDACT/.hoodie/20200609160339.clean
2020-06-09 17:04:07 6199 PREREDACT/.hoodie/20200609160339.commit
2020-06-09 17:05:35 3874 PREREDACT/.hoodie/20200609160515.commit
2020-06-09 17:07:17 3879 PREREDACT/.hoodie/20200609160644.commit
2020-06-09 17:08:45 3877 PREREDACT/.hoodie/20200609160822.commit
2020-06-09 17:10:18 3881 PREREDACT/.hoodie/20200609160950.commit
2020-06-09 17:11:49 3878 PREREDACT/.hoodie/20200609161125.commit
2020-06-09 17:13:24 3881 PREREDACT/.hoodie/20200609161254.commit
2020-06-09 17:15:12 3881 PREREDACT/.hoodie/20200609161444.commit
2020-06-09 17:16:42 3878 PREREDACT/.hoodie/20200609161618.commit
2020-06-11 10:15:10 6201 PREREDACT/.hoodie/20200611091443.commit
2020-06-11 10:16:51 3876 PREREDACT/.hoodie/20200611091630.commit
2020-06-11 10:18:26 3881 PREREDACT/.hoodie/20200611091758.commit
2020-06-11 10:20:06 3879 PREREDACT/.hoodie/20200611091936.commit
2020-06-11 10:21:43 3881 PREREDACT/.hoodie/20200611092117.commit
2020-06-11 10:23:38 3879 PREREDACT/.hoodie/20200611092249.commit
2020-06-11 10:25:37 3881 PREREDACT/.hoodie/20200611092452.commit
2020-06-11 10:29:37 3881 PREREDACT/.hoodie/20200611092814.commit
2020-06-11 10:31:25 3878 PREREDACT/.hoodie/20200611093049.commit
2020-06-15 21:02:11 3878 PREREDACT/.hoodie/20200615200144.commit
2020-06-15 21:03:42 3881 PREREDACT/.hoodie/20200615200312.commit
2020-06-15 21:05:22 3881 PREREDACT/.hoodie/20200615200453.commit
2020-06-15 21:06:45 3878 PREREDACT/.hoodie/20200615200620.commit
2020-06-15 21:08:14 3881 PREREDACT/.hoodie/20200615200745.commit
2020-06-15 21:09:39 3879 PREREDACT/.hoodie/20200615200914.commit
2020-06-01 17:24:32 194 PREREDACT/.hoodie/hoodie.properties
2020-06-01 16:35:19 93 PREREDACT/unknown/.hoodie_partition_metadata
2020-06-07 03:13:23 687069 PREREDACT/unknown/1ed20eac-618f-4b2b-ab09-9a8f620d52f7_0_20200607021313.parquet
2020-06-09 17:05:26 687126 PREREDACT/unknown/1ed20eac-618f-4b2b-ab09-9a8f620d52f7_0_20200609160515.parquet
2020-06-11 10:16:40 687164 PREREDACT/unknown/1ed20eac-618f-4b2b-ab09-9a8f620d52f7_0_20200611091630.parquet
2020-06-04 14:23:32 686896 PREREDACT/unknown/1ed20eac-618f-4b2b-ab09-9a8f620d52f7_2_20200604132316.parquet
2020-06-07 03:11:56 687025 PREREDACT/unknown/1ed20eac-618f-4b2b-ab09-9a8f620d52f7_2_20200607021138.parquet
2020-06-09 17:03:52 687125 PREREDACT/unknown/1ed20eac-618f-4b2b-ab09-9a8f620d52f7_2_20200609160339.parquet
2020-06-11 10:14:58 687163 PREREDACT/unknown/1ed20eac-618f-4b2b-ab09-9a8f620d52f7_2_20200611091443.parquet
2020-06-04 14:23:30 661660 PREREDACT/unknown/3a7e1b07-4d43-4cef-bad4-fb0b1945452b_0_20200604132316.parquet
2020-06-07 03:11:56 661763 PREREDACT/unknown/3a7e1b07-4d43-4cef-bad4-fb0b1945452b_0_20200607021138.parquet
2020-06-09 17:03:53 661807 PREREDACT/unknown/3a7e1b07-4d43-4cef-bad4-fb0b1945452b_1_20200609160339.parquet
2020-06-11 10:14:54 661856 PREREDACT/unknown/3a7e1b07-4d43-4cef-bad4-fb0b1945452b_1_20200611091443.parquet
2020-06-07 03:08:36 8481028 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200607020820.parquet
2020-06-07 03:14:56 8730524 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200607021437.parquet
2020-06-07 03:16:24 8731305 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200607021611.parquet
2020-06-07 03:18:01 8926445 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200607021742.parquet
2020-06-07 03:19:26 8926645 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200607021914.parquet
2020-06-07 03:21:02 9137267 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200607022043.parquet
2020-06-07 03:22:43 9179806 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200607022224.parquet
2020-06-07 03:24:02 9179883 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200607022350.parquet
2020-06-09 17:07:08 9180937 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200609160644.parquet
2020-06-09 17:08:37 9180889 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200609160822.parquet
2020-06-09 17:10:10 9182202 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200609160950.parquet
2020-06-09 17:11:37 9182209 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200609161125.parquet
2020-06-09 17:13:16 9183427 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200609161254.parquet
2020-06-09 17:15:03 9183651 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200609161444.parquet
2020-06-09 17:16:30 9183647 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200609161618.parquet
2020-06-11 10:18:19 9184774 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200611091758.parquet
2020-06-11 10:19:55 9184729 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200611091936.parquet
2020-06-11 10:21:35 9185833 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200611092117.parquet
2020-06-11 10:23:16 9185826 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200611092249.parquet
2020-06-11 10:25:19 9186894 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200611092452.parquet
2020-06-11 10:29:20 9187050 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200611092814.parquet
2020-06-11 10:31:03 9187045 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200611093049.parquet
2020-06-15 21:01:56 9187054 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200615200144.parquet
2020-06-15 21:03:34 9187292 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200615200312.parquet
2020-06-15 21:05:15 9189647 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200615200453.parquet
2020-06-15 21:06:32 9189649 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200615200620.parquet
2020-06-15 21:08:05 9191392 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200615200745.parquet
2020-06-15 21:09:30 9191397 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_0_20200615200914.parquet
2020-06-07 03:11:59 8481102 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_3_20200607021138.parquet
2020-06-09 17:03:58 9179924 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_3_20200609160339.parquet
2020-06-11 10:14:59 9183710 PREREDACT/unknown/7ed927a4-15bf-4e9f-a8a5-04f685a9004b_3_20200611091443.parquet
2020-06-09 17:03:52 657453 PREREDACT/unknown/ba3684e1-3ce8-4b5f-a077-2b8ba8e62787_0_20200609160339.parquet
2020-06-11 10:14:58 657515 PREREDACT/unknown/ba3684e1-3ce8-4b5f-a077-2b8ba8e62787_0_20200611091443.parquet
2020-06-04 14:23:30 657285 PREREDACT/unknown/ba3684e1-3ce8-4b5f-a077-2b8ba8e62787_1_20200604132316.parquet
2020-06-07 03:11:55 657385 PREREDACT/unknown/ba3684e1-3ce8-4b5f-a077-2b8ba8e62787_1_20200607021138.parquet
2020-06-01 13:05:46 0 PREREDACT/unknown/x
5 times retrying the job but always hit the same error.
Using spark 2.3.4 and hoodie 0.4.6, yes i know old but code path looks to be same:
https://github.com/apache/hudi/blob/hoodie-0.4.6/hoodie-client/src/main/java/com/uber/hoodie/io/HoodieCommitArchiveLog.java
vs https://github.com/apache/hudi/blob/master/hudi-client/src/main/java/org/apache/hudi/table/HoodieTimelineArchiveLog.java
https://github.com/apache/hudi/blob/hoodie-0.4.6/hoodie-common/src/main/java/com/uber/hoodie/common/util/FSUtils.java
vs https://github.com/apache/hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/fs/FSUtils.java
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar commented on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-657961520
@tooptoop4 : Also, As this is not being reported by others commonly , I am wondering if you can check whether the patch you have on top of hudi might be causing it ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar closed issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
bvaradar closed issue #1738:
URL: https://github.com/apache/hudi/issues/1738
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-653263255
@tooptoop4 can you please update us on the latest here ? are you able to consistently reproduce this
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar commented on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-660423950
Closing this due to inactivity.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] tooptoop4 edited a comment on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
tooptoop4 edited a comment on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-644440893
Looks like the last stage to run was consistency check
So seems below 3 stages never got chance to run (below 2 screens are example from a successful spark-submit on a different table):
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] tooptoop4 commented on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
tooptoop4 commented on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-644440893
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] tooptoop4 commented on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
tooptoop4 commented on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-653270736
can consistently reproduce this issue. maybe the code should create the archived folder if it does not exist, rather than erroring out ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-655184220
Based on slack discussion.. this is also resolved now?
if not, @tooptoop4 do you have a small snippet I can use to repro this locally? we can make the code create the folder if it does not exist, and retry may be.. Like to understand why this happens though..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] tooptoop4 commented on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
tooptoop4 commented on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-644790174
don't have anything set up to touch the folders, just plain spark-submits to do upsert/delete
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar commented on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-645542918
We have not seen this issue before and hence suspecting some external process is deleting it.
Is it possible an empty directory is automatically getting removed automatically. I am wondering if this is the first time archiving is kicking in and finding that an empty .archived folder is deleted.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on issue #1738: [SUPPORT] java.io.FileNotFoundException: No such file or directory: .hoodie/archived
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on issue #1738:
URL: https://github.com/apache/hudi/issues/1738#issuecomment-644741098
did some process delete that folder out of hudi by any chance? it gets created during table init i.e whenever `.hoodie` etc is first created..
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org