Re: Kylin 4.2 - cleanup not working

[ShaoFeng Shi <sh...@apache.org>]

Hello,

It seems that message should be a warning, instead of error:
https://github.com/apache/kylin/blob/main/server-base/src/main/java/org/apache/kylin/rest/job/StorageCleanupJob.java#L230

For each cube, it checks whether the path
"<project_name>/parquet/<cube_name>" exists or not. If exists, it will
further check its subfolders (corresponding to each segment). If not, it
just gives a warning.

Hope it helps.

Best regards,

Shaofeng Shi 史少锋
Apache Kylin PMC,
Apache Incubator PMC,
Email: shaofengshi@apache.org

Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html
Join Kylin user mail group: user-subscribe@kylin.apache.org
Join Kylin dev mail group: dev-subscribe@kylin.apache.org




Singh Sonu <so...@gmail.com> 于2023年5月15日周一 14:14写道：

> Hi Experts,
>
> Any help or suggestions will be appreciated.
>
> I am facing an issue while cleaning up unused parquet files in HDFS.
> command: bin/kylin.sh org.apache.kylin.tool.StorageCleanupJob --delete true
>
> Error: job.StorageCleanup Job:222: Cube path doesn't exist! The path is
> file:/apps/kylin/kylin_metadata/project_a/parquet/cube1
>
> During full cube build or incremental, kylin is not removing unwanted or
> unused segments from parquet folder under hdfs.
>
>
>
>  You can reach me out at
>  Email- sonusingh.javatech@gmail.com
>
>  with regards,
>  Sonu Kumar Singh
>