You are viewing a plain text version of this content. The canonical link for it is here.

Posted to commits@hudi.apache.org by "Raymond Xu (Jira)" <ji...@apache.org> on 2020/11/09 23:33:00 UTC

[jira] [Updated] (HUDI-96) Use Command line options instead of positional arguments when launching spark applications from various CLI commands

     [ https://issues.apache.org/jira/browse/HUDI-96?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Raymond Xu updated HUDI-96:
---------------------------
    Labels: gsoc2021 mentor newbie pull-request-available  (was: newbie pull-request-available)

> Use Command line options instead of positional arguments when launching spark applications from various CLI commands
> --------------------------------------------------------------------------------------------------------------------
>
>                 Key: HUDI-96
>                 URL: https://issues.apache.org/jira/browse/HUDI-96
>             Project: Apache Hudi
>          Issue Type: Improvement
>          Components: CLI, newbie
>            Reporter: Balaji Varadarajan
>            Assignee: Pratyaksh Sharma
>            Priority: Minor
>              Labels: gsoc2021, mentor, newbie, pull-request-available
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> Hoodie CLI commands like compaction/rollback/repair/savepoints/parquet-import relies on launching a spark application to perform their operations (look at SparkMain.java). 
> SparkMain (Look at SparkMain.main()) relies on positional arguments for passing  various CLI options. Instead we should define proper CLI options in SparkMain and use them (using Jcommander)  to improve readability and avoid accidental errors at call sites. For e.g : See com.uber.hoodie.utilities.HoodieCompactor



--
This message was sent by Atlassian Jira
(v8.3.4#803005)