[GitHub] [hudi] RajasekarSribalan edited a comment on issue #1823: [SUPPORT] MOR trigger compaction from Hudi CLI

[GitBox <gi...@apache.org>]

RajasekarSribalan edited a comment on issue #1823:
URL: https://github.com/apache/hudi/issues/1823#issuecomment-657337822


   Thank you for your response Bhavani. @bhasudha 
   
   1.May I know the purpose of compaction schedule and compaction run command
   from Hudi CLI?
   
   2. If inline compaction is only possible from spark streaming for MOR
   tables then it is similar to CopyOnWrite table? There is no difference in
   using them? Pls correct me if I am wrong.
   
   Kind regards,
   Rajasekar
   
   On Mon, 13 Jul 2020, 5:43 am Bhavani Sudha Saktheeswaran, <
   notifications@github.com> wrote:
   
   > @RajasekarSribalan <https://github.com/RajasekarSribalan> For your first
   > question, unfortunately currently in Spark Streaming writes only support
   > inline compaction is supported. So you have to enable that config. Good
   > news is, this PR <https://github.com/apache/hudi/pull/1752> is working on
   > enabling the async compaction for Spark Streaming and is in priority.
   >
   > For second question, couple clarifications.
   >
   >    - Hudi moved to Spark 2.4. I see that you are using spark 2.2.0. Could
   >    you try on spark 2.4* ?
   >    - Also, in your spark submit command are you passing in these jars and
   >    cones - https://hudi.apache.org/docs/quick-start-guide.html#setup
   >
   >
   >    1. The conf
   >    'spark.serializer=org.apache.spark.serializer.KryoSerializer'
   >    2. in addition to hudi-spark-bundle, you need to pass
   >    ,org.apache.spark:spark-avro_2.11:2.4.4 Note the spark-avro must match
   >    your spark version which is 2.4.4. This applies if you are using
   >    spark-shell as it does not ship with spark-avro explicitly.
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > <https://github.com/apache/hudi/issues/1823#issuecomment-657297717>, or
   > unsubscribe
   > <https://github.com/notifications/unsubscribe-auth/AFMO6I26RADG2PR3ZGDG7GDR3JGRVANCNFSM4OXMTUNQ>
   > .
   >
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org