You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/04/04 10:06:00 UTC

[jira] [Work logged] (HIVE-27068) Use Spark Accumulator V2 instead of the deprecated Accumulator V1

     [ https://issues.apache.org/jira/browse/HIVE-27068?focusedWorklogId=854742&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-854742 ]

ASF GitHub Bot logged work on HIVE-27068:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 04/Apr/23 10:05
            Start Date: 04/Apr/23 10:05
    Worklog Time Spent: 10m 
      Work Description: TuroczyX commented on PR #4052:
URL: https://github.com/apache/hive/pull/4052#issuecomment-1495693883

   Hi @nineinfra 
   I'm not familiar with Accumulator.  (I guess it is a serialization mechanism or belongs to that topic) 
   
   I have two question with this fix:
   1. If we allow to upgrade to AC2 that old version of spark (like spark 2) can use it or it will be incompatible? What would see the user if they start to use?
   2. If we do not upgrade to AC2 would be a problem with Spark3? 
   It would be nice if it would be clear that what could be the impact of this change
   
   cc: @ayushtkn @veghlaci05 
   




Issue Time Tracking
-------------------

    Worklog Id:     (was: 854742)
    Time Spent: 20m  (was: 10m)

> Use Spark Accumulator V2 instead of the deprecated Accumulator V1
> -----------------------------------------------------------------
>
>                 Key: HIVE-27068
>                 URL: https://issues.apache.org/jira/browse/HIVE-27068
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>    Affects Versions: 3.0.0
>            Reporter: Wei Yan
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.1.4
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> The  Accumulator V1 has been deprecated since the Spark version 2.0.0
> But in hive 3 series and below versions，the Accumulator V1 is still used.
> There will be compatibility problems when the new version of spark is used with hive,
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)