[jira] [Created] (IMPALA-10370) Thrift Message Incompatibility Detector

["junwen yang (Jira)" <ji...@apache.org>]

junwen yang created IMPALA-10370:
------------------------------------

             Summary: Thrift Message Incompatibility Detector
                 Key: IMPALA-10370
                 URL: https://issues.apache.org/jira/browse/IMPALA-10370
             Project: IMPALA
          Issue Type: Improvement
            Reporter: junwen yang
         Attachments: impala_thrift_incompatibility.txt

Regarding the issue  [IMPALA-8243|https://issues.apache.org/jira/browse/IMPALA-8234] caused by the incompatibility of thrift message, we have created a static checker which keeps track of the thrift file change, and detect potential incompatibility:
 # Add/delete required field.  The thrift guidelines suggests _Any new fields that you add should be optional_.
 # The tag number of a field has been changed. Also, the thrift guidelines suggests _Don’t change the numeric tags for any existing fields_.
 # A  required field has been changed to optional, or an optional field has been changed to required. According to the guidelines , _*Required Is Forever* You should be very careful about marking fields as required. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field — old readers will consider messages without this field to be incomplete and may reject or drop them unintentionally. You should consider writing application-specific custom validation routines for your buffers instead. Some have come to the conclusion that using required does more harm than good; they prefer to use only optional. However, this view is not universal._

We have applied our checker on the frequently maintained IMPALA versions: refs/tags/2.10.0, refs/tags/2.11.0, refs/tags/2.12.0, refs/tags/2.7.0, refs/tags/2.8.0, refs/tags/2.9.0, refs/tags/3.0.0, refs/tags/3.0.1, refs/tags/3.1.0, refs/tags/3.2.0, refs/tags/3.3.0, refs/tags/3.4.0, we found more than 1000 problems as attached. 

The results reported by our checker got confirmation from developers in HBASE, which can be found here HBASE-25340. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)