You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-issues@hadoop.apache.org by "junwen yang (Jira)" <ji...@apache.org> on 2020/11/30 21:10:00 UTC
[jira] [Updated] (YARN-10508) Protobuf Mesage Incompatibility
Detector
[ https://issues.apache.org/jira/browse/YARN-10508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
junwen yang updated YARN-10508:
-------------------------------
Attachment: yarn_proto_incompatibility.txt
> Protobuf Mesage Incompatibility Detector
> ----------------------------------------
>
> Key: YARN-10508
> URL: https://issues.apache.org/jira/browse/YARN-10508
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: junwen yang
> Priority: Major
> Attachments: yarn_proto_incompatibility.txt
>
>
> Regarding the issue YARN-5632 caused by the incompatibility of protobuf message, we have created a static checker which keeps track of the proto file change, and detect potential incompatibility:
> # Add/delete required field, which is the case reported in HBASE-25238.
> # The tag number of a field has been changed, as described in HDFS-9788. Also, the [protobuf guidelines |https://developers.google.com/protocol-buffers/docs/proto]suggests _each field in the message definition has a *unique number*. These numbers are used to identify your fields in the [message binary format|https://developers.google.com/protocol-buffers/docs/encoding], and should not be changed once your message type is in use_.
> # A required field has been changed to optional, or an optional field has been changed to required. According to the guidelines in [protobuf official website|https://developers.google.com/protocol-buffers/docs/proto], _*Required Is Forever* You should be very careful about marking fields as {{required}}. If at some point you wish to stop writing or sending a required field, it will be problematic to change the field to an optional field - old readers will consider messages without this field to be incomplete and may reject or drop them unintentionally. You should consider writing application-specific custom validation routines for your buffers instead._
> We have applied our checker on the frequently maintained HDFS versions: rel/release-2.6.4, rel/release-2.7.2, rel/release-2.8.0, rel/release-2.9.0, rel/release-2.10.0, rel/release-3.0.0, rel/release-3.1.0, rel/release-3.2.0, rel/release-3.3.0, and we found 28 potential problems as attached.
> The results reported by our checker got confirmation from developers in HBASE, which can be found here HBASE-25340.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org