You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Pavel Stepanov (Jira)" <ji...@apache.org> on 2025/01/21 14:06:00 UTC
[jira] [Created] (HDDS-12116) Customizable Protobuf shaded prefix in ozonefs-hadoop3-client
Pavel Stepanov created HDDS-12116:
-------------------------------------
Summary: Customizable Protobuf shaded prefix in ozonefs-hadoop3-client
Key: HDDS-12116
URL: https://issues.apache.org/jira/browse/HDDS-12116
Project: Apache Ozone
Issue Type: Improvement
Components: Ozone Client
Affects Versions: 2.0.0
Reporter: Pavel Stepanov
Assignee: Pavel Stepanov
I tried running Trino together with Ozone over OFS but encountered a Protobuf shading incompatibility between the [Trino Hadoop library|https://github.com/trinodb/trino-hadoop-apache/tree/master] and the Ozone Hadoop client:
{code:java}
2024-12-27T09:09:30.178Z DEBUG hive-hive-0 org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem Ozone URI for OFS initialization is ofs://ozone-om-1
2024-12-27T09:09:30.183Z WARN hive-hive-0 org.apache.hadoop.ozone.om.helpers.OzoneFSUtils Ignoring ozone.fs.hsync.enabled = true because HBase enhancements are disallowed. To enable it, set ozone.client.hbase.enhancements.allowed = true as well.
2024-12-27T09:09:30.184Z DEBUG hive-hive-0 org.apache.hadoop.fs.ozone.BasicRootedOzoneFileSystem hsyncEnabled = false
2024-12-27T09:09:30.473Z INFO dispatcher-query-3 io.trino.event.QueryMonitor TIMELINE: Query 20241227_090929_00003_x4ukx :: FINISHED :: elapsed 480ms :: planning 134ms :: waiting 23ms :: scheduling 202ms :: running 128ms :: finishing 16ms :: begin 2024-12-27T09:09:29.959Z :: end
2024-12-27T09:09:30.641Z INFO hive-hive-0 org.apache.hadoop.hdds.utils.LeakDetector Starting leak detector thread OzoneClientObject0.
2024-12-27T09:09:30.816Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.protocolPB.OmTransportFactory Loading OM transport implementation org.apache.hadoop.ozone.om.protocolPB.Hadoop3OmTransportFactory as specified by configuration.
2024-12-27T09:09:31.241Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.ha.OMFailoverProxyProviderBase RetryProxy: OM om1: class org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OMRequest cannot be cast to class io.trino.hadoop.$internal.com.google.protobuf.Message (org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OMRequest and io.trino.hadoop.$internal.com.google.protobuf.Message are in unnamed module of loader io.trino.filesystem.manager.HdfsClassLoader @43fffec2)
2024-12-27T09:09:31.243Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.ha.OMFailoverProxyProviderBase Incrementing OM proxy index to 0, nodeId: om1
2024-12-27T09:09:33.246Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.ha.OMFailoverProxyProviderBase Failing over OM from om1:0 to om1:0
2024-12-27T09:09:33.246Z DEBUG hive-hive-0 org.apache.hadoop.ozone.om.ha.OMFailoverProxyProviderBase RetryProxy: OM om1: null{code}
The simplest way to resolve this issue is to adjust the ozonefs-hadoop3-client module, which already shades Protobuf, but currently uses a fixed prefix. I suggest making the prefix configurable as a separate property to allow customization if needed.
With the proposed changes, running
{code:java}
mvn clean package -Dproto.shaded.prefix='io.trino.hadoop.\$internal' {code}
inside the ozonefs-hadoop3-client module will properly shade Protobuf and allow the output JAR to be used with Trino.
Related trino issue: https://github.com/trinodb/trino/discussions/18026
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org