You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Wei-Chiu Chuang (Jira)" <ji...@apache.org> on 2023/07/18 17:53:00 UTC
[jira] [Commented] (HDDS-8969) [Phoenix-Ozone] Failed to identify the fs No FileSystem for scheme "ofs"
[ https://issues.apache.org/jira/browse/HDDS-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17744308#comment-17744308 ]
Wei-Chiu Chuang commented on HDDS-8969:
---------------------------------------
For posterity. In the end we realized it was because Pheonix was not able to find the shaded protobuf classes used by Ozone.
To solve this, we would need to create a shaded uber jar similar to HDDS-6926, but with a different classpath prefix for Pheonix. Specifically,
{code:java}
org.apache.phoenix.shaded.{code}
> [Phoenix-Ozone] Failed to identify the fs No FileSystem for scheme "ofs"
> ------------------------------------------------------------------------
>
> Key: HDDS-8969
> URL: https://issues.apache.org/jira/browse/HDDS-8969
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Filesystem
> Reporter: Arun Sarin
> Priority: Major
>
> Seeing while running the phoenix ozone integration runs
> {code:java}
> org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "ofs" {code}
> Error Message :
> {code:java}
> 2023-06-18 10:41:03,441|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|Connecting to jdbc:phoenix:host-2.host.root.hwx.site:2181:/hbase1
> 2023-06-18 10:41:03,963|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|23/06/18 10:41:03 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-phoenix.properties,hadoop-metrics2.properties
> 2023-06-18 10:41:44,078|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|23/06/18 10:41:44 WARN util.DynamicClassLoader: Failed to identify the fs of dir ofs://ozone1/OzoneTmp/JavaJars, ignored
> 2023-06-18 10:41:44,079|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|org.apache.hadoop.fs.UnsupportedFileSystemException: No FileSystem for scheme "ofs"
> 2023-06-18 10:41:44,079|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3425)
> 2023-06-18 10:41:44,079|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3448)
> 2023-06-18 10:41:44,080|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.fs.FileSystem.access$300(FileSystem.java:161)
> 2023-06-18 10:41:44,080|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3556)
> 2023-06-18 10:41:44,080|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3503)
> 2023-06-18 10:41:44,080|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:521)
> 2023-06-18 10:41:44,080|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.fs.Path.getFileSystem(Path.java:361)
> 2023-06-18 10:41:44,080|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.util.DynamicClassLoader.initTempDir(DynamicClassLoader.java:124)
> 2023-06-18 10:41:44,081|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.util.DynamicClassLoader.<init>(DynamicClassLoader.java:95)
> 2023-06-18 10:41:44,081|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException$ClassLoaderHolder.lambda$static$0(RemoteWithExtrasException.java:54)
> 2023-06-18 10:41:44,081|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at java.security.AccessController.doPrivileged(Native Method)
> 2023-06-18 10:41:44,081|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException$ClassLoaderHolder.<clinit>(RemoteWithExtrasException.java:54)
> 2023-06-18 10:41:44,081|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.ipc.RemoteWithExtrasException.unwrapRemoteException(RemoteWithExtrasException.java:76)
> 2023-06-18 10:41:44,082|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.protobuf.ProtobufUtil.makeIOExceptionOfException(ProtobufUtil.java:276)
> 2023-06-18 10:41:44,082|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.protobuf.ProtobufUtil.handleRemoteException(ProtobufUtil.java:261)
> 2023-06-18 10:41:44,082|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.RegionServerCallable.call(RegionServerCallable.java:126)
> 2023-06-18 10:41:44,082|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104)
> 2023-06-18 10:41:44,082|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.HTable.get(HTable.java:370)
> 2023-06-18 10:41:44,082|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.HTable.get(HTable.java:343)
> 2023-06-18 10:41:44,083|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.MetaTableAccessor.getTableState(MetaTableAccessor.java:1140)
> 2023-06-18 10:41:44,083|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.HBaseAdmin$7.rpcCall(HBaseAdmin.java:496)
> 2023-06-18 10:41:44,083|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.HBaseAdmin$7.rpcCall(HBaseAdmin.java:493)
> 2023-06-18 10:41:44,083|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.RpcRetryingCallable.call(RpcRetryingCallable.java:57)
> 2023-06-18 10:41:44,083|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.RpcRetryingCallerImpl.callWithRetries(RpcRetryingCallerImpl.java:104)
> 2023-06-18 10:41:44,083|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3029)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3021)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.hadoop.hbase.client.HBaseAdmin.tableExists(HBaseAdmin.java:493)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.query.AdminUtilWithFallback.tableExists(AdminUtilWithFallback.java:53)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.query.ConnectionQueryServicesImpl.ensureTableCreated(ConnectionQueryServicesImpl.java:1361)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.query.ConnectionQueryServicesImpl.createTable(ConnectionQueryServicesImpl.java:1944)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.schema.MetaDataClient.createTableInternal(MetaDataClient.java:3147)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.schema.MetaDataClient.createTable(MetaDataClient.java:1111)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.compile.CreateTableCompiler$CreateTableMutationPlan.execute(CreateTableCompiler.java:420)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:445)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:427)
> 2023-06-18 10:41:44,084|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:426)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:414)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.jdbc.PhoenixStatement.executeUpdate(PhoenixStatement.java:1997)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:3319)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:3273)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:3273)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:144)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.DatabaseConnection.connect(DatabaseConnection.java:135)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.DatabaseConnection.getConnection(DatabaseConnection.java:192)
> 2023-06-18 10:41:44,085|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.Commands.connect(Commands.java:1364)
> 2023-06-18 10:41:44,086|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.Commands.connect(Commands.java:1244)
> 2023-06-18 10:41:44,086|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2023-06-18 10:41:44,086|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2023-06-18 10:41:44,086|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2023-06-18 10:41:44,086|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at java.lang.reflect.Method.invoke(Method.java:498)
> 2023-06-18 10:41:44,086|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38)
> 2023-06-18 10:41:44,086|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.SqlLine.dispatch(SqlLine.java:730)
> 2023-06-18 10:41:44,086|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.SqlLine.initArgs(SqlLine.java:410)
> 2023-06-18 10:41:44,086|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.SqlLine.begin(SqlLine.java:515)
> 2023-06-18 10:41:44,087|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.SqlLine.start(SqlLine.java:267)
> 2023-06-18 10:41:44,087|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|at sqlline.SqlLine.main(SqlLine.java:206)
> 2023-06-18 10:49:24,814|INFO|MainThread|machine.py:203 - run()||GUID=76d63bd5-6f8a-4945-8667-bb09756d204c|Error: Failed after attempts=11, exceptions: {code}
> On turning on the TRACE log issue seen as below:
> {code:java}
> DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcProtobufRequest, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@4b544732
> ...
> java.lang.ClassCastException: class org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OMRequest cannot be cast to class org.apache.phoenix.shaded.com.google.protobuf.Message (org.apache.hadoop.ozone.protocol.proto.OzoneManagerProtocolProtos$OMRequest and org.apache.phoenix.shaded.com.google.protobuf.Message are in unnamed module of loader 'app')
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:225)
> at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:118) {code}
> Need to identify if its an classpath issue or some other issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@ozone.apache.org
For additional commands, e-mail: issues-help@ozone.apache.org