You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2020/08/19 02:09:49 UTC
[GitHub] [hudi] Ac-Rush opened a new issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.
Ac-Rush opened a new issue #1982:
URL: https://github.com/apache/hudi/issues/1982
I got below error when saving data to ADLS Gen2 with hudi format, But I can save it to Delta Lake with the same credential.
abfss://xxxxx.dfs.core.windows.net/hudi-tables/xxx has invalid authority.
at shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.authorityParts(AzureBlobFileSystemStore.java:161)
at shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystemStore.<init>(AzureBlobFileSystemStore.java:137)
at shaded.databricks.v20180920_b33d810.org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem.initialize(AzureBlobFileSystem.java:106)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2669)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
at com.databricks.sql.transaction.tahoe.DeltaTableUtils$.findDeltaTableRoot(DeltaTable.scala:166)
at com.databricks.sql.transaction.tahoe.DeltaValidation$.validateNonDeltaWrite(DeltaValidation.scala:192)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:262)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:236)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar closed issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.
Posted by GitBox <gi...@apache.org>.
vinothchandar closed issue #1982:
URL: https://github.com/apache/hudi/issues/1982
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar commented on issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.
Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #1982:
URL: https://github.com/apache/hudi/issues/1982#issuecomment-676498672
@Ac-Rush : Hudi lets Hadoop FileSystem framework (FileSystem.get()) instantiate specific file-system objects. Is there anything special we need to do to support ADLS Gen2 ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on issue #1982:
URL: https://github.com/apache/hudi/issues/1982#issuecomment-926130028
There are plenty of other issues reporting usage on adls. So closing this for now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] n3nash commented on issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.
Posted by GitBox <gi...@apache.org>.
n3nash commented on issue #1982:
URL: https://github.com/apache/hudi/issues/1982#issuecomment-693555511
@Ac-Rush Could you please let us know if you are blocked ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] bvaradar commented on issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.
Posted by GitBox <gi...@apache.org>.
bvaradar commented on issue #1982:
URL: https://github.com/apache/hudi/issues/1982#issuecomment-691774166
@Ac-Rush : Are you still blocked by this ?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #1982:
URL: https://github.com/apache/hudi/issues/1982#issuecomment-767205667
@Ac-Rush : would you mind update the ticket.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] vinothchandar commented on issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.
Posted by GitBox <gi...@apache.org>.
vinothchandar commented on issue #1982:
URL: https://github.com/apache/hudi/issues/1982#issuecomment-855191779
Next step: Can we test this setup and reproduce this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #1982: [SUPPORT] Not able to write to ADLS Gen2 in Azure Databricks, with error has invalid authority.
Posted by GitBox <gi...@apache.org>.
nsivabalan commented on issue #1982:
URL: https://github.com/apache/hudi/issues/1982#issuecomment-767205667
@Ac-Rush : would you mind update the ticket.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org