You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ozone.apache.org by "Dustin Smith (Jira)" <ji...@apache.org> on 2020/08/10 04:37:00 UTC
[jira] [Updated] (HDDS-4092) Writing delta to Ozone hangs when
creating the _delta_log json
[ https://issues.apache.org/jira/browse/HDDS-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dustin Smith updated HDDS-4092:
-------------------------------
Environment: We are using Kubernetes k8s, Ozone 0.5.0beta, Spark 3.0.0, Hadoop 3.2, Scala 2.12.10, and io.delta:delta-core_2.12:0.7.0 (was: We are using Kubernetes k8s, Ozone 0.5.0beta, Spark 3.0.0, Hadoop 3.2, and Scala 2.12.10.)
> Writing delta to Ozone hangs when creating the _delta_log json
> --------------------------------------------------------------
>
> Key: HDDS-4092
> URL: https://issues.apache.org/jira/browse/HDDS-4092
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Filesystem
> Affects Versions: 0.5.0
> Environment: We are using Kubernetes k8s, Ozone 0.5.0beta, Spark 3.0.0, Hadoop 3.2, Scala 2.12.10, and io.delta:delta-core_2.12:0.7.0
> Reporter: Dustin Smith
> Priority: Major
> Labels: delta, filesystem, scala, spark
>
> I am testing writing delta, OSS not databricks, data to Ozone FS since my company is looking to replace Hadoop if feasible. However, whenever I write delta table, the parquet files are writing, the delta log directory is created, but the json is never writing.
> I am using the spark operator to submit a batch test job to write about 5mb of data.
> Neither on the driver nor on the executor is there an error. The driver never finishes since the creation of the json hangs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: ozone-issues-help@hadoop.apache.org