You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Tarique Anwer (Jira)" <ji...@apache.org> on 2025/04/25 10:39:00 UTC

[jira] [Updated] (SPARK-42198) spark.read fails to read filenames with accented characters

     [ https://issues.apache.org/jira/browse/SPARK-42198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tarique Anwer updated SPARK-42198:
----------------------------------
    Priority: Minor  (was: Major)

> spark.read fails to read filenames with accented characters
> -----------------------------------------------------------
>
>                 Key: SPARK-42198
>                 URL: https://issues.apache.org/jira/browse/SPARK-42198
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark
>    Affects Versions: 3.2.1
>            Reporter: Tarique Anwer
>            Priority: Minor
>
> Unable to read filenames with accented characters in the filename.
> *Sample error:*
> {code:java}
> org.apache.spark.SparkException: Job aborted due to stage failure: Task 43 in stage 1.0 failed 4 times, most recent failure: Lost task 43.3 in stage 1.0 (TID 105) (10.139.64.5 executor 0): java.io.FileNotFoundException: /4842022074360943/user/hive/warehouse/hls_cms_source.db/raw_files/synthea_mass/ccda/Amalia471_Magaña874_3912696a-0aef-492e-83ef-468262b82966.xml{code}
>  
> *{{Steps to reproduce error:}}*
> {code:java}
> %sh
> mkdir -p /dbfs/user/hive/warehouse/hls_cms_source.db/raw_files/synthea_mass
> wget  https://synthetichealth.github.io/synthea-sample-data/downloads/synthea_sample_data_ccda_sep2019.zip -O ./synthea_sample_data_ccda_sep2019.zip 
> unzip ./synthea_sample_data_ccda_sep2019.zip -d /dbfs/user/hive/warehouse/hls_cms_source.db/raw_files/synthea_mass/
> {code}
>  
> {code:java}
> spark.conf.set("spark.sql.caseSensitive", "true")
> df = (
>   spark.read.format('xml')
>    .option("rowTag", "ClinicalDocument")
>   .load('/user/hive/warehouse/hls_cms_source.db/raw_files/synthea_mass/ccda/')
> ){code}
> Is there a way to deal with this situation where I don't have control over the file names for some reason?



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org