You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hop.apache.org by "pauloalexsb (via GitHub)" <gi...@apache.org> on 2024/01/31 17:12:32 UTC
[I] Error reading parquet file[Bug]: (hop)
pauloalexsb opened a new issue, #3598:
URL: https://github.com/apache/hop/issues/3598
### Apache Hop version?
2.7.0
### Java version?
21.0.2
### Operating system
Windows
### What happened?
I'm using Parquet Input, following the documentation I started with Get File names and called Parquet Input File;
I chose the filename as Filename Field and looked for a parquet file to pull the fields, everything ok so far....but it presents the following error when trying to read the files:
### Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException : Relative path in absolute URI: ParquetStream of file 'E:%5CQSR%5CTEMP%5CSIPNI_COVID19_OUTROS_21012024_06_50.parquet'
I'm on a Windows machine and the absolute path of the file is:
### E:\QSR\TEMP\SIPNI_COVID19_OUTROS_21012024_06_50.parquet
I'm trying to read several files, and after some tests I realized that the error appeared in a single file.
I then tested searching for the fields only from this file and realized that the field types (Type) had two types that were different from the other files.
Example, in file 1 the column type was Integer and in file 2 the column was of type Number. After adjusting this and leaving everything as String, the error no longer appeared and I was able to read all the files.
I think the error message about not finding the file or about an error reading the files should be more explicit.
I received a correct error when, in the tests, the file did not exist, but when the files existed and the number and column names were the same, but the data type was different between the files, the error appeared.
### Issue Priority
Priority: 3
### Issue Component
Component: Hop Gui, Component: Pipelines
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Error reading parquet file (hop)
Posted by "hansva (via GitHub)" <gi...@apache.org>.
hansva commented on issue #3598:
URL: https://github.com/apache/hop/issues/3598#issuecomment-5778966121
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Error reading parquet file[Bug]: (hop)
Posted by "pauloalexsb (via GitHub)" <gi...@apache.org>.
pauloalexsb commented on issue #3598:
URL: https://github.com/apache/hop/issues/3598#issuecomment-1919554086
I forgot... to present the error, take two parquet files with the same name and number of columns, in file 1 a column must be integer, in file 2 a column must be of type number, with Get File Names look for the two files and in Input Parquet choose one of the two as file, perform the transformation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Error reading parquet file (hop)
Posted by "sramazzina (via GitHub)" <gi...@apache.org>.
sramazzina commented on issue #3598:
URL: https://github.com/apache/hop/issues/3598#issuecomment-1969994294
@pauloalexsb I was looking int your problem. First: replaced the backslashes in the path with slashes by mean of a replace transform. This way the error changed and was more clear because it is like this
`2024/02/28 19:26:04 - Parquet File Input.0 - Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 13: ParquetStream of file 'C:/java/hop/hop/config/projects/default/parquet_hpl_file/interger.parquet'`
Can you tell me from which platform you files were generated?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Error reading parquet file (hop)
Posted by "hansva (via GitHub)" <gi...@apache.org>.
hansva closed issue #3598: [Bug]: Error reading parquet file
URL: https://github.com/apache/hop/issues/3598
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Error reading parquet file (hop)
Posted by "pauloalexsb (via GitHub)" <gi...@apache.org>.
pauloalexsb commented on issue #3598:
URL: https://github.com/apache/hop/issues/3598#issuecomment-1924523117
Hello, here are two parquet files with two fields, the same names.
One of them has the **interger** type field, and the other has the same field with the **Number** type.
As you can see in the hpl file, I searched for both and chose one to search for the fields, the error occurs.
If I change the type to String, reading happens normally. Please note that the error apparently implies that it is a file location error, not a field type error.
[parquet_hpl_file.tar.gz](https://github.com/apache/hop/files/14144315/parquet_hpl_file.tar.gz)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug]: Error reading parquet file (hop)
Posted by "usbrandon (via GitHub)" <gi...@apache.org>.
usbrandon commented on issue #3598:
URL: https://github.com/apache/hop/issues/3598#issuecomment-1922765429
Would you please create two small parquet files and an hpl file that reproduce the issue. That will make it much easier to isolate the problem. The parquet files can be super tiny.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@hop.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org