You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "Kamilooosx (via GitHub)" <gi...@apache.org> on 2024/12/03 21:47:36 UTC
[I] Bug with import CSV in DATE format [superset]
Kamilooosx opened a new issue, #31272:
URL: https://github.com/apache/superset/issues/31272
### Bug description
CSV import loads string type every time, instead of date. I check the Columns to be parsed as dates option but it still doesn't work. I tried
Database:File Uploads
Schema: main
I checked all data types in CSV for example YYYY-MM-DD or 2024-07-09 09:15:05
Version: 4.1.0.3
### Screenshots/recordings
_No response_
### Superset version
4.1.1
### Python version
3.9
### Node version
16
### Browser
Chrome
### Additional context
_No response_
### Checklist
- [X] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [X] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Bug with import CSV in DATE format [superset]
Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2517924693
Did the same file + upload choices successfully get turned into a date using Superset version 4.0.2?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]
Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2536612385
With preset support i found a solution.
We must create SQL Lab queries with that function
**datetime("<date column>") AS <name that you want>**
Do note that you would still need to change the date column generated by the function above to a temporal field and then use it as a filter after that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]
Posted by "yousoph (via GitHub)" <gi...@apache.org>.
yousoph commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2521754765
Hi @Kamilooosx , once the upload has finished, if you're having trouble with the column type, you should be able to go into your dataset under the Columns tab and mark the time column as a temporal column to use it as a time column in charts
<img width="864" alt="image" src="https://github.com/user-attachments/assets/839cb484-14e4-42df-9793-1a42c4bd084d">
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Bug with import CSV in DATE format [superset]
Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2517977359
I hadn't used the CSV import feature before updating to version 4.1.0.3.
When I uploaded the CSV file to BigQuery and connected it to the database, everything worked correctly.
I am working with Preset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]
Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2521862440
Yeah ! It works ! But when i want to filter by Date it's generate error :C

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Bug with import CSV in DATE format [superset]
Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2515621189
<!-- Greeting -->
Hey @Kamilooosx! I'm here to help you with any bugs, questions, or contributions you have for Superset. Let's squash this bug together!
<!-- Answer -->
I found a similar unsolved discussion titled [Error message: data type 'Datetime' not understood](https://github.com/apache/superset/discussions/27662) which might be related to your issue <sup>[[1]](https://github.com/apache/superset/discussions/27662)</sup>.
Regarding your issue with the CSV import feature not recognizing date formats, ensure that the 'Columns to be parsed as dates' option is correctly set. The expected behavior is for this option to allow you to specify which columns should be interpreted as dates <sup>[[2]](https://github.com/apache/superset/blob/master/superset-frontend/src/features/databases/UploadDataModel/index.tsx)</sup>.
If the date parsing is failing, it might be due to incorrect settings in the CSV import process. Key settings to check include:
- The `parse_dates` option, which specifies which columns should be parsed as dates. Ensure the column names or indices match those in your CSV file.
- The `dayfirst` option, which affects how dates are interpreted. Make sure this is set according to your date format <sup>[[3]](https://github.com/apache/superset/blob/master/superset/commands/database/uploaders/csv_reader.py)</sup>.
If these settings are correct and the issue persists, it might be a limitation or bug in the current version.
<!-- Dosu Comment Footer -->
*To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Bug with import CSV in DATE format [superset]
Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2521139443
I can't replicate this in Superset 4.1.1. I created a CSV with a column called "datetime" and a value of `2024-07-09 09:15:05`. I then uploaded it into Superset, specifying that the column called "datetime" was a date time column. It loaded into my data warehouse with the correct column type and I was able to make time series charts in Superset with it.
My only guess is that there's some invalid date-time value lurking in that column and causing it to be a string. I once had this problem and finally figured out that a user had inputted some invalid date, like 2023-02-29 when that year wasn't a leap year for instance. Or an invalid time like 25:00. Once I deleted those few rows, it worked correctly.
If you are able to create a reproducible minimal example that indicates this is indeed a problem with Superset, please share it on this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]
Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2521640460
I created a CSV file with random data, where the date format is similar to yours, but the bug still persists.





[health_center_data (1).csv](https://github.com/user-attachments/files/18030580/health_center_data.1.csv)

--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]
Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx closed issue #31272: Importing CSV: the csv has a date column but it appears in the database as a string
URL: https://github.com/apache/superset/issues/31272
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org