You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by "Kamilooosx (via GitHub)" <gi...@apache.org> on 2024/12/03 21:47:36 UTC

[I] Bug with import CSV in DATE format [superset]

Kamilooosx opened a new issue, #31272:
URL: https://github.com/apache/superset/issues/31272

   ### Bug description
   
   CSV import loads string type every time, instead of date. I check the Columns to be parsed as dates option but it still doesn't work. I tried 
   Database:File Uploads 
   Schema: main
   I checked all data types in CSV for example YYYY-MM-DD or 2024-07-09 09:15:05
   Version: 4.1.0.3
   
   ### Screenshots/recordings
   
   _No response_
   
   ### Superset version
   
   4.1.1
   
   ### Python version
   
   3.9
   
   ### Node version
   
   16
   
   ### Browser
   
   Chrome
   
   ### Additional context
   
   _No response_
   
   ### Checklist
   
   - [X] I have searched Superset docs and Slack and didn't find a solution to my problem.
   - [X] I have searched the GitHub issue tracker and didn't find a similar bug report.
   - [ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Bug with import CSV in DATE format [superset]

Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2517924693

   Did the same file + upload choices successfully get turned into a date using Superset version 4.0.2?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]

Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2536612385

   With preset support i found a solution.
   We must create SQL Lab queries with that function 
   
   **datetime("<date column>") AS <name that you want>**
   Do note that you would still need to change the date column generated by the function above to a temporal field and then use it as a filter after that.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]

Posted by "yousoph (via GitHub)" <gi...@apache.org>.
yousoph commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2521754765

   Hi @Kamilooosx , once the upload has finished, if you're having trouble with the column type, you should be able to go into your dataset under the Columns tab and mark the time column as a temporal column to use it as a time column in charts
   
   <img width="864" alt="image" src="https://github.com/user-attachments/assets/839cb484-14e4-42df-9793-1a42c4bd084d">
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Bug with import CSV in DATE format [superset]

Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2517977359

   I hadn't used the CSV import feature before updating to version 4.1.0.3. 
   When I uploaded the CSV file to BigQuery and connected it to the database, everything worked correctly. 
   I am working with Preset.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]

Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2521862440

   Yeah ! It works ! But when i want to filter by Date it's generate error :C 
   ![image](https://github.com/user-attachments/assets/f79b02a8-eab9-4e81-982d-ea375c806b05)
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Bug with import CSV in DATE format [superset]

Posted by "dosubot[bot] (via GitHub)" <gi...@apache.org>.
dosubot[bot] commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2515621189

   <!-- Greeting -->
   Hey @Kamilooosx! I'm here to help you with any bugs, questions, or contributions you have for Superset. Let's squash this bug together!
   
   <!-- Answer -->
   I found a similar unsolved discussion titled [Error message: data type 'Datetime' not understood](https://github.com/apache/superset/discussions/27662) which might be related to your issue <sup>[[1]](https://github.com/apache/superset/discussions/27662)</sup>.
   
   Regarding your issue with the CSV import feature not recognizing date formats, ensure that the 'Columns to be parsed as dates' option is correctly set. The expected behavior is for this option to allow you to specify which columns should be interpreted as dates <sup>[[2]](https://github.com/apache/superset/blob/master/superset-frontend/src/features/databases/UploadDataModel/index.tsx)</sup>. 
   
   If the date parsing is failing, it might be due to incorrect settings in the CSV import process. Key settings to check include:
   - The `parse_dates` option, which specifies which columns should be parsed as dates. Ensure the column names or indices match those in your CSV file.
   - The `dayfirst` option, which affects how dates are interpreted. Make sure this is set according to your date format <sup>[[3]](https://github.com/apache/superset/blob/master/superset/commands/database/uploaders/csv_reader.py)</sup>.
   
   If these settings are correct and the issue persists, it might be a limitation or bug in the current version.
   
   
   <!-- Dosu Comment Footer -->
   *To continue talking to [Dosu](https://dosu.dev), mention @dosu.*
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Bug with import CSV in DATE format [superset]

Posted by "sfirke (via GitHub)" <gi...@apache.org>.
sfirke commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2521139443

   I can't replicate this in Superset 4.1.1. I created a CSV with a column called "datetime" and a value of `2024-07-09 09:15:05`. I then uploaded it into Superset, specifying that the column called "datetime" was a date time column. It loaded into my data warehouse with the correct column type and I was able to make time series charts in Superset with it.
   
   My only guess is that there's some invalid date-time value lurking in that column and causing it to be a string.  I once had this problem and finally figured out that a user had inputted some invalid date, like 2023-02-29 when that year wasn't a leap year for instance. Or an invalid time like 25:00. Once I deleted those few rows, it worked correctly.
   
   If you are able to create a reproducible minimal example that indicates this is indeed a problem with Superset, please share it on this issue.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]

Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx commented on issue #31272:
URL: https://github.com/apache/superset/issues/31272#issuecomment-2521640460

   I created a CSV file with random data, where the date format is similar to yours, but the bug still persists.
   ![image](https://github.com/user-attachments/assets/788b4a3f-92ad-494b-96c5-47263638df4f)
   ![image](https://github.com/user-attachments/assets/7a382a4f-6428-4c2d-85c8-c40368177351)
   ![image](https://github.com/user-attachments/assets/aae57a97-0e4b-481c-882d-65a688d5d440)
   ![image](https://github.com/user-attachments/assets/6e5d0f37-5665-4efd-9091-e1b6c476811e)
   ![image](https://github.com/user-attachments/assets/1b1a3dcb-b499-4971-82c2-7a0eb7177737)
   [health_center_data (1).csv](https://github.com/user-attachments/files/18030580/health_center_data.1.csv)
   ![image](https://github.com/user-attachments/assets/598a3496-2eb3-4ee5-bcda-43422822ffff)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org


Re: [I] Importing CSV: the csv has a date column but it appears in the database as a string [superset]

Posted by "Kamilooosx (via GitHub)" <gi...@apache.org>.
Kamilooosx closed issue #31272: Importing CSV: the csv has a date column but it appears in the database as a string
URL: https://github.com/apache/superset/issues/31272


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org