You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@streampipes.apache.org by "tenthe (via GitHub)" <gi...@apache.org> on 2026/03/12 14:37:05 UTC
[I] Add CSV dataset import with preview for Data Sets
tenthe opened a new issue, #4250:
URL: https://github.com/apache/streampipes/issues/4250
### Body
We want to add a new dataset import feature that allows users to upload CSV files directly in the Data Lake dataset view.
The feature should provide a simple UI flow in the dataset configuration screen where a user can:
- upload a CSV file
- define parsing options such as delimiter and decimal separator
- preview the parsed data to verify the file was read correctly
- choose a timestamp column
- define for each column whether it should be treated as a header, dimension, or measurement
- decide whether the data should be imported into an existing dataset or a new dataset
If a new dataset is created, the backend should infer the event schema from the uploaded data. If an existing dataset is selected, the backend should validate that the imported schema matches the existing one. The backend should also validate that a newly created dataset name is not already in use.
To support this, we should introduce dedicated REST endpoints for import preview and execution, for example:
- `POST /api/v4/datalake/import/preview`
- `POST /api/v4/datalake/import`
The preview endpoint should return parsed column information, inferred data types, and validation hints. The import endpoint should perform the actual dataset creation and/or data ingestion.
This API should not be designed only for the browser UI. It should also serve as the foundation for future programmatic usage, especially:
- client integrations such as a Python client
- e2e tests that need to load datasets into the system quickly
### StreamPipes Committer
I acknowledge that I am a maintainer/committer of the Apache StreamPipes project.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Add CSV dataset import with preview for Data Sets
Posted by "tenthe (via GitHub)" <gi...@apache.org>.
tenthe closed issue #4250: Add CSV dataset import with preview for Data Sets
URL: https://github.com/apache/streampipes/issues/4250
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@streampipes.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org