[GitHub] [incubator-superset] willbarrett commented on a change in pull request #8556: [WIP] [SIP] Proposal for Improvement on CSV Upload

[GitBox <gi...@apache.org>]

willbarrett commented on a change in pull request #8556: [WIP] [SIP] Proposal for Improvement on CSV Upload
URL: https://github.com/apache/incubator-superset/pull/8556#discussion_r346613651
 
 

 ##########
 File path: superset/views/core.py
 ##########
 @@ -3054,6 +3083,259 @@ def schemas_access_for_csv_upload(self):
                 stacktrace=utils.get_stacktrace(),
             )
 
+    @api
+    @has_access_api
+    @expose("/csvtodatabase/add", methods=["POST"])
+    def add(self) -> Response:
+        """ import a csv into a Table
+
+        Handles the logic for importing the csv into a Table in a given or newly created Database
+        returns HTTP Status Codes (200 for ok, 400 for errors)
+        Request body contains multipart/form-data
+        Form content:
+        form -- contains the properties for the table to be created
+        file -- csv file to be imported
+        """
+        form_data = request.form
+        csv_file = request.files["file"]
+        csv_filename = secure_filename(csv_file.filename)
+        if len(csv_filename) == 0:
+            return json_error_response("Filename is not allowed", status=400)
+        database_id = form_data["connectionId"]
+        # check for possible SQL-injection, filter_by does not sanitize the input therefore we have to check
+        # this beforehand
+        try:
+            database_id = int(database_id)
+        except ValueError:
+            message = _(
+                "Possible tampering detected, non-numeral character in database-id"
+            )
+            return json_error_response(message, status=400)
+
+        try:
+            if database_id != -1:
+                schema = None if not form_data["schema"] else form_data["schema"]
+                database = self._get_existing_database(database_id, schema)
+                db_name = database.database_name
+            else:
+                db_name = (
+                    csv_filename[:-4]
+                    if not form_data["databaseName"]
+                    else form_data["databaseName"]
+                )
+                db_name = secure_filename(db_name)
+                if len(db_name) == 0:
+                    return json_error_response(
+                        "Database name is not allowed", status=400
+                    )
+                database = self._create_database(db_name)
+        except Exception as e:
+            return json_error_response(e.args[0], status=400)
+
+        try:
+            path = self._check_and_save_csv(csv_file, csv_filename)
+            self._create_table(form_data, database, csv_filename)
+        except Exception as e:
+            try:
+                os.remove(os.getcwd() + "/" + db_name + ".db")
+            except OSError:
+                pass
+            try:
+                if database_id == -1:
+                    db.session.rollback()
+                    db.session.delete(database)
+                    db.session.commit()
+            except Exception:
+                pass
+            if hasattr(e, "orig"):
+                # pylint: disable=no-member
+                if isinstance(e.orig, IntegrityError):  # type: ignore
+                    message = "Table {0} could not be created".format(
+                        form_data["tableName"]
+                    )
+                # pylint: disable:no-member
+                elif isinstance(e.orig, OperationalError):  # type: ignore
+                    message = "Schema {0} is not allowed in a SQLite database".format(
+                        form_data["schema"]
+                    )
+                else:
+                    message = str(e)
+            return json_error_response(message, status=400)
+        finally:
+            try:
+                os.remove(path)
+            except OSError:
+                pass
+
+        stats_logger.incr("successful_csv_upload")
+        message = "{} imported into database {}".format(form_data["tableName"], db_name)
+        flash(message, "success")
+        return json_success(
+            '"{} imported into database {}"'.format(form_data["tableName"], db_name)
+        )
+
+    def _create_database(self, db_name: str):
 
 Review comment:
   This and the other introduced underscore methods would better live in a different module - something like a `CSVUploadHelper`? In general, it's good practice to separate business logic from controller/view code wherever possible.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org