You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@superset.apache.org by GitBox <gi...@apache.org> on 2021/05/03 18:05:20 UTC
[GitHub] [superset] exemplary-citizen opened a new pull request #14449: feat: Allow csv upload to accept parquet file
exemplary-citizen opened a new pull request #14449:
URL: https://github.com/apache/superset/pull/14449
### SUMMARY
Allow CSV upload form to accept parquet file. Went in this direction so as not to exacerbate what was brought up in #13834 by adding a new form specifically for parquet files. I believe small modifications can be made to this PR to accommodate `feather` and `orc` files.
### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A
### TEST PLAN
Added a test similar to the ones already in `csv_upload_tests.py`
### ADDITIONAL INFORMATION
Fixes #14020
- [x] Has associated issue: #14020
- [x] Changes UI
- [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351))
- [ ] Migration is atomic, supports rollback & is backwards-compatible
- [ ] Confirm DB migration upgrade and downgrade tested
- [ ] Runtime estimates and downtime expectations provided
- [x] Introduces new feature or API
- [ ] Removes existing feature or API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-891857154
@villebro do you have any lingering doubts? if not, we should be good to merge once you enable CI and it comes back green.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f209999) into [master](https://codecov.io/gh/apache/superset/commit/3456dd571ca705699766473e55ededd36c6f4dc7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3456dd5) will **decrease** coverage by `0.16%`.
> The diff coverage is `85.55%`.
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
- Coverage 77.54% 77.38% -0.17%
==========================================
Files 967 967
Lines 49759 49846 +87
Branches 6352 6352
==========================================
- Hits 38585 38571 -14
- Misses 10972 11073 +101
Partials 202 202
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `?` | |
| postgres | `81.74% <85.55%> (+0.01%)` | :arrow_up: |
| python | `81.79% <85.55%> (-0.33%)` | :arrow_down: |
| sqlite | `81.37% <85.55%> (+0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `87.76% <100.00%> (+0.08%)` | :arrow_up: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.18% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [superset/db\_engine\_specs/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2hpdmUucHk=) | `70.32% <0.00%> (-17.08%)` | :arrow_down: |
| [superset/db\_engine\_specs/mysql.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL215c3FsLnB5) | `92.85% <0.00%> (-4.29%)` | :arrow_down: |
| [superset/views/database/mixins.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvbWl4aW5zLnB5) | `81.03% <0.00%> (-1.73%)` | :arrow_down: |
| [superset/db\_engine\_specs/presto.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL3ByZXN0by5weQ==) | `83.36% <0.00%> (-1.06%)` | :arrow_down: |
| [superset/models/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvbW9kZWxzL2NvcmUucHk=) | `88.97% <0.00%> (-0.79%)` | :arrow_down: |
| ... and [4 more](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [3456dd5...f209999](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-891918965
@exemplary-citizen btw, it appears one of the csv upload integration tests is failing
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #14449:
URL: https://github.com/apache/superset/pull/14449#discussion_r681785283
##########
File path: superset/views/database/forms.py
##########
@@ -402,3 +417,130 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
'Use [""] for empty string.'
),
)
+
+
+class ColumnarToDatabaseForm(DynamicForm):
+ # pylint: disable=E0211
+ def columnar_allowed_dbs() -> List[Database]: # type: ignore
+ # TODO: change allow_csv_upload to allow_file_upload
+ columnar_enabled_dbs = (
+ db.session.query(Database).filter_by(allow_csv_upload=True).all()
+ )
+ return [
+ columnar_enabled_db
+ for columnar_enabled_db in columnar_enabled_dbs
+ if ColumnarToDatabaseForm.at_least_one_schema_is_allowed(
+ columnar_enabled_db
+ )
+ ]
Review comment:
There's unnecessary duplication here: we could abstract this into a `class UploadToDatabaseForm(DynamicForm)` with all the repeated boilerplate and then extend those into `class CsvToDatabaseForm(UploadToDatabaseForm)`, `class ColumnarToDatabaseForm(UploadToDatabaseForm)` etc. But this was here before you already, so not your fault! But if you feel like doing some valuable cleanup work, it would be great to do some of this cleanup in a follow-up PR.
##########
File path: superset/templates/superset/form_view/columnar_to_database_view/edit.html
##########
@@ -0,0 +1,64 @@
+{#
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+#}
+{% extends 'appbuilder/general/model/edit.html' %}
+
+{% block tail_js %}
+ {{ super() }}
+ <script>
+ var db = $("#con");
+ var schema = $("#schema");
+
+ // this element is a text input
+ // copy it here so it can be reused later
+ var any_schema_is_allowed = schema.clone();
+
+ update_schemas_allowed_for_columnar_upload(db.val());
+ db.change(function(){
+ update_schemas_allowed_for_columnar_upload(db.val());
+ });
+
+ function update_schemas_allowed_for_columnar_upload(db_id) {
+ $.ajax({
+ method: "GET",
+ url: "/superset/schemas_access_for_columnar_upload",
Review comment:
This endpoint is called `schemas_access_for_csv_upload` and will cause a 404 (this same error happens in the Excel upload form) - either use that or rename the original to `schemas_access_for_file_upload`.
##########
File path: superset/views/database/forms.py
##########
@@ -402,3 +417,130 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
'Use [""] for empty string.'
),
)
+
+
+class ColumnarToDatabaseForm(DynamicForm):
+ # pylint: disable=E0211
+ def columnar_allowed_dbs() -> List[Database]: # type: ignore
Review comment:
I wonder why this wasn't `@staticmethod`?? Can we update that here and in the other classes
##########
File path: superset/views/database/forms.py
##########
@@ -163,6 +169,15 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
_("Mangle Duplicate Columns"),
description=_('Specify duplicate columns as "X.0, X.1".'),
)
+ usecols = JsonListField(
+ _("Use Columns"),
+ default=None,
+ description=_(
+ "Json list of the column names that should be read. "
+ "If not None, only these columns will be read from the file."
+ ),
+ validators=[Optional()],
+ )
Review comment:
Yes, let's keep this, I think it's a great addition 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on a change in pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on a change in pull request #14449:
URL: https://github.com/apache/superset/pull/14449#discussion_r646243432
##########
File path: superset/views/database/forms.py
##########
@@ -163,6 +169,15 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
_("Mangle Duplicate Columns"),
description=_('Specify duplicate columns as "X.0, X.1".'),
)
+ usecols = JsonListField(
+ _("Use Columns"),
+ default=None,
+ description=_(
+ "Json list of the column names that should be read. "
+ "If not None, only these columns will be read from the file."
+ ),
+ validators=[Optional()],
+ )
Review comment:
@villebro do you still want to keep the `Use Columns` feature in the csv form? if so, should we keep it in this PR or move it to a separate one?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-906370157
@exemplary-citizen there's a linting error in one of the files. You can either setup pre-commit hooks or apply the diff below to fix the problem:
```diff
diff --git a/superset/views/database/views.py b/superset/views/database/views.py
index 8d0a92f6c..3863b165c 100644
--- a/superset/views/database/views.py
+++ b/superset/views/database/views.py
@@ -406,17 +406,23 @@ class ColumnarToDatabaseView(SimpleFormView):
def form_get(self, form: ColumnarToDatabaseForm) -> None:
form.if_exists.data = "fail"
- def form_post(self, form: ColumnarToDatabaseForm) -> Response: # pylint: disable=too-many-locals
+ def form_post(
+ self, form: ColumnarToDatabaseForm
+ ) -> Response: # pylint: disable=too-many-locals
database = form.con.data
columnar_table = Table(table=form.name.data, schema=form.schema.data)
files = form.columnar_file.data
file_type = {file.filename.split(".")[-1] for file in files}
if file_type == {"zip"}:
- zipfile_ob = zipfile.ZipFile(form.columnar_file.data[0]) # pylint: disable=consider-using-with
+ zipfile_ob = zipfile.ZipFile(
+ form.columnar_file.data[0]
+ ) # pylint: disable=consider-using-with
file_type = {filename.split(".")[-1] for filename in zipfile_ob.namelist()}
files = [
- io.BytesIO((zipfile_ob.open(filename).read(), filename)[0]) # pylint: disable=consider-using-with
+ io.BytesIO(
+ (zipfile_ob.open(filename).read(), filename)[0]
+ ) # pylint: disable=consider-using-with
for filename in zipfile_ob.namelist()
]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] arita37 commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
arita37 commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-889341475
This is a great PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on a change in pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on a change in pull request #14449:
URL: https://github.com/apache/superset/pull/14449#discussion_r694382318
##########
File path: superset/views/database/forms.py
##########
@@ -402,3 +417,130 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
'Use [""] for empty string.'
),
)
+
+
+class ColumnarToDatabaseForm(DynamicForm):
+ # pylint: disable=E0211
+ def columnar_allowed_dbs() -> List[Database]: # type: ignore
Review comment:
Adding that decorator will throw `TypeError: 'staticmethod' object is not callable`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on a change in pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on a change in pull request #14449:
URL: https://github.com/apache/superset/pull/14449#discussion_r636508662
##########
File path: superset/views/database/views.py
##########
@@ -128,6 +128,32 @@ def form_get(self, form: CsvToDatabaseForm) -> None:
def form_post(self, form: CsvToDatabaseForm) -> Response:
database = form.con.data
csv_table = Table(table=form.name.data, schema=form.schema.data)
+ file_type = form.csv_file.data.filename.split(".")[-1]
+ if file_type == "parquet":
+ read = pd.read_parquet
+ kwargs = {
+ "columns": form.usecols.data,
+ }
+ else:
+ read = pd.read_csv
+ kwargs = {
+ "chunksize": 1000,
+ "encoding": "utf-8",
+ "header": form.header.data if form.header.data else 0,
+ "index_col": form.index_col.data,
+ "infer_datetime_format": form.infer_datetime_format.data,
+ "iterator": True,
+ "keep_default_na": not form.null_values.data,
+ "mangle_dupe_cols": form.mangle_dupe_cols.data,
+ "usecols": form.usecols.data,
Review comment:
This appears to change the behavior of the existing CSV upload functionality by specifying columns. Can you add some tests around this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] github-actions[bot] commented on pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-845590065
@robdiciuccio Ephemeral environment spinning up at http://34.214.127.48:8080. Credentials are `admin`/`admin`. Please allow several minutes for bootstrapping and startup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] arita37 commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
arita37 commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-892261069
It will be a very very useful features
for many people.
> On Aug 3, 2021, at 23:08, Ville Brofeldt ***@***.***> wrote:
>
>
> @villebro requested changes on this pull request.
>
> Great work! Just a few (hopefully) last comments - I'll prioritize seeing this through, so as soon as you can make the necessary updates let me know!
>
> In superset/views/database/forms.py:
>
> > +class ColumnarToDatabaseForm(DynamicForm):
> + # pylint: disable=E0211
> + def columnar_allowed_dbs() -> List[Database]: # type: ignore
> + # TODO: change allow_csv_upload to allow_file_upload
> + columnar_enabled_dbs = (
> + db.session.query(Database).filter_by(allow_csv_upload=True).all()
> + )
> + return [
> + columnar_enabled_db
> + for columnar_enabled_db in columnar_enabled_dbs
> + if ColumnarToDatabaseForm.at_least_one_schema_is_allowed(
> + columnar_enabled_db
> + )
> + ]
> There's unnecessary duplication here: we could abstract this into a class UploadToDatabaseForm(DynamicForm) with all the repeated boilerplate and then extend those into class CsvToDatabaseForm(UploadToDatabaseForm), class ColumnarToDatabaseForm(UploadToDatabaseForm) etc. But this was here before you already, so not your fault! But if you feel like doing some valuable cleanup work, it would be great to do some of this cleanup in a follow-up PR.
>
> In superset/templates/superset/form_view/columnar_to_database_view/edit.html:
>
> > + var db = $("#con");
> + var schema = $("#schema");
> +
> + // this element is a text input
> + // copy it here so it can be reused later
> + var any_schema_is_allowed = schema.clone();
> +
> + update_schemas_allowed_for_columnar_upload(db.val());
> + db.change(function(){
> + update_schemas_allowed_for_columnar_upload(db.val());
> + });
> +
> + function update_schemas_allowed_for_columnar_upload(db_id) {
> + $.ajax({
> + method: "GET",
> + url: "/superset/schemas_access_for_columnar_upload",
> This endpoint is called schemas_access_for_csv_upload and will cause a 404 (this same error happens in the Excel upload form) - either use that or rename the original to schemas_access_for_file_upload.
>
> In superset/views/database/forms.py:
>
> > + # pylint: disable=E0211
> + def columnar_allowed_dbs() -> List[Database]: # type: ignore
> I wonder why this wasn't @staticmethod?? Can we update that here and in the other classes
>
> In superset/views/database/forms.py:
>
> > + usecols = JsonListField(
> + _("Use Columns"),
> + default=None,
> + description=_(
> + "Json list of the column names that should be read. "
> + "If not None, only these columns will be read from the file."
> + ),
> + validators=[Optional()],
> + )
> Yes, let's keep this, I think it's a great addition 👍
>
> —
> You are receiving this because you commented.
> Reply to this email directly, view it on GitHub, or unsubscribe.
> Triage notifications on the go with GitHub Mobile for iOS or Android.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (016fbea) into [master](https://codecov.io/gh/apache/superset/commit/18be181946a3e5b7e651dde7a3e5fec46504dd08?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (18be181) will **decrease** coverage by `0.19%`.
> The diff coverage is `85.86%`.
> :exclamation: Current head 016fbea differs from pull request most recent head 5b5eb74. Consider uploading reports for the commit 5b5eb74 to get more accurate results
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
- Coverage 76.63% 76.44% -0.20%
==========================================
Files 1002 1000 -2
Lines 53637 53574 -63
Branches 6853 6818 -35
==========================================
- Hits 41104 40953 -151
- Misses 12294 12383 +89
+ Partials 239 238 -1
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.54% <85.86%> (+0.01%)` | :arrow_up: |
| postgres | `81.52% <85.86%> (-0.07%)` | :arrow_down: |
| presto | `81.34% <85.86%> (-0.07%)` | :arrow_down: |
| python | `81.82% <85.86%> (-0.28%)` | :arrow_down: |
| sqlite | `81.17% <85.86%> (-0.03%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.36% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.17% <100.00%> (+0.08%)` | :arrow_up: |
| [superset/views/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvY29yZS5weQ==) | `75.30% <100.00%> (+0.01%)` | :arrow_up: |
| [superset-frontend/src/setup/setupPlugins.ts](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3NldHVwL3NldHVwUGx1Z2lucy50cw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset-frontend/src/setup/setupPluginsExtra.ts](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3NldHVwL3NldHVwUGx1Z2luc0V4dHJhLnRz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [...-frontend/src/visualizations/presets/MainPreset.js](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3Zpc3VhbGl6YXRpb25zL3ByZXNldHMvTWFpblByZXNldC5qcw==) | `0.00% <0.00%> (-80.00%)` | :arrow_down: |
| [...c/visualizations/FilterBox/FilterBoxChartPlugin.js](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3Zpc3VhbGl6YXRpb25zL0ZpbHRlckJveC9GaWx0ZXJCb3hDaGFydFBsdWdpbi5qcw==) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| ... and [41 more](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [18be181...5b5eb74](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-847622610
Yeah I agree that this probably belongs in a separate form. Just went in this direction because creating a new form would mean that we'd effectively be abandoning #13834. I'll go ahead and get started working on a new form for `parquet/directory/zip` unless @junlincc @srinify @Steejay have any ideas on how to better consolidate these forms. I believe `wtforms` has a `MultipleFileField` class that should allow us to upload entire directories. We can also introduce this change to the `CsvToDatabaseForm` so users can upload multiple CSVs at a time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-856634339
> Yeah I agree that this probably belongs in a separate form. Just went in this direction because creating a new form would mean that we'd effectively be abandoning #13834. I'll go ahead and get started working on a new form for `parquet/directory/zip` unless @junlincc @srinify @Steejay have any ideas on how to better consolidate these forms. I believe `wtforms` has a `MultipleFileField` class that should allow us to upload entire directories. We can also introduce this change to the `CsvToDatabaseForm` so users can upload multiple CSVs at a time.
I think we'll end up consolidating all upload forms into one at some point, but it probably won't happen just yet. So I personally vote for keeping these separate to avoid making one huge form that doesn't really specifically apply to any one format.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-855538821
Here is what the new form looks like:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (5b5eb74) into [master](https://codecov.io/gh/apache/superset/commit/18be181946a3e5b7e651dde7a3e5fec46504dd08?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (18be181) will **decrease** coverage by `0.20%`.
> The diff coverage is `85.86%`.
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
- Coverage 76.63% 76.42% -0.21%
==========================================
Files 1002 1002
Lines 53637 53724 +87
Branches 6853 6853
==========================================
- Hits 41104 41060 -44
- Misses 12294 12425 +131
Partials 239 239
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.53% <85.86%> (+0.01%)` | :arrow_up: |
| postgres | `81.60% <85.86%> (+0.01%)` | :arrow_up: |
| presto | `?` | |
| python | `81.68% <85.86%> (-0.42%)` | :arrow_down: |
| sqlite | `81.24% <85.86%> (+0.05%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.36% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.17% <100.00%> (+0.08%)` | :arrow_up: |
| [superset/views/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvY29yZS5weQ==) | `75.29% <100.00%> (ø)` | |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [superset/db\_engine\_specs/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2hpdmUucHk=) | `69.80% <0.00%> (-16.87%)` | :arrow_down: |
| [superset/db\_engine\_specs/presto.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL3ByZXN0by5weQ==) | `83.47% <0.00%> (-6.91%)` | :arrow_down: |
| [superset/views/database/mixins.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvbWl4aW5zLnB5) | `81.03% <0.00%> (-1.73%)` | :arrow_down: |
| [superset/connectors/sqla/models.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29ubmVjdG9ycy9zcWxhL21vZGVscy5weQ==) | `88.04% <0.00%> (-1.66%)` | :arrow_down: |
| ... and [3 more](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [18be181...5b5eb74](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro merged pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro merged pull request #14449:
URL: https://github.com/apache/superset/pull/14449
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (016fbea) into [master](https://codecov.io/gh/apache/superset/commit/486ef6b81fb0eab39a3e403bb7fdba784c5e8649?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (486ef6b) will **decrease** coverage by `0.11%`.
> The diff coverage is `85.86%`.
> :exclamation: Current head 016fbea differs from pull request most recent head 6bcc0d9. Consider uploading reports for the commit 6bcc0d9 to get more accurate results
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
- Coverage 76.55% 76.44% -0.12%
==========================================
Files 1000 1000
Lines 53487 53574 +87
Branches 6818 6818
==========================================
+ Hits 40948 40953 +5
- Misses 12301 12383 +82
Partials 238 238
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.54% <85.86%> (+0.01%)` | :arrow_up: |
| postgres | `81.52% <85.86%> (-0.03%)` | :arrow_down: |
| presto | `81.34% <85.86%> (+0.01%)` | :arrow_up: |
| python | `81.82% <85.86%> (-0.25%)` | :arrow_down: |
| sqlite | `81.17% <85.86%> (+0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.36% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.17% <100.00%> (+0.08%)` | :arrow_up: |
| [superset/views/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvY29yZS5weQ==) | `75.30% <100.00%> (ø)` | |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [superset/db\_engine\_specs/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2hpdmUucHk=) | `69.80% <0.00%> (-16.87%)` | :arrow_down: |
| [superset/connectors/sqla/models.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29ubmVjdG9ycy9zcWxhL21vZGVscy5weQ==) | `89.46% <0.00%> (-0.24%)` | :arrow_down: |
| [superset/utils/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvY29yZS5weQ==) | `89.86% <0.00%> (-0.13%)` | :arrow_down: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [486ef6b...6bcc0d9](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-908879786
@exemplary-citizen sorry to bother you again, but we've recently updated the version of `pylint` which appears to be causing some new linting issues. Do you want to fix this or is it ok if I jump in and resolve the remaining ones?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (016fbea) into [master](https://codecov.io/gh/apache/superset/commit/18be181946a3e5b7e651dde7a3e5fec46504dd08?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (18be181) will **decrease** coverage by `0.19%`.
> The diff coverage is `85.86%`.
> :exclamation: Current head 016fbea differs from pull request most recent head 360e17e. Consider uploading reports for the commit 360e17e to get more accurate results
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
- Coverage 76.63% 76.44% -0.20%
==========================================
Files 1002 1000 -2
Lines 53637 53574 -63
Branches 6853 6818 -35
==========================================
- Hits 41104 40953 -151
- Misses 12294 12383 +89
+ Partials 239 238 -1
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.54% <85.86%> (+0.01%)` | :arrow_up: |
| postgres | `81.52% <85.86%> (-0.07%)` | :arrow_down: |
| presto | `81.34% <85.86%> (-0.07%)` | :arrow_down: |
| python | `81.82% <85.86%> (-0.28%)` | :arrow_down: |
| sqlite | `81.17% <85.86%> (-0.03%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.36% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.17% <100.00%> (+0.08%)` | :arrow_up: |
| [superset/views/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvY29yZS5weQ==) | `75.30% <100.00%> (+0.01%)` | :arrow_up: |
| [superset-frontend/src/setup/setupPlugins.ts](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3NldHVwL3NldHVwUGx1Z2lucy50cw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset-frontend/src/setup/setupPluginsExtra.ts](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3NldHVwL3NldHVwUGx1Z2luc0V4dHJhLnRz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [...-frontend/src/visualizations/presets/MainPreset.js](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3Zpc3VhbGl6YXRpb25zL3ByZXNldHMvTWFpblByZXNldC5qcw==) | `0.00% <0.00%> (-80.00%)` | :arrow_down: |
| [...c/visualizations/FilterBox/FilterBoxChartPlugin.js](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3Zpc3VhbGl6YXRpb25zL0ZpbHRlckJveC9GaWx0ZXJCb3hDaGFydFBsdWdpbi5qcw==) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| ... and [41 more](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [18be181...360e17e](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-906152286
@villebro should come back green now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on a change in pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on a change in pull request #14449:
URL: https://github.com/apache/superset/pull/14449#discussion_r636508816
##########
File path: superset/views/database/forms.py
##########
@@ -163,6 +165,15 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
_("Mangle Duplicate Columns"),
description=_('Specify duplicate columns as "X.0, X.1".'),
)
+ usecols = JsonListField(
+ _("Use Columns"),
+ default=None,
+ description=_(
+ "Json list of the column names that should be read. "
+ "If not None, only these columns will be read from the file."
+ ),
+ validators=[Optional()],
+ )
Review comment:
Can you provide a screenshot of the updated form UI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro merged pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro merged pull request #14449:
URL: https://github.com/apache/superset/pull/14449
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-908879786
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-908972685
@villebro thx for pushing this through. yeah do tag me, i'll be sure to review. in the meantime i'll get started on #16046
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (016fbea) into [master](https://codecov.io/gh/apache/superset/commit/18be181946a3e5b7e651dde7a3e5fec46504dd08?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (18be181) will **decrease** coverage by `0.19%`.
> The diff coverage is `85.86%`.
> :exclamation: Current head 016fbea differs from pull request most recent head 5b5eb74. Consider uploading reports for the commit 5b5eb74 to get more accurate results
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
- Coverage 76.63% 76.44% -0.20%
==========================================
Files 1002 1000 -2
Lines 53637 53574 -63
Branches 6853 6818 -35
==========================================
- Hits 41104 40953 -151
- Misses 12294 12383 +89
+ Partials 239 238 -1
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.54% <85.86%> (+0.01%)` | :arrow_up: |
| postgres | `81.52% <85.86%> (-0.07%)` | :arrow_down: |
| presto | `81.34% <85.86%> (-0.07%)` | :arrow_down: |
| python | `81.82% <85.86%> (-0.28%)` | :arrow_down: |
| sqlite | `81.17% <85.86%> (-0.03%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.36% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.17% <100.00%> (+0.08%)` | :arrow_up: |
| [superset/views/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvY29yZS5weQ==) | `75.30% <100.00%> (+0.01%)` | :arrow_up: |
| [superset-frontend/src/setup/setupPlugins.ts](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3NldHVwL3NldHVwUGx1Z2lucy50cw==) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset-frontend/src/setup/setupPluginsExtra.ts](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3NldHVwL3NldHVwUGx1Z2luc0V4dHJhLnRz) | `0.00% <0.00%> (-100.00%)` | :arrow_down: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [...-frontend/src/visualizations/presets/MainPreset.js](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3Zpc3VhbGl6YXRpb25zL3ByZXNldHMvTWFpblByZXNldC5qcw==) | `0.00% <0.00%> (-80.00%)` | :arrow_down: |
| [...c/visualizations/FilterBox/FilterBoxChartPlugin.js](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQtZnJvbnRlbmQvc3JjL3Zpc3VhbGl6YXRpb25zL0ZpbHRlckJveC9GaWx0ZXJCb3hDaGFydFBsdWdpbi5qcw==) | `0.00% <0.00%> (-75.00%)` | :arrow_down: |
| ... and [41 more](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [18be181...5b5eb74](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861785894
Looks like the failing presto postgres test was fixed in #15163. CI should come back green now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-845352239
@exemplary-citizen sorry for dropping the ball on this - I'll have this reviewed within the next 24 hours
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on a change in pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on a change in pull request #14449:
URL: https://github.com/apache/superset/pull/14449#discussion_r636614194
##########
File path: superset/views/database/views.py
##########
@@ -128,6 +128,32 @@ def form_get(self, form: CsvToDatabaseForm) -> None:
def form_post(self, form: CsvToDatabaseForm) -> Response:
database = form.con.data
csv_table = Table(table=form.name.data, schema=form.schema.data)
+ file_type = form.csv_file.data.filename.split(".")[-1]
+ if file_type == "parquet":
+ read = pd.read_parquet
+ kwargs = {
+ "columns": form.usecols.data,
+ }
+ else:
+ read = pd.read_csv
+ kwargs = {
+ "chunksize": 1000,
+ "encoding": "utf-8",
+ "header": form.header.data if form.header.data else 0,
+ "index_col": form.index_col.data,
+ "infer_datetime_format": form.infer_datetime_format.data,
+ "iterator": True,
+ "keep_default_na": not form.null_values.data,
+ "mangle_dupe_cols": form.mangle_dupe_cols.data,
+ "usecols": form.usecols.data,
Review comment:
added a scenario to `test_import_csv` that tests uploading a CSV with specific columns
##########
File path: superset/views/database/forms.py
##########
@@ -163,6 +165,15 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
_("Mangle Duplicate Columns"),
description=_('Specify duplicate columns as "X.0, X.1".'),
)
+ usecols = JsonListField(
+ _("Use Columns"),
+ default=None,
+ description=_(
+ "Json list of the column names that should be read. "
+ "If not None, only these columns will be read from the file."
+ ),
+ validators=[Optional()],
+ )
Review comment:
Added a screenshot to the summary above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-908972685
@villebro thx for pushing this through. yeah do tag me, i'll be sure to review. in the meantime i'll get started on #16046
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on a change in pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on a change in pull request #14449:
URL: https://github.com/apache/superset/pull/14449#discussion_r681865870
##########
File path: superset/views/database/forms.py
##########
@@ -402,3 +417,130 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
'Use [""] for empty string.'
),
)
+
+
+class ColumnarToDatabaseForm(DynamicForm):
+ # pylint: disable=E0211
+ def columnar_allowed_dbs() -> List[Database]: # type: ignore
+ # TODO: change allow_csv_upload to allow_file_upload
+ columnar_enabled_dbs = (
+ db.session.query(Database).filter_by(allow_csv_upload=True).all()
+ )
+ return [
+ columnar_enabled_db
+ for columnar_enabled_db in columnar_enabled_dbs
+ if ColumnarToDatabaseForm.at_least_one_schema_is_allowed(
+ columnar_enabled_db
+ )
+ ]
Review comment:
I just opened https://github.com/apache/superset/issues/16046. Feel free to assign it to me. I'll try to open a PR for it sometime in the next week
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (de3509e) into [master](https://codecov.io/gh/apache/superset/commit/ad8336a5b4acff0fb3dc1f9ec5d8954933ce9e6a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ad8336a) will **increase** coverage by `0.03%`.
> The diff coverage is `85.86%`.
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
+ Coverage 76.45% 76.49% +0.03%
==========================================
Files 1002 1002
Lines 53693 53780 +87
Branches 6859 6859
==========================================
+ Hits 41053 41139 +86
- Misses 12403 12404 +1
Partials 237 237
```
| Flag | Coverage Δ | |
|---|---|---|
| mysql | `81.49% <85.86%> (?)` | |
| postgres | `81.56% <85.86%> (+0.01%)` | :arrow_up: |
| python | `81.65% <85.86%> (+0.05%)` | :arrow_up: |
| sqlite | `81.17% <85.86%> (+0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.36% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.17% <100.00%> (+0.08%)` | :arrow_up: |
| [superset/views/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvY29yZS5weQ==) | `75.29% <100.00%> (+0.43%)` | :arrow_up: |
| [superset/models/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvbW9kZWxzL2NvcmUucHk=) | `89.14% <0.00%> (+0.77%)` | :arrow_up: |
| [superset/db\_engine\_specs/mysql.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL215c3FsLnB5) | `97.61% <0.00%> (+3.57%)` | :arrow_up: |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [ad8336a...de3509e](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-908974184
> @villebro thx for pushing this through. yeah do tag me, i'll be sure to review. in the meantime i'll get started on #16046
That sounds great @exemplary-citizen! Oh, and I forgot; thanks so much for your patience with the review process!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-891858416
@exemplary-citizen thanks for the ping! I restarted CI, I'll re-review today 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-891941273
I think the failure might be coming from https://github.com/apache/superset/pull/15909. Rebasing should fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (f209999) into [master](https://codecov.io/gh/apache/superset/commit/3456dd571ca705699766473e55ededd36c6f4dc7?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (3456dd5) will **decrease** coverage by `0.13%`.
> The diff coverage is `85.55%`.
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
- Coverage 77.54% 77.40% -0.14%
==========================================
Files 967 967
Lines 49759 49846 +87
Branches 6352 6352
==========================================
- Hits 38585 38583 -2
- Misses 10972 11061 +89
Partials 202 202
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.73% <85.55%> (+0.01%)` | :arrow_up: |
| postgres | `81.74% <85.55%> (+0.01%)` | :arrow_up: |
| python | `81.83% <85.55%> (-0.28%)` | :arrow_down: |
| sqlite | `81.37% <85.55%> (+0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/app.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvYXBwLnB5) | `87.76% <100.00%> (+0.08%)` | :arrow_up: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.18% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [superset/db\_engine\_specs/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2hpdmUucHk=) | `70.32% <0.00%> (-17.08%)` | :arrow_down: |
| [superset/views/database/mixins.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvbWl4aW5zLnB5) | `81.03% <0.00%> (-1.73%)` | :arrow_down: |
| [superset/db\_engine\_specs/presto.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL3ByZXN0by5weQ==) | `83.36% <0.00%> (-1.06%)` | :arrow_down: |
| [superset/db\_engine\_specs/base.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2Jhc2UucHk=) | `87.97% <0.00%> (-0.41%)` | :arrow_down: |
| [superset/connectors/sqla/models.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29ubmVjdG9ycy9zcWxhL21vZGVscy5weQ==) | `88.15% <0.00%> (-0.24%)` | :arrow_down: |
| ... and [1 more](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [3456dd5...f209999](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-844336253
@john-bodley @villebro just wanted to follow up on this in case it got lost in the shuffle. How should we proceed on this PR?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #14449:
URL: https://github.com/apache/superset/pull/14449#discussion_r637711057
##########
File path: superset/views/database/forms.py
##########
@@ -95,7 +95,9 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
validators=[
FileRequired(),
FileAllowed(
- config["ALLOWED_EXTENSIONS"].intersection(config["CSV_EXTENSIONS"]),
+ config["ALLOWED_EXTENSIONS"].intersection(
+ config["CSV_EXTENSIONS"].union(config["OTHER_EXTENSIONS"])
+ ),
Review comment:
You need to add the `union` to the error message below.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (016fbea) into [master](https://codecov.io/gh/apache/superset/commit/486ef6b81fb0eab39a3e403bb7fdba784c5e8649?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (486ef6b) will **decrease** coverage by `0.20%`.
> The diff coverage is `85.86%`.
> :exclamation: Current head 016fbea differs from pull request most recent head 6bcc0d9. Consider uploading reports for the commit 6bcc0d9 to get more accurate results
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
- Coverage 76.55% 76.35% -0.21%
==========================================
Files 1000 1000
Lines 53487 53574 +87
Branches 6818 6818
==========================================
- Hits 40948 40904 -44
- Misses 12301 12432 +131
Partials 238 238
```
| Flag | Coverage Δ | |
|---|---|---|
| hive | `?` | |
| mysql | `81.54% <85.86%> (+0.01%)` | :arrow_up: |
| postgres | `81.52% <85.86%> (-0.03%)` | :arrow_down: |
| presto | `?` | |
| python | `81.65% <85.86%> (-0.42%)` | :arrow_down: |
| sqlite | `81.17% <85.86%> (+0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.36% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.17% <100.00%> (+0.08%)` | :arrow_up: |
| [superset/views/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvY29yZS5weQ==) | `75.30% <100.00%> (ø)` | |
| [superset/db\_engines/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lcy9oaXZlLnB5) | `0.00% <0.00%> (-82.15%)` | :arrow_down: |
| [superset/db\_engine\_specs/hive.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL2hpdmUucHk=) | `69.80% <0.00%> (-16.87%)` | :arrow_down: |
| [superset/db\_engine\_specs/presto.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvZGJfZW5naW5lX3NwZWNzL3ByZXN0by5weQ==) | `83.47% <0.00%> (-6.91%)` | :arrow_down: |
| [superset/views/database/mixins.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvbWl4aW5zLnB5) | `81.03% <0.00%> (-1.73%)` | :arrow_down: |
| [superset/connectors/sqla/models.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29ubmVjdG9ycy9zcWxhL21vZGVscy5weQ==) | `88.04% <0.00%> (-1.66%)` | :arrow_down: |
| ... and [3 more](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [486ef6b...6bcc0d9](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-904211661
@villebro can you restart CI?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] villebro commented on a change in pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
villebro commented on a change in pull request #14449:
URL: https://github.com/apache/superset/pull/14449#discussion_r682372284
##########
File path: superset/views/database/forms.py
##########
@@ -402,3 +417,130 @@ def at_least_one_schema_is_allowed(database: Database) -> bool:
'Use [""] for empty string.'
),
)
+
+
+class ColumnarToDatabaseForm(DynamicForm):
+ # pylint: disable=E0211
+ def columnar_allowed_dbs() -> List[Database]: # type: ignore
+ # TODO: change allow_csv_upload to allow_file_upload
+ columnar_enabled_dbs = (
+ db.session.query(Database).filter_by(allow_csv_upload=True).all()
+ )
+ return [
+ columnar_enabled_db
+ for columnar_enabled_db in columnar_enabled_dbs
+ if ColumnarToDatabaseForm.at_least_one_schema_is_allowed(
+ columnar_enabled_db
+ )
+ ]
Review comment:
Thanks - I assigned to you (will keep myself as assignee for review)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-906541593
@villebro took care of the code formatting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] exemplary-citizen commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
exemplary-citizen commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-859305598
Not sure why CI threw a 502 error. @villebro can you enable CI again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] codecov[bot] edited a comment on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
codecov[bot] edited a comment on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-861184955
# [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report
> Merging [#14449](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (de3509e) into [master](https://codecov.io/gh/apache/superset/commit/ad8336a5b4acff0fb3dc1f9ec5d8954933ce9e6a?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (ad8336a) will **increase** coverage by `0.01%`.
> The diff coverage is `85.86%`.
[](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
```diff
@@ Coverage Diff @@
## master #14449 +/- ##
==========================================
+ Coverage 76.45% 76.47% +0.01%
==========================================
Files 1002 1002
Lines 53693 53737 +44
Branches 6859 6859
==========================================
+ Hits 41053 41097 +44
Misses 12403 12403
Partials 237 237
```
| Flag | Coverage Δ | |
|---|---|---|
| mysql | `81.49% <85.86%> (?)` | |
| postgres | `81.53% <85.86%> (-0.02%)` | :arrow_down: |
| python | `81.62% <85.86%> (+0.02%)` | :arrow_up: |
| sqlite | `81.17% <85.86%> (+0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Impacted Files](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | |
|---|---|---|
| [superset/views/database/forms.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2UvZm9ybXMucHk=) | `87.09% <81.81%> (-1.80%)` | :arrow_down: |
| [superset/views/database/views.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvZGF0YWJhc2Uvdmlld3MucHk=) | `88.46% <85.93%> (-1.02%)` | :arrow_down: |
| [superset/config.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29uZmlnLnB5) | `91.36% <100.00%> (+0.02%)` | :arrow_up: |
| [superset/initialization/\_\_init\_\_.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvaW5pdGlhbGl6YXRpb24vX19pbml0X18ucHk=) | `88.08% <100.00%> (ø)` | |
| [superset/views/core.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdmlld3MvY29yZS5weQ==) | `75.29% <100.00%> (+0.43%)` | :arrow_up: |
| [superset/models/dashboard.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvbW9kZWxzL2Rhc2hib2FyZC5weQ==) | `74.53% <0.00%> (-2.16%)` | :arrow_down: |
| [superset/utils/sqllab\_execution\_context.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvc3FsbGFiX2V4ZWN1dGlvbl9jb250ZXh0LnB5) | `89.70% <0.00%> (-0.44%)` | :arrow_down: |
| [superset/utils/decorators.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvZGVjb3JhdG9ycy5weQ==) | `92.59% <0.00%> (-0.27%)` | :arrow_down: |
| [superset/utils/cache.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvdXRpbHMvY2FjaGUucHk=) | `74.28% <0.00%> (-0.25%)` | :arrow_down: |
| [superset/common/query\_context.py](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c3VwZXJzZXQvY29tbW9uL3F1ZXJ5X2NvbnRleHQucHk=) | `90.52% <0.00%> (-0.22%)` | :arrow_down: |
| ... and [12 more](https://codecov.io/gh/apache/superset/pull/14449/diff?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | |
------
[Continue to review full report at Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Last update [ad8336a...de3509e](https://codecov.io/gh/apache/superset/pull/14449?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] github-actions[bot] commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-908969380
Ephemeral environment shutdown and build artifacts deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] github-actions[bot] commented on pull request #14449: feat: Add parquet upload
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-908969380
Ephemeral environment shutdown and build artifacts deleted.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org
[GitHub] [superset] robdiciuccio commented on pull request #14449: feat: Allow csv upload to accept parquet file
Posted by GitBox <gi...@apache.org>.
robdiciuccio commented on pull request #14449:
URL: https://github.com/apache/superset/pull/14449#issuecomment-845506817
/testenv up
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org