[GitHub] mistercrunch closed pull request #6540: Make boto3/botocore installation optional

[GitBox <gi...@apache.org>]

mistercrunch closed pull request #6540: Make boto3/botocore installation optional
URL: https://github.com/apache/incubator-superset/pull/6540
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/UPDATING.md b/UPDATING.md
index af8561fffd..97ed71082a 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -4,7 +4,11 @@ This file documents any backwards-incompatible changes in Superset and
 assists people when migrating to a new version.
 
 ## Superset 0.31.0
-From 0.31.0 onwards, we recommend not using the npm package `yarn` in
+* boto3 / botocore was removed from the dependency list. If you use s3
+as a place to store your SQL Lab result set or Hive uploads, you may
+have to rely on an alternate requirements.txt file to install those
+dependencies.
+* From 0.31.0 onwards, we recommend not using the npm package `yarn` in
 favor of good old `npm install`. While yarn should still work just fine,
 you should probably align to guarantee builds similar to the ones we
 use in testing and across the community in general.
diff --git a/requirements.txt b/requirements.txt
index b728bf0a16..0b3c67b70c 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -10,8 +10,6 @@ asn1crypto==0.24.0        # via cryptography
 babel==2.6.0              # via flask-babel, flower
 billiard==3.5.0.4         # via celery
 bleach==3.0.2
-boto3==1.4.7
-botocore==1.7.48
 cachetools==3.0.0         # via google-auth
 cchardet==1.0.0           # via tabulator
 celery==4.2.0
@@ -25,7 +23,6 @@ croniter==0.3.25
 cryptography==1.9
 decorator==4.3.0          # via retry
 defusedxml==0.5.0         # via python3-openid
-docutils==0.14            # via botocore
 et-xmlfile==1.0.1         # via openpyxl
 flask-appbuilder==1.12.1
 flask-babel==0.11.1       # via flask-appbuilder
@@ -50,7 +47,6 @@ isodate==0.6.0
 itsdangerous==0.24        # via flask
 jdcal==1.4                # via openpyxl
 jinja2==2.10              # via flask, flask-babel
-jmespath==0.9.3           # via boto3, botocore
 jsonlines==1.2.0          # via tabulator
 jsonschema==2.6.0         # via tableschema
 kombu==4.2.1              # via celery
@@ -83,7 +79,6 @@ requests==2.20.0
 retry==0.9.2
 rfc3986==1.1.0            # via tableschema
 rsa==4.0                  # via google-auth
-s3transfer==0.1.13        # via boto3
 sasl==0.2.1               # via thrift-sasl
 selenium==3.14.0
 simplejson==3.15.0
diff --git a/setup.py b/setup.py
index b6ca7983ce..53804ad092 100644
--- a/setup.py
+++ b/setup.py
@@ -54,8 +54,6 @@ def get_git_sha():
     scripts=['superset/bin/superset'],
     install_requires=[
         'bleach>=3.0.2, <4.0.0',
-        'boto3>=1.4.7, <2.0.0',
-        'botocore>=1.7.0, <1.8.0',
         'celery>=4.2.0, <5.0.0',
         'click>=6.0, <7.0.0',  # click >=7 forces "-" instead of "_"
         'colorama',
diff --git a/superset/db_engine_specs.py b/superset/db_engine_specs.py
index 97b6439a36..3c09de5c90 100644
--- a/superset/db_engine_specs.py
+++ b/superset/db_engine_specs.py
@@ -20,7 +20,6 @@
 import textwrap
 import time
 
-import boto3
 from flask import g
 from flask_babel import lazy_gettext as _
 import pandas
@@ -1065,6 +1064,9 @@ def convert_to_hive_type(col_type):
                     convert_to_hive_type(column_info['type'])))
         schema_definition = ', '.join(column_name_and_type)
 
+        # Optional dependency
+        import boto3  # pylint: disable=import-error
+
         s3 = boto3.client('s3')
         location = os.path.join('s3a://', bucket_path, upload_prefix, table_name)
         s3.upload_file(


 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org