[GitHub] betodealmeida opened a new pull request #6863: [SIP-16] Proposal for loading additional polygon encodings

[GitBox <gi...@apache.org>]

betodealmeida opened a new pull request #6863: [SIP-16] Proposal for loading additional polygon encodings
URL: https://github.com/apache/incubator-superset/pull/6863
 
 
   This is work in progress, just as a reference.
   
   This PR implements https://github.com/apache/incubator-superset/issues/6814. It includes some work to bring configuration associated with a given feature flag, other than a boolean (cc: @xtinec) that should be implemented in a separate PR.
   
   The PR is also missing the relevant documentation, which will be added once the SIP has been approved.
   
   As a test, I add US ZIP codes to my `superset_config`:
   
   ```python
   class USZipCodes(PolygonEncoding):
   
       name = 'US ZIP codes'
   
       @staticmethod
       def to_polygon(codes, cache=None):
           user = os.environ.get('CREDENTIALS_LYFTPG_USER', '')
           password = os.environ.get('CREDENTIALS_LYFTPG_PASSWORD', '')
   
           url = (
               'postgresql+psycopg2://'
               '{user}:{password}@'
               'REDACTED:5432'
               '/platform'.format(user=user, password=password)
           )
   
           out = {}
           missing = set()
           for code in codes:
               cache_key = 'zipcode_geojson_{}'.format(code)
               geojson = cache and cache.get(cache_key)
               if geojson:
                   out[code] = geojson
               else:
                   missing.add(str(int(code)))
   
           if not missing:
               return out
   
           # fetch missing geojson from lyftpg
           in_clause = ', '.join(['%s'] * len(missing))
           query = (
               'SELECT zipcode, geojson FROM zip_codes WHERE zipcode IN ({0})'
               .format(in_clause))
           conn = sqlalchemy.create_engine(url, client_encoding='utf8')
           results = conn.execute(query, tuple(missing)).fetchall()
   
           for code, geojson in results:
               out[code] = geojson
               if cache and len(results) < 10000:  # avoid storing too much
                   cache_key = 'zipcode_geojson_{}'.format(code)
                   try:
                       cache.set(cache_key, geojson, timeout=86400)
                   except Exception:
                       pass
   
           return out
   
       @classmethod
       def get_deser(cls, codes):
           polygons = cls.to_polygon(codes)
   
           def deser(code):
               return polygons[str(int(code))]['coordinates'][0]
   
           return deser
   
   
   FEATURE_FLAGS = {
       'EXTRA_POLYGON_ENCODINGS': True,
   }
   
   CONF_KEYS = {
       'EXTRA_POLYGON_ENCODINGS': [USZipCodes],
   }
   ```
   
   Here's the result:
   
   <img width="1698" alt="screen shot 2019-02-12 at 5 48 07 pm" src="https://user-images.githubusercontent.com/1534870/52681017-4f0fae80-2eef-11e9-81fd-ce432bb15994.png">
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: notifications-unsubscribe@superset.apache.org
For additional commands, e-mail: notifications-help@superset.apache.org