Superset 0.20.4, Druid: Problem with DimensionSpec (ExtractionFN with Regex)

[Alexander TRICHLIN <al...@r-software.at>]

Hi, 

we have problems with Superset when using ExtractionFunction in 
DimensionSpec.

The Query produced looks like this:

// Two phase query
// Phase 1
{
  "metric": "count", 
  "aggregations": [
    {
      "type": "count", 
      "name": "count"
    }
  ], 
  "dimension": {
    "outputName": "kundennummer", 
    "extractionFn": {
      "replaceMissingValue": true, 
      "expr": "kundennummer=([^&]*)", 
      "type": "regex", 
      "replaceMissingValueWith": "foobar", 
      "index": 1
    }, 
    "type": "extraction", 
    "dimension": "pageQuery", 
    "outputType": "STRING"
  }, 
  "intervals": "2017-10-19T00:00:00+00:00/2017-11-02T16:24:59+00:00", 
  "dataSource": "drb_persistence_supervisor", 
  "granularity": "all", 
  "threshold": 50000, 
  "postAggregations": [], 
  "queryType": "topN"
}

We tested this query with Postman and it worked fine. In Superset we get 
an Exception which seems to be in PHASE 2 (which is missing in the query 
view).

Traceback (most recent call last):
  File "/opt/venv/lib/python2.7/site-packages/superset/viz.py", line 274, 
in get_payload
    df = self.get_df()
  File "/opt/venv/lib/python2.7/site-packages/superset/viz.py", line 97, 
in get_df
    self.results = self.datasource.query(query_obj)
  File 
"/opt/venv/lib/python2.7/site-packages/superset/connectors/druid/models.py", 
line 1069, in query
    client=client, query_obj=query_obj, phase=2)
  File 
"/opt/venv/lib/python2.7/site-packages/superset/connectors/druid/models.py", 
line 865, in get_query_str
    return self.run_query(client=client, phase=phase, **query_obj)
  File 
"/opt/venv/lib/python2.7/site-packages/superset/connectors/druid/models.py", 
line 1006, in run_query
    qry['dimensions'], filters)
  File 
"/opt/venv/lib/python2.7/site-packages/superset/connectors/druid/models.py", 
line 874, in _add_filter_from_pre_query_data
    f = Dimension(dim) == row[dim]
  File "/opt/venv/lib/python2.7/site-packages/pandas/core/series.py", line 
642, in __getitem__
    return self._get_with(key)
  File "/opt/venv/lib/python2.7/site-packages/pandas/core/series.py", line 
683, in _get_with
    return self.loc[key]
  File "/opt/venv/lib/python2.7/site-packages/pandas/core/indexing.py", 
line 1328, in __getitem__
    return self._getitem_axis(key, axis=0)
  File "/opt/venv/lib/python2.7/site-packages/pandas/core/indexing.py", 
line 1541, in _getitem_axis
    return self._getitem_iterable(key, axis=axis)
  File "/opt/venv/lib/python2.7/site-packages/pandas/core/indexing.py", 
line 1081, in _getitem_iterable
    self._has_valid_type(key, axis)
  File "/opt/venv/lib/python2.7/site-packages/pandas/core/indexing.py", 
line 1418, in _has_valid_type
    (key, self.obj._get_axis_name(axis)))
KeyError: "None of [[u'outputName', u'extractionFn', u'type', 
u'dimension', u'outputType']] are in the [index]"


Why is this happening? 




----------------------------------------
Raiffeisen Software GmbH
Firmenbuchnummer: 86804d, Landesgericht Linz
DVR: 0572934, UID: ATU 36961105

Der Austausch von Nachrichten mit dem oben angeführten Absender via E-Mail dient ausschließlich Informationszwecken. Rechtsgeschäftliche Erklärungen dürfen über dieses Medium nicht ausgetauscht werden.
Correspondence with above mentioned sender via e-mail is only for information purposes. This medium is not to be used for the exchange of legally-binding communications.
----------------------------------------