You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@spark.apache.org by "thierry accart (Jira)" <ji...@apache.org> on 2019/12/11 16:59:00 UTC

[jira] [Created] (SPARK-30224) Configurable field and record separators for filters

thierry accart created SPARK-30224:
--------------------------------------

             Summary: Configurable field and record separators for filters
                 Key: SPARK-30224
                 URL: https://issues.apache.org/jira/browse/SPARK-30224
             Project: Spark
          Issue Type: Improvement
          Components: Web UI
    Affects Versions: 2.4.4
            Reporter: thierry accart


The method addfilter of JettyUtils is parsing sparkConf using strict field and record separator: at this time, it's not possible to have = or , signs in a parameter value. This brings impossibility to configure for exampe Hadoop's LDAP authenticator filter, as some of its parameters contain comma and equal signs.

To fix this problem, there are two solutions:

either we allow spark.<filter>.params.<key>=<value> in sparkConf
or 
we allow spark.<filter>.params.rs=<record separator char> and spark.<filter>.params.fs=<field separator char> 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org