You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (JIRA)" <ji...@apache.org> on 2016/11/27 04:51:58 UTC
[jira] [Updated] (SPARK-18593) Unable to use filter function if the
data length is one.
[ https://issues.apache.org/jira/browse/SPARK-18593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun updated SPARK-18593:
----------------------------------
Affects Version/s: 1.6.3
> Unable to use filter function if the data length is one.
> --------------------------------------------------------
>
> Key: SPARK-18593
> URL: https://issues.apache.org/jira/browse/SPARK-18593
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.2, 1.6.3
> Reporter: Durga Prasad Gunturu
> Priority: Minor
>
> when using filter using jdbc.. unable to retrieve data if the column value is having only value of length one.
> for example:
> Below code works because the filter condition data is of length > 1
> val dataFromTable = sqlContext.read.jdbc("jdbc:postgresql://localhost:5433/postgres", "stocknames", defaultProps)
> dataFromTable.filter(dataFromTable("grouptype")==="ZP").show()
> result:
> +------------+----------+--------------------+------+---------+---------+------------+--------------------+----------+
> |securitycode|securityid| securityname|status|grouptype|facevalue| isinno| industry|instrument|
> +------------+----------+--------------------+------+---------+---------+------------+--------------------+----------+
> | 504370| ZARDIINV|ARDI INVESTMENT &...|Active| ZP| 10.0| |Misc.Commercial S...| Equity|
> | 512221| TARCF|TARRIF CINE &...|Active| ZP| 10.0|INE177E01010|Finance (includin...| Equity|
> | 512505| MEENST|MEENAKSHI STEEL I...|Active| ZP| 10.0| |Iron & Steel ...| Equity|
> | 514060| EVERTEX|Evergreen Textile...|Active| ZP| 10.0|INE229N01010| Textiles| Equity|
> | 526085| SGARRES|SAGAR TOURIST RES...|Active| ZP| 10.0|NA | Hotels| Equity|
> +------------+----------+--------------------+------+---------+---------+------------+--------------------+----------+
> But if the same filter condition code is of length 1, no data is filtered.
> eg:
> val dataFromTable = sqlContext.read.jdbc("jdbc:postgresql://localhost:5433/postgres", "stocknames", defaultProps)
> dataFromTable.filter(dataFromTable("grouptype")==="A").show()
> not getting any output
> +------------+----------+------------+------+---------+---------+------+--------+----------+
> |securitycode|securityid|securityname|status|grouptype|facevalue|isinno|industry|instrument|
> +------------+----------+------------+------+---------+---------+------+--------+----------+
> +------------+----------+------------+------+---------+---------+------+--------+----------+
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org