You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2016/01/14 10:13:40 UTC
[jira] [Commented] (FLINK-2984) Support lenient parsing of SVMLight
input files
[ https://issues.apache.org/jira/browse/FLINK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097860#comment-15097860 ]
ASF GitHub Bot commented on FLINK-2984:
---------------------------------------
GitHub user chiwanpark opened a pull request:
https://github.com/apache/flink/pull/1504
[FLINK-2984] [ml] Extend libSVM file format support
Currently, FlinkML libSVM file reader cannot read some datasets (such as splice-site) in libSVM official site. This PR extends libSVM file format to liblinear equivalent. To avoid re-compiling regex pattern in every line, I changed the mapper from scala lambda function to `RichFlatMapFunction`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chiwanpark/flink FLINK-2984
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1504.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1504
----
commit 200e8335ff157cb621fa9d385ebb36531de3bd68
Author: Chiwan Park <ch...@apache.org>
Date: 2016-01-13T08:34:11Z
[FLINK-2984] [ml] Extend libSVM file format support
----
> Support lenient parsing of SVMLight input files
> -----------------------------------------------
>
> Key: FLINK-2984
> URL: https://issues.apache.org/jira/browse/FLINK-2984
> Project: Flink
> Issue Type: Improvement
> Components: Machine Learning Library
> Affects Versions: 0.9.1
> Reporter: Johannes
> Assignee: Chiwan Park
> Priority: Trivial
> Labels: easyfix
>
> The current implementation for the reader assumes that the format follows the exact specification.
> The [splice-site Dataset| https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html#splice-site] dataset is formatted slightly different
> Example
> {noformat}
> -1 1:0.381846 2:0.163648 3:0.245472 4:0.627318
> {noformat}
> note the two spaces after the label.
> Currently MLUtils.scala splits on single spaces.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)