[jira] [Commented] (FLINK-5886) Python API for streaming applications

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/FLINK-5886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124914#comment-16124914 ] 

ASF GitHub Bot commented on FLINK-5886:
---------------------------------------

Github user zohar-mizrahi commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3838#discussion_r132841381
  
    --- Diff: flink-libraries/flink-streaming-python/pom.xml ---
    @@ -0,0 +1,104 @@
    +<!--
    +Licensed to the Apache Software Foundation (ASF) under one
    +or more contributor license agreements.  See the NOTICE file
    +distributed with this work for additional information
    +regarding copyright ownership.  The ASF licenses this file
    +to you under the Apache License, Version 2.0 (the
    +"License"); you may not use this file except in compliance
    +with the License.  You may obtain a copy of the License at
    +
    +  http://www.apache.org/licenses/LICENSE-2.0
    +
    +Unless required by applicable law or agreed to in writing,
    +software distributed under the License is distributed on an
    +"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
    +KIND, either express or implied.  See the License for the
    +specific language governing permissions and limitations
    +under the License.
    +-->
    +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    +		 xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    +	<modelVersion>4.0.0</modelVersion>
    +
    +	<parent>
    +		<groupId>org.apache.flink</groupId>
    +		<artifactId>flink-libraries</artifactId>
    +		<version>1.4-SNAPSHOT</version>
    +		<relativePath>..</relativePath>
    +	</parent>
    +
    +	<artifactId>flink-streaming-python_${scala.binary.version}</artifactId>
    +	<name>flink-streaming-python</name>
    +	<packaging>jar</packaging>
    +
    +	<build>
    +		<plugins>
    +			<plugin>
    +				<groupId>org.apache.maven.plugins</groupId>
    +				<artifactId>maven-jar-plugin</artifactId>
    +				<configuration>
    +					<descriptorRefs>
    +						<descriptorRef>jar-with-dependencies</descriptorRef>
    +					</descriptorRefs>
    +					<archive>
    +						<manifest>
    +							<addClasspath>true</addClasspath>
    +							<mainClass>org.apache.flink.streaming.python.api.PythonStreamBinder</mainClass>
    +						</manifest>
    +					</archive>
    +				</configuration>
    +			</plugin>
    +		</plugins>
    +	</build>
    +
    +	<dependencies>
    +
    +		<!-- core dependencies -->
    +
    +		<dependency>
    +			<groupId>org.apache.flink</groupId>
    +			<artifactId>flink-core</artifactId>
    +			<version>${project.version}</version>
    +            <scope>provided</scope>
    +		</dependency>
    +		<dependency>
    +			<groupId>org.apache.flink</groupId>
    +			<artifactId>flink-java</artifactId>
    +			<version>${project.version}</version>
    +            <scope>provided</scope>
    +		</dependency>
    +		<dependency>
    +			<groupId>org.apache.flink</groupId>
    +			<artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
    +			<version>${project.version}</version>
    +            <scope>provided</scope>
    +		</dependency>
    +		<dependency>
    +			<groupId>org.apache.flink</groupId>
    +			<artifactId>flink-runtime_${scala.binary.version}</artifactId>
    +			<version>${project.version}</version>
    +            <scope>provided</scope>
    +		</dependency>
    +		<dependency>
    +			<groupId>org.python</groupId>
    +			<artifactId>jython-standalone</artifactId>
    +			<version>2.7.0</version>
    +		</dependency>
    +		<dependency>
    +			<groupId>org.apache.flink</groupId>
    +			<artifactId>flink-connector-kafka-0.9_2.10</artifactId>
    --- End diff --
    
    Done for the Scala version. 
    
    As for not being able to run the tests:
    1. I fixed the issue with the ```TypeError: object of type 'java.lang.Class' has no len()```.
    2. I still can't reproduce the main issue, concerning an import of java class from the Python module:
    File: ```flink-libraries/flink-streaming-python/src/test/python/org/apache/flink/streaming/python/api/utils/python_test_base.py```
    Line:19: ```from org.apache.flink.api.java.utils import ParameterTool```
    
    The given class (ParameterTool) resides in different project `flink-java` and the jython module cannot find it. Probably, It somehow concerns the CLASSPATH. 
    
    Any suggestion for how to reproduce it?


> Python API for streaming applications
> -------------------------------------
>
>                 Key: FLINK-5886
>                 URL: https://issues.apache.org/jira/browse/FLINK-5886
>             Project: Flink
>          Issue Type: New Feature
>          Components: Python API
>            Reporter: Zohar Mizrahi
>            Assignee: Zohar Mizrahi
>
> A work in progress to provide python interface for Flink streaming APIs. The core technology is based on jython and thus imposes two limitations: a. user defined functions cannot use python extensions. b. the python version is 2.x
> The branch is based on Flink release 1.2.0, as can be found here:
> https://github.com/zohar-pm/flink/tree/python-streaming
> In order to test it, someone can use IntelliJ IDE. Assuming IntelliJ was setup properly (see: https://ci.apache.org/projects/flink/flink-docs-release-1.3/internals/ide_setup.html), one can run/debug {{org.apache.flink.python.api.PythonStreamBinderTest}}, which in return will execute all the tests under {{/Users/zohar/dev/pm-flink/flink-libraries/flink-python/src/test/python/org/apache/flink/python/api/streaming}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)