[GitHub] [flink] dianfu commented on a change in pull request #12903: [FLINK-18492][python] Extract the implementation logic of Beam in Operations

[GitBox <gi...@apache.org>]

dianfu commented on a change in pull request #12903:
URL: https://github.com/apache/flink/pull/12903#discussion_r456265064



##########
File path: flink-python/pyflink/fn_execution/beam/beam_operations.pyx
##########
@@ -0,0 +1,186 @@
+################################################################################
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+# cython: language_level = 3
+# cython: infer_types = True
+# cython: profile=True
+# cython: boundscheck=False, wraparound=False, initializedcheck=False, cdivision=True
+

Review comment:
       What about change the naming conversion as following:
   ```
   beam_slow_operations.py -> beam_operations_slow.py
   beam_operations.pyd/beam_operations.pyx -> beam_operations_fast.pyd/beam_operations_fast.pyx
   ```
   
   Need also update the coder classes.

##########
File path: flink-python/pyflink/fn_execution/operations_utils.py
##########
@@ -0,0 +1,111 @@
+################################################################################

Review comment:
       rename to operation_utils

##########
File path: flink-python/pyflink/fn_execution/fast_operations.pyx
##########
@@ -18,101 +18,34 @@
 # cython: infer_types = True
 # cython: profile=True
 # cython: boundscheck=False, wraparound=False, initializedcheck=False, cdivision=True
+cimport libc.stdint
 
 import datetime
 
 import cloudpickle
-from apache_beam.runners.worker import bundle_processor
-from apache_beam.runners.worker import operation_specs
-from apache_beam.utils.windowed_value cimport WindowedValue
 
-from pyflink.fn_execution.beam.beam_stream cimport BeamInputStream, BeamOutputStream
-from pyflink.fn_execution.beam.beam_coder_impl cimport InputStreamWrapper
-from pyflink.fn_execution import flink_fn_execution_pb2
-from pyflink.metrics.metricbase import GenericMetricGroup
 from pyflink.serializers import PickleSerializer
-from pyflink.table import FunctionContext
 from pyflink.table.udf import DelegatingScalarFunction, DelegationTableFunction
 
 SCALAR_FUNCTION_URN = "flink:transform:scalar_function:v1"
 TABLE_FUNCTION_URN = "flink:transform:table_function:v1"
 
-cdef class StatelessFunctionOperation(Operation):
+cdef class UserDefinedFunctionOperation:
+    pass
+
+cdef class StatelessFunctionOperation(UserDefinedFunctionOperation):

Review comment:
       Move to a utils class such as `operation_utils`. Besides, these methods are not performance critical and there is no need to use Cython.

##########
File path: flink-python/pyflink/fn_execution/operations_utils.py
##########
@@ -0,0 +1,111 @@
+################################################################################
+#  Licensed to the Apache Software Foundation (ASF) under one
+#  or more contributor license agreements.  See the NOTICE file
+#  distributed with this work for additional information
+#  regarding copyright ownership.  The ASF licenses this file
+#  to you under the Apache License, Version 2.0 (the
+#  "License"); you may not use this file except in compliance
+#  with the License.  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+# limitations under the License.
+################################################################################
+import datetime
+
+import cloudpickle
+
+from pyflink.serializers import PickleSerializer
+from pyflink.table.udf import DelegationTableFunction, DelegatingScalarFunction
+
+SCALAR_FUNCTION_URN = "flink:transform:scalar_function:v1"
+TABLE_FUNCTION_URN = "flink:transform:table_function:v1"
+
+
+class FunctionFactory:

Review comment:
       There is no need to wrap the functions in a class.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org