You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@spark.apache.org by ue...@apache.org on 2018/08/02 18:18:56 UTC
spark git commit: [BUILD] Fix lint-python.
Repository: spark
Updated Branches:
refs/heads/master 38e4699c9 -> 0df6bf882
[BUILD] Fix lint-python.
## What changes were proposed in this pull request?
This pr fixes lint-python.
```
./python/pyspark/accumulators.py:231:9: E306 expected 1 blank line before a nested definition, found 0
./python/pyspark/accumulators.py:257:101: E501 line too long (107 > 100 characters)
./python/pyspark/accumulators.py:264:1: E302 expected 2 blank lines, found 1
./python/pyspark/accumulators.py:281:1: E302 expected 2 blank lines, found 1
```
## How was this patch tested?
Executed lint-python manually.
Author: Takuya UESHIN <ue...@databricks.com>
Closes #21973 from ueshin/issues/build/1/fix_lint-python.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/0df6bf88
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/0df6bf88
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/0df6bf88
Branch: refs/heads/master
Commit: 0df6bf882907d7d76572f513168a144067d0e0ec
Parents: 38e4699
Author: Takuya UESHIN <ue...@databricks.com>
Authored: Fri Aug 3 03:18:46 2018 +0900
Committer: Takuya UESHIN <ue...@databricks.com>
Committed: Fri Aug 3 03:18:46 2018 +0900
----------------------------------------------------------------------
python/pyspark/accumulators.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/0df6bf88/python/pyspark/accumulators.py
----------------------------------------------------------------------
diff --git a/python/pyspark/accumulators.py b/python/pyspark/accumulators.py
index 1276c31..30ad042 100644
--- a/python/pyspark/accumulators.py
+++ b/python/pyspark/accumulators.py
@@ -228,6 +228,7 @@ class _UpdateRequestHandler(SocketServer.StreamRequestHandler):
def handle(self):
from pyspark.accumulators import _accumulatorRegistry
auth_token = self.server.auth_token
+
def poll(func):
while not self.server.server_shutdown:
# Poll every 1 second for new data -- don't block in case of shutdown.
@@ -254,13 +255,15 @@ class _UpdateRequestHandler(SocketServer.StreamRequestHandler):
# we've authenticated, we can break out of the first loop now
return True
else:
- raise Exception("The value of the provided token to the AccumulatorServer is not correct.")
+ raise Exception(
+ "The value of the provided token to the AccumulatorServer is not correct.")
# first we keep polling till we've received the authentication token
poll(authenticate_and_accum_updates)
# now we've authenticated, don't need to check for the token anymore
poll(accum_updates)
+
class AccumulatorServer(SocketServer.TCPServer):
def __init__(self, server_address, RequestHandlerClass, auth_token):
@@ -278,6 +281,7 @@ class AccumulatorServer(SocketServer.TCPServer):
SocketServer.TCPServer.shutdown(self)
self.server_close()
+
def _start_update_server(auth_token):
"""Start a TCP server to receive accumulator updates in a daemon thread, and returns it"""
server = AccumulatorServer(("localhost", 0), _UpdateRequestHandler, auth_token)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org