[jira] [Created] (BAHIR-122) [PubSub] Make "ServiceAccountCredentials" really broadcastable

["Ire Sun (JIRA)" <ji...@apache.org>]

Ire Sun created BAHIR-122:
-----------------------------

             Summary: [PubSub] Make "ServiceAccountCredentials" really broadcastable
                 Key: BAHIR-122
                 URL: https://issues.apache.org/jira/browse/BAHIR-122
             Project: Bahir
          Issue Type: Improvement
          Components: Spark Streaming Connectors
            Reporter: Ire Sun


The origin implementation broadcast the key file path to Spark cluster, then the executor read key file with the broadcasted path. Which is absurd, if you are using a shared Spark cluster in a group/company, you certainly not want to (and have no right to) put your key file on each instance of the cluster.

If you store the key file on driver node and submit your job to a remote cluster. You would get the following warning:
{{WARN ReceiverTracker: Error reported by receiver for stream 0: Failed to pull messages - java.io.FileNotFoundException}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)