You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/10/07 12:01:55 UTC

[GitHub] [pulsar] kjaggann opened a new issue #8212: spark Java

kjaggann opened a new issue #8212:
URL: https://github.com/apache/pulsar/issues/8212


   **Is your enhancement request related to a problem? Please describe.**
   Need help in configuring the tls certs in spark java for consumer code
   
   **Describe the solution you'd like**
   Sample sparkReceiver in java but unable to configure the tls certs 
   String serviceUrl =  "pulsar+ssl://abc-wss-us-east-1-non.abc.com:1234/";
       String inputTopic = "non-persistent://gdfvyv/test-test/ConsumerApproval-Test-Test-3m";
       String subscription = "vmb_test";
       System.out.println("Parameters:");
       System.out.println("\tServiceUrl:\t" + serviceUrl);
       System.out.println("\tTopic:\t" + inputTopic);
       System.out.println("\tSubscription:\t" + subscription);
       SparkConf sparkConf = new SparkConf().setAppName("Pulsar Spark Example");
       JavaStreamingContext jsc = new JavaStreamingContext(sparkConf, Durations.seconds(60));
       ConsumerConfigurationData<byte[]> pulsarConf = new ConsumerConfigurationData();
       Set<String> set = new HashSet<String>();
       set.add(inputTopic);
   pulsarConf.setTopicNames(set);
       pulsarConf.setSubscriptionName(subscription);
       SparkStreamingPulsarReceiver pulsarReceiver = new SparkStreamingPulsarReceiver(
           serviceUrl,
           pulsarConf,
           new AuthenticationDisabled());
       JavaReceiverInputDStream<byte[]> lineDStream = jsc.receiverStream(pulsarReceiver);
       JavaPairDStream<String, Integer> result = lineDStream.flatMap(Consumer::call)
           .mapToPair(x -> new Tuple2<String, Integer>((String) x, 1))
           .reduceByKey((x, y) -> x + y);
       result.print();
       jsc.start();
       jsc.awaitTermination();
   
   **Describe alternatives you've considered**
   Nothing yet stuck with it 
   
   **Additional context**
   Add any other context or screenshots about the feature request here.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org