You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2021/06/29 10:51:32 UTC

[GitHub] [pulsar] HistoryGift opened a new issue #11147: flume io connector can not run normally

HistoryGift opened a new issue #11147:
URL: https://github.com/apache/pulsar/issues/11147


   **Describe the bug**
   when flume io connector run , Exception: 
   java.lang.IllegalArgumentException: Source class org.apache.pulsar.io.flume.sink.SourceOfFlume does not implement the correct interface
   
   **To Reproduce**
   Steps to reproduce the behavior:
   1. I am running Pulsar in docker with apachepulsar/pulsar-all image
   2.  #vim flume-connector.yaml
       configs:
          name: a1
          confFile: source.conf
          noReloadConf: false
          zkConnString: ""
          zkBasePath: ""
   
   3. #vim source.conf
       a1.sources = r1
   a1.sinks = k1
   a1.channels = c1
   
   # Describe/configure the source
   a1.sources.r1.type = org.apache.pulsar.io.flume.sink.SourceOfFlume
   
   # Describe the sink
   a1.sinks.k1.type = avro
   a1.sinks.k1.hostname = 127.0.0.1
   a1.sinks.k1.port = 44444
   # Use a channel which buffers events in memory
   a1.channels.c1.type = memory
   a1.channels.c1.capacity = 1000
   a1.channels.c1.transactionCapacity = 1000
   
   # Bind the source and sink to the channel
   a1.sources.r1.channels = c1
   
   4. run command 
   #bin/pulsar-admin sources localrun --archive connectors/pulsar-io-flume-2.8.0.nar  --classname org.apache.pulsar.io.flume.sink.SourceOfFlume   --name flume-test --destination-topic-name    my-topic --source-config-file /pulsar/flume-connector.yaml
   
   then I can see the exception "java.lang.IllegalArgumentException: Source class org.apache.pulsar.io.flume.sink.SourceOfFlume does not implement the correct interface" in console
   
   What do I need to do ?
   And there is another question "Isn't pulsar flume io connector supported taildirSource ?"
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] lhotari commented on issue #11147: flume io connector can not run normally

Posted by GitBox <gi...@apache.org>.
lhotari commented on issue #11147:
URL: https://github.com/apache/pulsar/issues/11147#issuecomment-871939876


   The stack trace shows that Pulsar 2.7.1 is used to run the Pulsar Flume connector that is of version 2.8.0 . @HistoryGift Please use a connector version that matches the Pulsar version.
   For Pulsar 2.7.1, the Flume connector is https://www.apache.org/dyn/mirrors/mirrors.cgi?action=download&filename=pulsar/pulsar-2.7.1/connectors/pulsar-io-flume-2.7.1.nar


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] HistoryGift commented on issue #11147: flume io connector can not run normally

Posted by GitBox <gi...@apache.org>.
HistoryGift commented on issue #11147:
URL: https://github.com/apache/pulsar/issues/11147#issuecomment-870571748


   I download and copy pulsar-flume-io.jar to lib,I get a new issue "JavaInstanceRunnable - Encountered exception in sink write: 
   java.lang.NullPointerException: null" 
   
   Exception description: 
   JavaInstanceRunnable - Encountered exception in sink write: 
   java.lang.NullPointerException: null
           at org.apache.pulsar.io.flume.source.AbstractSource$FlumeRecord.getKey(AbstractSource.java:121) ~[pulsar-io-flume-2.8.0.jar:?]
           at org.apache.pulsar.functions.instance.SinkRecord.getKey(SinkRecord.java:53) ~[org.apache.pulsar-pulsar-functions-instance-2.7.1.jar:2.7.1]
           at org.apache.pulsar.functions.sink.PulsarSink.write(PulsarSink.java:354) ~[org.apache.pulsar-pulsar-functions-instance-2.7.1.jar:2.7.1]
           at org.apache.pulsar.functions.instance.JavaInstanceRunnable.sendOutputMessage(JavaInstanceRunnable.java:372) ~[org.apache.pulsar-pulsar-functions-instance-2.7.1.jar:2.7.1]
           at org.apache.pulsar.functions.instance.JavaInstanceRunnable.lambda$processResult$0(JavaInstanceRunnable.java:354) ~[org.apache.pulsar-pulsar-functions-instance-2.7.1.jar:2.7.1]
           at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) [?:1.8.0_291]
           at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) [?:1.8.0_291]
           at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) [?:1.8.0_291]
           at org.apache.pulsar.functions.instance.JavaInstanceRunnable.processResult(JavaInstanceRunnable.java:345) [org.apache.pulsar-
   pulsar-functions-instance-2.7.1.jar:2.7.1]
           at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:269) [org.apache.pulsar-pulsar-fun
   ctions-instance-2.7.1.jar:2.7.1]
   
   
   I look the code in Record.java
       /**
        * Return a key if the key has one associated.
        */
       default Optional<String> getKey() {
           return Optional.empty();  //
       }
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] HistoryGift closed issue #11147: flume io connector can not run normally

Posted by GitBox <gi...@apache.org>.
HistoryGift closed issue #11147:
URL: https://github.com/apache/pulsar/issues/11147


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [pulsar] HistoryGift commented on issue #11147: flume io connector can not run normally

Posted by GitBox <gi...@apache.org>.
HistoryGift commented on issue #11147:
URL: https://github.com/apache/pulsar/issues/11147#issuecomment-870571748


   I download and copy pulsar-flume-io.jar to lib,I get a new issue "JavaInstanceRunnable - Encountered exception in sink write: 
   java.lang.NullPointerException: null" 
   
   Exception description: 
   JavaInstanceRunnable - Encountered exception in sink write: 
   java.lang.NullPointerException: null
           at org.apache.pulsar.io.flume.source.AbstractSource$FlumeRecord.getKey(AbstractSource.java:121) ~[pulsar-io-flume-2.8.0.jar:?]
           at org.apache.pulsar.functions.instance.SinkRecord.getKey(SinkRecord.java:53) ~[org.apache.pulsar-pulsar-functions-instance-2.7.1.jar:2.7.1]
           at org.apache.pulsar.functions.sink.PulsarSink.write(PulsarSink.java:354) ~[org.apache.pulsar-pulsar-functions-instance-2.7.1.jar:2.7.1]
           at org.apache.pulsar.functions.instance.JavaInstanceRunnable.sendOutputMessage(JavaInstanceRunnable.java:372) ~[org.apache.pulsar-pulsar-functions-instance-2.7.1.jar:2.7.1]
           at org.apache.pulsar.functions.instance.JavaInstanceRunnable.lambda$processResult$0(JavaInstanceRunnable.java:354) ~[org.apache.pulsar-pulsar-functions-instance-2.7.1.jar:2.7.1]
           at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) [?:1.8.0_291]
           at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(CompletableFuture.java:792) [?:1.8.0_291]
           at java.util.concurrent.CompletableFuture.whenComplete(CompletableFuture.java:2153) [?:1.8.0_291]
           at org.apache.pulsar.functions.instance.JavaInstanceRunnable.processResult(JavaInstanceRunnable.java:345) [org.apache.pulsar-
   pulsar-functions-instance-2.7.1.jar:2.7.1]
           at org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:269) [org.apache.pulsar-pulsar-fun
   ctions-instance-2.7.1.jar:2.7.1]
   
   
   I look the code in Record.java
       /**
        * Return a key if the key has one associated.
        */
       default Optional<String> getKey() {
           return Optional.empty();  //
       }
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org