You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by GitBox <gi...@apache.org> on 2020/04/09 14:37:37 UTC

[GitHub] [pulsar] itaross opened a new issue #6701: Cannot create Pulsar-IO connector

itaross opened a new issue #6701: Cannot create Pulsar-IO connector
URL: https://github.com/apache/pulsar/issues/6701
 
 
   **Describe the bug**
   Ni-fi maven plugin is not copying /resources/META-INF/services/pulsar-io.yaml
   Pulsar cannot load the connector because a yaml file is not inside the .nar archive
   
   **To Reproduce**
   Steps to reproduce the behavior:
   1.  Clone an existing pulsar-io connector
   2. mvn compile
   3. Use nar plugin to generate .nar
   4. Copy the nar under connectors folder
   5. Type the command: pulsar-admin sink reload
   6. Type the command: pulsar-admin sinks available-sinks
   
   **Expected behavior**
   Should see the plug-in available
   
   **Screenshots**
   ![image](https://user-images.githubusercontent.com/695970/78906950-4e859000-7a80-11ea-982b-dad685ee82ba.png)
   
   **Desktop (please complete the following information):**
    - macOS
   
   **Additional context**
   Add any other context about the problem here.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] itaross commented on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin

Posted by GitBox <gi...@apache.org>.
itaross commented on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin
URL: https://github.com/apache/pulsar/issues/6701#issuecomment-611734506
 
 
   I do know if it is too specific, but I'd love too see it in the official doc here: https://pulsar.apache.org/docs/en/io-develop/

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] itaross commented on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin

Posted by GitBox <gi...@apache.org>.
itaross commented on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin
URL: https://github.com/apache/pulsar/issues/6701#issuecomment-612372347
 
 
   > @itaross it seems like a bug. Do you want to contribute a fix?
   
   I am not quite sure it is a bug from Pulsar side.
   The Pulsar API expects to find "pulsar-io.yaml" inside the .nar archive, but when building it is not copied under target/classes/META-INF/services.
   From Ni-Fi plugin side I understand that they could not be aware that it is required to copy that pulsar-io.yaml. Yesterday I digged down a bit and I have found that on apache/nifi-maven there is this method: 
   https://github.com/apache/nifi-maven/blob/b81db233a200847998a57ac9b1f5b6218ab66a1b/src/main/java/org/apache/nifi/extension/definition/extraction/ExtensionDefinitionFactory.java#L104
   I am not quite sure if it is referred to the target folder or resources under src.
   
   Anyway, the use of that maven goal resources:resources I came up with overcomes the missing file in the target directory.
   So a correct maven pipeline is:
   ```
   mvn clean compile resources:resources nifi-nar:nar
   ```
   
   I don't know if this is really a bug or not, if you can suggest one I can try to fix it.
   For now I think the first action is to update the documentation with the use of this resources plugin for maven.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] itaross edited a comment on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin

Posted by GitBox <gi...@apache.org>.
itaross edited a comment on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin
URL: https://github.com/apache/pulsar/issues/6701#issuecomment-612372347
 
 
   > @itaross it seems like a bug. Do you want to contribute a fix?
   
   I am not quite sure it is a bug from Pulsar side.
   The Pulsar API expects to find "pulsar-io.yaml" inside the .nar archive, but when building it is not copied under target/classes/META-INF/services.
   From Ni-Fi plugin side I understand that they could not be aware that it is required to copy that pulsar-io.yaml. Yesterday I digged down a bit and I have found that on apache/nifi-maven there is this method: 
   https://github.com/apache/nifi-maven/blob/b81db233a200847998a57ac9b1f5b6218ab66a1b/src/main/java/org/apache/nifi/extension/definition/extraction/ExtensionDefinitionFactory.java#L104
   I am not quite sure if it is referred to the target folder or resources under src.
   
   Anyway, the use of that maven goal resources:resources I came up with overcomes the missing file in the target directory.
   So a correct maven pipeline is:
   ```
   mvn clean compile resources:resources nifi-nar:nar
   ```
   
   I don't know if this is really a bug or not, if you can suggest one I can try to fix it.
   For now I think the first action is to update the documentation with the use of this resources plugin for maven.
   
   PS. I wonder how people wrote connectors until now. I discovered that one of my colleagues was copying the file manually inside the folder before using the nar plugin :sunglasses: cheeky move :sunglasses:

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin

Posted by GitBox <gi...@apache.org>.
sijie commented on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin
URL: https://github.com/apache/pulsar/issues/6701#issuecomment-611732077
 
 
   Thank you @itaross 

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] itaross closed issue #6701: Cannot create Pulsar-IO connector

Posted by GitBox <gi...@apache.org>.
itaross closed issue #6701: Cannot create Pulsar-IO connector
URL: https://github.com/apache/pulsar/issues/6701
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] sijie commented on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin

Posted by GitBox <gi...@apache.org>.
sijie commented on issue #6701: Cannot create Pulsar-IO connector with nar maven plugin
URL: https://github.com/apache/pulsar/issues/6701#issuecomment-611981927
 
 
   @itaross it seems like a bug. Do you want to contribute a fix?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

[GitHub] [pulsar] itaross commented on issue #6701: Cannot create Pulsar-IO connector

Posted by GitBox <gi...@apache.org>.
itaross commented on issue #6701: Cannot create Pulsar-IO connector
URL: https://github.com/apache/pulsar/issues/6701#issuecomment-611568592
 
 
   Ok sorry @sijie you can close this.
   I have found the solution.
   I needed to add this inside the build tag in pom.xml:
   ```
     <resources>
               <resource>
                   <directory>src/main/resources/META-INF/services</directory>
                   <targetPath>META-INF/services</targetPath>
               </resource>
           </resources>
           <plugins>
               <plugin>
                   <groupId>org.apache.maven.plugins</groupId>
                   <artifactId>maven-resources-plugin</artifactId>
                   <version>3.1.0</version>
               </plugin>
   ```
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services