You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@nifi.apache.org by Valentina Ivanova <va...@ri.se> on 2020/02/17 15:20:49 UTC

Nifi protobuf processor

Hi!
I am trying to use nifi-protobuf-procesor to decode gtfs real time messages. However, I got a number of issues:
 - I could not compile the schema .desc from the existing gtfs-realtime.proto due to "Unable to read schema file: Unsupported platform: protoc-3.4.0-linux-x86_64.exe" although I am using Linux.
 - I managed to compile the schema using the protobuf binaries but when I tried to use it with the processor it requires proto.messageType property. If I create a custom property I got error saying that there is no validator for it.
- I managed to modify the processor's code and compiled it again in order to specify the proto.messageType but I am not sure what message type I have to enter. I tried with Vehicle Position - one of the message types in the gtfs real time. I got some broken output containing two JSON elements and that was everything. It seems to me that the message type should be detected automatically and changed dynamically as one file might contain different types such as VehicleDescriptor, TripDescriptor, etc but I don't know how to achieve this.

Has anyone used this processor and knows how complex schemata is possible to decode with it? Any help is appreciated.

Thanks & all the best

Valentina

Re: Nifi protobuf processor

Posted by Valentina Ivanova <va...@ri.se>.
I figured it out - I have used a wrong message type. It should be the top most type - in my case FeedMessage.


________________________________
From: Valentina Ivanova <va...@ri.se>
Sent: Monday, 17 February 2020 21:38
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Re: Nifi protobuf processor

A clarification - this is the processor I meant:
https://github.com/whiver/nifi-protobuf-processor

Thanks for the help in advance

Valentina
________________________________
From: Valentina Ivanova <va...@ri.se>
Sent: Monday, 17 February 2020 16:20
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Nifi protobuf processor

Hi!
I am trying to use nifi-protobuf-procesor to decode gtfs real time messages. However, I got a number of issues:
 - I could not compile the schema .desc from the existing gtfs-realtime.proto due to "Unable to read schema file: Unsupported platform: protoc-3.4.0-linux-x86_64.exe" although I am using Linux.
 - I managed to compile the schema using the protobuf binaries but when I tried to use it with the processor it requires proto.messageType property. If I create a custom property I got error saying that there is no validator for it.
- I managed to modify the processor's code and compiled it again in order to specify the proto.messageType but I am not sure what message type I have to enter. I tried with Vehicle Position - one of the message types in the gtfs real time. I got some broken output containing two JSON elements and that was everything. It seems to me that the message type should be detected automatically and changed dynamically as one file might contain different types such as VehicleDescriptor, TripDescriptor, etc but I don't know how to achieve this.

Has anyone used this processor and knows how complex schemata is possible to decode with it? Any help is appreciated.

Thanks & all the best

Valentina

Re: Nifi protobuf processor

Posted by Valentina Ivanova <va...@ri.se>.
A clarification - this is the processor I meant:
https://github.com/whiver/nifi-protobuf-processor

Thanks for the help in advance

Valentina
________________________________
From: Valentina Ivanova <va...@ri.se>
Sent: Monday, 17 February 2020 16:20
To: users@nifi.apache.org <us...@nifi.apache.org>
Subject: Nifi protobuf processor

Hi!
I am trying to use nifi-protobuf-procesor to decode gtfs real time messages. However, I got a number of issues:
 - I could not compile the schema .desc from the existing gtfs-realtime.proto due to "Unable to read schema file: Unsupported platform: protoc-3.4.0-linux-x86_64.exe" although I am using Linux.
 - I managed to compile the schema using the protobuf binaries but when I tried to use it with the processor it requires proto.messageType property. If I create a custom property I got error saying that there is no validator for it.
- I managed to modify the processor's code and compiled it again in order to specify the proto.messageType but I am not sure what message type I have to enter. I tried with Vehicle Position - one of the message types in the gtfs real time. I got some broken output containing two JSON elements and that was everything. It seems to me that the message type should be detected automatically and changed dynamically as one file might contain different types such as VehicleDescriptor, TripDescriptor, etc but I don't know how to achieve this.

Has anyone used this processor and knows how complex schemata is possible to decode with it? Any help is appreciated.

Thanks & all the best

Valentina