ProtoBuf generated files manually checked in in REEF .NET

[Chenxi Zhao <Ch...@microsoft.com.INVALID>]

Hi all,

I noticed that in REEF .NET protobuf files are defined in reef\lang\cs\Org.Apache.REEF.Common\Protobuf\Proto and generated CS files are in reef\lang\cs\Org.Apache.REEF.Common\Protobuf\ReefProtocol. However, .pb.cs files in in reef\lang\cs\Org.Apache.REEF.Common\Protobuf\ReefProtocol are manually checked in instead of being generated by build script. I wonder what is the command line and version of protobuf to generate those files when we merged these files?

I tried to use protoc-3.0.0 (e.g. protoc.exe ./reef_service_protos.proto --csharp_out=./) to regenerate those files but the format looks quite different.

What I am working on is trying to add a new proto file in REEF .NET, and I want to make sure the generated file format are consistent. Could someone help clarify on the how .pb.cs files are generated? And I think we may want to have them changed to be generated by build script in future.

Thanks,
Chenxi

RE: ProtoBuf generated files manually checked in in REEF .NET

[Chenxi Zhao <Ch...@microsoft.com.INVALID>]

After spending a day I finally got what happened to the ProtoBuffer tool we used in .NET REEF. I would like share my finding here in case others get into the same issue:

The .NET side ProtoBuffer tool we were using seems to be "protobuf-net" (https://code.google.com/archive/p/protobuf-net/), which seems to be a deprecated project. The last release is 09/30/2013. I confirmed this by searching and comparing the regenerated files of existing code.

On the java side, we are using Protofol-buffers (https://developers.google.com/protocol-buffers/), which has C# language support now. REEF .NET project is not compatible using it yet.

Regards,
Chenxi


From: Chenxi Zhao
Sent: Friday, May 11, 2018 5:00 PM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: ProtoBuf generated files manually checked in in REEF .NET

Hi all,

I noticed that in REEF .NET protobuf files are defined in reef\lang\cs\Org.Apache.REEF.Common\Protobuf\Proto and generated CS files are in reef\lang\cs\Org.Apache.REEF.Common\Protobuf\ReefProtocol. However, .pb.cs files in in reef\lang\cs\Org.Apache.REEF.Common\Protobuf\ReefProtocol are manually checked in instead of being generated by build script. I wonder what is the command line and version of protobuf to generate those files when we merged these files?

I tried to use protoc-3.0.0 (e.g. protoc.exe ./reef_service_protos.proto --csharp_out=./) to regenerate those files but the format looks quite different.

What I am working on is trying to add a new proto file in REEF .NET, and I want to make sure the generated file format are consistent. Could someone help clarify on the how .pb.cs files are generated? And I think we may want to have them changed to be generated by build script in future.

Thanks,
Chenxi