[GitHub] maximann opened a new pull request #270: Fix binary encoding

[GitBox <gi...@apache.org>]

maximann opened a new pull request #270: Fix binary encoding
URL: https://github.com/apache/incubator-openwhisk-package-kafka/pull/270
 
 
   This addresses #269 
   Note: I'm not a python coder, so I'm sure there might be better ways to achieve this. 
   
   When encoding binary data (ascii string) as utf with variable length encoding only 7 bits are preserved. The 8th bit has a special meaning to indicate the variable encoding "continuation bit". This will of course corrupt any true binary data that has values larger than 127.
   
   A secondary issue addressed in this MR relates to the base 64 encoding mechanism. The method called previously inserts newline characters in the encoded string every 76 characters, something that's not typically expected in newer encoding libraries. I've added a new flag which allows encoding without newline characters (which I assume is what most people will expect).
   
   Finally, each message is encoded twice in the current implementation, once to retrieve the size and a second time to actually trigger the function. 
   
   This fix has the potential to break existing functions that may rely on the old behavior. I'm not sure how to address that and would appreciate feedback. A new parameter to trigger the fixed binary encoding may be necessary.
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services