msgpack pull request

[Jean-Claude Cote <jc...@gmail.com>]

Hey Paul,

In my pull request you mentioned handling splits.. I put a comment in the
pull request but essentially msgpack files are a list of records so
technically they can be split. However I'm not sure if that's beneficial
because I'm not sure how the splitting process works..

The other thing you mentioned is using TupleSchema. I have not looked at it
yet but in my initial design I had tried to use it but ran into
limitations. I will try it again and let you know what the issue is (if I
run into it again).

I did a major refactoring of my msgpack format reader and added more
documentation/explanations. Hopefully it will be easier to follow. Could
you have a look when you have a moment?

Thanks
jc

Re: msgpack pull request

[Paul Rogers <pa...@yahoo.com.INVALID>]

Hi JC,

Thanks much for the updates. I’ll take another look over the weekend.

- Paul

Sent from my iPhone

> On Nov 9, 2018, at 2:02 PM, Jean-Claude Cote <jc...@gmail.com> wrote:
> 
> Hey Paul,
> 
> In my pull request you mentioned handling splits.. I put a comment in the
> pull request but essentially msgpack files are a list of records so
> technically they can be split. However I'm not sure if that's beneficial
> because I'm not sure how the splitting process works..
> 
> The other thing you mentioned is using TupleSchema. I have not looked at it
> yet but in my initial design I had tried to use it but ran into
> limitations. I will try it again and let you know what the issue is (if I
> run into it again).
> 
> I did a major refactoring of my msgpack format reader and added more
> documentation/explanations. Hopefully it will be easier to follow. Could
> you have a look when you have a moment?
> 
> Thanks
> jc