You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Ivan Pavlukhin <vo...@gmail.com> on 2019/10/16 08:49:08 UTC
Binary object format KB article
Igniters,
I published a document about Binary format in cwiki [1]. Please share
your feedback. I feel that there is a lack of pictures on the page.
Need to figure out what aspects will be more clear with pictures.
[1] https://cwiki.apache.org/confluence/display/IGNITE/Binary+object+format
--
Best regards,
Ivan Pavlukhin
Re: Binary object format KB article
Posted by Dmitriy Pavlov <dp...@apache.org>.
Hi Ivan,
Thank you for expanding our knowledge base.
Sincerely,
Dmitriy Pavlov
пт, 18 окт. 2019 г. в 12:39, Igor Sapego <is...@apache.org>:
> Great job,
>
> I think we should have details like this in documentation, not only in wiki
>
> Denis, what do you think?
>
> Best Regards,
> Igor
>
>
> On Wed, Oct 16, 2019 at 7:19 PM Ivan Pavlukhin <vo...@gmail.com>
> wrote:
>
> > Sergey,
> >
> > Thank you for a review!
> >
> > > It seems to me that document tries to focus on details of the format
> > itself but other aspects of this functionality leak into the explanation
> > and confuses reader.
> >
> > You are absolutely right, it was an original idea. I will try to
> > define a terminology and clarify things about schemas.
> >
> > ср, 16 окт. 2019 г. в 16:49, Sergey Chugunov <sergey.chugunov@gmail.com
> >:
> > >
> > > Then I would suggest to define good terminology at the very beginning
> of
> > > the article.
> > >
> > > Right in introduction section I see a lot of terms like "Binary object
> > > format", "Binary object container format" (is it the same thing?),
> > "Binary
> > > serialization format". In the next section "binary type" pops up. What
> > are
> > > the relations between them?
> > >
> > > Schemes part needs more examples. What is scheme? How it is related to
> > > binary type? Is it a one-to-one relationship? One-to-many? When a new
> > > scheme is created? Why type and scheme should be registered on a
> receiver
> > > side? And if the receiver exists then who is the sender?
> > >
> > > It seems to me that document tries to focus on details of the format
> > itself
> > > but other aspects of this functionality leak into the explanation and
> > > confuses reader.
> > >
> > > On Wed, Oct 16, 2019 at 2:52 PM Ivan Pavlukhin <vo...@gmail.com>
> > wrote:
> > >
> > > > Pavel, Sergey,
> > > >
> > > > Thank you for your feedback!
> > > >
> > > > To be exact the document does not describe broad picture (including
> > > > metadata exchange) and is not a formal format specification
> > > > intentionally. I wanted to create a lightweight article giving an
> > > > intuition about binary object structure to a reader. And yes,
> > > > intuition about metadata registration is definitely an important,
> > > > related but slightly different subject.
> > > >
> > > > ср, 16 окт. 2019 г. в 14:23, Sergey Chugunov <
> > sergey.chugunov@gmail.com>:
> > > > >
> > > > > Ivan, thank you for documenting this functionality, agree with
> Pavel
> > > > here.
> > > > >
> > > > > I think this document is a good starting point and contains a lot
> of
> > > > > low-level details and great examples but from my perspective it
> > doesn't
> > > > > show how binary objects fit into a broader picture.
> > > > >
> > > > > It worth adding higher-level details and restructure the document
> > into a
> > > > > top-down article starting from where binary format is used
> > > > (representation
> > > > > of objects in cache, binary protocol for communications with thin
> > > > clients)
> > > > > and down to lower details like binary metadata exchange and
> > serialization
> > > > > and container formats.
> > > > >
> > > > > Another option would be to leave the document focused on a
> low-level
> > > > > details as it is now but build around it drafts for documents
> > describing
> > > > > other aspects of Binary Objects.
> > > > > This will make our documentation much more solid and useful for
> > readers.
> > > > >
> > > > > On Wed, Oct 16, 2019 at 2:07 PM Pavel Tupitsyn <
> ptupitsyn@apache.org
> > >
> > > > wrote:
> > > > >
> > > > > > Ivan, great job, thanks for putting this together.
> > > > > >
> > > > > > I think we also need a more formal description of the format,
> > including
> > > > > > binary metadata exchange mechanics.
> > > > > > It was done (partially) for IEP-9 Thin Client Protocol, we should
> > > > probably
> > > > > > copy from there:
> > > > > >
> > > > > >
> > > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol#IEP-9ThinClientProtocol-BinaryObjects
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Oct 16, 2019 at 11:49 AM Ivan Pavlukhin <
> > vololo100@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I published a document about Binary format in cwiki [1]. Please
> > share
> > > > > > > your feedback. I feel that there is a lack of pictures on the
> > page.
> > > > > > > Need to figure out what aspects will be more clear with
> pictures.
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Binary+object+format
> > > > > > >
> > > > > > > --
> > > > > > > Best regards,
> > > > > > > Ivan Pavlukhin
> > > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>
Re: Binary object format KB article
Posted by Igor Sapego <is...@apache.org>.
Great job,
I think we should have details like this in documentation, not only in wiki
Denis, what do you think?
Best Regards,
Igor
On Wed, Oct 16, 2019 at 7:19 PM Ivan Pavlukhin <vo...@gmail.com> wrote:
> Sergey,
>
> Thank you for a review!
>
> > It seems to me that document tries to focus on details of the format
> itself but other aspects of this functionality leak into the explanation
> and confuses reader.
>
> You are absolutely right, it was an original idea. I will try to
> define a terminology and clarify things about schemas.
>
> ср, 16 окт. 2019 г. в 16:49, Sergey Chugunov <se...@gmail.com>:
> >
> > Then I would suggest to define good terminology at the very beginning of
> > the article.
> >
> > Right in introduction section I see a lot of terms like "Binary object
> > format", "Binary object container format" (is it the same thing?),
> "Binary
> > serialization format". In the next section "binary type" pops up. What
> are
> > the relations between them?
> >
> > Schemes part needs more examples. What is scheme? How it is related to
> > binary type? Is it a one-to-one relationship? One-to-many? When a new
> > scheme is created? Why type and scheme should be registered on a receiver
> > side? And if the receiver exists then who is the sender?
> >
> > It seems to me that document tries to focus on details of the format
> itself
> > but other aspects of this functionality leak into the explanation and
> > confuses reader.
> >
> > On Wed, Oct 16, 2019 at 2:52 PM Ivan Pavlukhin <vo...@gmail.com>
> wrote:
> >
> > > Pavel, Sergey,
> > >
> > > Thank you for your feedback!
> > >
> > > To be exact the document does not describe broad picture (including
> > > metadata exchange) and is not a formal format specification
> > > intentionally. I wanted to create a lightweight article giving an
> > > intuition about binary object structure to a reader. And yes,
> > > intuition about metadata registration is definitely an important,
> > > related but slightly different subject.
> > >
> > > ср, 16 окт. 2019 г. в 14:23, Sergey Chugunov <
> sergey.chugunov@gmail.com>:
> > > >
> > > > Ivan, thank you for documenting this functionality, agree with Pavel
> > > here.
> > > >
> > > > I think this document is a good starting point and contains a lot of
> > > > low-level details and great examples but from my perspective it
> doesn't
> > > > show how binary objects fit into a broader picture.
> > > >
> > > > It worth adding higher-level details and restructure the document
> into a
> > > > top-down article starting from where binary format is used
> > > (representation
> > > > of objects in cache, binary protocol for communications with thin
> > > clients)
> > > > and down to lower details like binary metadata exchange and
> serialization
> > > > and container formats.
> > > >
> > > > Another option would be to leave the document focused on a low-level
> > > > details as it is now but build around it drafts for documents
> describing
> > > > other aspects of Binary Objects.
> > > > This will make our documentation much more solid and useful for
> readers.
> > > >
> > > > On Wed, Oct 16, 2019 at 2:07 PM Pavel Tupitsyn <ptupitsyn@apache.org
> >
> > > wrote:
> > > >
> > > > > Ivan, great job, thanks for putting this together.
> > > > >
> > > > > I think we also need a more formal description of the format,
> including
> > > > > binary metadata exchange mechanics.
> > > > > It was done (partially) for IEP-9 Thin Client Protocol, we should
> > > probably
> > > > > copy from there:
> > > > >
> > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol#IEP-9ThinClientProtocol-BinaryObjects
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Oct 16, 2019 at 11:49 AM Ivan Pavlukhin <
> vololo100@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I published a document about Binary format in cwiki [1]. Please
> share
> > > > > > your feedback. I feel that there is a lack of pictures on the
> page.
> > > > > > Need to figure out what aspects will be more clear with pictures.
> > > > > >
> > > > > > [1]
> > > > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Binary+object+format
> > > > > >
> > > > > > --
> > > > > > Best regards,
> > > > > > Ivan Pavlukhin
> > > > > >
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
Re: Binary object format KB article
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Sergey,
Thank you for a review!
> It seems to me that document tries to focus on details of the format itself but other aspects of this functionality leak into the explanation and confuses reader.
You are absolutely right, it was an original idea. I will try to
define a terminology and clarify things about schemas.
ср, 16 окт. 2019 г. в 16:49, Sergey Chugunov <se...@gmail.com>:
>
> Then I would suggest to define good terminology at the very beginning of
> the article.
>
> Right in introduction section I see a lot of terms like "Binary object
> format", "Binary object container format" (is it the same thing?), "Binary
> serialization format". In the next section "binary type" pops up. What are
> the relations between them?
>
> Schemes part needs more examples. What is scheme? How it is related to
> binary type? Is it a one-to-one relationship? One-to-many? When a new
> scheme is created? Why type and scheme should be registered on a receiver
> side? And if the receiver exists then who is the sender?
>
> It seems to me that document tries to focus on details of the format itself
> but other aspects of this functionality leak into the explanation and
> confuses reader.
>
> On Wed, Oct 16, 2019 at 2:52 PM Ivan Pavlukhin <vo...@gmail.com> wrote:
>
> > Pavel, Sergey,
> >
> > Thank you for your feedback!
> >
> > To be exact the document does not describe broad picture (including
> > metadata exchange) and is not a formal format specification
> > intentionally. I wanted to create a lightweight article giving an
> > intuition about binary object structure to a reader. And yes,
> > intuition about metadata registration is definitely an important,
> > related but slightly different subject.
> >
> > ср, 16 окт. 2019 г. в 14:23, Sergey Chugunov <se...@gmail.com>:
> > >
> > > Ivan, thank you for documenting this functionality, agree with Pavel
> > here.
> > >
> > > I think this document is a good starting point and contains a lot of
> > > low-level details and great examples but from my perspective it doesn't
> > > show how binary objects fit into a broader picture.
> > >
> > > It worth adding higher-level details and restructure the document into a
> > > top-down article starting from where binary format is used
> > (representation
> > > of objects in cache, binary protocol for communications with thin
> > clients)
> > > and down to lower details like binary metadata exchange and serialization
> > > and container formats.
> > >
> > > Another option would be to leave the document focused on a low-level
> > > details as it is now but build around it drafts for documents describing
> > > other aspects of Binary Objects.
> > > This will make our documentation much more solid and useful for readers.
> > >
> > > On Wed, Oct 16, 2019 at 2:07 PM Pavel Tupitsyn <pt...@apache.org>
> > wrote:
> > >
> > > > Ivan, great job, thanks for putting this together.
> > > >
> > > > I think we also need a more formal description of the format, including
> > > > binary metadata exchange mechanics.
> > > > It was done (partially) for IEP-9 Thin Client Protocol, we should
> > probably
> > > > copy from there:
> > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol#IEP-9ThinClientProtocol-BinaryObjects
> > > >
> > > >
> > > >
> > > > On Wed, Oct 16, 2019 at 11:49 AM Ivan Pavlukhin <vo...@gmail.com>
> > > > wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I published a document about Binary format in cwiki [1]. Please share
> > > > > your feedback. I feel that there is a lack of pictures on the page.
> > > > > Need to figure out what aspects will be more clear with pictures.
> > > > >
> > > > > [1]
> > > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Binary+object+format
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Ivan Pavlukhin
> > > > >
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
--
Best regards,
Ivan Pavlukhin
Re: Binary object format KB article
Posted by Sergey Chugunov <se...@gmail.com>.
Then I would suggest to define good terminology at the very beginning of
the article.
Right in introduction section I see a lot of terms like "Binary object
format", "Binary object container format" (is it the same thing?), "Binary
serialization format". In the next section "binary type" pops up. What are
the relations between them?
Schemes part needs more examples. What is scheme? How it is related to
binary type? Is it a one-to-one relationship? One-to-many? When a new
scheme is created? Why type and scheme should be registered on a receiver
side? And if the receiver exists then who is the sender?
It seems to me that document tries to focus on details of the format itself
but other aspects of this functionality leak into the explanation and
confuses reader.
On Wed, Oct 16, 2019 at 2:52 PM Ivan Pavlukhin <vo...@gmail.com> wrote:
> Pavel, Sergey,
>
> Thank you for your feedback!
>
> To be exact the document does not describe broad picture (including
> metadata exchange) and is not a formal format specification
> intentionally. I wanted to create a lightweight article giving an
> intuition about binary object structure to a reader. And yes,
> intuition about metadata registration is definitely an important,
> related but slightly different subject.
>
> ср, 16 окт. 2019 г. в 14:23, Sergey Chugunov <se...@gmail.com>:
> >
> > Ivan, thank you for documenting this functionality, agree with Pavel
> here.
> >
> > I think this document is a good starting point and contains a lot of
> > low-level details and great examples but from my perspective it doesn't
> > show how binary objects fit into a broader picture.
> >
> > It worth adding higher-level details and restructure the document into a
> > top-down article starting from where binary format is used
> (representation
> > of objects in cache, binary protocol for communications with thin
> clients)
> > and down to lower details like binary metadata exchange and serialization
> > and container formats.
> >
> > Another option would be to leave the document focused on a low-level
> > details as it is now but build around it drafts for documents describing
> > other aspects of Binary Objects.
> > This will make our documentation much more solid and useful for readers.
> >
> > On Wed, Oct 16, 2019 at 2:07 PM Pavel Tupitsyn <pt...@apache.org>
> wrote:
> >
> > > Ivan, great job, thanks for putting this together.
> > >
> > > I think we also need a more formal description of the format, including
> > > binary metadata exchange mechanics.
> > > It was done (partially) for IEP-9 Thin Client Protocol, we should
> probably
> > > copy from there:
> > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol#IEP-9ThinClientProtocol-BinaryObjects
> > >
> > >
> > >
> > > On Wed, Oct 16, 2019 at 11:49 AM Ivan Pavlukhin <vo...@gmail.com>
> > > wrote:
> > >
> > > > Igniters,
> > > >
> > > > I published a document about Binary format in cwiki [1]. Please share
> > > > your feedback. I feel that there is a lack of pictures on the page.
> > > > Need to figure out what aspects will be more clear with pictures.
> > > >
> > > > [1]
> > > >
> https://cwiki.apache.org/confluence/display/IGNITE/Binary+object+format
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
Re: Binary object format KB article
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Pavel, Sergey,
Thank you for your feedback!
To be exact the document does not describe broad picture (including
metadata exchange) and is not a formal format specification
intentionally. I wanted to create a lightweight article giving an
intuition about binary object structure to a reader. And yes,
intuition about metadata registration is definitely an important,
related but slightly different subject.
ср, 16 окт. 2019 г. в 14:23, Sergey Chugunov <se...@gmail.com>:
>
> Ivan, thank you for documenting this functionality, agree with Pavel here.
>
> I think this document is a good starting point and contains a lot of
> low-level details and great examples but from my perspective it doesn't
> show how binary objects fit into a broader picture.
>
> It worth adding higher-level details and restructure the document into a
> top-down article starting from where binary format is used (representation
> of objects in cache, binary protocol for communications with thin clients)
> and down to lower details like binary metadata exchange and serialization
> and container formats.
>
> Another option would be to leave the document focused on a low-level
> details as it is now but build around it drafts for documents describing
> other aspects of Binary Objects.
> This will make our documentation much more solid and useful for readers.
>
> On Wed, Oct 16, 2019 at 2:07 PM Pavel Tupitsyn <pt...@apache.org> wrote:
>
> > Ivan, great job, thanks for putting this together.
> >
> > I think we also need a more formal description of the format, including
> > binary metadata exchange mechanics.
> > It was done (partially) for IEP-9 Thin Client Protocol, we should probably
> > copy from there:
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol#IEP-9ThinClientProtocol-BinaryObjects
> >
> >
> >
> > On Wed, Oct 16, 2019 at 11:49 AM Ivan Pavlukhin <vo...@gmail.com>
> > wrote:
> >
> > > Igniters,
> > >
> > > I published a document about Binary format in cwiki [1]. Please share
> > > your feedback. I feel that there is a lack of pictures on the page.
> > > Need to figure out what aspects will be more clear with pictures.
> > >
> > > [1]
> > > https://cwiki.apache.org/confluence/display/IGNITE/Binary+object+format
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
> >
--
Best regards,
Ivan Pavlukhin
Re: Binary object format KB article
Posted by Sergey Chugunov <se...@gmail.com>.
Ivan, thank you for documenting this functionality, agree with Pavel here.
I think this document is a good starting point and contains a lot of
low-level details and great examples but from my perspective it doesn't
show how binary objects fit into a broader picture.
It worth adding higher-level details and restructure the document into a
top-down article starting from where binary format is used (representation
of objects in cache, binary protocol for communications with thin clients)
and down to lower details like binary metadata exchange and serialization
and container formats.
Another option would be to leave the document focused on a low-level
details as it is now but build around it drafts for documents describing
other aspects of Binary Objects.
This will make our documentation much more solid and useful for readers.
On Wed, Oct 16, 2019 at 2:07 PM Pavel Tupitsyn <pt...@apache.org> wrote:
> Ivan, great job, thanks for putting this together.
>
> I think we also need a more formal description of the format, including
> binary metadata exchange mechanics.
> It was done (partially) for IEP-9 Thin Client Protocol, we should probably
> copy from there:
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol#IEP-9ThinClientProtocol-BinaryObjects
>
>
>
> On Wed, Oct 16, 2019 at 11:49 AM Ivan Pavlukhin <vo...@gmail.com>
> wrote:
>
> > Igniters,
> >
> > I published a document about Binary format in cwiki [1]. Please share
> > your feedback. I feel that there is a lack of pictures on the page.
> > Need to figure out what aspects will be more clear with pictures.
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/IGNITE/Binary+object+format
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
>
Re: Binary object format KB article
Posted by Pavel Tupitsyn <pt...@apache.org>.
Ivan, great job, thanks for putting this together.
I think we also need a more formal description of the format, including
binary metadata exchange mechanics.
It was done (partially) for IEP-9 Thin Client Protocol, we should probably
copy from there:
https://cwiki.apache.org/confluence/display/IGNITE/IEP-9+Thin+Client+Protocol#IEP-9ThinClientProtocol-BinaryObjects
On Wed, Oct 16, 2019 at 11:49 AM Ivan Pavlukhin <vo...@gmail.com> wrote:
> Igniters,
>
> I published a document about Binary format in cwiki [1]. Please share
> your feedback. I feel that there is a lack of pictures on the page.
> Need to figure out what aspects will be more clear with pictures.
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/Binary+object+format
>
> --
> Best regards,
> Ivan Pavlukhin
>