You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by Charan Reddy G <re...@gmail.com> on 2016/03/18 01:10:22 UTC
Question/Issue regarding formatEntry in BookieShell
Currently in BookieShell for formatting Entry (in formatEntry method), to
skip digest, position of recBuff is set to (32+8) (rebuff.position(32 +
8);). This works fine if the Digest is of type CRC32Digest where the
MacCodeLength is 8, but it wouldn’t work well for MacDigest since its
MacCodeLength is 20 or for any other future DigestManager if its
MacCodeLength != 8.
Hence when we execute ReadLedgerCmd or ReadjournalCmd, for CRC32 it prints
data correctly for e.g.: “EntryNData”. But for MacDigest it would print
something like the following - "....N<..}.`rEntryNData”, here first 12
bytes is part of Digest of that Entry.
I tried to see if there is easy way to fix it but couldn’t find any.
Basically I’m trying to find out if there is any easy way to find the
DigestType or the MacCodeLength of that entry so that we can skip those
bytes. The entryLength we receive as part of 'recBuff’ is not actual length
of the entry but the accumulative length of the data until this entry, so
it won’t be helpful to us. Any help would be appreciated, to print the
actual entry data without digest noise.
Thanks,
Charan.
Re: Question/Issue regarding formatEntry in BookieShell
Posted by Charan Reddy G <re...@gmail.com>.
Thanks Sijie. I was thinking much in the lines of possibility of
identifying the DigestType looking at the persisted entry data. But
providing option to mention DigestType is good enough solution.
Thanks,
Charan.
On Mon, Mar 21, 2016 at 12:02 PM, Sijie Guo <si...@apache.org> wrote:
> unfortunately don't know... we used crc32 at Twitter.
>
> - Sijie
>
> On Mon, Mar 21, 2016 at 10:32 AM, Venkateswara Rao Jujjuri <
> jujjuri@gmail.com> wrote:
>
> > Right that works, but most of the folks stick to crc32. :)
> >
> > Sijie do you know if anyone in production uses mac?
> >
> > On Mon, Mar 21, 2016 at 10:28 AM, Sijie Guo <si...@apache.org> wrote:
> >
> > > How about passing a flag to those commands to specify the digest types?
> > By
> > > default, use crc32.
> > >
> > > - Sijie
> > >
> > > On Thu, Mar 17, 2016 at 5:10 PM, Charan Reddy G <
> reddycharan18@gmail.com
> > >
> > > wrote:
> > >
> > > > Currently in BookieShell for formatting Entry (in formatEntry
> method),
> > to
> > > > skip digest, position of recBuff is set to (32+8)
> (rebuff.position(32 +
> > > > 8);). This works fine if the Digest is of type CRC32Digest where the
> > > > MacCodeLength is 8, but it wouldn’t work well for MacDigest since its
> > > > MacCodeLength is 20 or for any other future DigestManager if its
> > > > MacCodeLength != 8.
> > > >
> > > > Hence when we execute ReadLedgerCmd or ReadjournalCmd, for CRC32 it
> > > prints
> > > > data correctly for e.g.: “EntryNData”. But for MacDigest it would
> print
> > > > something like the following - "....N<..}.`rEntryNData”, here first
> 12
> > > > bytes is part of Digest of that Entry.
> > > >
> > > > I tried to see if there is easy way to fix it but couldn’t find any.
> > > > Basically I’m trying to find out if there is any easy way to find the
> > > > DigestType or the MacCodeLength of that entry so that we can skip
> those
> > > > bytes. The entryLength we receive as part of 'recBuff’ is not actual
> > > length
> > > > of the entry but the accumulative length of the data until this
> entry,
> > so
> > > > it won’t be helpful to us. Any help would be appreciated, to print
> the
> > > > actual entry data without digest noise.
> > > >
> > > > Thanks,
> > > >
> > > > Charan.
> > > >
> > >
> >
> >
> >
> > --
> > Jvrao
> > ---
> > First they ignore you, then they laugh at you, then they fight you, then
> > you win. - Mahatma Gandhi
> >
>
Re: Question/Issue regarding formatEntry in BookieShell
Posted by Sijie Guo <si...@apache.org>.
unfortunately don't know... we used crc32 at Twitter.
- Sijie
On Mon, Mar 21, 2016 at 10:32 AM, Venkateswara Rao Jujjuri <
jujjuri@gmail.com> wrote:
> Right that works, but most of the folks stick to crc32. :)
>
> Sijie do you know if anyone in production uses mac?
>
> On Mon, Mar 21, 2016 at 10:28 AM, Sijie Guo <si...@apache.org> wrote:
>
> > How about passing a flag to those commands to specify the digest types?
> By
> > default, use crc32.
> >
> > - Sijie
> >
> > On Thu, Mar 17, 2016 at 5:10 PM, Charan Reddy G <reddycharan18@gmail.com
> >
> > wrote:
> >
> > > Currently in BookieShell for formatting Entry (in formatEntry method),
> to
> > > skip digest, position of recBuff is set to (32+8) (rebuff.position(32 +
> > > 8);). This works fine if the Digest is of type CRC32Digest where the
> > > MacCodeLength is 8, but it wouldn’t work well for MacDigest since its
> > > MacCodeLength is 20 or for any other future DigestManager if its
> > > MacCodeLength != 8.
> > >
> > > Hence when we execute ReadLedgerCmd or ReadjournalCmd, for CRC32 it
> > prints
> > > data correctly for e.g.: “EntryNData”. But for MacDigest it would print
> > > something like the following - "....N<..}.`rEntryNData”, here first 12
> > > bytes is part of Digest of that Entry.
> > >
> > > I tried to see if there is easy way to fix it but couldn’t find any.
> > > Basically I’m trying to find out if there is any easy way to find the
> > > DigestType or the MacCodeLength of that entry so that we can skip those
> > > bytes. The entryLength we receive as part of 'recBuff’ is not actual
> > length
> > > of the entry but the accumulative length of the data until this entry,
> so
> > > it won’t be helpful to us. Any help would be appreciated, to print the
> > > actual entry data without digest noise.
> > >
> > > Thanks,
> > >
> > > Charan.
> > >
> >
>
>
>
> --
> Jvrao
> ---
> First they ignore you, then they laugh at you, then they fight you, then
> you win. - Mahatma Gandhi
>
Re: Question/Issue regarding formatEntry in BookieShell
Posted by Venkateswara Rao Jujjuri <ju...@gmail.com>.
Right that works, but most of the folks stick to crc32. :)
Sijie do you know if anyone in production uses mac?
On Mon, Mar 21, 2016 at 10:28 AM, Sijie Guo <si...@apache.org> wrote:
> How about passing a flag to those commands to specify the digest types? By
> default, use crc32.
>
> - Sijie
>
> On Thu, Mar 17, 2016 at 5:10 PM, Charan Reddy G <re...@gmail.com>
> wrote:
>
> > Currently in BookieShell for formatting Entry (in formatEntry method), to
> > skip digest, position of recBuff is set to (32+8) (rebuff.position(32 +
> > 8);). This works fine if the Digest is of type CRC32Digest where the
> > MacCodeLength is 8, but it wouldn’t work well for MacDigest since its
> > MacCodeLength is 20 or for any other future DigestManager if its
> > MacCodeLength != 8.
> >
> > Hence when we execute ReadLedgerCmd or ReadjournalCmd, for CRC32 it
> prints
> > data correctly for e.g.: “EntryNData”. But for MacDigest it would print
> > something like the following - "....N<..}.`rEntryNData”, here first 12
> > bytes is part of Digest of that Entry.
> >
> > I tried to see if there is easy way to fix it but couldn’t find any.
> > Basically I’m trying to find out if there is any easy way to find the
> > DigestType or the MacCodeLength of that entry so that we can skip those
> > bytes. The entryLength we receive as part of 'recBuff’ is not actual
> length
> > of the entry but the accumulative length of the data until this entry, so
> > it won’t be helpful to us. Any help would be appreciated, to print the
> > actual entry data without digest noise.
> >
> > Thanks,
> >
> > Charan.
> >
>
--
Jvrao
---
First they ignore you, then they laugh at you, then they fight you, then
you win. - Mahatma Gandhi
Re: Question/Issue regarding formatEntry in BookieShell
Posted by Sijie Guo <si...@apache.org>.
How about passing a flag to those commands to specify the digest types? By
default, use crc32.
- Sijie
On Thu, Mar 17, 2016 at 5:10 PM, Charan Reddy G <re...@gmail.com>
wrote:
> Currently in BookieShell for formatting Entry (in formatEntry method), to
> skip digest, position of recBuff is set to (32+8) (rebuff.position(32 +
> 8);). This works fine if the Digest is of type CRC32Digest where the
> MacCodeLength is 8, but it wouldn’t work well for MacDigest since its
> MacCodeLength is 20 or for any other future DigestManager if its
> MacCodeLength != 8.
>
> Hence when we execute ReadLedgerCmd or ReadjournalCmd, for CRC32 it prints
> data correctly for e.g.: “EntryNData”. But for MacDigest it would print
> something like the following - "....N<..}.`rEntryNData”, here first 12
> bytes is part of Digest of that Entry.
>
> I tried to see if there is easy way to fix it but couldn’t find any.
> Basically I’m trying to find out if there is any easy way to find the
> DigestType or the MacCodeLength of that entry so that we can skip those
> bytes. The entryLength we receive as part of 'recBuff’ is not actual length
> of the entry but the accumulative length of the data until this entry, so
> it won’t be helpful to us. Any help would be appreciated, to print the
> actual entry data without digest noise.
>
> Thanks,
>
> Charan.
>