You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@gora.apache.org by "Ferdy Galema (Created) (JIRA)" <ji...@apache.org> on 2012/01/30 14:52:10 UTC
[jira] [Created] (GORA-88) HBaseByteInterface not thread safe
HBaseByteInterface not thread safe
----------------------------------
Key: GORA-88
URL: https://issues.apache.org/jira/browse/GORA-88
Project: Gora
Issue Type: Bug
Components: storage-hbase
Reporter: Ferdy Galema
The recent attempt to make HBaseStore thread safe overlooked the utility class HBaseByteInterface. This class currently uses a single SpecificDatumReader and SpecificDatumWriter for the serialization of embedded records (schemas) within the schema. These reader and writer classes are not thread safe when used with a single schema (at least not yet with the current Avro library in use), much less when using multiple schema's (i.e. writer.setSchema(schema)).
I will attach a patch that makes proper (re)use of these writer and reader classes, as well as the closely related decoders and encoders. Some workarounds are provided that would be unnecessary with newer versions of Avro, but these can be removed whenever we update this library.
This class does not break compatibilty; the end result data in HBase is unchanged.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Closed] (GORA-88) HBaseByteInterface not thread safe
Posted by Lewis John Mcgibbney <le...@gmail.com>.
excellent Ferdy, great work. From what I can see with this one you've
cracked a couple of TODO's as well.
Lewis
On Sat, Feb 18, 2012 at 11:11 PM, Ferdy Galema (Closed) (JIRA) <
jira@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/GORA-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> Ferdy Galema closed GORA-88.
> ----------------------------
>
> Resolution: Fixed
>
> committed
>
> > HBaseByteInterface not thread safe
> > ----------------------------------
> >
> > Key: GORA-88
> > URL: https://issues.apache.org/jira/browse/GORA-88
> > Project: Apache Gora
> > Issue Type: Bug
> > Components: storage-hbase
> > Affects Versions: 0.2
> > Reporter: Ferdy Galema
> > Fix For: 0.2
> >
> > Attachments: GORA-88-v2.patch, GORA-88.patch
> >
> >
> > The recent attempt to make HBaseStore thread safe overlooked the utility
> class HBaseByteInterface. This class currently uses a single
> SpecificDatumReader and SpecificDatumWriter for the serialization of
> embedded records (schemas) within the schema. These reader and writer
> classes are not thread safe when used with a single schema (at least not
> yet with the current Avro library in use), much less when using multiple
> schema's (i.e. writer.setSchema(schema)).
> > I will attach a patch that makes proper (re)use of these writer and
> reader classes, as well as the closely related decoders and encoders. Some
> workarounds are provided that would be unnecessary with newer versions of
> Avro, but these can be removed whenever we update this library.
> > This class does not break compatibilty; the end result data in HBase is
> unchanged.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
--
*Lewis*
[jira] [Commented] (GORA-88) HBaseByteInterface not thread safe
Posted by "Ferdy Galema (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-88?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13210983#comment-13210983 ]
Ferdy Galema commented on GORA-88:
----------------------------------
Almost done with a patch that includes a test. Will commit in a few days
> HBaseByteInterface not thread safe
> ----------------------------------
>
> Key: GORA-88
> URL: https://issues.apache.org/jira/browse/GORA-88
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-hbase
> Affects Versions: 0.2
> Reporter: Ferdy Galema
> Fix For: 0.2
>
> Attachments: GORA-88.patch
>
>
> The recent attempt to make HBaseStore thread safe overlooked the utility class HBaseByteInterface. This class currently uses a single SpecificDatumReader and SpecificDatumWriter for the serialization of embedded records (schemas) within the schema. These reader and writer classes are not thread safe when used with a single schema (at least not yet with the current Avro library in use), much less when using multiple schema's (i.e. writer.setSchema(schema)).
> I will attach a patch that makes proper (re)use of these writer and reader classes, as well as the closely related decoders and encoders. Some workarounds are provided that would be unnecessary with newer versions of Avro, but these can be removed whenever we update this library.
> This class does not break compatibilty; the end result data in HBase is unchanged.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-88) HBaseByteInterface not thread safe
Posted by "Lewis John McGibbney (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Lewis John McGibbney updated GORA-88:
-------------------------------------
Affects Version/s: 0.2
Fix Version/s: 0.2
> HBaseByteInterface not thread safe
> ----------------------------------
>
> Key: GORA-88
> URL: https://issues.apache.org/jira/browse/GORA-88
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-hbase
> Affects Versions: 0.2
> Reporter: Ferdy Galema
> Fix For: 0.2
>
> Attachments: GORA-88.patch
>
>
> The recent attempt to make HBaseStore thread safe overlooked the utility class HBaseByteInterface. This class currently uses a single SpecificDatumReader and SpecificDatumWriter for the serialization of embedded records (schemas) within the schema. These reader and writer classes are not thread safe when used with a single schema (at least not yet with the current Avro library in use), much less when using multiple schema's (i.e. writer.setSchema(schema)).
> I will attach a patch that makes proper (re)use of these writer and reader classes, as well as the closely related decoders and encoders. Some workarounds are provided that would be unnecessary with newer versions of Avro, but these can be removed whenever we update this library.
> This class does not break compatibilty; the end result data in HBase is unchanged.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-88) HBaseByteInterface not thread safe
Posted by "Hudson (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-88?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211170#comment-13211170 ]
Hudson commented on GORA-88:
----------------------------
Integrated in gora-trunk #149 (See [https://builds.apache.org/job/gora-trunk/149/])
GORA-88 HBaseByteInterface not thread safe (Revision 1290902)
Result = SUCCESS
ferdy :
Files :
* /gora/trunk/CHANGES.txt
* /gora/trunk/gora-hbase/src/main/java/org/apache/gora/hbase/util/HBaseByteInterface.java
* /gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/util
* /gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/util/TestHBaseByteInterface.java
> HBaseByteInterface not thread safe
> ----------------------------------
>
> Key: GORA-88
> URL: https://issues.apache.org/jira/browse/GORA-88
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-hbase
> Affects Versions: 0.2
> Reporter: Ferdy Galema
> Fix For: 0.2
>
> Attachments: GORA-88-v2.patch, GORA-88.patch
>
>
> The recent attempt to make HBaseStore thread safe overlooked the utility class HBaseByteInterface. This class currently uses a single SpecificDatumReader and SpecificDatumWriter for the serialization of embedded records (schemas) within the schema. These reader and writer classes are not thread safe when used with a single schema (at least not yet with the current Avro library in use), much less when using multiple schema's (i.e. writer.setSchema(schema)).
> I will attach a patch that makes proper (re)use of these writer and reader classes, as well as the closely related decoders and encoders. Some workarounds are provided that would be unnecessary with newer versions of Avro, but these can be removed whenever we update this library.
> This class does not break compatibilty; the end result data in HBase is unchanged.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-88) HBaseByteInterface not thread safe
Posted by "Ferdy Galema (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated GORA-88:
-----------------------------
Attachment: GORA-88-v2.patch
Finished patch. It tests HBaseByteInterface by directly calling the encode/decode methods using various schemas, both single and multithreaded. (The HBaseByteInterface implementation prior to this patch fails on the multithreaded test, as expected. It throws indexoutofbounds and class casting exceptions).
> HBaseByteInterface not thread safe
> ----------------------------------
>
> Key: GORA-88
> URL: https://issues.apache.org/jira/browse/GORA-88
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-hbase
> Affects Versions: 0.2
> Reporter: Ferdy Galema
> Fix For: 0.2
>
> Attachments: GORA-88-v2.patch, GORA-88.patch
>
>
> The recent attempt to make HBaseStore thread safe overlooked the utility class HBaseByteInterface. This class currently uses a single SpecificDatumReader and SpecificDatumWriter for the serialization of embedded records (schemas) within the schema. These reader and writer classes are not thread safe when used with a single schema (at least not yet with the current Avro library in use), much less when using multiple schema's (i.e. writer.setSchema(schema)).
> I will attach a patch that makes proper (re)use of these writer and reader classes, as well as the closely related decoders and encoders. Some workarounds are provided that would be unnecessary with newer versions of Avro, but these can be removed whenever we update this library.
> This class does not break compatibilty; the end result data in HBase is unchanged.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (GORA-88) HBaseByteInterface not thread safe
Posted by "Ferdy Galema (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema updated GORA-88:
-----------------------------
Attachment: GORA-88.patch
> HBaseByteInterface not thread safe
> ----------------------------------
>
> Key: GORA-88
> URL: https://issues.apache.org/jira/browse/GORA-88
> Project: Gora
> Issue Type: Bug
> Components: storage-hbase
> Reporter: Ferdy Galema
> Attachments: GORA-88.patch
>
>
> The recent attempt to make HBaseStore thread safe overlooked the utility class HBaseByteInterface. This class currently uses a single SpecificDatumReader and SpecificDatumWriter for the serialization of embedded records (schemas) within the schema. These reader and writer classes are not thread safe when used with a single schema (at least not yet with the current Avro library in use), much less when using multiple schema's (i.e. writer.setSchema(schema)).
> I will attach a patch that makes proper (re)use of these writer and reader classes, as well as the closely related decoders and encoders. Some workarounds are provided that would be unnecessary with newer versions of Avro, but these can be removed whenever we update this library.
> This class does not break compatibilty; the end result data in HBase is unchanged.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (GORA-88) HBaseByteInterface not thread safe
Posted by "Ferdy Galema (Closed) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-88?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ferdy Galema closed GORA-88.
----------------------------
Resolution: Fixed
committed
> HBaseByteInterface not thread safe
> ----------------------------------
>
> Key: GORA-88
> URL: https://issues.apache.org/jira/browse/GORA-88
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-hbase
> Affects Versions: 0.2
> Reporter: Ferdy Galema
> Fix For: 0.2
>
> Attachments: GORA-88-v2.patch, GORA-88.patch
>
>
> The recent attempt to make HBaseStore thread safe overlooked the utility class HBaseByteInterface. This class currently uses a single SpecificDatumReader and SpecificDatumWriter for the serialization of embedded records (schemas) within the schema. These reader and writer classes are not thread safe when used with a single schema (at least not yet with the current Avro library in use), much less when using multiple schema's (i.e. writer.setSchema(schema)).
> I will attach a patch that makes proper (re)use of these writer and reader classes, as well as the closely related decoders and encoders. Some workarounds are provided that would be unnecessary with newer versions of Avro, but these can be removed whenever we update this library.
> This class does not break compatibilty; the end result data in HBase is unchanged.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (GORA-88) HBaseByteInterface not thread safe
Posted by "Lewis John McGibbney (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/GORA-88?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206882#comment-13206882 ]
Lewis John McGibbney commented on GORA-88:
------------------------------------------
Has anyone been able to test yet on HBase cluster? I am not using HBase just now, and therefore can't test :0|
> HBaseByteInterface not thread safe
> ----------------------------------
>
> Key: GORA-88
> URL: https://issues.apache.org/jira/browse/GORA-88
> Project: Apache Gora
> Issue Type: Bug
> Components: storage-hbase
> Reporter: Ferdy Galema
> Attachments: GORA-88.patch
>
>
> The recent attempt to make HBaseStore thread safe overlooked the utility class HBaseByteInterface. This class currently uses a single SpecificDatumReader and SpecificDatumWriter for the serialization of embedded records (schemas) within the schema. These reader and writer classes are not thread safe when used with a single schema (at least not yet with the current Avro library in use), much less when using multiple schema's (i.e. writer.setSchema(schema)).
> I will attach a patch that makes proper (re)use of these writer and reader classes, as well as the closely related decoders and encoders. Some workarounds are provided that would be unnecessary with newer versions of Avro, but these can be removed whenever we update this library.
> This class does not break compatibilty; the end result data in HBase is unchanged.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira