You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "stack (JIRA)" <ji...@apache.org> on 2007/06/16 00:13:25 UTC
[jira] Created: (HADOOP-1498) Use primitives -- long, byte [] -- in
place of Long, BytesWritable, etc.
Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
------------------------------------------------------------------------
Key: HADOOP-1498
URL: https://issues.apache.org/jira/browse/HADOOP-1498
Project: Hadoop
Issue Type: Improvement
Components: contrib/hbase
Reporter: stack
Assignee: stack
Priority: Minor
Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
+ HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
+ Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1498) Use primitives -- long, byte [] --
in place of Long, BytesWritable, etc.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12505424 ]
Hadoop QA commented on HADOOP-1498:
-----------------------------------
-1, build or testing failed
2 attempts failed to build and test the latest attachment http://issues.apache.org/jira/secure/attachment/12359915/primitives.patch against trunk revision r547811.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/291/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/291/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1498) Use primitives -- long, byte [] -- in
place of Long, BytesWritable, etc.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HADOOP-1498:
--------------------------
Attachment: primitives-v2.patch
Version 2. Fixes failing TestHRegion unit test.
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives-v2.patch, primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1498) Use primitives -- long, byte [] --
in place of Long, BytesWritable, etc.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506291 ]
Hadoop QA commented on HADOOP-1498:
-----------------------------------
-1, could not apply patch.
The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12359950/primitives-v2.patch as a patch to trunk revision r548794.
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/302/console
Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives-v2.patch, primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1498) Use primitives -- long, byte [] -- in
place of Long, BytesWritable, etc.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HADOOP-1498:
--------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Patch committed closing.
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives-v2.patch, primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1498) Use primitives -- long, byte [] -- in
place of Long, BytesWritable, etc.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HADOOP-1498:
--------------------------
Attachment: primitives.patch
Commit message.
HADOOP-1498 Use primitives -- long, byte [] -- in place of Long,
BytesWritable, etc.
General pattern of changes in the below is:
+ Replace BytesWriable instances with byte [].
+ Remove unnecessary copying of byte arrays.
+ Use ImmutableByteWritable where we need to pass a Comparable.
+ Add in javadoc.
+ New style for loop.
+ Use List instead of Vector where no need of synchronization.
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestHMemcache.java
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestScanner2.java
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/AbstractMergeTestBase.java
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestHLog.java
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestScanner.java
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestHRegion.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HStore.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMerge.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HScannerInterface.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HClient.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionServer.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMaster.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMemcache.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegiondirReader.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HInternalScannerInterface.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HAbstractScanner.java
All above got the general treatment.
Below got general treatment plus whatever we call out below.
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestHStoreKey.java
Added extractFamily tests.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HLogKey.java
Added comment.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/Leases.java
Have leases created, renewed and cancelled using longs rather than
Text. Makes for large savings in object creation.
Javadoc edit. Changed leases Map from TreeMap to HashMap.
(getLeaseId, createLeaseId): Renamed.
(LeaseName, creatLeaseName, getLeaseName): Added.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HStoreKey.java
Added override so can extractFamily with colon appended.
Redid extraction so we used Text#find rather than String.indexOf
Makes for object creations savings.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionServer.java
Pass scannerId long instead of ScannerName creating, cancelling, etc.,
leases for scanners.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HLog.java
Comment.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMaster.java
Use server name hashCode for lease naming.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegion.java
servers, rowsToLocks, locksToRows, stores do not need to be
TreeMaps (They are more expensive than hashmaps).
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HLogKey.java
Added Comment.
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestGet.java
Add in some assertions about what the expected name of the server
is meant to be when we read HRegion.
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestHBaseCluster.java
(testHBaseCluster): Remove catching of exceptions. Just let them out.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HStoreFile.java
(doReconstructionLog): Move bunch of constructor into its own method.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HConstants.java
Added ImmutableBytesWriable defines for DELETE_BYTES and COPMLETE_CACHEFLUSH
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionInterface.java
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HMasterRegionInterface.java
Formatting.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HServerAddress.java
Added intValue to extract port from Integer.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/io/ImmutableBytesWritable.java
Added.
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1498) Use primitives -- long, byte [] -- in
place of Long, BytesWritable, etc.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HADOOP-1498:
--------------------------
Status: Patch Available (was: Open)
Passes local tests. See if Hudson agrees.
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1498) Use primitives -- long, byte [] -- in
place of Long, BytesWritable, etc.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HADOOP-1498:
--------------------------
Status: Patch Available (was: In Progress)
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives-v2.patch, primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1498) Use primitives -- long, byte [] --
in place of Long, BytesWritable, etc.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506041 ]
Hadoop QA commented on HADOOP-1498:
-----------------------------------
Integrated in Hadoop-Nightly #127 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/127/])
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives-v2.patch, primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1498) Use primitives -- long, byte [] --
in place of Long, BytesWritable, etc.
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12505570 ]
Hadoop QA commented on HADOOP-1498:
-----------------------------------
+1
http://issues.apache.org/jira/secure/attachment/12359950/primitives-v2.patch applied and successfully tested against trunk revision r547811.
Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/292/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/292/console
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives-v2.patch, primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-1498) Use primitives -- long, byte [] -- in
place of Long, BytesWritable, etc.
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HADOOP-1498:
--------------------------
Status: In Progress (was: Patch Available)
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-1498) Use primitives -- long, byte [] --
in place of Long, BytesWritable, etc.
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-1498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506258 ]
Hudson commented on HADOOP-1498:
--------------------------------
Integrated in Hadoop-Nightly #129 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/129/])
> Use primitives -- long, byte [] -- in place of Long, BytesWritable, etc.
> ------------------------------------------------------------------------
>
> Key: HADOOP-1498
> URL: https://issues.apache.org/jira/browse/HADOOP-1498
> Project: Hadoop
> Issue Type: Improvement
> Components: contrib/hbase
> Reporter: stack
> Assignee: stack
> Priority: Minor
> Attachments: primitives-v2.patch, primitives.patch
>
>
> Profiling, bulk of allocations are short-lived object wrapping of primitives. Examples include:
> + HADOOP-1483 'Use long primitives creating, renewing, and cancelling leases' now subsumed by this issue.
> + Published APIs for region server, master and clients return ByteWritables or arrays of ByteWritables when could be returning byte [] or byte [][].
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.