You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "stack (Created) (JIRA)" <ji...@apache.org> on 2011/12/16 20:46:30 UTC
[jira] [Created] (HBASE-5056) [performance] On upload
[performance] On upload
------------------------
Key: HBASE-5056
URL: https://issues.apache.org/jira/browse/HBASE-5056
Project: HBase
Issue Type: Improvement
Reporter: stack
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5056) [performance] On upload, 4% of
memory allocations are NullInstance
Posted by "Chris Trezzo (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445485#comment-13445485 ]
Chris Trezzo commented on HBASE-5056:
-------------------------------------
[~stack] Since HFilev2 is the default now, can we consider this issue resolved?
> [performance] On upload, 4% of memory allocations are NullInstance
> ------------------------------------------------------------------
>
> Key: HBASE-5056
> URL: https://issues.apache.org/jira/browse/HBASE-5056
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Reporter: stack
> Labels: noob
> Attachments: NullInstanceHotSpotMemAllocation.html
>
>
> Profiling a PE upload, I see that 4% of memory allocations are instances of NullInstance (caveat profiler's give distorted view of running app as does PE when it comes to real-world workloads). See attached profiler output. Looks like we could save a bunch on object creation if we had our own WritableFactories instance that treated stuff like NullInstance special creating a Singleton NullInstance per declared class type returning these instead of creating new ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5056) [performance] On upload, 4% of
memory allocations are NullInstance
Posted by "stack (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445635#comment-13445635 ]
stack commented on HBASE-5056:
------------------------------
@Chris I don't think so. When I look at attached profiler output, it looks like the NullInstance is allocated in HbaseObjectWritable. It might be gone in trunk because we should be by-passing HbaseObjectWritable now we are pb all the time but would have to check in profiler again (Not sure how the hfile2 relates after this much time has passed).
> [performance] On upload, 4% of memory allocations are NullInstance
> ------------------------------------------------------------------
>
> Key: HBASE-5056
> URL: https://issues.apache.org/jira/browse/HBASE-5056
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Reporter: stack
> Labels: noob
> Attachments: NullInstanceHotSpotMemAllocation.html
>
>
> Profiling a PE upload, I see that 4% of memory allocations are instances of NullInstance (caveat profiler's give distorted view of running app as does PE when it comes to real-world workloads). See attached profiler output. Looks like we could save a bunch on object creation if we had our own WritableFactories instance that treated stuff like NullInstance special creating a Singleton NullInstance per declared class type returning these instead of creating new ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5056) [performance] On upload, 4% of memory
allocations are NullInstance
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5056:
-------------------------
Attachment: NullInstanceHotSpotMemAllocation.html
> [performance] On upload, 4% of memory allocations are NullInstance
> ------------------------------------------------------------------
>
> Key: HBASE-5056
> URL: https://issues.apache.org/jira/browse/HBASE-5056
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Reporter: stack
> Labels: noob
> Attachments: NullInstanceHotSpotMemAllocation.html
>
>
> Profiling a PE upload, I see that 4% of memory allocations are instances of NullInstance (caveat profiler's give distorted view of running app as does PE when it comes to real-world workloads). See attached profiler output. Looks like we could save a bunch on object creation if we had our own WritableFactories instance that treated stuff like NullInstance special creating a Singleton NullInstance per declared class type returning these instead of creating new ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5056) [performance] On upload, 4% of
memory allocations are NullInstance
Posted by "stack (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13172455#comment-13172455 ]
stack commented on HBASE-5056:
------------------------------
Diff in performance seems to be a missing bufferedinput if you go the hfilev1 non-compressed path. Apparently hfilev2 doesn't have this issue. J-D was going to write the list.
> [performance] On upload, 4% of memory allocations are NullInstance
> ------------------------------------------------------------------
>
> Key: HBASE-5056
> URL: https://issues.apache.org/jira/browse/HBASE-5056
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Reporter: stack
> Labels: noob
> Attachments: NullInstanceHotSpotMemAllocation.html
>
>
> Profiling a PE upload, I see that 4% of memory allocations are instances of NullInstance (caveat profiler's give distorted view of running app as does PE when it comes to real-world workloads). See attached profiler output. Looks like we could save a bunch on object creation if we had our own WritableFactories instance that treated stuff like NullInstance special creating a Singleton NullInstance per declared class type returning these instead of creating new ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5056) [performance] On upload, 4% of
memory allocations are NullInstance
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171183#comment-13171183 ]
Zhihong Yu commented on HBASE-5056:
-----------------------------------
0.90 has the same code:
{code}
instanceObj = new NullInstance(declClass, conf);
{code}
So I guess this wouldn't explain the difference in performance.
> [performance] On upload, 4% of memory allocations are NullInstance
> ------------------------------------------------------------------
>
> Key: HBASE-5056
> URL: https://issues.apache.org/jira/browse/HBASE-5056
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Reporter: stack
> Labels: noob
> Attachments: NullInstanceHotSpotMemAllocation.html
>
>
> Profiling a PE upload, I see that 4% of memory allocations are instances of NullInstance (caveat profiler's give distorted view of running app as does PE when it comes to real-world workloads). See attached profiler output. Looks like we could save a bunch on object creation if we had our own WritableFactories instance that treated stuff like NullInstance special creating a Singleton NullInstance per declared class type returning these instead of creating new ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HBASE-5056) [performance] On upload, 4% of memory
allocations are NullInstance
Posted by "stack (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
stack updated HBASE-5056:
-------------------------
Component/s: performance
Description: Profiling a PE upload, I see that 4% of memory allocations are instances of NullInstance (caveat profiler's give distorted view of running app as does PE when it comes to real-world workloads). See attached profiler output. Looks like we could save a bunch on object creation if we had our own WritableFactories instance that treated stuff like NullInstance special creating a Singleton NullInstance per declared class type returning these instead of creating new ones.
Summary: [performance] On upload, 4% of memory allocations are NullInstance (was: [performance] On upload )
> [performance] On upload, 4% of memory allocations are NullInstance
> ------------------------------------------------------------------
>
> Key: HBASE-5056
> URL: https://issues.apache.org/jira/browse/HBASE-5056
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Reporter: stack
> Labels: noob
> Attachments: NullInstanceHotSpotMemAllocation.html
>
>
> Profiling a PE upload, I see that 4% of memory allocations are instances of NullInstance (caveat profiler's give distorted view of running app as does PE when it comes to real-world workloads). See attached profiler output. Looks like we could save a bunch on object creation if we had our own WritableFactories instance that treated stuff like NullInstance special creating a Singleton NullInstance per declared class type returning these instead of creating new ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5056) [performance] On upload, 4% of
memory allocations are NullInstance
Posted by "Zhihong Yu (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HBASE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13171170#comment-13171170 ]
Zhihong Yu commented on HBASE-5056:
-----------------------------------
+1 on proposed change.
> [performance] On upload, 4% of memory allocations are NullInstance
> ------------------------------------------------------------------
>
> Key: HBASE-5056
> URL: https://issues.apache.org/jira/browse/HBASE-5056
> Project: HBase
> Issue Type: Improvement
> Components: performance
> Reporter: stack
> Labels: noob
> Attachments: NullInstanceHotSpotMemAllocation.html
>
>
> Profiling a PE upload, I see that 4% of memory allocations are instances of NullInstance (caveat profiler's give distorted view of running app as does PE when it comes to real-world workloads). See attached profiler output. Looks like we could save a bunch on object creation if we had our own WritableFactories instance that treated stuff like NullInstance special creating a Singleton NullInstance per declared class type returning these instead of creating new ones.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira