You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@harmony.apache.org by Okonechnikov Konstantin <ok...@gmail.com> on 2008/03/20 07:22:06 UTC
[general] GSoC 2008 Refactor Java Bytecode Translator
Hello everyone!
Currently I am solving a problem of optimizing callee-saved register
spilling in JiT opt codegenerator. I find Apache
Harmony project very perspective and
I am intrested in the idea "Refactor Java Bytecode Translator in Harmony
JIT".
The goal is to move all optimizations to HL optimizer or there's
something else? What data structures need to be simplified?
And one more question: do I have to insert wiki into
http://wiki.apache.org/general/SummerOfCode2008 or an appliacation woud be
enough to present my ideas?
With respect,
Okonechnikov Konstantin.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
I'm sorry. Please, disregard the previous letter.
On Fri, Mar 28, 2008 at 1:09 AM, Alexei Fedotov
<al...@gmail.com> wrote:
> And if these baby steps
>
>
>
> On 27 Mar 2008 21:17:58 +0300, Egor Pasko <eg...@gmail.com> wrote:
> > On the 0x413 day of Apache Harmony Okonechnikov Konstantin wrote:
> > > Well, there are some questions.
> > > [IrBuilder]
> > > If we clean out simplification, it wont affect the functionality of
> > > instructions, will it? (i.e.will OPT work without these simplifications?
> > > I ve tried to clean some instructions, it seems everything is OK (HelloWorld
> > > works) )
> >
> > Ideally everything should work if we disable simplification. Although
> > I remember some problems running Eclipse solely on OPT with
> > simplification disabled. That was a long time ago, and if it appears,
> > that's a bug.
> >
> >
> > > I 've written sort of plan. Check this out.
> > > 1) Clean up IRBuilder
> > > -remove Simplifier and its methods
> >
> > sounds good
> >
> >
> > > 2) Move all simplification to Optimizer stage
> > > (this is quite big task as I suppose, lots of instructions use
> > > simplification. Not sure if we should create separate action for every
> > > simplification....)
> > > 1 and 2 should be implemented concurrently
> >
> > actually there is a stage called 'simplify' already that in fact uses
> > the same simplifier. So, it is not a huge work, need to ask somebody
> > to check performance of disabling simplification in translator and
> > running it from HLO right after the translator (with dead code
> > eliminator, blablabla).
> >
> >
> > > 3) Develop the translator
> > > - new PrePass (maybe from Jet?)
> > >
> > > - implement data analisys, parser etc...
> > >
> > > Where am I mistaken? I will appreciate any additions and(or) corrections.
> >
> > Kostya, thanks a lot for the draft plan! I have been quite busy and
> > could not discuss it with you earlier. Sorry for that. I like your
> > pro-active approach very much, please continue behave like that :)
> >
> > Although, I have somewhat different views on the plan, those views are
> > more like discussable than final :) So, plz, do not take personally if
> > I just ruined your best dreams about the process))
> >
> > TA-DAA! Here my suggestion goes..
> >
> > I would recomend to hack the whole code starting from a small and
> > limited implementation and grow functionality doing small baby
> > steps. So, you would not need to implement the rock solid data flow
> > analysis at once. Just write one that looks beautiful by design, does
> > not support non-interesting instructions as well as difficult stuff
> > like exceptions, loops and JSR/RET, put asserts in places, where you
> > do not support something.
> >
> > Then grow functionality until you can run HelloWorld. Constantly
> > ensuring that the code looks good and we love it.
> >
> > This can be split into several sub-steps:
> >
> > 0. move simplifier out of IRBuilder, basically fix HARMONY-5500 or
> > determine that it is not a part of our work if it unexpectedly
> > wants to eat too much time :)
> >
> > 1. draft code that does prepass and data flow analysis in code _without_
> > loops, exceptions, call instructions
> > 1.1. prepass
> > 1.2. data flow analysis
> > 1.3. bytecode mapping
> >
> > 2. support call instructions and lazy resolution
> >
> > 3. with loops
> >
> > 4. with exceptions and loops
> >
> > 5. make sure Hello World works and fix bugs when it does not ;)
> >
> > 6. JSR/RET support
> >
> > 7. inliner support
> >
> > 8. make sure all tests and eclipse work in -Xem:opt mode
> > 8.1. bytecode without JSR/RET
> > 8.2. bytecode obtained by ECJ (with JSR/RET)
> >
> > 9. write barriers (?), other features...
> >
> > 10. what I forgot
> >
> > (I am yet not sure about the order, though, pick what you want to do
> > first)
> >
> > Let's review the code between steps, speak about it, think what we do
> > next and how. It also makes sense to review the code more often, once
> > noticeable features are implemented or code is refactored.
> >
> > My picture is that we will commit it into a separate directory, your
> > code can be running via tweaked emconf file. That is because I do not
> > like branches :)
> >
> > I separated exceptions, loops, resolution stuff because I think it
> > will let us better concentrate on coding in the early stages (what
> > else to separate?). Additionally, I think, exception support needs
> > little change. It has been completely rewritten once, looks not so bad
> > (although scattered around) and no bugs were found in it since
> > then. So, just move the code carefully. Same picture with JSR/RET
> > support. Looks like no need to rewrite.
> >
> > Why is this approach worth trying? Because you can test the code while
> > writing. Make an emconf that uses your code on methods with supported
> > bytecodes and use JET or older OPT for others.
> >
> > Sounds good?
> >
> > --
> > Egor Pasko
> >
> >
>
>
>
> --
> With best regards,
> Alexei
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
And if these baby steps
On 27 Mar 2008 21:17:58 +0300, Egor Pasko <eg...@gmail.com> wrote:
> On the 0x413 day of Apache Harmony Okonechnikov Konstantin wrote:
> > Well, there are some questions.
> > [IrBuilder]
> > If we clean out simplification, it wont affect the functionality of
> > instructions, will it? (i.e.will OPT work without these simplifications?
> > I ve tried to clean some instructions, it seems everything is OK (HelloWorld
> > works) )
>
> Ideally everything should work if we disable simplification. Although
> I remember some problems running Eclipse solely on OPT with
> simplification disabled. That was a long time ago, and if it appears,
> that's a bug.
>
>
> > I 've written sort of plan. Check this out.
> > 1) Clean up IRBuilder
> > -remove Simplifier and its methods
>
> sounds good
>
>
> > 2) Move all simplification to Optimizer stage
> > (this is quite big task as I suppose, lots of instructions use
> > simplification. Not sure if we should create separate action for every
> > simplification....)
> > 1 and 2 should be implemented concurrently
>
> actually there is a stage called 'simplify' already that in fact uses
> the same simplifier. So, it is not a huge work, need to ask somebody
> to check performance of disabling simplification in translator and
> running it from HLO right after the translator (with dead code
> eliminator, blablabla).
>
>
> > 3) Develop the translator
> > - new PrePass (maybe from Jet?)
> >
> > - implement data analisys, parser etc...
> >
> > Where am I mistaken? I will appreciate any additions and(or) corrections.
>
> Kostya, thanks a lot for the draft plan! I have been quite busy and
> could not discuss it with you earlier. Sorry for that. I like your
> pro-active approach very much, please continue behave like that :)
>
> Although, I have somewhat different views on the plan, those views are
> more like discussable than final :) So, plz, do not take personally if
> I just ruined your best dreams about the process))
>
> TA-DAA! Here my suggestion goes..
>
> I would recomend to hack the whole code starting from a small and
> limited implementation and grow functionality doing small baby
> steps. So, you would not need to implement the rock solid data flow
> analysis at once. Just write one that looks beautiful by design, does
> not support non-interesting instructions as well as difficult stuff
> like exceptions, loops and JSR/RET, put asserts in places, where you
> do not support something.
>
> Then grow functionality until you can run HelloWorld. Constantly
> ensuring that the code looks good and we love it.
>
> This can be split into several sub-steps:
>
> 0. move simplifier out of IRBuilder, basically fix HARMONY-5500 or
> determine that it is not a part of our work if it unexpectedly
> wants to eat too much time :)
>
> 1. draft code that does prepass and data flow analysis in code _without_
> loops, exceptions, call instructions
> 1.1. prepass
> 1.2. data flow analysis
> 1.3. bytecode mapping
>
> 2. support call instructions and lazy resolution
>
> 3. with loops
>
> 4. with exceptions and loops
>
> 5. make sure Hello World works and fix bugs when it does not ;)
>
> 6. JSR/RET support
>
> 7. inliner support
>
> 8. make sure all tests and eclipse work in -Xem:opt mode
> 8.1. bytecode without JSR/RET
> 8.2. bytecode obtained by ECJ (with JSR/RET)
>
> 9. write barriers (?), other features...
>
> 10. what I forgot
>
> (I am yet not sure about the order, though, pick what you want to do
> first)
>
> Let's review the code between steps, speak about it, think what we do
> next and how. It also makes sense to review the code more often, once
> noticeable features are implemented or code is refactored.
>
> My picture is that we will commit it into a separate directory, your
> code can be running via tweaked emconf file. That is because I do not
> like branches :)
>
> I separated exceptions, loops, resolution stuff because I think it
> will let us better concentrate on coding in the early stages (what
> else to separate?). Additionally, I think, exception support needs
> little change. It has been completely rewritten once, looks not so bad
> (although scattered around) and no bugs were found in it since
> then. So, just move the code carefully. Same picture with JSR/RET
> support. Looks like no need to rewrite.
>
> Why is this approach worth trying? Because you can test the code while
> writing. Make an emconf that uses your code on methods with supported
> bytecodes and use JET or older OPT for others.
>
> Sounds good?
>
> --
> Egor Pasko
>
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x41D day of Apache Harmony Okonechnikov Konstantin wrote:
> Egor, Ian thanks for helpful information.
>
> I have some good news: Simplifier and CSE removal from IRBuilder is
> accomplished.
Great news, Kostya! I am eager to see how it works ;)
> 1) What is done:
> - Simplifier* IrBuilder::simplifier and all connected methods are removed or
> modified
> - CSEHashtable IrBuilder::cseHashTable and its methods are removed
> - Instruction generation doesn't include simplification and hashing anymore
> - DoSimplify, DoCSE flags are removed from IRBuilderFlags
all cool stuff!
Also consider removing other flags like expandNullChecks it pollutes
the code and does not make much sence. Once I needed to remove all
checks I just fixed some bits in the corresponding optpass :
> 2) Results: WinXP/x86/ia32
> Jvmti, CUnit, Smoke tests passed
> Some kernel tests failed, but they appear to be the same as with
> unmodified IRBuilder, I am investigating this problem.
good good good
I am also thinking about the ways to compare the code after translator
and optimizations, to not kill performance accidentally. The thinking
probably smells like bad news for you since it is probably a lot of
unexpected work.. Sorry for that. Will try to keep the amount of work
reasonable.
The testing methodology sounds like that to me:
1. disable inliner
2. run non-modified version Harmony on HelloWorld in OPT mode, collect all logs
3. run new translator, collect all logs
4. compare the logs recursively
5. classify and analyse the differences
I hope 5 would not be hard, but no guarantee here. Need a short
(perl?) script that compares the logs, ignoring non-essential differencies.
opt path should probably be:
newtranslator,cse,simplify,uce,dce,uce
for the fair comparison probably need to run old version with path:
translator,uce,dce,uce
I am really sorry that so much extra work needs to be done, but we
should be very careful here, it is easy to kill performance in
translator (or gain it?:). I would not insist on very-very tricky ways
to test the changes, just a sanity check to see in practice how it
works. Is ot OK with you?
> 3) TODO: rebuild on Linux; play with OPT chain;
feel free to ask questions on it, even the dummiest ones. I hope to be
quicker answering than now :(
> examine carefully instructions, determine which are not required in IrBuilder
> (as I suppose so called SIMPLIFIER GEN instructions have to be carefully
> investigated); make more thorough testing
> I 've read information on how to commit, but didn't find some facts. What
> are the requirments for commiting the
> code?
basically you need to pass smoke, unit tests. in case of translator I
suggest to run all in OPT mode, also regression tests. (I'll tell how)
AFAIR, Eclipse is also a good catcher of translator bugs, so, I
suggest to run it too, create a new project, type in Hello World
application there, run it. All in OPT mode, of course :)
> Are there any branches for unstable code?
no, we do not like to keep branches of unstable code. You can keep
patches of code in JIRA. When there is a lot of them, we try to
stabilize and commit. Aligned with the general strategy: commit early
and often (though, I cannot say I myself conform to this strategy
completely, eh)
We run more aggressive testing in stabilization periods before
releasing stable builds. Do not need to care about those tests when
committing.
GIT [1] is a powerful version control system to operate on patches,
slightly brain-damaging in the beginning, but I find it extremely
useful. I can give you hints on it.
> Unfortunately I have bad news: due to some technical problems I can't
> access temporarily SVN repository and checkout the code. I was modifying the
> IRBuilder using stable snapshot (r629320). As I found out from browsing
> Harmony Repository
> online, since r629320 2 patches were implemented to IRBuilder ( r636777,
> r645073). So I suppose, when I am able to checkout the code, I will
> need to implement them. I am not sure, maybe this is not a big problem,
> nevertheless... hope it's not that difficult to handle :)
I sent you a gtalk invite to discuss this crap. Feel free to accept it
:)
[1] http://git.or.cz/
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x423 day of Apache Harmony Okonechnikov Konstantin wrote:
> >
> > Okay, so, HARMONY-5500 is to unify the two IRBuilders. Nice. Good idea
> > too)) What is the time estimate for Kostya to do this? we should
> > consider it when deciding the scope of the work.
> >
> > I like that Kostya is now aware of what IRBuilder does, how it
> > interacts with translator, simplifier and InstFactory. This is
> > important to know either way. So the time spent on this is not lost
> > anyway.
> >
> >
> > > I agree this is difficult to imagine an HLO pass which needs
> > > simplification on the fly. But I insist it is needed at the
> > > translation.
> >
> >
> > now I see your point even better
> >
> > --
> >
> > Egor Pasko
> >
> >
> Due to latest discussion I have modified my proposal wiki
> http://wiki.apache.org/general/WikiTranslatorRefactoringProposal
> It's pity that I wasn't in time to do the same with GSoC proposal, but it
> was too late already.
no problem, your original roposal suits well. As far as I understood
you removed the IRBuilder separation, which is probably a too strong
change. You can formulate it as "investigate the output stage of the
Translator by moving IRBuilder optimizations out, study the
feasibility of this refactoring part". You already did most of this,
although, I would like to see the report: did you manage to run
HelloWorld? any other progress indicators?
Some notes on the proposal:
1. I would move bytecode mapping to some later stage, bcmapping is not
necessary to run HelloWorld in OPT mode. (I suspect you mean that
HelloWorld should be checked in OPT mode:)
2. (well, I should have said that before) I do not like the words like
"define specifications", "elaborate development scheme" and "work
out project model". Too much buzz. Please, focus on
functionality. You will move step by step in your
implementation. The proposal outlines the steps, more detailed
steps will be discovered during the fun process. This is good.
> Nevertheless now I focus my efforts on translator (first steps are
> prepass and data analysis).
Hey, I support your aim at this! That should be the most interesting
part of the project. Really good that you can jump into some work
based on reasonable decicions of yours. I will try be guiding you here
and there, but your decisions are always most valuable in terms of
SoC, and it is really important that you are active. Thanks!
> BTW, perhaps this wiki requires some modification too (item 4)
> http://wiki.apache.org/harmony/JIT_Development_Tasks
did you already speak with George on this? I do think it is still TBD,
we will converge to the final decision, I believe. Just not
immediately.
P.S.: I ranked your application, committers, plz, follow me :)
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
>
> Okay, so, HARMONY-5500 is to unify the two IRBuilders. Nice. Good idea
> too)) What is the time estimate for Kostya to do this? we should
> consider it when deciding the scope of the work.
>
> I like that Kostya is now aware of what IRBuilder does, how it
> interacts with translator, simplifier and InstFactory. This is
> important to know either way. So the time spent on this is not lost
> anyway.
>
>
> > I agree this is difficult to imagine an HLO pass which needs
> > simplification on the fly. But I insist it is needed at the
> > translation.
>
>
> now I see your point even better
>
> --
>
> Egor Pasko
>
>
Due to latest discussion I have modified my proposal wiki
http://wiki.apache.org/general/WikiTranslatorRefactoringProposal
It's pity that I wasn't in time to do the same with GSoC proposal, but it
was too late already.
Nevertheless now I focus
my efforts on translator (first steps are prepass and data analysis).
BTW, perhaps this wiki requires some modification too (item 4)
http://wiki.apache.org/harmony/JIT_Development_Tasks
--
Best regards,
Okonechnikov Konstantin
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
George,
Thank you for paying attention to the student projects. They require
someone with your experience. The previous discussion was quite
consistent for me. Egor and Kostya were pretty flexible about my
concern. I'm not a JIT professional, but I'm eager to learn. Could you
please support your disagreement with arguments to help me
understanding that better?
You noticed that it is much easier to destroy than to improve.
Generally, removing the code from a big system may be more difficult
then adding a new code. DRLVM relies on ICU and HyPort in
localization, but I find it painful to remove apr-iconv dependency
(still hope to complete it one day).
Thanks.
On Mon, Apr 7, 2008 at 7:31 AM, George Timoshenko
<ge...@gmail.com> wrote:
> Okonechnikov Konstantin wrote:
>
>
> > I have some good news: Simplifier and CSE removal from IRBuilder is
> > accomplished.
> > 1) What is done:
> > - Simplifier* IrBuilder::simplifier and all connected methods are removed
> or
> > modified
> > - CSEHashtable IrBuilder::cseHashTable and its methods are removed
> > - Instruction generation doesn't include simplification and hashing
> anymore
> > - DoSimplify, DoCSE flags are removed from IRBuilderFlags
> >
>
> That's awful. I see the discussion too late.
>
> This is much easier to destroy rather than to improve.
> Does irBuilder simplify/hvn ability prevents you from writing new
> translator?
>
> I am strongly against the idea of removing simplify/hvn from IRBuilder.
>
>
> In HARMONY-5500 there is NOTHING about this removing.
>
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x420 day of Apache Harmony Alexei Fedotov wrote:
> > physicist
> AFAIK, NSU graduates in physics become incomparable programmers.
I am a mathematician, do not start a holywar here :)
> On Tue, Apr 8, 2008 at 1:36 PM, Okonechnikov Konstantin
> <ok...@gmail.com> wrote:
> > On 4/8/08, George Timoshenko <ge...@gmail.com> wrote:
> > >
> > > Egor Pasko wrote:
> > >
> > > But what is the reason not to?
> > > >
> > >
> > > The main value of simplifying-on-the-fly in my opinion is reducing the
> > > number of generated instructions. Of course, they will be eliminated later
> > > by separate "simplify", but then you get gaps in instruction ids.
> > >
> > > Basing on quite long debugging experience I can say it is a problem when
> > > you see a lot of instruction sets with non-continuous ids. Such region looks
> > > suspicious: the instructions were created by different passes, and you have
> > > to ensure their order and effects are correct. "simplifying-on-the-fly"
> > > allows to reduce the number of such regions.
> > >
> > > The situation reminds me the joke when physicist and mathematician boil
> > > water on the camp-fire. Removing simplifier and CSE from IRBuilder is a
> > > mathematician's approach for me.
> > >
> > > From another side I understand that from the design point of view The idea
> > > of IRBuilder cleanup makes translation phase more clear for new participants
> > > of Jitrino development.
> > > I can not agree this makes translator easier to debug: just switch OFF the
> > > flags and you get pure IRBuilder...
> >
> >
> > Good arguments. I should take them into account.
> > And as a physicist maybe I have to :)
> > So, the problem is not exatly with the IrBuilder itself. Well, at least,
> > It is OK to use it in new translator implementation.
> >
> >
> >
> > > The main example of IRBuilder availability is HLOAPIMagic pass.
> > > It is needed there to manage InstructionFactory, a number of nodes for
> > > generating instructions and labels for them...
> > > The part of IRBuilder is just copy/pasted there - that is what I want to
> > > get rid of.
> > > I agree this is difficult to imagine an HLO pass which needs
> > > simplification on the fly. But I insist it is needed at the translation.
> >
> >
> > Pehaps working on replacing HLOAPIMagicIRBuilder in HLOAPIMagic with
> > IrBuilder or its derived class will
> > help me to get even better understanding on how IrBuilder works (IMO I got
> > familiar with it while I was "cleaning it up"). But of course, I will
> > require community help & support. Also it's not clear for me yet how
> > much time will it take me to manage this task, if I am assigned to it.
> > And I am not quite sure if this task is in the scope of the translator
> > project.
> >
>
>
>
> --
> With best regards,
> Alexei
>
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
> physicist
AFAIK, NSU graduates in physics become incomparable programmers.
On Tue, Apr 8, 2008 at 1:36 PM, Okonechnikov Konstantin
<ok...@gmail.com> wrote:
> On 4/8/08, George Timoshenko <ge...@gmail.com> wrote:
> >
> > Egor Pasko wrote:
> >
> > But what is the reason not to?
> > >
> >
> > The main value of simplifying-on-the-fly in my opinion is reducing the
> > number of generated instructions. Of course, they will be eliminated later
> > by separate "simplify", but then you get gaps in instruction ids.
> >
> > Basing on quite long debugging experience I can say it is a problem when
> > you see a lot of instruction sets with non-continuous ids. Such region looks
> > suspicious: the instructions were created by different passes, and you have
> > to ensure their order and effects are correct. "simplifying-on-the-fly"
> > allows to reduce the number of such regions.
> >
> > The situation reminds me the joke when physicist and mathematician boil
> > water on the camp-fire. Removing simplifier and CSE from IRBuilder is a
> > mathematician's approach for me.
> >
> > From another side I understand that from the design point of view The idea
> > of IRBuilder cleanup makes translation phase more clear for new participants
> > of Jitrino development.
> > I can not agree this makes translator easier to debug: just switch OFF the
> > flags and you get pure IRBuilder...
>
>
> Good arguments. I should take them into account.
> And as a physicist maybe I have to :)
> So, the problem is not exatly with the IrBuilder itself. Well, at least,
> It is OK to use it in new translator implementation.
>
>
>
> > The main example of IRBuilder availability is HLOAPIMagic pass.
> > It is needed there to manage InstructionFactory, a number of nodes for
> > generating instructions and labels for them...
> > The part of IRBuilder is just copy/pasted there - that is what I want to
> > get rid of.
> > I agree this is difficult to imagine an HLO pass which needs
> > simplification on the fly. But I insist it is needed at the translation.
>
>
> Pehaps working on replacing HLOAPIMagicIRBuilder in HLOAPIMagic with
> IrBuilder or its derived class will
> help me to get even better understanding on how IrBuilder works (IMO I got
> familiar with it while I was "cleaning it up"). But of course, I will
> require community help & support. Also it's not clear for me yet how
> much time will it take me to manage this task, if I am assigned to it.
> And I am not quite sure if this task is in the scope of the translator
> project.
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
On 4/8/08, George Timoshenko <ge...@gmail.com> wrote:
>
> Egor Pasko wrote:
>
> But what is the reason not to?
> >
>
> The main value of simplifying-on-the-fly in my opinion is reducing the
> number of generated instructions. Of course, they will be eliminated later
> by separate "simplify", but then you get gaps in instruction ids.
>
> Basing on quite long debugging experience I can say it is a problem when
> you see a lot of instruction sets with non-continuous ids. Such region looks
> suspicious: the instructions were created by different passes, and you have
> to ensure their order and effects are correct. "simplifying-on-the-fly"
> allows to reduce the number of such regions.
>
> The situation reminds me the joke when physicist and mathematician boil
> water on the camp-fire. Removing simplifier and CSE from IRBuilder is a
> mathematician's approach for me.
>
> From another side I understand that from the design point of view The idea
> of IRBuilder cleanup makes translation phase more clear for new participants
> of Jitrino development.
> I can not agree this makes translator easier to debug: just switch OFF the
> flags and you get pure IRBuilder...
Good arguments. I should take them into account.
And as a physicist maybe I have to :)
So, the problem is not exatly with the IrBuilder itself. Well, at least,
It is OK to use it in new translator implementation.
> The main example of IRBuilder availability is HLOAPIMagic pass.
> It is needed there to manage InstructionFactory, a number of nodes for
> generating instructions and labels for them...
> The part of IRBuilder is just copy/pasted there - that is what I want to
> get rid of.
> I agree this is difficult to imagine an HLO pass which needs
> simplification on the fly. But I insist it is needed at the translation.
Pehaps working on replacing HLOAPIMagicIRBuilder in HLOAPIMagic with
IrBuilder or its derived class will
help me to get even better understanding on how IrBuilder works (IMO I got
familiar with it while I was "cleaning it up"). But of course, I will
require community help & support. Also it's not clear for me yet how
much time will it take me to manage this task, if I am assigned to it.
And I am not quite sure if this task is in the scope of the translator
project.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
On 4/7/08, George Timoshenko <ge...@gmail.com> wrote:
>
> Okonechnikov Konstantin wrote:
>
> I have some good news: Simplifier and CSE removal from IRBuilder is
> > accomplished.
> > 1) What is done:
> > - Simplifier* IrBuilder::simplifier and all connected methods are
> > removed or
> > modified
> > - CSEHashtable IrBuilder::cseHashTable and its methods are removed
> > - Instruction generation doesn't include simplification and hashing
> > anymore
> > - DoSimplify, DoCSE flags are removed from IRBuilderFlags
> >
>
> That's awful. I see the discussion too late.
>
> This is much easier to destroy rather than to improve.
> Does irBuilder simplify/hvn ability prevents you from writing new
> translator?
I thought that the problem was that it
is not easy to reuse IRBuilder, and if simplifier and hvn has the same
funcionality as IrBuilder built-in optimizations, why it is so
improtant to leave it?
I am a bit confused. As Egor and Alexei
I will appreciate clarification and explanation very much. What is the
better way to improve IrBuilder?
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x41F day of Apache Harmony George Timoshenko wrote:
> Okonechnikov Konstantin wrote:
>
> > I have some good news: Simplifier and CSE removal from IRBuilder is
> > accomplished.
> > 1) What is done:
> > - Simplifier* IrBuilder::simplifier and all connected methods are removed or
> > modified
> > - CSEHashtable IrBuilder::cseHashTable and its methods are removed
> > - Instruction generation doesn't include simplification and hashing anymore
> > - DoSimplify, DoCSE flags are removed from IRBuilderFlags
>
> That's awful. I see the discussion too late.
BTW, so good to see you here, George :)
> This is much easier to destroy rather than to improve.
> Does irBuilder simplify/hvn ability prevents you from writing new
> translator?
>
> I am strongly against the idea of removing simplify/hvn from IRBuilder.
>
>
> In HARMONY-5500 there is NOTHING about this removing.
>
>
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x420 day of Apache Harmony George Timoshenko wrote:
> Egor Pasko wrote:
>
> > But what is the reason not to?
>
> The main value of simplifying-on-the-fly in my opinion is reducing the
> number of generated instructions. Of course, they will be eliminated
> later by separate "simplify", but then you get gaps in instruction ids.
>
> Basing on quite long debugging experience I can say it is a problem
> when you see a lot of instruction sets with non-continuous ids. Such
> region looks suspicious: the instructions were created by different
> passes, and you have to ensure their order and effects are
> correct. "simplifying-on-the-fly" allows to reduce the number of such
> regions.
>
> The situation reminds me the joke when physicist and mathematician
> boil water on the camp-fire. Removing simplifier and CSE from
> IRBuilder is a mathematician's approach for me.
>
> From another side I understand that from the design point of view The
> idea of IRBuilder cleanup makes translation phase more clear for new
> participants of Jitrino development.
> I can not agree this makes translator easier to debug: just switch OFF
> the flags and you get pure IRBuilder...
now I see the reason of your complains. Thanks, George. I have no
perfect plan in my mind yet. How do you like the idea to renumber
instructions thus making them not so suspicious? :)
BTW, we will see nothing suspicious after translator, then
suspiciousness will grow, but the growth is trackable and
controllable, thus making it not so suspicious :)
>
> > please, elaborate more on how you see the HARMONY-5500. Especially this:
> > All HLO optimizations creates necessary instructions "manually". And
> > sometimes it is not very convenient.
> > what are examples of not very convenient? I suspect on-the-fly
> > optimization is not very convenient, but I might be wrong.
>
> The main example of IRBuilder availability is HLOAPIMagic pass.
> It is needed there to manage InstructionFactory, a number of nodes for
> generating instructions and labels for them...
> The part of IRBuilder is just copy/pasted there - that is what I want
> to get rid of.
Okay, so, HARMONY-5500 is to unify the two IRBuilders. Nice. Good idea
too)) What is the time estimate for Kostya to do this? we should
consider it when deciding the scope of the work.
I like that Kostya is now aware of what IRBuilder does, how it
interacts with translator, simplifier and InstFactory. This is
important to know either way. So the time spent on this is not lost
anyway.
> I agree this is difficult to imagine an HLO pass which needs
> simplification on the fly. But I insist it is needed at the
> translation.
now I see your point even better
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by George Timoshenko <ge...@gmail.com>.
Egor Pasko wrote:
> But what is the reason not to?
The main value of simplifying-on-the-fly in my opinion is reducing the
number of generated instructions. Of course, they will be eliminated
later by separate "simplify", but then you get gaps in instruction ids.
Basing on quite long debugging experience I can say it is a problem when
you see a lot of instruction sets with non-continuous ids. Such region
looks suspicious: the instructions were created by different passes, and
you have to ensure their order and effects are correct.
"simplifying-on-the-fly" allows to reduce the number of such regions.
The situation reminds me the joke when physicist and mathematician boil
water on the camp-fire. Removing simplifier and CSE from IRBuilder is a
mathematician's approach for me.
From another side I understand that from the design point of view The
idea of IRBuilder cleanup makes translation phase more clear for new
participants of Jitrino development.
I can not agree this makes translator easier to debug: just switch OFF
the flags and you get pure IRBuilder...
> please, elaborate more on how you see the HARMONY-5500. Especially this:
>
> All HLO optimizations creates necessary instructions "manually". And
> sometimes it is not very convenient.
>
> what are examples of not very convenient? I suspect on-the-fly
> optimization is not very convenient, but I might be wrong.
The main example of IRBuilder availability is HLOAPIMagic pass.
It is needed there to manage InstructionFactory, a number of nodes for
generating instructions and labels for them...
The part of IRBuilder is just copy/pasted there - that is what I want to
get rid of.
I agree this is difficult to imagine an HLO pass which needs
simplification on the fly. But I insist it is needed at the translation.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x41F day of Apache Harmony George Timoshenko wrote:
> Okonechnikov Konstantin wrote:
>
> > I have some good news: Simplifier and CSE removal from IRBuilder is
> > accomplished.
> > 1) What is done:
> > - Simplifier* IrBuilder::simplifier and all connected methods are removed or
> > modified
> > - CSEHashtable IrBuilder::cseHashTable and its methods are removed
> > - Instruction generation doesn't include simplification and hashing anymore
> > - DoSimplify, DoCSE flags are removed from IRBuilderFlags
>
> That's awful. I see the discussion too late.
>
> This is much easier to destroy rather than to improve.
> Does irBuilder simplify/hvn ability prevents you from writing new
> translator?
no, it does not prevent. Kostya started from this task to better
understand what translator does and improve the output side of
translator.
> I am strongly against the idea of removing simplify/hvn from IRBuilder.
>
>
> In HARMONY-5500 there is NOTHING about this removing.
yep, nothing. But what is the reason not to? I see only one: faster
translator on systems without JET. This is not a good reason, IMHO.
I remember Mikhail Fursov complaining about optimizing nature of
translator that makes it harder to debug bytecode -> IR mapping.
please, elaborate more on how you see the HARMONY-5500. Especially this:
All HLO optimizations creates necessary instructions "manually". And
sometimes it is not very convenient.
what are examples of not very convenient? I suspect on-the-fly
optimization is not very convenient, but I might be wrong.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by George Timoshenko <ge...@gmail.com>.
Okonechnikov Konstantin wrote:
> I have some good news: Simplifier and CSE removal from IRBuilder is
> accomplished.
> 1) What is done:
> - Simplifier* IrBuilder::simplifier and all connected methods are removed or
> modified
> - CSEHashtable IrBuilder::cseHashTable and its methods are removed
> - Instruction generation doesn't include simplification and hashing anymore
> - DoSimplify, DoCSE flags are removed from IRBuilderFlags
That's awful. I see the discussion too late.
This is much easier to destroy rather than to improve.
Does irBuilder simplify/hvn ability prevents you from writing new
translator?
I am strongly against the idea of removing simplify/hvn from IRBuilder.
In HARMONY-5500 there is NOTHING about this removing.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
Egor, Ian thanks for helpful information.
I have some good news: Simplifier and CSE removal from IRBuilder is
accomplished.
1) What is done:
- Simplifier* IrBuilder::simplifier and all connected methods are removed or
modified
- CSEHashtable IrBuilder::cseHashTable and its methods are removed
- Instruction generation doesn't include simplification and hashing anymore
- DoSimplify, DoCSE flags are removed from IRBuilderFlags
2) Results: WinXP/x86/ia32
Jvmti, CUnit, Smoke tests passed
Some kernel tests failed, but they appear to be the same as with
unmodified IRBuilder, I am investigating this problem.
3) TODO: rebuild on Linux; play with OPT chain;
examine carefully instructions, determine which are not required in IrBuilder
(as I suppose so called SIMPLIFIER GEN instructions have to be carefully
investigated); make more thorough testing
I 've read information on how to commit, but didn't find some facts. What
are the requirments for commiting the
code? Are there any branches for unstable code?
Unfortunately I have bad news: due to some technical problems I can't
access temporarily SVN repository and checkout the code. I was modifying the
IRBuilder using stable snapshot (r629320). As I found out from browsing
Harmony Repository
online, since r629320 2 patches were implemented to IRBuilder ( r636777,
r645073). So I suppose, when I am able to checkout the code, I will
need to implement them. I am not sure, maybe this is not a big problem,
nevertheless... hope it's not that difficult to handle :)
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x41D day of Apache Harmony Ian Rogers wrote:
> Egor Pasko wrote:
> > On the 0x41C day of Apache Harmony Okonechnikov Konstantin wrote:
> >
> >> Continue working on removing SCE. Have some questions:
> >> What are "tau operations"? What does "tau prefix" mean?
> >> And what about those nasty
> >> warnings (necessary simplification) in genTauCheckNull etc?
> >>
> >
> > Oh, yeah, this is a big story :)) Everybody asks me about it, and I
> > often forget the tricky details, so, referring to some explanations
> > back in time sounds reasonable to me.
> >
>
> Tau operations sound similar to guarded operations:
> http://www.research.ibm.com/jalapeno/publication.html#paste99
I cannot find the definition of guarded operations in the article. The
concept of Factored Control Flow Graph looks like anti-tau :)
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Ian Rogers <ro...@gmail.com>.
Egor Pasko wrote:
> On the 0x41C day of Apache Harmony Okonechnikov Konstantin wrote:
>
>> Continue working on removing SCE. Have some questions:
>> What are "tau operations"? What does "tau prefix" mean?
>> And what about those nasty
>> warnings (necessary simplification) in genTauCheckNull etc?
>>
>
> Oh, yeah, this is a big story :)) Everybody asks me about it, and I
> often forget the tricky details, so, referring to some explanations
> back in time sounds reasonable to me.
>
Tau operations sound similar to guarded operations:
http://www.research.ibm.com/jalapeno/publication.html#paste99
Regards,
Ian
--
http://www.cs.man.ac.uk/~irogers/
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x41C day of Apache Harmony Okonechnikov Konstantin wrote:
> Continue working on removing SCE. Have some questions:
> What are "tau operations"? What does "tau prefix" mean?
> And what about those nasty
> warnings (necessary simplification) in genTauCheckNull etc?
Oh, yeah, this is a big story :)) Everybody asks me about it, and I
often forget the tricky details, so, referring to some explanations
back in time sounds reasonable to me.
Here the references are. One short explanation of TauDiv: [1]. Patent
application for the tau framework [2] is more interesting.
It is easy to explain the situation using chknull instruction
(Op_TauCheckNull) as an example.
suppose we have invokevirtual bytecode instruction. When invoked it
should throw a NullPointerException if the class is null, you
know. Here how it is represented in Jitrino HIR:
chknull class1 -) tau1
ldvtable ((tau1)) class1 -) vtable1
ldvfnslot ((tau1)) vtable1::method -) address1
call ((tau1)) address1 params
ldvtable and ldvfnslot abstract away the means to get the address of
the method.
chknull is of course at the end of the basic block, it throws the
exception if class1 is null.
so, why tau?
suppose we want to move instructions in HIR to optimize the code. But
we need to somehow keep in mind what instructions can be moved and how
far. Here call instruction has a tau source operand produced by
chknull ensuring that call is not moved above the definition of tau1,
which is _exactly_ the same semantics as "nullcheck has to precede the
call". Thus it is the way to describe such precedence rules in HIR.
another example:
-----------------------
chknull class1 -) tau1
...
call ((tau1)) ...
chknull class2 -> tau2
...
call ((tau2)) ...
-----------------------
CSE converts it to:
-----------------------
chknull class1 -) tau1
...
call ((tau1)) ...
...
call ((tau1)) ...
-----------------------
which is a magic. Nullcheck was eliminated with CSE without prior
knowledge about semantics of the nullchecks. The idea was invented
specifically for such things: describing control flow dependencies
using data flow dependencies.
In the code you are referring to translator tries to optimise on the
fly. For example, replacing code like this:
MyClass m = null;
m.run();
with:
throw new NullPointerException();
And this is not a job of translator to do things like that. Simplifier
does that, actually.
TauSafe means that you canmove your instruction anywhere. TauUnsafe
pins the instruction position.
There are also tauedge and taupoint, I tend to forget these details,
you can find more info in [2].
[1] http://thread.gmane.org/gmane.comp.java.harmony.devel/24471/focus=24474
[2] http://www.google.com/patents?id=Ht-XAAAAEBAJ&printsec=abstract&zoom=4&dq=tau+operations+compiler
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
Continue working on removing SCE. Have some questions:
What are "tau operations"? What does "tau prefix" mean?
And what about those nasty
warnings (necessary simplification) in genTauCheckNull etc?
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x41A day of Apache Harmony Okonechnikov Konstantin wrote:
> On 02 Apr 2008 01:53:18 +0400, Egor Pasko <eg...@gmail.com> wrote:
> >
> > what do you mean by requirements? they should optimize the code, that
> > is one requirement, but if they make things slower in some cases, that
> > happens too :)
> >
> >
> I meant, is there anything specific about those passes that I should know?
Ah, I see, should've guessed that
> do they require ssa, for example, or any other passes to be run right before
> them? (compatibility etc.)
HVN requires SSA, Simplifier can work on both SSA and non-SSA. From
the top of my head I cannot remember any other requirements for hvn
and simplify. They can raise once you run tests in debug mode :/
If anybody remembers any problems with hvn and simplify not used
"properly", plz, speak up.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
On 02 Apr 2008 01:53:18 +0400, Egor Pasko <eg...@gmail.com> wrote:
>
> what do you mean by requirements? they should optimize the code, that
> is one requirement, but if they make things slower in some cases, that
> happens too :)
>
>
I meant, is there anything specific about those passes that I should know?
do they require ssa, for example, or any other passes to be run right before
them? (compatibility etc.)
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x41A day of Apache Harmony Alexei Fedotov wrote:
> Egor, Kostya,
>
> Let me make my point a bit more precise: I'm afraid that the abstract
> grammar is the only thing we may reuse among components efficiently.
> So, reusability and generators are not two different tasks: generator
> is a possible approach to reusability.
>
> From a technical point of view ASN.1 is used for parsing a binary
> stream in runtime. It is quite efficient to populate C structures with
> data.
in case of bytecode ASN.1 structures will be linear cequences. No
tricky nested structures like those ASN.1 is good to parse. I see no
point in ASN.1 here. But you still have a chance to onvince me
otherwise :)
> Thanks!
>
> On 02 Apr 2008 11:52:25 +0400, Egor Pasko <eg...@gmail.com> wrote:
> >
> > On the 0x41A day of Apache Harmony Okonechnikov Konstantin wrote:
> > > On 4/2/08, Alexei Fedotov <al...@gmail.com> wrote:
> > > >
> > > > One more idea on further rewriting.
> > > > 1. An old verifier built some IR on the top of bytecode.
> > > > 2. A new verifier also does parsing, but is more like JET.
> > > >
> > > > You may think of building a reusable component. I wonder if
> > > > conventional parser generators or ASN.1 are applicable here. Note,
> > > > thinking is different from starting doing, there are cases when a
> > > > specific solution is better than reusable.
> > > >
> > > >
> > > >
> > > > On Wed, Apr 2, 2008 at 3:02 AM, Alexei Fedotov <al...@gmail.com>
> > > > wrote:
> > > > > Hello, Kostya,
> > > > >
> > > > > Do you think it would be good to reflect Egor's (and mine) idea of an
> > > > > evolutionary path during learning period during the first step of your
> > > > > plan? Namely, incremental approach to IR separation are subsequent
> > > > > patches to the existing code base. Each of the patches does the
> > > > > following:
> > > > > 1. Removes some optimization from the parsing stage.
> > > > > 2. Adds necessary infrastructure to HIR.
> > > > > 3. Adds or ensures that the optimization is done on HIR.
> > > > >
> > > > > This would help attacking the problem and learning a parser, HIR and
> > > > > tools incrementally. On completion of this step you would have your
> > > > > code committed to Harmony and get more confidence. Moreover the code
> > > > > which enhances HIR would be close to final. In process you will learn
> > > > > why the existing IR builder is bad and grow a desire to attack it
> > > > > fully rewriting the code.
> > > > >
> > > > > What do you think?
> > > > >
> > > >
> > > > Hi, Alexei!
> > > I like this approach with "first steps".
> > > Actually I have already started with IRBuilder refactoring and completely
> > > removed Simplifier from the code, tested on HelloWorld. (maybe I
> > > should try more complicated tests) Now
> > > I work on removing SCE now. Then there will be the first patch.
> > > But it suprises me that IRBuilder should be completely rewritten!
> >
> > I did not see anybody say so. I do not think that IRBuilder should be
> > rewritten. As you figured out, removing CSE and Simplifier from
> > IRBuilder makes IRBuilder better. So, let's do just that.
> >
> >
> > > I thought the problem was only with translator... I suppose when I am
> > > finished with IRBuilder refactoring, to start prepass and data analisys (as
> > > Egor advised), so on... and later to use IRBuilder in new translator.
> >
> > sounds like a plan :)
> >
> >
> > > P.S. About verifier and parser generators: I will ivestigate this
> > > problem carefully.
> >
> > My opinion on parser generators: Java bytecode structure was
> > specifically designed for the ease and speed of parsing. Parser
> > generators are usually used to parse tricker and much tricker
> > grammars. Sounds like an overkill to use them.
> >
> > --
> > Egor Pasko
> >
> >
>
>
>
> --
> With best regards,
> Alexei
>
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x41A day of Apache Harmony Okonechnikov Konstantin wrote:
> On 4/2/08, Alexei Fedotov <al...@gmail.com> wrote:
> >
> > > Actually I have already started with IRBuilder refactoring and
> > completely
> > removed Simplifier from the code, tested on HelloWorld. (maybe I
> > should try more complicated tests)
> >
> >
> > Sorry for a dumb question. Have you turned OPT on? (May be this does
> > not matter, I'm not a JIT guru)
> >
> > It matters! OPT translator (which is supposed to be refactored) uses
> IRBuilder to create HIR.
> Currently I use Xem:opt for testing ( standard opt emoconf file without any
> modifications).
Kostya, I had no doubt you do so :)
> Again about generators: can't say yet anything
> particular, I am not very familiar with this technique,
> need to take look at it
> . Specific translator implementing is clear to my mind, and I think there are
> ways to make it maybe not reusable, but good-structured and
> convinient for modification and evolution.
this might take a while. And probably time to open the Aho and Ullman book :)
This technique/theory might seem difficult at the first glance
(basically the course takes one semester). You probably would not have
as much time on this, and that is totally OK, since not a part of your
SoC project. I suspect we will encounter many other cases of
ideally-would-be-good-to-fix, but we should not let them loose our
focus.
I must admit that you seem to like the idea to look into possibilities
to reuse parser generator code. I would not recomend against that. If
you happen to quickly prototype it, that's super-cool, but if not, no
problem, just skip.
Thanks for being so active! this is a key not just to open source
projects :)
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
On 4/2/08, Alexei Fedotov <al...@gmail.com> wrote:
>
> > Actually I have already started with IRBuilder refactoring and
> completely
> removed Simplifier from the code, tested on HelloWorld. (maybe I
> should try more complicated tests)
>
>
> Sorry for a dumb question. Have you turned OPT on? (May be this does
> not matter, I'm not a JIT guru)
>
> It matters! OPT translator (which is supposed to be refactored) uses
IRBuilder to create HIR.
Currently I use Xem:opt for testing ( standard opt emoconf file without any
modifications).
Again about generators: can't say yet anything
particular, I am not very familiar with this technique,
need to take look at it
. Specific translator implementing is clear to my mind, and I think there are
ways to make it maybe not reusable, but good-structured and
convinient for modification and evolution.
Best regards,
Okonechnikov Konstantin.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
> Actually I have already started with IRBuilder refactoring and completely
removed Simplifier from the code, tested on HelloWorld. (maybe I
should try more complicated tests)
Sorry for a dumb question. Have you turned OPT on? (May be this does
not matter, I'm not a JIT guru)
On Wed, Apr 2, 2008 at 11:37 AM, Okonechnikov Konstantin
<ok...@gmail.com> wrote:
>
> On 4/2/08, Alexei Fedotov <al...@gmail.com> wrote:
> >
> > One more idea on further rewriting.
> > 1. An old verifier built some IR on the top of bytecode.
> > 2. A new verifier also does parsing, but is more like JET.
> >
> > You may think of building a reusable component. I wonder if
> > conventional parser generators or ASN.1 are applicable here. Note,
> > thinking is different from starting doing, there are cases when a
> > specific solution is better than reusable.
> >
> >
> >
> > On Wed, Apr 2, 2008 at 3:02 AM, Alexei Fedotov <al...@gmail.com>
> > wrote:
> > > Hello, Kostya,
> > >
> > > Do you think it would be good to reflect Egor's (and mine) idea of an
> > > evolutionary path during learning period during the first step of your
> > > plan? Namely, incremental approach to IR separation are subsequent
> > > patches to the existing code base. Each of the patches does the
> > > following:
> > > 1. Removes some optimization from the parsing stage.
> > > 2. Adds necessary infrastructure to HIR.
> > > 3. Adds or ensures that the optimization is done on HIR.
> > >
> > > This would help attacking the problem and learning a parser, HIR and
> > > tools incrementally. On completion of this step you would have your
> > > code committed to Harmony and get more confidence. Moreover the code
> > > which enhances HIR would be close to final. In process you will learn
> > > why the existing IR builder is bad and grow a desire to attack it
> > > fully rewriting the code.
> > >
> > > What do you think?
> > >
> >
> > Hi, Alexei!
> I like this approach with "first steps".
> Actually I have already started with IRBuilder refactoring and completely
> removed Simplifier from the code, tested on HelloWorld. (maybe I
> should try more complicated tests) Now
> I work on removing SCE now. Then there will be the first patch.
> But it suprises me that IRBuilder should be completely rewritten!
> I thought the problem was only with translator... I suppose when I am
> finished with IRBuilder refactoring, to start prepass and data analisys (as
> Egor advised), so on... and later to use IRBuilder in new translator.
> P.S. About verifier and parser generators: I will ivestigate this
> problem carefully.
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
Kostya,
You are doing great. I believe reflecting your progress in the
application would add some weight to it.
On Wed, Apr 2, 2008 at 11:37 AM, Okonechnikov Konstantin
<ok...@gmail.com> wrote:
>
> On 4/2/08, Alexei Fedotov <al...@gmail.com> wrote:
> >
> > One more idea on further rewriting.
> > 1. An old verifier built some IR on the top of bytecode.
> > 2. A new verifier also does parsing, but is more like JET.
> >
> > You may think of building a reusable component. I wonder if
> > conventional parser generators or ASN.1 are applicable here. Note,
> > thinking is different from starting doing, there are cases when a
> > specific solution is better than reusable.
> >
> >
> >
> > On Wed, Apr 2, 2008 at 3:02 AM, Alexei Fedotov <al...@gmail.com>
> > wrote:
> > > Hello, Kostya,
> > >
> > > Do you think it would be good to reflect Egor's (and mine) idea of an
> > > evolutionary path during learning period during the first step of your
> > > plan? Namely, incremental approach to IR separation are subsequent
> > > patches to the existing code base. Each of the patches does the
> > > following:
> > > 1. Removes some optimization from the parsing stage.
> > > 2. Adds necessary infrastructure to HIR.
> > > 3. Adds or ensures that the optimization is done on HIR.
> > >
> > > This would help attacking the problem and learning a parser, HIR and
> > > tools incrementally. On completion of this step you would have your
> > > code committed to Harmony and get more confidence. Moreover the code
> > > which enhances HIR would be close to final. In process you will learn
> > > why the existing IR builder is bad and grow a desire to attack it
> > > fully rewriting the code.
> > >
> > > What do you think?
> > >
> >
> > Hi, Alexei!
> I like this approach with "first steps".
> Actually I have already started with IRBuilder refactoring and completely
> removed Simplifier from the code, tested on HelloWorld. (maybe I
> should try more complicated tests) Now
> I work on removing SCE now. Then there will be the first patch.
> But it suprises me that IRBuilder should be completely rewritten!
> I thought the problem was only with translator... I suppose when I am
> finished with IRBuilder refactoring, to start prepass and data analisys (as
> Egor advised), so on... and later to use IRBuilder in new translator.
> P.S. About verifier and parser generators: I will ivestigate this
> problem carefully.
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
Egor, Kostya,
Let me make my point a bit more precise: I'm afraid that the abstract
grammar is the only thing we may reuse among components efficiently.
So, reusability and generators are not two different tasks: generator
is a possible approach to reusability.
>From a technical point of view ASN.1 is used for parsing a binary
stream in runtime. It is quite efficient to populate C structures with
data.
Thanks!
On 02 Apr 2008 11:52:25 +0400, Egor Pasko <eg...@gmail.com> wrote:
>
> On the 0x41A day of Apache Harmony Okonechnikov Konstantin wrote:
> > On 4/2/08, Alexei Fedotov <al...@gmail.com> wrote:
> > >
> > > One more idea on further rewriting.
> > > 1. An old verifier built some IR on the top of bytecode.
> > > 2. A new verifier also does parsing, but is more like JET.
> > >
> > > You may think of building a reusable component. I wonder if
> > > conventional parser generators or ASN.1 are applicable here. Note,
> > > thinking is different from starting doing, there are cases when a
> > > specific solution is better than reusable.
> > >
> > >
> > >
> > > On Wed, Apr 2, 2008 at 3:02 AM, Alexei Fedotov <al...@gmail.com>
> > > wrote:
> > > > Hello, Kostya,
> > > >
> > > > Do you think it would be good to reflect Egor's (and mine) idea of an
> > > > evolutionary path during learning period during the first step of your
> > > > plan? Namely, incremental approach to IR separation are subsequent
> > > > patches to the existing code base. Each of the patches does the
> > > > following:
> > > > 1. Removes some optimization from the parsing stage.
> > > > 2. Adds necessary infrastructure to HIR.
> > > > 3. Adds or ensures that the optimization is done on HIR.
> > > >
> > > > This would help attacking the problem and learning a parser, HIR and
> > > > tools incrementally. On completion of this step you would have your
> > > > code committed to Harmony and get more confidence. Moreover the code
> > > > which enhances HIR would be close to final. In process you will learn
> > > > why the existing IR builder is bad and grow a desire to attack it
> > > > fully rewriting the code.
> > > >
> > > > What do you think?
> > > >
> > >
> > > Hi, Alexei!
> > I like this approach with "first steps".
> > Actually I have already started with IRBuilder refactoring and completely
> > removed Simplifier from the code, tested on HelloWorld. (maybe I
> > should try more complicated tests) Now
> > I work on removing SCE now. Then there will be the first patch.
> > But it suprises me that IRBuilder should be completely rewritten!
>
> I did not see anybody say so. I do not think that IRBuilder should be
> rewritten. As you figured out, removing CSE and Simplifier from
> IRBuilder makes IRBuilder better. So, let's do just that.
>
>
> > I thought the problem was only with translator... I suppose when I am
> > finished with IRBuilder refactoring, to start prepass and data analisys (as
> > Egor advised), so on... and later to use IRBuilder in new translator.
>
> sounds like a plan :)
>
>
> > P.S. About verifier and parser generators: I will ivestigate this
> > problem carefully.
>
> My opinion on parser generators: Java bytecode structure was
> specifically designed for the ease and speed of parsing. Parser
> generators are usually used to parse tricker and much tricker
> grammars. Sounds like an overkill to use them.
>
> --
> Egor Pasko
>
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x41A day of Apache Harmony Okonechnikov Konstantin wrote:
> On 4/2/08, Alexei Fedotov <al...@gmail.com> wrote:
> >
> > One more idea on further rewriting.
> > 1. An old verifier built some IR on the top of bytecode.
> > 2. A new verifier also does parsing, but is more like JET.
> >
> > You may think of building a reusable component. I wonder if
> > conventional parser generators or ASN.1 are applicable here. Note,
> > thinking is different from starting doing, there are cases when a
> > specific solution is better than reusable.
> >
> >
> >
> > On Wed, Apr 2, 2008 at 3:02 AM, Alexei Fedotov <al...@gmail.com>
> > wrote:
> > > Hello, Kostya,
> > >
> > > Do you think it would be good to reflect Egor's (and mine) idea of an
> > > evolutionary path during learning period during the first step of your
> > > plan? Namely, incremental approach to IR separation are subsequent
> > > patches to the existing code base. Each of the patches does the
> > > following:
> > > 1. Removes some optimization from the parsing stage.
> > > 2. Adds necessary infrastructure to HIR.
> > > 3. Adds or ensures that the optimization is done on HIR.
> > >
> > > This would help attacking the problem and learning a parser, HIR and
> > > tools incrementally. On completion of this step you would have your
> > > code committed to Harmony and get more confidence. Moreover the code
> > > which enhances HIR would be close to final. In process you will learn
> > > why the existing IR builder is bad and grow a desire to attack it
> > > fully rewriting the code.
> > >
> > > What do you think?
> > >
> >
> > Hi, Alexei!
> I like this approach with "first steps".
> Actually I have already started with IRBuilder refactoring and completely
> removed Simplifier from the code, tested on HelloWorld. (maybe I
> should try more complicated tests) Now
> I work on removing SCE now. Then there will be the first patch.
> But it suprises me that IRBuilder should be completely rewritten!
I did not see anybody say so. I do not think that IRBuilder should be
rewritten. As you figured out, removing CSE and Simplifier from
IRBuilder makes IRBuilder better. So, let's do just that.
> I thought the problem was only with translator... I suppose when I am
> finished with IRBuilder refactoring, to start prepass and data analisys (as
> Egor advised), so on... and later to use IRBuilder in new translator.
sounds like a plan :)
> P.S. About verifier and parser generators: I will ivestigate this
> problem carefully.
My opinion on parser generators: Java bytecode structure was
specifically designed for the ease and speed of parsing. Parser
generators are usually used to parse tricker and much tricker
grammars. Sounds like an overkill to use them.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
On 4/2/08, Alexei Fedotov <al...@gmail.com> wrote:
>
> One more idea on further rewriting.
> 1. An old verifier built some IR on the top of bytecode.
> 2. A new verifier also does parsing, but is more like JET.
>
> You may think of building a reusable component. I wonder if
> conventional parser generators or ASN.1 are applicable here. Note,
> thinking is different from starting doing, there are cases when a
> specific solution is better than reusable.
>
>
>
> On Wed, Apr 2, 2008 at 3:02 AM, Alexei Fedotov <al...@gmail.com>
> wrote:
> > Hello, Kostya,
> >
> > Do you think it would be good to reflect Egor's (and mine) idea of an
> > evolutionary path during learning period during the first step of your
> > plan? Namely, incremental approach to IR separation are subsequent
> > patches to the existing code base. Each of the patches does the
> > following:
> > 1. Removes some optimization from the parsing stage.
> > 2. Adds necessary infrastructure to HIR.
> > 3. Adds or ensures that the optimization is done on HIR.
> >
> > This would help attacking the problem and learning a parser, HIR and
> > tools incrementally. On completion of this step you would have your
> > code committed to Harmony and get more confidence. Moreover the code
> > which enhances HIR would be close to final. In process you will learn
> > why the existing IR builder is bad and grow a desire to attack it
> > fully rewriting the code.
> >
> > What do you think?
> >
>
> Hi, Alexei!
I like this approach with "first steps".
Actually I have already started with IRBuilder refactoring and completely
removed Simplifier from the code, tested on HelloWorld. (maybe I
should try more complicated tests) Now
I work on removing SCE now. Then there will be the first patch.
But it suprises me that IRBuilder should be completely rewritten!
I thought the problem was only with translator... I suppose when I am
finished with IRBuilder refactoring, to start prepass and data analisys (as
Egor advised), so on... and later to use IRBuilder in new translator.
P.S. About verifier and parser generators: I will ivestigate this
problem carefully.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
One more idea on further rewriting.
1. An old verifier built some IR on the top of bytecode.
2. A new verifier also does parsing, but is more like JET.
You may think of building a reusable component. I wonder if
conventional parser generators or ASN.1 are applicable here. Note,
thinking is different from starting doing, there are cases when a
specific solution is better than reusable.
On Wed, Apr 2, 2008 at 3:02 AM, Alexei Fedotov <al...@gmail.com> wrote:
> Hello, Kostya,
>
> Do you think it would be good to reflect Egor's (and mine) idea of an
> evolutionary path during learning period during the first step of your
> plan? Namely, incremental approach to IR separation are subsequent
> patches to the existing code base. Each of the patches does the
> following:
> 1. Removes some optimization from the parsing stage.
> 2. Adds necessary infrastructure to HIR.
> 3. Adds or ensures that the optimization is done on HIR.
>
> This would help attacking the problem and learning a parser, HIR and
> tools incrementally. On completion of this step you would have your
> code committed to Harmony and get more confidence. Moreover the code
> which enhances HIR would be close to final. In process you will learn
> why the existing IR builder is bad and grow a desire to attack it
> fully rewriting the code.
>
> What do you think?
>
>
>
>
> On 02 Apr 2008 01:53:18 +0400, Egor Pasko <eg...@gmail.com> wrote:
> > On the 0x419 day of Apache Harmony Okonechnikov Konstantin wrote:
> > > > Let's not remove copy propagation, it does not make the code
> > > > difficult, does not add lots of lines, but still very functional.
> > > >
> > > > OK?
> > > >
> > > > --
> > > >
> > > > Egor Pasko
> > > >
> > > >
> > > OK, got it.
> > > What are the requirments for simplify and hvn passes?
> >
> > what do you mean by requirements? they should optimize the code, that
> > is one requirement, but if they make things slower in some cases, that
> > happens too :)
> >
> >
> > > Currently I am testing using this optimizer
> > > chain: ssa,devirt,inline,purge,simplify,dce,uce... (didn't include hvn yet)
> >
> > we can check later whether we need hvn here or not
> >
> >
> > > BTW, I' ve updated my application and put it on the wiki
> > > http://wiki.apache.org/general/WikiTranslatorRefactoringProposal.<http://wiki.apache.org/general/WikiTranslatorRefactoringProposal>
> >
> > doing good, probably somebody can give more valuable comments now.
> >
> > --
> > Egor Pasko
> >
> >
>
>
>
> --
> With best regards,
> Alexei
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
Hello, Kostya,
Do you think it would be good to reflect Egor's (and mine) idea of an
evolutionary path during learning period during the first step of your
plan? Namely, incremental approach to IR separation are subsequent
patches to the existing code base. Each of the patches does the
following:
1. Removes some optimization from the parsing stage.
2. Adds necessary infrastructure to HIR.
3. Adds or ensures that the optimization is done on HIR.
This would help attacking the problem and learning a parser, HIR and
tools incrementally. On completion of this step you would have your
code committed to Harmony and get more confidence. Moreover the code
which enhances HIR would be close to final. In process you will learn
why the existing IR builder is bad and grow a desire to attack it
fully rewriting the code.
What do you think?
On 02 Apr 2008 01:53:18 +0400, Egor Pasko <eg...@gmail.com> wrote:
> On the 0x419 day of Apache Harmony Okonechnikov Konstantin wrote:
> > > Let's not remove copy propagation, it does not make the code
> > > difficult, does not add lots of lines, but still very functional.
> > >
> > > OK?
> > >
> > > --
> > >
> > > Egor Pasko
> > >
> > >
> > OK, got it.
> > What are the requirments for simplify and hvn passes?
>
> what do you mean by requirements? they should optimize the code, that
> is one requirement, but if they make things slower in some cases, that
> happens too :)
>
>
> > Currently I am testing using this optimizer
> > chain: ssa,devirt,inline,purge,simplify,dce,uce... (didn't include hvn yet)
>
> we can check later whether we need hvn here or not
>
>
> > BTW, I' ve updated my application and put it on the wiki
> > http://wiki.apache.org/general/WikiTranslatorRefactoringProposal.<http://wiki.apache.org/general/WikiTranslatorRefactoringProposal>
>
> doing good, probably somebody can give more valuable comments now.
>
> --
> Egor Pasko
>
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x419 day of Apache Harmony Okonechnikov Konstantin wrote:
> > Let's not remove copy propagation, it does not make the code
> > difficult, does not add lots of lines, but still very functional.
> >
> > OK?
> >
> > --
> >
> > Egor Pasko
> >
> >
> OK, got it.
> What are the requirments for simplify and hvn passes?
what do you mean by requirements? they should optimize the code, that
is one requirement, but if they make things slower in some cases, that
happens too :)
> Currently I am testing using this optimizer
> chain: ssa,devirt,inline,purge,simplify,dce,uce... (didn't include hvn yet)
we can check later whether we need hvn here or not
> BTW, I' ve updated my application and put it on the wiki
> http://wiki.apache.org/general/WikiTranslatorRefactoringProposal.<http://wiki.apache.org/general/WikiTranslatorRefactoringProposal>
doing good, probably somebody can give more valuable comments now.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
> Let's not remove copy propagation, it does not make the code
> difficult, does not add lots of lines, but still very functional.
>
> OK?
>
> --
>
> Egor Pasko
>
>
OK, got it.
What are the requirments for simplify and hvn passes?
Currently I am testing using this optimizer
chain: ssa,devirt,inline,purge,simplify,dce,uce... (didn't include hvn yet)
BTW, I' ve updated my application and put it on the wiki
http://wiki.apache.org/general/WikiTranslatorRefactoringProposal.<http://wiki.apache.org/general/WikiTranslatorRefactoringProposal>
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x418 day of Apache Harmony Okonechnikov Konstantin wrote:
> On 31 Mar 2008 16:52:41 +0400, Egor Pasko <eg...@gmail.com> wrote:
> >
> > On the 0x417 day of Apache Harmony Okonechnikov Konstantin wrote:
> > > In addition to previous letter:
> > > - I removed DoSimplify flag from IrBuilderFlags and all code connected
> > to it
> > > , but didn't removed simplifier yet.
> > > First reason is that,there are some problems with removing
> > simplification
> > > from
> > > IRBuilder::genTauCheckNull(Opnd* opnd, bool &alwaysThrows)
> > > IRBuilder::genTauCheckZero(Opnd* opnd, bool &alwaysThrows)
> > > IRBuilder::simplifyTauCheckDivOpnds(Opnd* src1, Opnd* src2, bool
> > > &alwaysThrows)
> >
> >
> > what is with these methods?
> >
> >
> > > Well, I figured out that simply removing simplification here doesn't
> > work
> > > :)
> >
> >
> > what happens? :)
>
>
> It looks like some dark magic or maybe late night hours
> :), now everyting seems to be OK, I removed
> simplification. Nevertheless, I beware of nasty warning inside
> IRBuilder::genTauCheckNull(...):
> ...
> // Not advisable to turn off simplification of checknull because
> // IRBuilder calls genTauCheckNull redundantly many times
this comment is obsolete. I see nothing wrong in duplicated chknulls
during translation. Should be eliminated by copy propagation and CSE.
> // ...then it does simplify without asking the flag
> ...
yeah, funny
I also liked this output parameter namely alwaysThrows, it is never used :)
>
> I am working now on removing propgate copy and SCE. Testing on HelloWorld.
> If there were such code:
>
> Opnd* IRBuilder::genAdd(Type* dstType, Modifier mod, Opnd* src1, Opnd*
> src2) {
> src1 = propagateCopy(src1);
> src2 = propagateCopy(src2);
> Operation operation(Op_Add, dstType->tag, mod);
> uint32 hashcode = operation.encodeForHashing();
> Opnd* dst = lookupHash(hashcode, src1, src2);
> if (dst) return dst;
> if (!dst) {
> dst = createOpnd(dstType);
> Inst *newi = instFactory->makeAdd(mod, dst, src1, src2);
> appendInst(newi);
> }
> insertHash(hashcode, src1, src2, dst->getInst());
> return dst;
> }
>
> It becomes like this:
>
> Opnd* IRBuilder::genAdd(Type* dstType, Modifier mod, Opnd* src1, Opnd* src2)
> {
> Operation operation(Op_Add, dstType->tag, mod);
> Opnd* dst = createOpnd(dstType);
> Inst *newi = instFactory->makeAdd(mod, dst, src1, src2);
> appendInst(newi);
> return dst;
> }
>
> Is everything OK?
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x418 day of Apache Harmony Okonechnikov Konstantin wrote:
> On 31 Mar 2008 16:52:41 +0400, Egor Pasko <eg...@gmail.com> wrote:
> >
> > On the 0x417 day of Apache Harmony Okonechnikov Konstantin wrote:
> > > In addition to previous letter:
> > > - I removed DoSimplify flag from IrBuilderFlags and all code connected
> > to it
> > > , but didn't removed simplifier yet.
> > > First reason is that,there are some problems with removing
> > simplification
> > > from
> > > IRBuilder::genTauCheckNull(Opnd* opnd, bool &alwaysThrows)
> > > IRBuilder::genTauCheckZero(Opnd* opnd, bool &alwaysThrows)
> > > IRBuilder::simplifyTauCheckDivOpnds(Opnd* src1, Opnd* src2, bool
> > > &alwaysThrows)
> >
> >
> > what is with these methods?
> >
> >
> > > Well, I figured out that simply removing simplification here doesn't
> > work
> > > :)
> >
> >
> > what happens? :)
>
>
> It looks like some dark magic or maybe late night hours
> :), now everyting seems to be OK, I removed
> simplification. Nevertheless, I beware of nasty warning inside
> IRBuilder::genTauCheckNull(...):
> ...
> // Not advisable to turn off simplification of checknull because
> // IRBuilder calls genTauCheckNull redundantly many times
I have no idea what this means, but I will take a look! plz, remind me
if I do not do it in a couple of days :)
> // ...then it does simplify without asking the flag
> ...
>
> I am working now on removing propgate copy and SCE. Testing on HelloWorld.
> If there were such code:
>
> Opnd* IRBuilder::genAdd(Type* dstType, Modifier mod, Opnd* src1, Opnd*
> src2) {
> src1 = propagateCopy(src1);
> src2 = propagateCopy(src2);
> Operation operation(Op_Add, dstType->tag, mod);
> uint32 hashcode = operation.encodeForHashing();
> Opnd* dst = lookupHash(hashcode, src1, src2);
> if (dst) return dst;
> if (!dst) {
> dst = createOpnd(dstType);
> Inst *newi = instFactory->makeAdd(mod, dst, src1, src2);
> appendInst(newi);
> }
> insertHash(hashcode, src1, src2, dst->getInst());
> return dst;
> }
>
> It becomes like this:
>
> Opnd* IRBuilder::genAdd(Type* dstType, Modifier mod, Opnd* src1, Opnd* src2)
> {
> Operation operation(Op_Add, dstType->tag, mod);
> Opnd* dst = createOpnd(dstType);
> Inst *newi = instFactory->makeAdd(mod, dst, src1, src2);
> appendInst(newi);
> return dst;
> }
>
> Is everything OK?
Let's not remove copy propagation, it does not make the code
difficult, does not add lots of lines, but still very functional.
OK?
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
On 31 Mar 2008 16:52:41 +0400, Egor Pasko <eg...@gmail.com> wrote:
>
> On the 0x417 day of Apache Harmony Okonechnikov Konstantin wrote:
> > In addition to previous letter:
> > - I removed DoSimplify flag from IrBuilderFlags and all code connected
> to it
> > , but didn't removed simplifier yet.
> > First reason is that,there are some problems with removing
> simplification
> > from
> > IRBuilder::genTauCheckNull(Opnd* opnd, bool &alwaysThrows)
> > IRBuilder::genTauCheckZero(Opnd* opnd, bool &alwaysThrows)
> > IRBuilder::simplifyTauCheckDivOpnds(Opnd* src1, Opnd* src2, bool
> > &alwaysThrows)
>
>
> what is with these methods?
>
>
> > Well, I figured out that simply removing simplification here doesn't
> work
> > :)
>
>
> what happens? :)
It looks like some dark magic or maybe late night hours
:), now everyting seems to be OK, I removed
simplification. Nevertheless, I beware of nasty warning inside
IRBuilder::genTauCheckNull(...):
...
// Not advisable to turn off simplification of checknull because
// IRBuilder calls genTauCheckNull redundantly many times
// ...then it does simplify without asking the flag
...
I am working now on removing propgate copy and SCE. Testing on HelloWorld.
If there were such code:
Opnd* IRBuilder::genAdd(Type* dstType, Modifier mod, Opnd* src1, Opnd*
src2) {
src1 = propagateCopy(src1);
src2 = propagateCopy(src2);
Operation operation(Op_Add, dstType->tag, mod);
uint32 hashcode = operation.encodeForHashing();
Opnd* dst = lookupHash(hashcode, src1, src2);
if (dst) return dst;
if (!dst) {
dst = createOpnd(dstType);
Inst *newi = instFactory->makeAdd(mod, dst, src1, src2);
appendInst(newi);
}
insertHash(hashcode, src1, src2, dst->getInst());
return dst;
}
It becomes like this:
Opnd* IRBuilder::genAdd(Type* dstType, Modifier mod, Opnd* src1, Opnd* src2)
{
Operation operation(Op_Add, dstType->tag, mod);
Opnd* dst = createOpnd(dstType);
Inst *newi = instFactory->makeAdd(mod, dst, src1, src2);
appendInst(newi);
return dst;
}
Is everything OK?
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x417 day of Apache Harmony Okonechnikov Konstantin wrote:
> In addition to previous letter:
> - I removed DoSimplify flag from IrBuilderFlags and all code connected to it
> , but didn't removed simplifier yet.
> First reason is that,there are some problems with removing simplification
> from
> IRBuilder::genTauCheckNull(Opnd* opnd, bool &alwaysThrows)
> IRBuilder::genTauCheckZero(Opnd* opnd, bool &alwaysThrows)
> IRBuilder::simplifyTauCheckDivOpnds(Opnd* src1, Opnd* src2, bool
> &alwaysThrows)
what is with these methods?
> Well, I figured out that simply removing simplification here doesn't work
> :)
what happens? :)
> Continued investigating. Maybe it's time to open Muchnik's book...
> And the other reason is Simplifier::PropogateCopy(). Again about it: what
> does it do, why do we use it?
just always get the propagated operand instead of the actual
operand. Makes copy propagation built in.
Of course Simplifier nature is that it only creates simplified
expressions. Old expressions have to be dead-code-eliminated
afterwards.
> - About CSE: we should use [optimizer] HNV , right?
HVN is more advanced, it keeps information about control flow, CSE in
IRBuilder operates only locally within basic blocks (that are
typically small in Java)
> - How do we configure opt emconf?
You probably want to run your optpass for a set of methods, and not
run it for others, right? In this case you better take a
client.emconf, which currently differentiates <clinit> methods, and do
something similar.
For example, your custom.emconf file might start with:
==========================================
chains=chain1,chain2
chain1.jits=MY_OPT
chain2.jits=JET
chain1.filter=+.MyMethod1
chain1.filter=+.MyMethod2
chain1.filter=-
JET.file=jitrino
MY_OPT.file=jitrino
==========================================
take configuration of MY_OPT from client.emconf and change it as you
want. Remove all profiling settings for simplicity. OPT will compile
all methods _starting with_ MyMethod1 and MyMethod2.
> How to reorganize optimizer passes correctly?
This is an open question. Some optimizations cannot work after
others. Sometimes it is not what it should be, but sometimes it is
intentional. For example, "sse" pass converts HIR to SSA form, some
optimizations require SSA form and cannot work without it.
It would be nice to have a single place to keep the information about
each optpass requirements, compatibilities, recommendations. But
currently we keep this info only in our minds, mailing lists,
comments, JIRA. Many incompatibilities are completely not obvious. A
set of not-tested flags for each optpass make the picture even more
complicated.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
In addition to previous letter:
- I removed DoSimplify flag from IrBuilderFlags and all code connected to it
, but didn't removed simplifier yet.
First reason is that,there are some problems with removing simplification
from
IRBuilder::genTauCheckNull(Opnd* opnd, bool &alwaysThrows)
IRBuilder::genTauCheckZero(Opnd* opnd, bool &alwaysThrows)
IRBuilder::simplifyTauCheckDivOpnds(Opnd* src1, Opnd* src2, bool
&alwaysThrows)
Well, I figured out that simply removing simplification here doesn't work
:) Continued investigating. Maybe it's time to open Muchnik's book...
And the other reason is Simplifier::PropogateCopy(). Again about it: what
does it do, why do we use it?
- About CSE: we should use [optimizer] HNV , right?
- How do we configure opt emconf? How to reorganize optimizer passes
correctly?
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x417 day of Apache Harmony Okonechnikov Konstantin wrote:
> On 29 Mar 2008 15:11:14 +0300, Egor Pasko <eg...@gmail.com> wrote:
>
>
> > Another idea: if you prefer not to start from the IRBuilder
> > refactoring, there is an alternative way. The task is to separate
> > JavaLabelPrepass into two passes:
> > 1. marking basic blocks
> > 2. data flow analysis
>
>
> Well, I have decided to start with IRBuilder.
nice! I love that.
> I suppose that developing the translator will require much more
> time,
truly
> and it will be better not to interrupt this process.
pretty logical :)
> But if it is essentially to divide JavaLabelPrepass
> at first, I could start with it.
obviously, not essential, I would rather recomment to start from IRBuilder.
> And I have some more questions:
> 1) What for do we need LabelInst,
there is a convention that LabelInst stands as the first instruction
in each basic block, stores the block name, type and reference-to.
You can find it in printed HIR like "I44:L13: bcmap:5"
For example, you can write:
(LabelInst*)node->getFirstInst())->isCatchLabel() to know if the block
is a catch block.
Catch blocks correspond to exception handlers. CatchLabelInst (which
is a LabelInst in catch block) keeps information about the exact type
of the exception handler)
> what is IRBuilder::currentLabel? Is it some kind of node index?
yes! IRBuilder is stateful, it appends instructions to the current
node denoted by currentLabel.
> ( IrBuilder::AppenInst uses it to get the node for appending
> instructions) 2) As I understood CSE is a part of optimization. Do
> we need to remove it?
>
> If yes, IrBuilder::CSEHashTable serves only this purpose, doesn't
> it?
yes, it is private. CSE is done locally in IRBuilder, cseHashTable is
cleared on each new basic block.
> 3) What for does Simplifier need to propgate source operands
> copies?
well, to always propagate is just a cheap optimization. Reduces
expression sizes and makes successful cseHashTable searches more
often.
> Didn't find out yet how the translator interacts with IrBuilder, continue
> working on this subject.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
On 29 Mar 2008 15:11:14 +0300, Egor Pasko <eg...@gmail.com> wrote:
> Another idea: if you prefer not to start from the IRBuilder
> refactoring, there is an alternative way. The task is to separate
> JavaLabelPrepass into two passes:
> 1. marking basic blocks
> 2. data flow analysis
Well, I have decided to start with IRBuilder. I suppose that developing the
translator will require much more time, and it will be better not to
interrupt this process. But if it is essentially to divide JavaLabelPrepass
at first, I could start with it.
And I have some more questions:
1) What for do we need LabelInst, what is
IRBuilder::currentLabel? Is it some kind of node index? (
IrBuilder::AppenInst uses it to get the node for appending instructions)
2) As I understood CSE is a part of optimization. Do we need to remove it?
If yes, IrBuilder::CSEHashTable serves only this purpose, doesn't it?
3) What for does Simplifier need to propgate source operands copies?
Didn't find out yet how the translator interacts with IrBuilder, continue
working on this subject.
Best regards,
Okonechnikov Konstantin.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x416 day of Apache Harmony Okonechnikov Konstantin wrote:
> Alexei, Egor, thanks for helpful discussion.
>
>
> So, there are two
>
> basic approaches:
>
> 1. rewrite the current impl, introduce limited functionality, grow
> functionality in evolutionary way
>
> 2. take the current impl, evolutionary move it to something better and
> > better
> >
> > Yes, I preferred (1) because changing (2) requires understanding the
> > current impl, which is not so fun.
>
>
> It seems to me I won't avoid digging into the code.
Somewhat -- yes, but do not spend a lot of time on this! this code is
time eater. You will basically need to find out _what_ the code does,
but not _how_ it does. Try using the JET _how_.
> And I don't want to make the same mistakes again.
You won't! We are here to prevent that. Release your patches "early
and often", this is one of the good properties of open source
development (releasing early and often).
> I think I have to get familiar with current implementation to
> understand better its problems. Maybe not so detailed, but I believe
> that I could use some of this code.
please, refer to the above notes about _what_ and _how_ while doing
this. I do not want this brain-damaging implementation to hurt you :)
> But the big plus of (2) is that no matter of how much work is done,
> > the impl becomes better :)
>
>
> Could it happen so that the only possible improvment is total refactoring?
This question is not easy to answer. On one hand total refactoring is
not necessary in some places. We already discussed that support of
some features will most likely remain the same (Exception support,
JSR/RET, Simplifier).
But on the other hand JavaTranslator has to be eliminated
completely. It affects a lot.
Another idea: if you prefer not to start from the IRBuilder
refactoring, there is an alternative way. The task is to separate
JavaLabelPrepass into two passes:
1. marking basic blocks
2. data flow analysis
try doing it in no more than one week. And do not feel upset if you do
not complete it, it is a step to get some knowledge on _what_ is done
there (OK, and you will have to find out the _how_, but here it is
unavoidable at the moment)
P.S.: It is so good that you share your opinion often, because it
makes me think about the subject more, and emit (crappy:) ideas. So,
keep updating us on your thoughts and plans.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
Alexei, Egor, thanks for helpful discussion.
So, there are two
basic approaches:
1. rewrite the current impl, introduce limited functionality, grow
functionality in evolutionary way
2. take the current impl, evolutionary move it to something better and
> better
>
> Yes, I preferred (1) because changing (2) requires understanding the
> current impl, which is not so fun.
It seems to me I won't avoid digging into the code. And I don't want to
make the same mistakes again. I think I have to get familiar
with current implementation to understand better its problems. Maybe not so
detailed, but I believe that I could use some of this code.
But the big plus of (2) is that no matter of how much work is done,
> the impl becomes better :)
Could it happen so that the only possible improvment is total refactoring?
Best regards,
Okonechnikov Konstantin.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x415 day of Apache Harmony Egor Pasko wrote:
> On the 0x415 day of Apache Harmony Alexei Fedotov wrote:
> > Egor,
> > Do I understand correctly that you are suggesting Kostya to design the
> > whole thing from scratch rather than evolutionary approach?
>
> Why? I suggest the evolutionary approach. Stop fooling me, April 1st
> is not there yet.
Alexei, sorry, misunderstood you.
So, there are two basic approaches:
1. rewrite the current impl, introduce limited functionality, grow
functionality in evolutionary way
2. take the current impl, evolutionary move it to something better and
better
Yes, I preferred (1) because changing (2) requires understanding the
current impl, which is not so fun.
But the big plus of (2) is that no matter of how much work is done,
the impl becomes better :)
Need to think about some hybrid approach. Actually my suggestion to
start from moving Simplifier away from IRBuilder makes it somewhat hybrid.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x415 day of Apache Harmony Alexei Fedotov wrote:
> Egor,
> Do I understand correctly that you are suggesting Kostya to design the
> whole thing from scratch rather than evolutionary approach?
Why? I suggest the evolutionary approach. Stop fooling me, April 1st
is not there yet.
> Thanks!
>
>
> On 28 Mar 2008 13:26:31 +0300, Egor Pasko <eg...@gmail.com> wrote:
> > On the 0x415 day of Apache Harmony Okonechnikov Konstantin wrote:
> > > > ....
> > > >
> > > > Why is this approach worth trying? Because you can test the code while
> > > > writing. Make an emconf that uses your code on methods with supported
> > > > bytecodes and use JET or older OPT for others.
> > > >
> > > > Sounds good?
> > > >
> > > > --
> > > >
> > > > Egor Pasko
> > > >
> > > >
> > > Sounds great! :)
> > > I like the idea with specialized emoconf file and "step-by-step"
> > > development. Thanks for detailed plan, I know what things I should
> > > ivenstigate at first.
> > > I suppose when developing architecture, one should consider all
> > > complicated features... But it's better to discuss later, when
> > > I will study how these features should work ))
> >
> > cool cool! Top-down architecture is usually hard to do right because
> > our small brains are so limited :) BTW, JavaTranslator is a good
> > example of top-down design done wrong.
> >
> > I know many complicated details, but I do not want to write a top-down
> > design for you. This is all for your fun, after all, I should not eat
> > a part of your pie. So, it is quite logical to rise from bottom. In
> > this approach, if you are stuck, you know where you are stuck.
> >
> >
> > > I need to deliberate and investigate for some time. Then I come up
> > > with questions.
> >
> > Great!
> >
> > Looks like IRBuilder refactoring will help you on the first steps. So,
> > feel free to take it. And to refresh the priorities, data flow
> > analysis is the most difficult. So, try not be stuck on other topics,
> > plan most of your time for the data flow.
> >
> > --
> > Egor Pasko
> >
> >
>
>
>
> --
> With best regards,
> Alexei
>
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexei Fedotov <al...@gmail.com>.
Egor,
Do I understand correctly that you are suggesting Kostya to design the
whole thing from scratch rather than evolutionary approach?
Thanks!
On 28 Mar 2008 13:26:31 +0300, Egor Pasko <eg...@gmail.com> wrote:
> On the 0x415 day of Apache Harmony Okonechnikov Konstantin wrote:
> > > ....
> > >
> > > Why is this approach worth trying? Because you can test the code while
> > > writing. Make an emconf that uses your code on methods with supported
> > > bytecodes and use JET or older OPT for others.
> > >
> > > Sounds good?
> > >
> > > --
> > >
> > > Egor Pasko
> > >
> > >
> > Sounds great! :)
> > I like the idea with specialized emoconf file and "step-by-step"
> > development. Thanks for detailed plan, I know what things I should
> > ivenstigate at first.
> > I suppose when developing architecture, one should consider all
> > complicated features... But it's better to discuss later, when
> > I will study how these features should work ))
>
> cool cool! Top-down architecture is usually hard to do right because
> our small brains are so limited :) BTW, JavaTranslator is a good
> example of top-down design done wrong.
>
> I know many complicated details, but I do not want to write a top-down
> design for you. This is all for your fun, after all, I should not eat
> a part of your pie. So, it is quite logical to rise from bottom. In
> this approach, if you are stuck, you know where you are stuck.
>
>
> > I need to deliberate and investigate for some time. Then I come up
> > with questions.
>
> Great!
>
> Looks like IRBuilder refactoring will help you on the first steps. So,
> feel free to take it. And to refresh the priorities, data flow
> analysis is the most difficult. So, try not be stuck on other topics,
> plan most of your time for the data flow.
>
> --
> Egor Pasko
>
>
--
With best regards,
Alexei
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x415 day of Apache Harmony Okonechnikov Konstantin wrote:
> > ....
> >
> > Why is this approach worth trying? Because you can test the code while
> > writing. Make an emconf that uses your code on methods with supported
> > bytecodes and use JET or older OPT for others.
> >
> > Sounds good?
> >
> > --
> >
> > Egor Pasko
> >
> >
> Sounds great! :)
> I like the idea with specialized emoconf file and "step-by-step"
> development. Thanks for detailed plan, I know what things I should
> ivenstigate at first.
> I suppose when developing architecture, one should consider all
> complicated features... But it's better to discuss later, when
> I will study how these features should work ))
cool cool! Top-down architecture is usually hard to do right because
our small brains are so limited :) BTW, JavaTranslator is a good
example of top-down design done wrong.
I know many complicated details, but I do not want to write a top-down
design for you. This is all for your fun, after all, I should not eat
a part of your pie. So, it is quite logical to rise from bottom. In
this approach, if you are stuck, you know where you are stuck.
> I need to deliberate and investigate for some time. Then I come up
> with questions.
Great!
Looks like IRBuilder refactoring will help you on the first steps. So,
feel free to take it. And to refresh the priorities, data flow
analysis is the most difficult. So, try not be stuck on other topics,
plan most of your time for the data flow.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
> ....
>
> Why is this approach worth trying? Because you can test the code while
> writing. Make an emconf that uses your code on methods with supported
> bytecodes and use JET or older OPT for others.
>
> Sounds good?
>
> --
>
> Egor Pasko
>
>
Sounds great! :)
I like the idea with specialized emoconf file and "step-by-step"
development. Thanks for detailed plan, I know what things I should
ivenstigate at first.
I suppose when developing architecture, one should consider all
complicated features... But it's better to discuss later, when
I will study how these features should work ))
I need
to deliberate and investigate for some time. Then I come up with questions.
Best regards,
Okonechnikov Konstantin.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x413 day of Apache Harmony Okonechnikov Konstantin wrote:
> Well, there are some questions.
> [IrBuilder]
> If we clean out simplification, it wont affect the functionality of
> instructions, will it? (i.e.will OPT work without these simplifications?
> I ve tried to clean some instructions, it seems everything is OK (HelloWorld
> works) )
Ideally everything should work if we disable simplification. Although
I remember some problems running Eclipse solely on OPT with
simplification disabled. That was a long time ago, and if it appears,
that's a bug.
> I 've written sort of plan. Check this out.
> 1) Clean up IRBuilder
> -remove Simplifier and its methods
sounds good
> 2) Move all simplification to Optimizer stage
> (this is quite big task as I suppose, lots of instructions use
> simplification. Not sure if we should create separate action for every
> simplification....)
> 1 and 2 should be implemented concurrently
actually there is a stage called 'simplify' already that in fact uses
the same simplifier. So, it is not a huge work, need to ask somebody
to check performance of disabling simplification in translator and
running it from HLO right after the translator (with dead code
eliminator, blablabla).
> 3) Develop the translator
> - new PrePass (maybe from Jet?)
>
> - implement data analisys, parser etc...
>
> Where am I mistaken? I will appreciate any additions and(or) corrections.
Kostya, thanks a lot for the draft plan! I have been quite busy and
could not discuss it with you earlier. Sorry for that. I like your
pro-active approach very much, please continue behave like that :)
Although, I have somewhat different views on the plan, those views are
more like discussable than final :) So, plz, do not take personally if
I just ruined your best dreams about the process))
TA-DAA! Here my suggestion goes..
I would recomend to hack the whole code starting from a small and
limited implementation and grow functionality doing small baby
steps. So, you would not need to implement the rock solid data flow
analysis at once. Just write one that looks beautiful by design, does
not support non-interesting instructions as well as difficult stuff
like exceptions, loops and JSR/RET, put asserts in places, where you
do not support something.
Then grow functionality until you can run HelloWorld. Constantly
ensuring that the code looks good and we love it.
This can be split into several sub-steps:
0. move simplifier out of IRBuilder, basically fix HARMONY-5500 or
determine that it is not a part of our work if it unexpectedly
wants to eat too much time :)
1. draft code that does prepass and data flow analysis in code _without_
loops, exceptions, call instructions
1.1. prepass
1.2. data flow analysis
1.3. bytecode mapping
2. support call instructions and lazy resolution
3. with loops
4. with exceptions and loops
5. make sure Hello World works and fix bugs when it does not ;)
6. JSR/RET support
7. inliner support
8. make sure all tests and eclipse work in -Xem:opt mode
8.1. bytecode without JSR/RET
8.2. bytecode obtained by ECJ (with JSR/RET)
9. write barriers (?), other features...
10. what I forgot
(I am yet not sure about the order, though, pick what you want to do
first)
Let's review the code between steps, speak about it, think what we do
next and how. It also makes sense to review the code more often, once
noticeable features are implemented or code is refactored.
My picture is that we will commit it into a separate directory, your
code can be running via tweaked emconf file. That is because I do not
like branches :)
I separated exceptions, loops, resolution stuff because I think it
will let us better concentrate on coding in the early stages (what
else to separate?). Additionally, I think, exception support needs
little change. It has been completely rewritten once, looks not so bad
(although scattered around) and no bugs were found in it since
then. So, just move the code carefully. Same picture with JSR/RET
support. Looks like no need to rewrite.
Why is this approach worth trying? Because you can test the code while
writing. Make an emconf that uses your code on methods with supported
bytecodes and use JET or older OPT for others.
Sounds good?
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
Well, there are some questions.
[IrBuilder]
If we clean out simplification, it wont affect the functionality of
instructions, will it? (i.e.will OPT work without these simplifications?
I ve tried to clean some instructions, it seems everything is OK (HelloWorld
works) )
I 've written sort of plan. Check this out.
1) Clean up IRBuilder
-remove Simplifier and its methods
2) Move all simplification to Optimizer stage
(this is quite big task as I suppose, lots of instructions use
simplification. Not sure if we should create separate action for every
simplification....)
1 and 2 should be implemented concurrently
3) Develop the translator
- new PrePass (maybe from Jet?)
- implement data analisys, parser etc...
Where am I mistaken? I will appreciate any additions and(or) corrections.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x411 day of Apache Harmony Okonechnikov Konstantin wrote:
> If we need to exclude simplification from IRBuilder, is it OK to use it? I
> mean IRBuilder should be only modified or this functionality should be
> written from scratch?
Yes, for the IRBuilder reimplementation is not required. We need to
move the simplification functionality out of the IRBuilder and make
IRBuilder usable to create instructions in HLO.
> And one more question: is there any specification of HIR, HIR <-> Java
> bytecode correspondence?
No, we do not have a spec. This is bad.
For now you can refer to Opcode.cpp to see the full instructions in
HIR and, yes, you will have to look into the JavaByteCodeTranslator to
see how HIR instructions are generated from bytecode instructions.
As for your application form, I am looking into it, sorry for taking
so long.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
If we need to exclude simplification from IRBuilder, is it OK to use it? I
mean IRBuilder should be only modified or this functionality should be
written from scratch?
And one more question: is there any specification of HIR, HIR <-> Java
bytecode correspondence?
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x40E day of Apache Harmony Okonechnikov Konstantin wrote:
> Thanks for help!
> There are a lot of things to think about
> here. I am looking forward to investigate :)
> And one more question. I suppose it is adderessed to Egor, as possible mentor.
> If I have some questions about my GSoC application, is it OK to ask in
> dev mailing list or is it better to write directly?
dev list
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
Thanks for help!
There are a lot of things to think about
here. I am looking forward to investigate :)
And one more question. I suppose it is adderessed to Egor, as possible mentor.
If I have some questions about my GSoC application, is it OK to ask in
dev mailing list or is it better to write directly?
Best regards,
Okonechnikov Konstantin.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x40E day of Apache Harmony Alexey Varlamov wrote:
> 21 Mar 2008 12:00:22 +0300, Egor Pasko <eg...@gmail.com>:
> > On the 0x40D day of Apache Harmony Okonechnikov Konstantin wrote:
> > > >hm, and how is it going? :)
> > >
> > > well, right now I modify [codegen]->stacklayout action. Callee-save
> > > registers are always being saved/restored in the begining/end of the
> > > method. But sometimes not all pathes use callee saved regs.
> > > Motion of the spill code inside CFG can reduce memory calling rate.
> > > (So called "shrink wrapping"
> > > technique). I have
> > > already implemented functions that collect info about callee save regs
> > > usage in each block and find proper postion for spill code,
> > > hope to finish the optimization to the end of April.
> >
> > wow, this is a very interesting work, assuming some tradeoffs to
> > consider. I love this :)
> >
> > > >seriously, what is your experience with the code?
> > >
> > > I am familiar with common principles ( DRVLM,HIR, LIR, CFG etc)
> > > and have expierience with codegenerator.
> >
> > nice nice nice
> >
> > > > ... there is a JavaLabelPrepass class that namely should only mark
> > > >boundaries of basic blocks. But it does a lot more, for example,
> > > >inferring types of variables. It is very complicated, makes
> > > >unpredictable number of traversals through the bytecode, not easy to
> > > >support.
> > >
> > > Some questions.
> > > As I
> > > understood TranslatatorSession::translate() starts parsing calling
> > > JavaTranslator::translateMethod(), right?
> > > Then we parse bytecode using JavaByteCodeTranslator as parser
> > > callback. When are JavaLabelPrepass methods called?
> >
> > I agree, this is not very logical. You are now closer to the crappy
> > code, just start feeling it :)
> >
> > in constructor of JavaByteCodeTranslator:
> > JavaByteCodeTranslator.cpp:142: initLocalVars();
> >
> > in initLocalVars:220:
> > //
> > // perform label prepass in this method
> > //
> > parser.parse(&prepass);
> > prepass.parseDone();
> >
> > it was supposed that parser incapsulates generic code for parsing,
> > but it does not have a lot of code to incapsulate and only makes things
> > more complicated.
> >
> > > What is VariableIncarnation? What is SlotVar?
> >
> > In brief, VariableIncarnation will refer to a single operand in
> > HIR. But during translation phase we do not know which variables
> > (local variables and stack variables) will represent the same operand,
> > so, during translation VariableIncarnations will be connected in
> > chains of equivalent vars. One chain represent a future operand.
> >
> > SlotVar .. you know, slot is an element of java operand stack. Just to
> > keep corresponding VariableIncarnations connected to the slot in
> > operation.
> >
> > Details later, if you wish.
> IMO the JavaLabelPrepass quite closely implements data-flow analysis
> as described in JVMS [1], and VariableIncarnations represent merged
> state of variables for each instruction, as Egor said.
> [1] http://java.sun.com/docs/books/jvms/second_edition/html/ClassFile.doc.html#9801
Alexey, thanks for the handy link! Most of JavaLabelPrepass
functionality is data-flow analysis. I forgot to mention it :(
> > I think you should not dive into details of JavaLabelPrepass. Better
> > try avoid it. And implement a readable and nice translator from scratch.
> >
> > > > ...IRBuilder separation necessity is reported as HARMONY-5500.
> > > Looks like refactoring of JavaLabelPrepass and IRBuilder separation are
> > > quite independent tasks. And separation is more important, isn't it?
> >
> > JavaLabelPrepass has nothing in common with IRBuilder. Though, both
> > are a part of the translator. I cannot judge the importance here. Both
> > improve design, readability, etc. IRBuilder separation is easier to
> > implement, and, probably, this is what you want to start from.
> >
> > > >A good example of clean code that does similar things is Jitrino.JET
> > > >compiler, but I did not investigate if it could be reused. Maybe you
> > > >can first try to build IR by extending Jitrino.JET.
> > > You mean modify or extend
> > > JET in order to use instead of JavaByteCodeTranslator?
> >
> > I meant take the basic concept and implement translator using this
> > concept/code-patterns. Then decide if there is a common part between
> > two and whether we need to separate it.
> >
> > I suggest to look more in JET code than in prepass or translator when
> > implementing the next big thing :)
>
> JFYI, I recall there are peculiarities related to subroutines (JSRs)
> processing, and current JET does not handle them completely
> (HARMONY-4740). So you might want to fix that as well ;)
Yes, JSR and RET instructions are tricky. Current aproach in Jitrino
is to keep the info about JSR+RET pairs until the CFG is fully
generated. And then inline them as if it is a CFG optpass. I suggest
to keep this code. So, only pre-CFG hefactoring for this functionality.
HARMONY-4740 suggests a similar approach, though, with CFG it is easier.
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Alexey Varlamov <al...@gmail.com>.
21 Mar 2008 12:00:22 +0300, Egor Pasko <eg...@gmail.com>:
> On the 0x40D day of Apache Harmony Okonechnikov Konstantin wrote:
> > >hm, and how is it going? :)
> >
> > well, right now I modify [codegen]->stacklayout action. Callee-save
> > registers are always being saved/restored in the begining/end of the
> > method. But sometimes not all pathes use callee saved regs.
> > Motion of the spill code inside CFG can reduce memory calling rate.
> > (So called "shrink wrapping"
> > technique). I have
> > already implemented functions that collect info about callee save regs
> > usage in each block and find proper postion for spill code,
> > hope to finish the optimization to the end of April.
>
> wow, this is a very interesting work, assuming some tradeoffs to
> consider. I love this :)
>
> > >seriously, what is your experience with the code?
> >
> > I am familiar with common principles ( DRVLM,HIR, LIR, CFG etc)
> > and have expierience with codegenerator.
>
> nice nice nice
>
> > > ... there is a JavaLabelPrepass class that namely should only mark
> > >boundaries of basic blocks. But it does a lot more, for example,
> > >inferring types of variables. It is very complicated, makes
> > >unpredictable number of traversals through the bytecode, not easy to
> > >support.
> >
> > Some questions.
> > As I
> > understood TranslatatorSession::translate() starts parsing calling
> > JavaTranslator::translateMethod(), right?
> > Then we parse bytecode using JavaByteCodeTranslator as parser
> > callback. When are JavaLabelPrepass methods called?
>
> I agree, this is not very logical. You are now closer to the crappy
> code, just start feeling it :)
>
> in constructor of JavaByteCodeTranslator:
> JavaByteCodeTranslator.cpp:142: initLocalVars();
>
> in initLocalVars:220:
> //
> // perform label prepass in this method
> //
> parser.parse(&prepass);
> prepass.parseDone();
>
> it was supposed that parser incapsulates generic code for parsing,
> but it does not have a lot of code to incapsulate and only makes things
> more complicated.
>
> > What is VariableIncarnation? What is SlotVar?
>
> In brief, VariableIncarnation will refer to a single operand in
> HIR. But during translation phase we do not know which variables
> (local variables and stack variables) will represent the same operand,
> so, during translation VariableIncarnations will be connected in
> chains of equivalent vars. One chain represent a future operand.
>
> SlotVar .. you know, slot is an element of java operand stack. Just to
> keep corresponding VariableIncarnations connected to the slot in
> operation.
>
> Details later, if you wish.
IMO the JavaLabelPrepass quite closely implements data-flow analysis
as described in JVMS [1], and VariableIncarnations represent merged
state of variables for each instruction, as Egor said.
[1] http://java.sun.com/docs/books/jvms/second_edition/html/ClassFile.doc.html#9801
> I think you should not dive into details of JavaLabelPrepass. Better
> try avoid it. And implement a readable and nice translator from scratch.
>
> > > ...IRBuilder separation necessity is reported as HARMONY-5500.
> > Looks like refactoring of JavaLabelPrepass and IRBuilder separation are
> > quite independent tasks. And separation is more important, isn't it?
>
> JavaLabelPrepass has nothing in common with IRBuilder. Though, both
> are a part of the translator. I cannot judge the importance here. Both
> improve design, readability, etc. IRBuilder separation is easier to
> implement, and, probably, this is what you want to start from.
>
> > >A good example of clean code that does similar things is Jitrino.JET
> > >compiler, but I did not investigate if it could be reused. Maybe you
> > >can first try to build IR by extending Jitrino.JET.
> > You mean modify or extend
> > JET in order to use instead of JavaByteCodeTranslator?
>
> I meant take the basic concept and implement translator using this
> concept/code-patterns. Then decide if there is a common part between
> two and whether we need to separate it.
>
> I suggest to look more in JET code than in prepass or translator when
> implementing the next big thing :)
JFYI, I recall there are peculiarities related to subroutines (JSRs)
processing, and current JET does not handle them completely
(HARMONY-4740). So you might want to fix that as well ;)
--
Alexey Varlamov
>
> --
> Egor Pasko
>
>
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x40D day of Apache Harmony Okonechnikov Konstantin wrote:
> >hm, and how is it going? :)
>
> well, right now I modify [codegen]->stacklayout action. Callee-save
> registers are always being saved/restored in the begining/end of the
> method. But sometimes not all pathes use callee saved regs.
> Motion of the spill code inside CFG can reduce memory calling rate.
> (So called "shrink wrapping"
> technique). I have
> already implemented functions that collect info about callee save regs
> usage in each block and find proper postion for spill code,
> hope to finish the optimization to the end of April.
wow, this is a very interesting work, assuming some tradeoffs to
consider. I love this :)
> >seriously, what is your experience with the code?
>
> I am familiar with common principles ( DRVLM,HIR, LIR, CFG etc)
> and have expierience with codegenerator.
nice nice nice
> > ... there is a JavaLabelPrepass class that namely should only mark
> >boundaries of basic blocks. But it does a lot more, for example,
> >inferring types of variables. It is very complicated, makes
> >unpredictable number of traversals through the bytecode, not easy to
> >support.
>
> Some questions.
> As I
> understood TranslatatorSession::translate() starts parsing calling
> JavaTranslator::translateMethod(), right?
> Then we parse bytecode using JavaByteCodeTranslator as parser
> callback. When are JavaLabelPrepass methods called?
I agree, this is not very logical. You are now closer to the crappy
code, just start feeling it :)
in constructor of JavaByteCodeTranslator:
JavaByteCodeTranslator.cpp:142: initLocalVars();
in initLocalVars:220:
//
// perform label prepass in this method
//
parser.parse(&prepass);
prepass.parseDone();
it was supposed that parser incapsulates generic code for parsing,
but it does not have a lot of code to incapsulate and only makes things
more complicated.
> What is VariableIncarnation? What is SlotVar?
In brief, VariableIncarnation will refer to a single operand in
HIR. But during translation phase we do not know which variables
(local variables and stack variables) will represent the same operand,
so, during translation VariableIncarnations will be connected in
chains of equivalent vars. One chain represent a future operand.
SlotVar .. you know, slot is an element of java operand stack. Just to
keep corresponding VariableIncarnations connected to the slot in
operation.
Details later, if you wish.
I think you should not dive into details of JavaLabelPrepass. Better
try avoid it. And implement a readable and nice translator from scratch.
> > ...IRBuilder separation necessity is reported as HARMONY-5500.
> Looks like refactoring of JavaLabelPrepass and IRBuilder separation are
> quite independent tasks. And separation is more important, isn't it?
JavaLabelPrepass has nothing in common with IRBuilder. Though, both
are a part of the translator. I cannot judge the importance here. Both
improve design, readability, etc. IRBuilder separation is easier to
implement, and, probably, this is what you want to start from.
> >A good example of clean code that does similar things is Jitrino.JET
> >compiler, but I did not investigate if it could be reused. Maybe you
> >can first try to build IR by extending Jitrino.JET.
> You mean modify or extend
> JET in order to use instead of JavaByteCodeTranslator?
I meant take the basic concept and implement translator using this
concept/code-patterns. Then decide if there is a common part between
two and whether we need to separate it.
I suggest to look more in JET code than in prepass or translator when
implementing the next big thing :)
--
Egor Pasko
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Okonechnikov Konstantin <ok...@gmail.com>.
>hm, and how is it going? :)
well, right now I modify [codegen]->stacklayout action. Callee-save
registers are always being saved/restored in the begining/end of the
method. But sometimes not all pathes use callee saved regs.
Motion of the spill code inside CFG can reduce memory calling rate.
(So called "shrink wrapping"
technique). I have
already implemented functions that collect info about callee save regs
usage in each block and find proper postion for spill code,
hope to finish the optimization to the end of April.
>seriously, what is your experience with the code?
I am familiar with common principles ( DRVLM,HIR, LIR, CFG etc)
and have expierience with codegenerator.
> ... there is a JavaLabelPrepass class that namely should only mark
>boundaries of basic blocks. But it does a lot more, for example,
>inferring types of variables. It is very complicated, makes
>unpredictable number of traversals through the bytecode, not easy to
>support.
Some questions.
As I
understood TranslatatorSession::translate() starts parsing calling
JavaTranslator::translateMethod(), right?
Then we parse bytecode using JavaByteCodeTranslator as parser
callback. When are JavaLabelPrepass methods called?
What is VariableIncarnation? What is SlotVar?
> ...IRBuilder separation necessity is reported as HARMONY-5500.
Looks like refactoring of JavaLabelPrepass and IRBuilder separation are
quite independent tasks. And separation is more important, isn't it?
>A good example of clean code that does similar things is Jitrino.JET
>compiler, but I did not investigate if it could be reused. Maybe you
>can first try to build IR by extending Jitrino.JET.
You mean modify or extend
JET in order to use instead of JavaByteCodeTranslator?
Best regards,
Okonechnikov Konstantin.
Re: [general] GSoC 2008 Refactor Java Bytecode Translator
Posted by Egor Pasko <eg...@gmail.com>.
On the 0x40D day of Apache Harmony Okonechnikov Konstantin wrote:
> Hello everyone!
Hello, Konstantin! Thank you for your interest in the project!
> Currently I am solving a problem of optimizing callee-saved register
> spilling in JiT opt codegenerator.
hm, and how is it going? :)
seriously, what is your experience with the code?
> I find Apache Harmony project very perspective and I am intrested in
> the idea "Refactor Java Bytecode Translator in Harmony JIT".
good!
> The goal is to move all optimizations to HL optimizer or there's
> something else? What data structures need to be simplified?
the idea is to replace the code that parses java bytecode (converts it
from stack-based to register-like based IR) with a completely new
implementation.
Namely, there is a JavaLabelPrepass class that namely should only mark
boundaries of basic blocks. But it does a lot more, for example,
inferring types of variables. It is very complicated, makes
unpredictable number of traversals through the bytecode, not easy to
support.
So, the steps of marking basic blocks and type inference should be
separated.
Then JavaByteCodeTranslator makes a single-pass through each basic
block, invoking IRBuilder, that invokes Simplifier. That was done long
time ago, when we did not have a fast compiler, now we have JET, we
need this code to be cleaned. I.e. do not perform simplification
on-the-fly, much better solution is to invoke it afterwards, when we
have HIR. IRBuilder separation necessity is reported as HARMONY-5500.
A good example of clean code that does similar things is Jitrino.JET
compiler, but I did not investigate if it could be reused. Maybe you
can first try to build IR by extending Jitrino.JET.
Feel free to ask more questions on this.
> And one more question: do I have to insert wiki into
> http://wiki.apache.org/general/SummerOfCode2008 or an appliacation woud be
> enough to present my ideas?
no. you do not need to update the page.
--
Egor Pasko