You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@qpid.apache.org by Justin Ross <ju...@gmail.com> on 2016/04/07 02:02:30 UTC
Qpid source code reorg update
Proposal: https://github.com/ssorj/qpid-svn-reorg
Previous discussion:
http://qpid.2158936.n2.nabble.com/Qpid-Subversion-reorganization-proposal-td7639094.html
Hi, everyone.
Since my last update, I have been trying to get the revamped C++ tests to
function under Windows. I've had only mixed success, but now I believe
it's time to press on.
To recap, in addition to reorganizing the source modules for independent
releases, my work overhauled the C++ tests. This means that there is now
greater potential for Qpid C++ testing on Windows, but that potential will
require more work to realize.
You can see some representative output here:
https://ci.appveyor.com/project/ssorj/qpid-svn-reorg/build/trunk.66
High level summary: the unit tests are not working[*] (and perhaps most of
all, I'd like go get *them* working), but many of the other tests are
working as expected. I am satisfied that most of the infrastructure for
making those tests work is in place, and we can start to tackle these
things as individual test issues, not test suite design issues.
On Linux (or rather on my F23 box), the tests are in relatively good
shape. I don't believe the test situation wrt Windows is a regression.
Rather, even though there are many problems to address, I think it may be a
net improvement.
Aside: there have been some related usability improvements as well. The
install mechanisms now include batch files so that important tools such as
qpid-config and qpid-python-test are executable from the Windows command
line.
What's next? I'm now up against the scheduled release of Proton 0.13.0.
Here's the sequence I have in mind:
- Merge my changes to "trunk" (see also
https://github.com/ssorj/qpid-svn-reorg#sequence-with-respect-to-git-migration
)
- Start the Proton 0.13.0 release process
- Meanwhile, address new issues arising from the reorg
- Complete the Proton 0.13.0 release
- Start and complete the Qpid C++ 1.35.0 and Qpid Python 1.35.0 releases,
using Proton 0.13.0 as a tested dependency
Thanks,
Justin
---
[*] In the appveyor output above, the part where the unit tests run, you
see a failure about not finding the appveyor-vm cert. Qpidd on Windows by
default tries to find a cert in CurrentUser/My. The PowerShell scripting I
use to set up the appveyor cert can be adapted to add one there, but it
produces a dialog box warning that I have not yet been able to suppress
(grumble). I have also tried to configure the unit tests to load the cert
from LocalMachine/My using QPID_* environment variables. While those
variables have an effect on the daemon, they do not seem to have any effect
on the unit tests binary.
In any case, when I run the unit tests on a local windows machine where I
can setup the certs correctly, I see another and more cryptic failure.
Some related pieces. Please let me know if you have suggestions. I would
really like to make this work under appveyor.
https://github.com/ssorj/qpid-svn-reorg/blob/trunk/appveyor.yml#L40
https://github.com/ssorj/qpid-svn-reorg/blob/trunk/scripts/InstallSelfSignedCert.ps1
https://github.com/ssorj/qpid-svn-reorg/blob/trunk/qpid/cpp/src/tests/run_unit_tests#L32
Re: Qpid source code reorg update
Posted by Justin Ross <ju...@gmail.com>.
Another update. CI is now in decent, if not perfect, shape. I've added
lots of suppressions for apparently real memory leaks. I'll send a
separate email regarding those.
Now that the vote on the migration to git has passed, I've raised two INFRA
jiras to start the migration process:
https://issues.apache.org/jira/browse/INFRA-11794
https://issues.apache.org/jira/browse/INFRA-11795
On Tue, Apr 26, 2016 at 7:34 AM, Justin Ross <ju...@gmail.com> wrote:
> Hi, everyone. I'm now in the midst of getting the various CI environments
> working, adding some temporary valgrind suppressions and fixing
> incompatibilities with various versions of the buildchain tools.
>
> After CI is in suitably good shape, I would like to start the process to
> migrate qpid-cpp and qpid-python to git. That will require a vote, which I
> will kick off shortly.
>
> On Thu, Apr 21, 2016 at 7:06 AM, Justin Ross <ju...@gmail.com>
> wrote:
>
>> I've committed the major pieces of this, and I'm now doing more testing
>> and improving the documentation.
>>
>> These changes are likely going to affect CI and test configurations. In
>> particular, the cpp tree now depends on an installed qpid python. I've
>> added new install instructions to the python tree.
>>
>> If you have a chance, and you think you may be affected, please take a
>> look at the new state of things and let me know if there's any trouble.
>> I'll be working on addressing problems today while the dust settles.
>>
>> Justin
>>
>> On Mon, Apr 18, 2016 at 9:44 AM, Justin Ross <ju...@gmail.com>
>> wrote:
>>
>>> On Mon, Apr 18, 2016 at 9:00 AM, Robbie Gemmell <
>>> robbie.gemmell@gmail.com> wrote:
>>>>
>>>> Looks good. Again, goes much further than I was originally expecting
>>>> initially :)
>>>>
>>>> I'm not sure I would copy the specs dir though, at least not until
>>>> some future point if particular need arises, since little/nothing will
>>>> really reference the copy.
>>>>
>>>
>>> Okay, I'll hold off on that.
>>>
>>>
>>>> The readme for the update no longer mentions the Java QMF bits,
>>>> suggesting they aren't getting moved, but I see they are still in the
>>>> reorg fork at a previously-moved-there location, so just in case: if
>>>> nobody is committing to update and release them, as seems to be the
>>>> case currently, then I'd also leave them where they are in the current
>>>> repo until cause arises to do otherwise.
>>>>
>>>
>>> Yes. It's been ambiguous in my mind as well as in the proposal. I will
>>> leave them as-is for now and wait for a response from Fraser. After a
>>> time, if I don't hear from him, I'll proceed with removing them in a
>>> second-round cleanup.
>>>
>>>
>>>> The above said, perhaps their current nested structure is the main
>>>> reason they were moved in the fork, in which case perhaps removing
>>>> them from trunk first is the way to go. Ditto the WCF bits.
>>>>
>>>> Might be best to create a pre-reorg tag of everything before commencing.
>>>>
>>>
>>> Definitely. Good idea.
>>>
>>>
>>>> Finally, I'd suggest to use svn directly to do the significant
>>>> moves/copies, as using git-svn for example sometimes wont end up doing
>>>> exactly what you wanted/expected in those cases.
>>>>
>>>
>>> Will do. Thanks for the feedback and guidance.
>>>
>>> I've made a dry run and met with success, so I will kick this off
>>> shortly.
>>>
>>> Justin
>>>
>>
>>
>
Re: Qpid source code reorg update
Posted by Justin Ross <ju...@gmail.com>.
Hi, everyone. I'm now in the midst of getting the various CI environments
working, adding some temporary valgrind suppressions and fixing
incompatibilities with various versions of the buildchain tools.
After CI is in suitably good shape, I would like to start the process to
migrate qpid-cpp and qpid-python to git. That will require a vote, which I
will kick off shortly.
On Thu, Apr 21, 2016 at 7:06 AM, Justin Ross <ju...@gmail.com> wrote:
> I've committed the major pieces of this, and I'm now doing more testing
> and improving the documentation.
>
> These changes are likely going to affect CI and test configurations. In
> particular, the cpp tree now depends on an installed qpid python. I've
> added new install instructions to the python tree.
>
> If you have a chance, and you think you may be affected, please take a
> look at the new state of things and let me know if there's any trouble.
> I'll be working on addressing problems today while the dust settles.
>
> Justin
>
> On Mon, Apr 18, 2016 at 9:44 AM, Justin Ross <ju...@gmail.com>
> wrote:
>
>> On Mon, Apr 18, 2016 at 9:00 AM, Robbie Gemmell <robbie.gemmell@gmail.com
>> > wrote:
>>>
>>> Looks good. Again, goes much further than I was originally expecting
>>> initially :)
>>>
>>> I'm not sure I would copy the specs dir though, at least not until
>>> some future point if particular need arises, since little/nothing will
>>> really reference the copy.
>>>
>>
>> Okay, I'll hold off on that.
>>
>>
>>> The readme for the update no longer mentions the Java QMF bits,
>>> suggesting they aren't getting moved, but I see they are still in the
>>> reorg fork at a previously-moved-there location, so just in case: if
>>> nobody is committing to update and release them, as seems to be the
>>> case currently, then I'd also leave them where they are in the current
>>> repo until cause arises to do otherwise.
>>>
>>
>> Yes. It's been ambiguous in my mind as well as in the proposal. I will
>> leave them as-is for now and wait for a response from Fraser. After a
>> time, if I don't hear from him, I'll proceed with removing them in a
>> second-round cleanup.
>>
>>
>>> The above said, perhaps their current nested structure is the main
>>> reason they were moved in the fork, in which case perhaps removing
>>> them from trunk first is the way to go. Ditto the WCF bits.
>>>
>>> Might be best to create a pre-reorg tag of everything before commencing.
>>>
>>
>> Definitely. Good idea.
>>
>>
>>> Finally, I'd suggest to use svn directly to do the significant
>>> moves/copies, as using git-svn for example sometimes wont end up doing
>>> exactly what you wanted/expected in those cases.
>>>
>>
>> Will do. Thanks for the feedback and guidance.
>>
>> I've made a dry run and met with success, so I will kick this off shortly.
>>
>> Justin
>>
>
>
Re: Qpid source code reorg update
Posted by Justin Ross <ju...@gmail.com>.
I've committed the major pieces of this, and I'm now doing more testing and
improving the documentation.
These changes are likely going to affect CI and test configurations. In
particular, the cpp tree now depends on an installed qpid python. I've
added new install instructions to the python tree.
If you have a chance, and you think you may be affected, please take a look
at the new state of things and let me know if there's any trouble. I'll be
working on addressing problems today while the dust settles.
Justin
On Mon, Apr 18, 2016 at 9:44 AM, Justin Ross <ju...@gmail.com> wrote:
> On Mon, Apr 18, 2016 at 9:00 AM, Robbie Gemmell <ro...@gmail.com>
> wrote:
>>
>> Looks good. Again, goes much further than I was originally expecting
>> initially :)
>>
>> I'm not sure I would copy the specs dir though, at least not until
>> some future point if particular need arises, since little/nothing will
>> really reference the copy.
>>
>
> Okay, I'll hold off on that.
>
>
>> The readme for the update no longer mentions the Java QMF bits,
>> suggesting they aren't getting moved, but I see they are still in the
>> reorg fork at a previously-moved-there location, so just in case: if
>> nobody is committing to update and release them, as seems to be the
>> case currently, then I'd also leave them where they are in the current
>> repo until cause arises to do otherwise.
>>
>
> Yes. It's been ambiguous in my mind as well as in the proposal. I will
> leave them as-is for now and wait for a response from Fraser. After a
> time, if I don't hear from him, I'll proceed with removing them in a
> second-round cleanup.
>
>
>> The above said, perhaps their current nested structure is the main
>> reason they were moved in the fork, in which case perhaps removing
>> them from trunk first is the way to go. Ditto the WCF bits.
>>
>> Might be best to create a pre-reorg tag of everything before commencing.
>>
>
> Definitely. Good idea.
>
>
>> Finally, I'd suggest to use svn directly to do the significant
>> moves/copies, as using git-svn for example sometimes wont end up doing
>> exactly what you wanted/expected in those cases.
>>
>
> Will do. Thanks for the feedback and guidance.
>
> I've made a dry run and met with success, so I will kick this off shortly.
>
> Justin
>
Re: Qpid source code reorg update
Posted by Justin Ross <ju...@gmail.com>.
On Mon, Apr 18, 2016 at 9:00 AM, Robbie Gemmell <ro...@gmail.com>
wrote:
>
> Looks good. Again, goes much further than I was originally expecting
> initially :)
>
> I'm not sure I would copy the specs dir though, at least not until
> some future point if particular need arises, since little/nothing will
> really reference the copy.
>
Okay, I'll hold off on that.
> The readme for the update no longer mentions the Java QMF bits,
> suggesting they aren't getting moved, but I see they are still in the
> reorg fork at a previously-moved-there location, so just in case: if
> nobody is committing to update and release them, as seems to be the
> case currently, then I'd also leave them where they are in the current
> repo until cause arises to do otherwise.
>
Yes. It's been ambiguous in my mind as well as in the proposal. I will
leave them as-is for now and wait for a response from Fraser. After a
time, if I don't hear from him, I'll proceed with removing them in a
second-round cleanup.
> The above said, perhaps their current nested structure is the main
> reason they were moved in the fork, in which case perhaps removing
> them from trunk first is the way to go. Ditto the WCF bits.
>
> Might be best to create a pre-reorg tag of everything before commencing.
>
Definitely. Good idea.
> Finally, I'd suggest to use svn directly to do the significant
> moves/copies, as using git-svn for example sometimes wont end up doing
> exactly what you wanted/expected in those cases.
>
Will do. Thanks for the feedback and guidance.
I've made a dry run and met with success, so I will kick this off shortly.
Justin
Re: Qpid source code reorg update
Posted by Robbie Gemmell <ro...@gmail.com>.
On 7 April 2016 at 01:02, Justin Ross <ju...@gmail.com> wrote:
> Proposal: https://github.com/ssorj/qpid-svn-reorg
> Previous discussion:
> http://qpid.2158936.n2.nabble.com/Qpid-Subversion-reorganization-proposal-td7639094.html
>
> Hi, everyone.
>
> Since my last update, I have been trying to get the revamped C++ tests to
> function under Windows. I've had only mixed success, but now I believe
> it's time to press on.
>
> To recap, in addition to reorganizing the source modules for independent
> releases, my work overhauled the C++ tests. This means that there is now
> greater potential for Qpid C++ testing on Windows, but that potential will
> require more work to realize.
>
> You can see some representative output here:
>
> https://ci.appveyor.com/project/ssorj/qpid-svn-reorg/build/trunk.66
>
> High level summary: the unit tests are not working[*] (and perhaps most of
> all, I'd like go get *them* working), but many of the other tests are
> working as expected. I am satisfied that most of the infrastructure for
> making those tests work is in place, and we can start to tackle these
> things as individual test issues, not test suite design issues.
>
> On Linux (or rather on my F23 box), the tests are in relatively good
> shape. I don't believe the test situation wrt Windows is a regression.
> Rather, even though there are many problems to address, I think it may be a
> net improvement.
>
> Aside: there have been some related usability improvements as well. The
> install mechanisms now include batch files so that important tools such as
> qpid-config and qpid-python-test are executable from the Windows command
> line.
>
> What's next? I'm now up against the scheduled release of Proton 0.13.0.
> Here's the sequence I have in mind:
>
> - Merge my changes to "trunk" (see also
> https://github.com/ssorj/qpid-svn-reorg#sequence-with-respect-to-git-migration
> )
> - Start the Proton 0.13.0 release process
> - Meanwhile, address new issues arising from the reorg
> - Complete the Proton 0.13.0 release
> - Start and complete the Qpid C++ 1.35.0 and Qpid Python 1.35.0 releases,
> using Proton 0.13.0 as a tested dependency
>
> Thanks,
> Justin
>
> ---
>
> [*] In the appveyor output above, the part where the unit tests run, you
> see a failure about not finding the appveyor-vm cert. Qpidd on Windows by
> default tries to find a cert in CurrentUser/My. The PowerShell scripting I
> use to set up the appveyor cert can be adapted to add one there, but it
> produces a dialog box warning that I have not yet been able to suppress
> (grumble). I have also tried to configure the unit tests to load the cert
> from LocalMachine/My using QPID_* environment variables. While those
> variables have an effect on the daemon, they do not seem to have any effect
> on the unit tests binary.
>
> In any case, when I run the unit tests on a local windows machine where I
> can setup the certs correctly, I see another and more cryptic failure.
>
> Some related pieces. Please let me know if you have suggestions. I would
> really like to make this work under appveyor.
>
> https://github.com/ssorj/qpid-svn-reorg/blob/trunk/appveyor.yml#L40
> https://github.com/ssorj/qpid-svn-reorg/blob/trunk/scripts/InstallSelfSignedCert.ps1
> https://github.com/ssorj/qpid-svn-reorg/blob/trunk/qpid/cpp/src/tests/run_unit_tests#L32
Looks good. Again, goes much further than I was originally expecting
initially :)
I'm not sure I would copy the specs dir though, at least not until
some future point if particular need arises, since little/nothing will
really reference the copy.
The readme for the update no longer mentions the Java QMF bits,
suggesting they aren't getting moved, but I see they are still in the
reorg fork at a previously-moved-there location, so just in case: if
nobody is committing to update and release them, as seems to be the
case currently, then I'd also leave them where they are in the current
repo until cause arises to do otherwise.
The above said, perhaps their current nested structure is the main
reason they were moved in the fork, in which case perhaps removing
them from trunk first is the way to go. Ditto the WCF bits.
Might be best to create a pre-reorg tag of everything before commencing.
Finally, I'd suggest to use svn directly to do the significant
moves/copies, as using git-svn for example sometimes wont end up doing
exactly what you wanted/expected in those cases.
Robbie
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org