You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mynewt.apache.org by Cris Frusina <cr...@frusina.com> on 2017/01/04 13:53:34 UTC
Compile and build time, any way to speed up?
Hi All,
I'm using the Docker method to run myNewt.
I have tried giving the VM more resources doesn't seem to do anything in terms of speed.
Is there any way to speed up the compile and build time?
Would the native method be substantially faster?
Thanks,
Cris
Re: Compile and build time, any way to speed up?
Posted by Cris Frusina <cr...@frusina.com>.
Thanks Marko, I'll try it out when I get a chance.
Cheers,
Cris
> On Jan 4, 2017, at 1:55 PM, marko kiiskila <ma...@runtime.io> wrote:
>
> Hi,
>
> those are merged in, and are present in develop branch.
>
> https://github.com/apache/incubator-mynewt-core/pull/150 <https://github.com/apache/incubator-mynewt-core/pull/150>
> https://github.com/apache/incubator-mynewt-newt/pull/29 <https://github.com/apache/incubator-mynewt-newt/pull/29>
>
>
>> On Jan 4, 2017, at 10:35 AM, Cris Frusina <cr...@frusina.com> wrote:
>>
>> Hi Marko,
>>
>> Where do I find Simon's changes? I'd like to give it a try.
>>
>> Thanks
>> Cris
>>
>>> On Jan 4, 2017, at 1:21 PM, marko kiiskila <ma...@runtime.io> wrote:
>>>
>>> Hi Cris,
>>>
>>> Simon Ratner has made a number of improvements to newt which should
>>> help with the native Windows support.
>>>
>>> I have not had a chance to try those out (as I don’t have Windows setup),
>>> so I don’t know what the exact state is. But they did seem pretty good.
>>> I do recommend trying that out. And improving those, if necessary ;)
>>>
>>>> On Jan 4, 2017, at 10:07 AM, Cris Frusina <cr...@frusina.com> wrote:
>>>>
>>>> Hi Chris,
>>>>
>>>> Wondering if this is a Windows issue with docker. Those are good additions, looking forward to the implementation!
>>>>
>>>> Hopefully David has some free time and can give it a try on his end yo see if he gets the same performance issues.
>>>>
>>>> I'm going to try setting everything up on another Windows machine as a sanity check.
>>>>
>>>> Btw, thanks to everyone for actively helping out.
>>>>
>>>> Cheers,
>>>> Cris
>>>>
>>>>
>>>>
>>>>> On Jan 4, 2017, at 11:45 AM, Christopher Collins <cc...@apache.org> wrote:
>>>>>
>>>>> Hi Cris,
>>>>>
>>>>>> On Wed, Jan 04, 2017 at 09:32:37AM -0500, Cris Frusina wrote:
>>>>>> I'm building my code on top of the bleprph app. I haven't done too
>>>>>> much to it as I'm still just playing around with myNewt. I've added
>>>>>> the nffs dependency and some custom code, most of it is still the
>>>>>> same.
>>>>>>
>>>>>> It depends on the changes I make to the code, if I only change the
>>>>>> main.c file the compile, build and sign would be about 2 to 3 min for
>>>>>> any small change. This can go up to 5-7 min if the change requires to
>>>>>> recompile some other dependencies. A full compile and build is about
>>>>>> 20 to 30 min.
>>>>>
>>>>> Whoa, that's insane! A build definitely should not take that long. The
>>>>> docker solution on non-Linux machines will be slower than native, but
>>>>> not that much slower! It has been a while since I have used docker, so
>>>>> I don't recall the exact build times, but I wouldn't expect a full
>>>>> rebuild of bleprph to take longer than five minutes. If you're only
>>>>> changing one file, the build time definitely should not exceed 10
>>>>> seconds.
>>>>>
>>>>> There are two planned speed improvements for newt. I don't think these
>>>>> will fix the problem you're experiencing, but I wanted to mention them:
>>>>>
>>>>> 1. Multithreaded builds. This is analogous to make's "-j" option. This
>>>>> is an obvious improvement that should greatly reduce build times,
>>>>> particularly for full rebuilds.
>>>>>
>>>>> 2. Smarter YAML parsing. Newt has to do a lot of inefficient YAML file
>>>>> processing to determine which files need to be rebuilt and how to build
>>>>> them. This is especially noticeable in builds with a lot of syscfg
>>>>> settings (particularly bluetooth). I can think of a change that should
>>>>> give a pretty good speed improvement to this process. This should help
>>>>> most in cases where only one or a few files need to be rebuilt.
>>>>>
>>>>> Chris
>>>>>
>>>>
>>>
>>>
>>
>
Re: Compile and build time, any way to speed up?
Posted by marko kiiskila <ma...@runtime.io>.
Hi,
those are merged in, and are present in develop branch.
https://github.com/apache/incubator-mynewt-core/pull/150 <https://github.com/apache/incubator-mynewt-core/pull/150>
https://github.com/apache/incubator-mynewt-newt/pull/29 <https://github.com/apache/incubator-mynewt-newt/pull/29>
> On Jan 4, 2017, at 10:35 AM, Cris Frusina <cr...@frusina.com> wrote:
>
> Hi Marko,
>
> Where do I find Simon's changes? I'd like to give it a try.
>
> Thanks
> Cris
>
>> On Jan 4, 2017, at 1:21 PM, marko kiiskila <ma...@runtime.io> wrote:
>>
>> Hi Cris,
>>
>> Simon Ratner has made a number of improvements to newt which should
>> help with the native Windows support.
>>
>> I have not had a chance to try those out (as I don’t have Windows setup),
>> so I don’t know what the exact state is. But they did seem pretty good.
>> I do recommend trying that out. And improving those, if necessary ;)
>>
>>> On Jan 4, 2017, at 10:07 AM, Cris Frusina <cr...@frusina.com> wrote:
>>>
>>> Hi Chris,
>>>
>>> Wondering if this is a Windows issue with docker. Those are good additions, looking forward to the implementation!
>>>
>>> Hopefully David has some free time and can give it a try on his end yo see if he gets the same performance issues.
>>>
>>> I'm going to try setting everything up on another Windows machine as a sanity check.
>>>
>>> Btw, thanks to everyone for actively helping out.
>>>
>>> Cheers,
>>> Cris
>>>
>>>
>>>
>>>> On Jan 4, 2017, at 11:45 AM, Christopher Collins <cc...@apache.org> wrote:
>>>>
>>>> Hi Cris,
>>>>
>>>>> On Wed, Jan 04, 2017 at 09:32:37AM -0500, Cris Frusina wrote:
>>>>> I'm building my code on top of the bleprph app. I haven't done too
>>>>> much to it as I'm still just playing around with myNewt. I've added
>>>>> the nffs dependency and some custom code, most of it is still the
>>>>> same.
>>>>>
>>>>> It depends on the changes I make to the code, if I only change the
>>>>> main.c file the compile, build and sign would be about 2 to 3 min for
>>>>> any small change. This can go up to 5-7 min if the change requires to
>>>>> recompile some other dependencies. A full compile and build is about
>>>>> 20 to 30 min.
>>>>
>>>> Whoa, that's insane! A build definitely should not take that long. The
>>>> docker solution on non-Linux machines will be slower than native, but
>>>> not that much slower! It has been a while since I have used docker, so
>>>> I don't recall the exact build times, but I wouldn't expect a full
>>>> rebuild of bleprph to take longer than five minutes. If you're only
>>>> changing one file, the build time definitely should not exceed 10
>>>> seconds.
>>>>
>>>> There are two planned speed improvements for newt. I don't think these
>>>> will fix the problem you're experiencing, but I wanted to mention them:
>>>>
>>>> 1. Multithreaded builds. This is analogous to make's "-j" option. This
>>>> is an obvious improvement that should greatly reduce build times,
>>>> particularly for full rebuilds.
>>>>
>>>> 2. Smarter YAML parsing. Newt has to do a lot of inefficient YAML file
>>>> processing to determine which files need to be rebuilt and how to build
>>>> them. This is especially noticeable in builds with a lot of syscfg
>>>> settings (particularly bluetooth). I can think of a change that should
>>>> give a pretty good speed improvement to this process. This should help
>>>> most in cases where only one or a few files need to be rebuilt.
>>>>
>>>> Chris
>>>>
>>>
>>
>>
>
Re: Compile and build time, any way to speed up?
Posted by Cris Frusina <cr...@frusina.com>.
Hi Marko,
Where do I find Simon's changes? I'd like to give it a try.
Thanks
Cris
> On Jan 4, 2017, at 1:21 PM, marko kiiskila <ma...@runtime.io> wrote:
>
> Hi Cris,
>
> Simon Ratner has made a number of improvements to newt which should
> help with the native Windows support.
>
> I have not had a chance to try those out (as I don’t have Windows setup),
> so I don’t know what the exact state is. But they did seem pretty good.
> I do recommend trying that out. And improving those, if necessary ;)
>
>> On Jan 4, 2017, at 10:07 AM, Cris Frusina <cr...@frusina.com> wrote:
>>
>> Hi Chris,
>>
>> Wondering if this is a Windows issue with docker. Those are good additions, looking forward to the implementation!
>>
>> Hopefully David has some free time and can give it a try on his end yo see if he gets the same performance issues.
>>
>> I'm going to try setting everything up on another Windows machine as a sanity check.
>>
>> Btw, thanks to everyone for actively helping out.
>>
>> Cheers,
>> Cris
>>
>>
>>
>>> On Jan 4, 2017, at 11:45 AM, Christopher Collins <cc...@apache.org> wrote:
>>>
>>> Hi Cris,
>>>
>>>> On Wed, Jan 04, 2017 at 09:32:37AM -0500, Cris Frusina wrote:
>>>> I'm building my code on top of the bleprph app. I haven't done too
>>>> much to it as I'm still just playing around with myNewt. I've added
>>>> the nffs dependency and some custom code, most of it is still the
>>>> same.
>>>>
>>>> It depends on the changes I make to the code, if I only change the
>>>> main.c file the compile, build and sign would be about 2 to 3 min for
>>>> any small change. This can go up to 5-7 min if the change requires to
>>>> recompile some other dependencies. A full compile and build is about
>>>> 20 to 30 min.
>>>
>>> Whoa, that's insane! A build definitely should not take that long. The
>>> docker solution on non-Linux machines will be slower than native, but
>>> not that much slower! It has been a while since I have used docker, so
>>> I don't recall the exact build times, but I wouldn't expect a full
>>> rebuild of bleprph to take longer than five minutes. If you're only
>>> changing one file, the build time definitely should not exceed 10
>>> seconds.
>>>
>>> There are two planned speed improvements for newt. I don't think these
>>> will fix the problem you're experiencing, but I wanted to mention them:
>>>
>>> 1. Multithreaded builds. This is analogous to make's "-j" option. This
>>> is an obvious improvement that should greatly reduce build times,
>>> particularly for full rebuilds.
>>>
>>> 2. Smarter YAML parsing. Newt has to do a lot of inefficient YAML file
>>> processing to determine which files need to be rebuilt and how to build
>>> them. This is especially noticeable in builds with a lot of syscfg
>>> settings (particularly bluetooth). I can think of a change that should
>>> give a pretty good speed improvement to this process. This should help
>>> most in cases where only one or a few files need to be rebuilt.
>>>
>>> Chris
>>>
>>
>
>
Re: Compile and build time, any way to speed up?
Posted by marko kiiskila <ma...@runtime.io>.
Hi Cris,
Simon Ratner has made a number of improvements to newt which should
help with the native Windows support.
I have not had a chance to try those out (as I don’t have Windows setup),
so I don’t know what the exact state is. But they did seem pretty good.
I do recommend trying that out. And improving those, if necessary ;)
> On Jan 4, 2017, at 10:07 AM, Cris Frusina <cr...@frusina.com> wrote:
>
> Hi Chris,
>
> Wondering if this is a Windows issue with docker. Those are good additions, looking forward to the implementation!
>
> Hopefully David has some free time and can give it a try on his end yo see if he gets the same performance issues.
>
> I'm going to try setting everything up on another Windows machine as a sanity check.
>
> Btw, thanks to everyone for actively helping out.
>
> Cheers,
> Cris
>
>
>
>> On Jan 4, 2017, at 11:45 AM, Christopher Collins <cc...@apache.org> wrote:
>>
>> Hi Cris,
>>
>>> On Wed, Jan 04, 2017 at 09:32:37AM -0500, Cris Frusina wrote:
>>> I'm building my code on top of the bleprph app. I haven't done too
>>> much to it as I'm still just playing around with myNewt. I've added
>>> the nffs dependency and some custom code, most of it is still the
>>> same.
>>>
>>> It depends on the changes I make to the code, if I only change the
>>> main.c file the compile, build and sign would be about 2 to 3 min for
>>> any small change. This can go up to 5-7 min if the change requires to
>>> recompile some other dependencies. A full compile and build is about
>>> 20 to 30 min.
>>
>> Whoa, that's insane! A build definitely should not take that long. The
>> docker solution on non-Linux machines will be slower than native, but
>> not that much slower! It has been a while since I have used docker, so
>> I don't recall the exact build times, but I wouldn't expect a full
>> rebuild of bleprph to take longer than five minutes. If you're only
>> changing one file, the build time definitely should not exceed 10
>> seconds.
>>
>> There are two planned speed improvements for newt. I don't think these
>> will fix the problem you're experiencing, but I wanted to mention them:
>>
>> 1. Multithreaded builds. This is analogous to make's "-j" option. This
>> is an obvious improvement that should greatly reduce build times,
>> particularly for full rebuilds.
>>
>> 2. Smarter YAML parsing. Newt has to do a lot of inefficient YAML file
>> processing to determine which files need to be rebuilt and how to build
>> them. This is especially noticeable in builds with a lot of syscfg
>> settings (particularly bluetooth). I can think of a change that should
>> give a pretty good speed improvement to this process. This should help
>> most in cases where only one or a few files need to be rebuilt.
>>
>> Chris
>>
>
Re: Compile and build time, any way to speed up?
Posted by Alan Graves <ag...@deltacontrols.com>.
I had a similar environment with Xubuntu VM on the Windows 7 laptop machine, but ran into problems connecting to the USB JLink port on my nRF52832 DK board. I never did figure out the USB problem, but I might have a PC motherboard issue that I'm in the process of dealing with.
I like the idea of the VM and would like to be able to getting it all going at some point. In fact making a VMware or VirtualBox image available to newbie developers (like me) can speed up the mynewt setup. :)
ALan
________________________________________
From: Cris Frusina <cr...@frusina.com>
Sent: Thursday, January 05, 2017 6:00 AM
To: dev@mynewt.incubator.apache.org
Subject: Re: Compile and build time, any way to speed up?
Hi David,
Sounds like a pain!
So I got an Ubuntu VM running on the Windows machine and installed the myNewt natively. I didn't really see much of a speed bump (needs a bit further testing). I'll try a different computer later today. Worst case I'll find a Mac to try it on.
Cris
> On Jan 5, 2017, at 8:34 AM, David G. Simmons <sa...@mac.com> wrote:
>
> Well, I spent the entire day yesterday working on this, and unfortunately, I don't have any real results.
>
> My only Windows setup is a VM, and it turns out that running a Docker instance on a Windows VM on top of Mac OS X is a fairly unrewarding experience in and of itself, and getting the mynewt stuff running in that setup is, to a first approximation, not easily achieved.
>
> I'll see if I can resurrect a Windows box here to try again, but I'm afraid I'm not able to get any further down this road.
>
> dg
>
>> On Jan 4, 2017, at 1:07 PM, Cris Frusina <cr...@frusina.com> wrote:
>>
>> Hopefully David has some free time and can give it a try on his end yo see if he gets the same performance issues.
>
> --
> David G. Simmons
> (919) 534-5099
> Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
> /** Message digitally signed for security and authenticity.
> * If you cannot read the PGP.sig attachment, please go to
> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
> * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
> **/
> ♺ This email uses 100% recycled electrons. Don't blow it by printing!
>
> There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
>
>
Re: Compile and build time, any way to speed up?
Posted by Jacob Rosenthal <ja...@gmail.com>.
I can confirm that newt is unusably slow via docker on osx as well. I also
rebuilt from develop so nothing upstream helping there. (btw the jlink
dependency isnt documented currently)
On Fri, Jan 6, 2017 at 11:56 AM, Christopher Collins <cc...@apache.org>
wrote:
> Hi Cris,
>
> On Fri, Jan 06, 2017 at 12:54:03PM -0500, Cris Frusina wrote:
> > Fixed the TCP issue by restarting Virtualbox, did an newt upgrade and a
> > newt install with the 0-dev in the project file.
> >
> > Still missing something as -j is not recognized.
>
> The -j option is only supported in the develop branch. To get the
> develop version of newt:
>
> cd $GOPATH/src/mynewt.apache.org/newt/newt &&
> git checkout develop
> git pull origin develop
> go install
>
> But be careful: the develop version of newt is not compatible with
> Mynewt version 1.0.0-b1. If you use newt develop, then you'll also need
> to use Mynewt develop. You can upgrade your Mynewt repositories to the
> develop branch as follows:
>
> 1. Open your project.yml in an editor.
> 2. Change all the "vers" fields to 0-dev (they are probably
> currently set to 0-latest).
> 3. Run "newt upgrade"
>
> If you ever want to revert back to 1.0.0-b1, you can reverse the above
> process, i.e., set the vers fields to 0-latest and run "newt upgrade".
>
> Chris
>
> >
> > ~/newt version
> > Apache Newt (incubating) version: 1.0.0-b1
> >
> > ~/newt build -j 5 ble_usb
> > Error: unknown shorthand flag: 'j' in -j
> > Usage:
> > newt build <target-name> [target-names...] [flags]
> >
> > Global Flags:
> > -l, --loglevel string Log level (default "WARN")
> > -o, --outfile string Filename to tee output to
> > -q, --quiet Be quiet; only display error output
> > -s, --silent Be silent; don't output anything
> > -v, --verbose Enable verbose output when executing commands
> >
> > Cris
> >
> >
> >
> > On 1/6/2017 12:27 PM, David G. Simmons wrote:
> > > That looks like your network isn't working from your Docker instance.
> If that's actually the case, it could also (possibly) explain the
> torturously long compile times. Maybe.
> > >
> > > dg
> > >> On Jan 6, 2017, at 12:24 PM, Cris Frusina <cr...@frusina.com> wrote:
> > >>
> > >> David,
> > >>
> > >> I've tried what you suggested I'm getting this error:
> > >>
> > >> *apache-mynewt-core**
> > >> **Error: Get https://api.github.com/repos/
> apache/incubator-mynewt-core/contents/repository.yml?ref=master: dial
> tcp: lookup api.github.com on 10.0.2.3:53: read udp 172.17.0.2:38728->
> 10.0.2.3:53: i/o timeout*
> > >>
> > >>
> > >> My project file looks like this now:
> > >>
> > >> project.name: "my_project"
> > >>
> > >> project.repositories:
> > >> - apache-mynewt-core
> > >>
> > >> # Use github's distribution mechanism for core ASF libraries.
> > >> # This provides mirroring automatically for us.
> > >> #
> > >> repository.apache-mynewt-core:
> > >> type: github
> > >> vers: 0-dev
> > >> user: apache
> > >> repo: incubator-mynewt-core
> > >>
> > >>
> > >> Am I missing something?
> > >>
> > >> Thanks,
> > >> Cris
> > >>
> > >> On 1/6/2017 12:05 PM, David G. Simmons wrote:
> > >>> You could do that, but if you don't want to remake the whole Docker
> instance, you should be able to simply edit the project.yml file and change
> the vers strin for repository.apache-mynewt-core to 0-dev and then run newt
> install
> > >>>
> > >>> Best regards,
> > >>> dg
> > >>>
> > >>>> On Jan 6, 2017, at 5:23 AM, Fabio Utzig <ut...@utzig.org> wrote:
> > >>>>
> > >>>> I'm not using the docker version myself, but I think you could try
> > >>>> rebuilding the docker image:
> > >>>>
> > >>>> https://github.com/runtimeinc/newt-docker <https://github.com/
> runtimeinc/newt-docker>
> > >>>>
> > >>>> Inside the Makefile, change the branch that is checked out from "-b
> > >>>> mynewt_1_0_0_b1_tag" to "-b develop":
> > >>>>
> > >>>> https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15
> <https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15>
> > >>>>
> > >>>> Att,
> > >>>> Fabio Utzig
> > >>> --
> > >>> David G. Simmons
> > >>> (919) 534-5099
> > >>> Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog>
> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <
> http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
> > >>> /** Message digitally signed for security and authenticity.
> > >>> * If you cannot read the PGP.sig attachment, please go to
> > >>> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your
> email!!!
> > >>> * Public key available at keyserver.pgp.com <
> http://keyserver.pgp.com/>
> > >>> **/
> > >>> ♺ This email uses 100% recycled electrons. Don't blow it by printing!
> > >>>
> > >>> There are only 2 hard things in computer science: Cache
> invalidation, naming things, and off-by-one errors.
> > >>>
> > >>>
> > >>>
> > > --
> > > David G. Simmons
> > > (919) 534-5099
> > > Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog>
> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <
> http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
> > > /** Message digitally signed for security and authenticity.
> > > * If you cannot read the PGP.sig attachment, please go to
> > > * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
> > > * Public key available at keyserver.pgp.com <
> http://keyserver.pgp.com/>
> > > **/
> > > ♺ This email uses 100% recycled electrons. Don't blow it by printing!
> > >
> > > There are only 2 hard things in computer science: Cache invalidation,
> naming things, and off-by-one errors.
> > >
> > >
> > >
> >
>
Re: Compile and build time, any way to speed up?
Posted by Christopher Collins <cc...@apache.org>.
Hi Cris,
On Fri, Jan 06, 2017 at 12:54:03PM -0500, Cris Frusina wrote:
> Fixed the TCP issue by restarting Virtualbox, did an newt upgrade and a
> newt install with the 0-dev in the project file.
>
> Still missing something as -j is not recognized.
The -j option is only supported in the develop branch. To get the
develop version of newt:
cd $GOPATH/src/mynewt.apache.org/newt/newt &&
git checkout develop
git pull origin develop
go install
But be careful: the develop version of newt is not compatible with
Mynewt version 1.0.0-b1. If you use newt develop, then you'll also need
to use Mynewt develop. You can upgrade your Mynewt repositories to the
develop branch as follows:
1. Open your project.yml in an editor.
2. Change all the "vers" fields to 0-dev (they are probably
currently set to 0-latest).
3. Run "newt upgrade"
If you ever want to revert back to 1.0.0-b1, you can reverse the above
process, i.e., set the vers fields to 0-latest and run "newt upgrade".
Chris
>
> ~/newt version
> Apache Newt (incubating) version: 1.0.0-b1
>
> ~/newt build -j 5 ble_usb
> Error: unknown shorthand flag: 'j' in -j
> Usage:
> newt build <target-name> [target-names...] [flags]
>
> Global Flags:
> -l, --loglevel string Log level (default "WARN")
> -o, --outfile string Filename to tee output to
> -q, --quiet Be quiet; only display error output
> -s, --silent Be silent; don't output anything
> -v, --verbose Enable verbose output when executing commands
>
> Cris
>
>
>
> On 1/6/2017 12:27 PM, David G. Simmons wrote:
> > That looks like your network isn't working from your Docker instance. If that's actually the case, it could also (possibly) explain the torturously long compile times. Maybe.
> >
> > dg
> >> On Jan 6, 2017, at 12:24 PM, Cris Frusina <cr...@frusina.com> wrote:
> >>
> >> David,
> >>
> >> I've tried what you suggested I'm getting this error:
> >>
> >> *apache-mynewt-core**
> >> **Error: Get https://api.github.com/repos/apache/incubator-mynewt-core/contents/repository.yml?ref=master: dial tcp: lookup api.github.com on 10.0.2.3:53: read udp 172.17.0.2:38728->10.0.2.3:53: i/o timeout*
> >>
> >>
> >> My project file looks like this now:
> >>
> >> project.name: "my_project"
> >>
> >> project.repositories:
> >> - apache-mynewt-core
> >>
> >> # Use github's distribution mechanism for core ASF libraries.
> >> # This provides mirroring automatically for us.
> >> #
> >> repository.apache-mynewt-core:
> >> type: github
> >> vers: 0-dev
> >> user: apache
> >> repo: incubator-mynewt-core
> >>
> >>
> >> Am I missing something?
> >>
> >> Thanks,
> >> Cris
> >>
> >> On 1/6/2017 12:05 PM, David G. Simmons wrote:
> >>> You could do that, but if you don't want to remake the whole Docker instance, you should be able to simply edit the project.yml file and change the vers strin for repository.apache-mynewt-core to 0-dev and then run newt install
> >>>
> >>> Best regards,
> >>> dg
> >>>
> >>>> On Jan 6, 2017, at 5:23 AM, Fabio Utzig <ut...@utzig.org> wrote:
> >>>>
> >>>> I'm not using the docker version myself, but I think you could try
> >>>> rebuilding the docker image:
> >>>>
> >>>> https://github.com/runtimeinc/newt-docker <https://github.com/runtimeinc/newt-docker>
> >>>>
> >>>> Inside the Makefile, change the branch that is checked out from "-b
> >>>> mynewt_1_0_0_b1_tag" to "-b develop":
> >>>>
> >>>> https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15 <https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15>
> >>>>
> >>>> Att,
> >>>> Fabio Utzig
> >>> --
> >>> David G. Simmons
> >>> (919) 534-5099
> >>> Web <https://davidgs.com/> \u2022 Blog <https://davidgs.com/davidgs_blog> \u2022 Linkedin <http://linkedin.com/in/davidgsimmons> \u2022 Twitter <http://twitter.com/TechEvangelist1> \u2022 GitHub <http://github.com/davidgs>
> >>> /** Message digitally signed for security and authenticity.
> >>> * If you cannot read the PGP.sig attachment, please go to
> >>> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
> >>> * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
> >>> **/
> >>> \u267a This email uses 100% recycled electrons. Don't blow it by printing!
> >>>
> >>> There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
> >>>
> >>>
> >>>
> > --
> > David G. Simmons
> > (919) 534-5099
> > Web <https://davidgs.com/> \u2022 Blog <https://davidgs.com/davidgs_blog> \u2022 Linkedin <http://linkedin.com/in/davidgsimmons> \u2022 Twitter <http://twitter.com/TechEvangelist1> \u2022 GitHub <http://github.com/davidgs>
> > /** Message digitally signed for security and authenticity.
> > * If you cannot read the PGP.sig attachment, please go to
> > * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
> > * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
> > **/
> > \u267a This email uses 100% recycled electrons. Don't blow it by printing!
> >
> > There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
> >
> >
> >
>
Re: Compile and build time, any way to speed up?
Posted by Cris Frusina <cr...@frusina.com>.
Fixed the TCP issue by restarting Virtualbox, did an newt upgrade and a
newt install with the 0-dev in the project file.
Still missing something as -j is not recognized.
~/newt version
Apache Newt (incubating) version: 1.0.0-b1
~/newt build -j 5 ble_usb
Error: unknown shorthand flag: 'j' in -j
Usage:
newt build <target-name> [target-names...] [flags]
Global Flags:
-l, --loglevel string Log level (default "WARN")
-o, --outfile string Filename to tee output to
-q, --quiet Be quiet; only display error output
-s, --silent Be silent; don't output anything
-v, --verbose Enable verbose output when executing commands
Cris
On 1/6/2017 12:27 PM, David G. Simmons wrote:
> That looks like your network isn't working from your Docker instance. If that's actually the case, it could also (possibly) explain the torturously long compile times. Maybe.
>
> dg
>> On Jan 6, 2017, at 12:24 PM, Cris Frusina <cr...@frusina.com> wrote:
>>
>> David,
>>
>> I've tried what you suggested I'm getting this error:
>>
>> *apache-mynewt-core**
>> **Error: Get https://api.github.com/repos/apache/incubator-mynewt-core/contents/repository.yml?ref=master: dial tcp: lookup api.github.com on 10.0.2.3:53: read udp 172.17.0.2:38728->10.0.2.3:53: i/o timeout*
>>
>>
>> My project file looks like this now:
>>
>> project.name: "my_project"
>>
>> project.repositories:
>> - apache-mynewt-core
>>
>> # Use github's distribution mechanism for core ASF libraries.
>> # This provides mirroring automatically for us.
>> #
>> repository.apache-mynewt-core:
>> type: github
>> vers: 0-dev
>> user: apache
>> repo: incubator-mynewt-core
>>
>>
>> Am I missing something?
>>
>> Thanks,
>> Cris
>>
>> On 1/6/2017 12:05 PM, David G. Simmons wrote:
>>> You could do that, but if you don't want to remake the whole Docker instance, you should be able to simply edit the project.yml file and change the vers strin for repository.apache-mynewt-core to 0-dev and then run newt install
>>>
>>> Best regards,
>>> dg
>>>
>>>> On Jan 6, 2017, at 5:23 AM, Fabio Utzig <ut...@utzig.org> wrote:
>>>>
>>>> I'm not using the docker version myself, but I think you could try
>>>> rebuilding the docker image:
>>>>
>>>> https://github.com/runtimeinc/newt-docker <https://github.com/runtimeinc/newt-docker>
>>>>
>>>> Inside the Makefile, change the branch that is checked out from "-b
>>>> mynewt_1_0_0_b1_tag" to "-b develop":
>>>>
>>>> https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15 <https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15>
>>>>
>>>> Att,
>>>> Fabio Utzig
>>> --
>>> David G. Simmons
>>> (919) 534-5099
>>> Web <https://davidgs.com/> \u2022 Blog <https://davidgs.com/davidgs_blog> \u2022 Linkedin <http://linkedin.com/in/davidgsimmons> \u2022 Twitter <http://twitter.com/TechEvangelist1> \u2022 GitHub <http://github.com/davidgs>
>>> /** Message digitally signed for security and authenticity.
>>> * If you cannot read the PGP.sig attachment, please go to
>>> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
>>> * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
>>> **/
>>> \u267a This email uses 100% recycled electrons. Don't blow it by printing!
>>>
>>> There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
>>>
>>>
>>>
> --
> David G. Simmons
> (919) 534-5099
> Web <https://davidgs.com/> \u2022 Blog <https://davidgs.com/davidgs_blog> \u2022 Linkedin <http://linkedin.com/in/davidgsimmons> \u2022 Twitter <http://twitter.com/TechEvangelist1> \u2022 GitHub <http://github.com/davidgs>
> /** Message digitally signed for security and authenticity.
> * If you cannot read the PGP.sig attachment, please go to
> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
> * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
> **/
> \u267a This email uses 100% recycled electrons. Don't blow it by printing!
>
> There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
>
>
>
Re: Compile and build time, any way to speed up?
Posted by "David G. Simmons" <sa...@mac.com>.
That looks like your network isn't working from your Docker instance. If that's actually the case, it could also (possibly) explain the torturously long compile times. Maybe.
dg
> On Jan 6, 2017, at 12:24 PM, Cris Frusina <cr...@frusina.com> wrote:
>
> David,
>
> I've tried what you suggested I'm getting this error:
>
> *apache-mynewt-core**
> **Error: Get https://api.github.com/repos/apache/incubator-mynewt-core/contents/repository.yml?ref=master: dial tcp: lookup api.github.com on 10.0.2.3:53: read udp 172.17.0.2:38728->10.0.2.3:53: i/o timeout*
>
>
> My project file looks like this now:
>
> project.name: "my_project"
>
> project.repositories:
> - apache-mynewt-core
>
> # Use github's distribution mechanism for core ASF libraries.
> # This provides mirroring automatically for us.
> #
> repository.apache-mynewt-core:
> type: github
> vers: 0-dev
> user: apache
> repo: incubator-mynewt-core
>
>
> Am I missing something?
>
> Thanks,
> Cris
>
> On 1/6/2017 12:05 PM, David G. Simmons wrote:
>> You could do that, but if you don't want to remake the whole Docker instance, you should be able to simply edit the project.yml file and change the vers strin for repository.apache-mynewt-core to 0-dev and then run newt install
>>
>> Best regards,
>> dg
>>
>>> On Jan 6, 2017, at 5:23 AM, Fabio Utzig <ut...@utzig.org> wrote:
>>>
>>> I'm not using the docker version myself, but I think you could try
>>> rebuilding the docker image:
>>>
>>> https://github.com/runtimeinc/newt-docker <https://github.com/runtimeinc/newt-docker>
>>>
>>> Inside the Makefile, change the branch that is checked out from "-b
>>> mynewt_1_0_0_b1_tag" to "-b develop":
>>>
>>> https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15 <https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15>
>>>
>>> Att,
>>> Fabio Utzig
>> --
>> David G. Simmons
>> (919) 534-5099
>> Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
>> /** Message digitally signed for security and authenticity.
>> * If you cannot read the PGP.sig attachment, please go to
>> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
>> * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
>> **/
>> ♺ This email uses 100% recycled electrons. Don't blow it by printing!
>>
>> There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
>>
>>
>>
>
--
David G. Simmons
(919) 534-5099
Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
* http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
* Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!
There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
Re: Compile and build time, any way to speed up?
Posted by Cris Frusina <cr...@frusina.com>.
David,
I've tried what you suggested I'm getting this error:
*apache-mynewt-core**
**Error: Get
https://api.github.com/repos/apache/incubator-mynewt-core/contents/repository.yml?ref=master:
dial tcp: lookup api.github.com on 10.0.2.3:53: read udp
172.17.0.2:38728->10.0.2.3:53: i/o timeout*
My project file looks like this now:
project.name: "my_project"
project.repositories:
- apache-mynewt-core
# Use github's distribution mechanism for core ASF libraries.
# This provides mirroring automatically for us.
#
repository.apache-mynewt-core:
type: github
vers: 0-dev
user: apache
repo: incubator-mynewt-core
Am I missing something?
Thanks,
Cris
On 1/6/2017 12:05 PM, David G. Simmons wrote:
> You could do that, but if you don't want to remake the whole Docker instance, you should be able to simply edit the project.yml file and change the vers strin for repository.apache-mynewt-core to 0-dev and then run newt install
>
> Best regards,
> dg
>
>> On Jan 6, 2017, at 5:23 AM, Fabio Utzig <ut...@utzig.org> wrote:
>>
>> I'm not using the docker version myself, but I think you could try
>> rebuilding the docker image:
>>
>> https://github.com/runtimeinc/newt-docker <https://github.com/runtimeinc/newt-docker>
>>
>> Inside the Makefile, change the branch that is checked out from "-b
>> mynewt_1_0_0_b1_tag" to "-b develop":
>>
>> https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15 <https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15>
>>
>> Att,
>> Fabio Utzig
> --
> David G. Simmons
> (919) 534-5099
> Web <https://davidgs.com/> \u2022 Blog <https://davidgs.com/davidgs_blog> \u2022 Linkedin <http://linkedin.com/in/davidgsimmons> \u2022 Twitter <http://twitter.com/TechEvangelist1> \u2022 GitHub <http://github.com/davidgs>
> /** Message digitally signed for security and authenticity.
> * If you cannot read the PGP.sig attachment, please go to
> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
> * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
> **/
> \u267a This email uses 100% recycled electrons. Don't blow it by printing!
>
> There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
>
>
>
Re: Compile and build time, any way to speed up?
Posted by "David G. Simmons" <sa...@mac.com>.
You could do that, but if you don't want to remake the whole Docker instance, you should be able to simply edit the project.yml file and change the vers strin for repository.apache-mynewt-core to 0-dev and then run newt install
Best regards,
dg
> On Jan 6, 2017, at 5:23 AM, Fabio Utzig <ut...@utzig.org> wrote:
>
> I'm not using the docker version myself, but I think you could try
> rebuilding the docker image:
>
> https://github.com/runtimeinc/newt-docker <https://github.com/runtimeinc/newt-docker>
>
> Inside the Makefile, change the branch that is checked out from "-b
> mynewt_1_0_0_b1_tag" to "-b develop":
>
> https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15 <https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15>
>
> Att,
> Fabio Utzig
--
David G. Simmons
(919) 534-5099
Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
* http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
* Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!
There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
Re: Compile and build time, any way to speed up?
Posted by Fabio Utzig <ut...@utzig.org>.
I'm not using the docker version myself, but I think you could try
rebuilding the docker image:
https://github.com/runtimeinc/newt-docker
Inside the Makefile, change the branch that is checked out from "-b
mynewt_1_0_0_b1_tag" to "-b develop":
https://github.com/runtimeinc/newt-docker/blob/master/Makefile#L15
Att,
Fabio Utzig
On Fri, Jan 6, 2017, at 01:14 AM, Cris Frusina wrote:
> Stupid question... Is there any way to try out the development branch
> while using the docker method on a Windows machine?
>
> Cris
>
>
> On 1/5/2017 8:20 PM, Sterling Hughes wrote:
> > This is awesome, timing on the new build (after a clean):
> >
> > Target successfully built: targets/nrf52
> >
> > real 0m6.327s
> > user 0m26.972s
> > sys 0m9.276s
> > $ newt target show nrf52
> > targets/nrf52
> > app=apps/slinky
> > bsp=hw/bsp/nrf52dk
> > build_profile=debug
> >
> > 6 seconds for a full build. Not bad!
> >
> > Sterling
> >
> > On 5 Jan 2017, at 11:57, Christopher Collins wrote:
> >
> >> On Thu, Jan 05, 2017 at 09:00:11AM -0500, Cris Frusina wrote:
> >>> Hi David,
> >>>
> >>> Sounds like a pain!
> >>>
> >>> So I got an Ubuntu VM running on the Windows machine and installed
> >>> the myNewt natively. I didn't really see much of a speed bump (needs
> >>> a bit further testing). I'll try a different computer later today.
> >>> Worst case I'll find a Mac to try it on.
> >>
> >> Just an note - I ended up adding multithreaded build support to newt
> >> yesterday. In the develop branch, you can enable it with the
> >> "-j <num-parallel-jobs>" command line option, e.g.,
> >>
> >> newt -j 5 build my_blinky_sim
> >>
> >> A word of warning, though: don't upgrade your newt to develop if you
> >> are using Mynewt 1.0-b1 and aren't prepared to upgrade that as well.
> >> Since the 1.0-b1 release, some backwards-compatible changes were made to
> >> the newt tool.
> >>
> >> Thanks,
> >> Chris
> >
> >
>
Re: Compile and build time, any way to speed up?
Posted by Cris Frusina <cr...@frusina.com>.
Stupid question... Is there any way to try out the development branch
while using the docker method on a Windows machine?
Cris
On 1/5/2017 8:20 PM, Sterling Hughes wrote:
> This is awesome, timing on the new build (after a clean):
>
> Target successfully built: targets/nrf52
>
> real 0m6.327s
> user 0m26.972s
> sys 0m9.276s
> $ newt target show nrf52
> targets/nrf52
> app=apps/slinky
> bsp=hw/bsp/nrf52dk
> build_profile=debug
>
> 6 seconds for a full build. Not bad!
>
> Sterling
>
> On 5 Jan 2017, at 11:57, Christopher Collins wrote:
>
>> On Thu, Jan 05, 2017 at 09:00:11AM -0500, Cris Frusina wrote:
>>> Hi David,
>>>
>>> Sounds like a pain!
>>>
>>> So I got an Ubuntu VM running on the Windows machine and installed
>>> the myNewt natively. I didn't really see much of a speed bump (needs
>>> a bit further testing). I'll try a different computer later today.
>>> Worst case I'll find a Mac to try it on.
>>
>> Just an note - I ended up adding multithreaded build support to newt
>> yesterday. In the develop branch, you can enable it with the
>> "-j <num-parallel-jobs>" command line option, e.g.,
>>
>> newt -j 5 build my_blinky_sim
>>
>> A word of warning, though: don't upgrade your newt to develop if you
>> are using Mynewt 1.0-b1 and aren't prepared to upgrade that as well.
>> Since the 1.0-b1 release, some backwards-compatible changes were made to
>> the newt tool.
>>
>> Thanks,
>> Chris
>
>
Re: Compile and build time, any way to speed up?
Posted by Sterling Hughes <st...@apache.org>.
This is awesome, timing on the new build (after a clean):
Target successfully built: targets/nrf52
real 0m6.327s
user 0m26.972s
sys 0m9.276s
$ newt target show nrf52
targets/nrf52
app=apps/slinky
bsp=hw/bsp/nrf52dk
build_profile=debug
6 seconds for a full build. Not bad!
Sterling
On 5 Jan 2017, at 11:57, Christopher Collins wrote:
> On Thu, Jan 05, 2017 at 09:00:11AM -0500, Cris Frusina wrote:
>> Hi David,
>>
>> Sounds like a pain!
>>
>> So I got an Ubuntu VM running on the Windows machine and installed
>> the myNewt natively. I didn't really see much of a speed bump (needs
>> a bit further testing). I'll try a different computer later today.
>> Worst case I'll find a Mac to try it on.
>
> Just an note - I ended up adding multithreaded build support to newt
> yesterday. In the develop branch, you can enable it with the
> "-j <num-parallel-jobs>" command line option, e.g.,
>
> newt -j 5 build my_blinky_sim
>
> A word of warning, though: don't upgrade your newt to develop if you
> are using Mynewt 1.0-b1 and aren't prepared to upgrade that as well.
> Since the 1.0-b1 release, some backwards-compatible changes were made
> to
> the newt tool.
>
> Thanks,
> Chris
Re: Compile and build time, any way to speed up?
Posted by Christopher Collins <cc...@apache.org>.
On Thu, Jan 05, 2017 at 09:00:11AM -0500, Cris Frusina wrote:
> Hi David,
>
> Sounds like a pain!
>
> So I got an Ubuntu VM running on the Windows machine and installed the myNewt natively. I didn't really see much of a speed bump (needs a bit further testing). I'll try a different computer later today. Worst case I'll find a Mac to try it on.
Just an note - I ended up adding multithreaded build support to newt
yesterday. In the develop branch, you can enable it with the
"-j <num-parallel-jobs>" command line option, e.g.,
newt -j 5 build my_blinky_sim
A word of warning, though: don't upgrade your newt to develop if you
are using Mynewt 1.0-b1 and aren't prepared to upgrade that as well.
Since the 1.0-b1 release, some backwards-compatible changes were made to
the newt tool.
Thanks,
Chris
Re: Compile and build time, any way to speed up?
Posted by Cris Frusina <cr...@frusina.com>.
Hi David,
Sounds like a pain!
So I got an Ubuntu VM running on the Windows machine and installed the myNewt natively. I didn't really see much of a speed bump (needs a bit further testing). I'll try a different computer later today. Worst case I'll find a Mac to try it on.
Cris
> On Jan 5, 2017, at 8:34 AM, David G. Simmons <sa...@mac.com> wrote:
>
> Well, I spent the entire day yesterday working on this, and unfortunately, I don't have any real results.
>
> My only Windows setup is a VM, and it turns out that running a Docker instance on a Windows VM on top of Mac OS X is a fairly unrewarding experience in and of itself, and getting the mynewt stuff running in that setup is, to a first approximation, not easily achieved.
>
> I'll see if I can resurrect a Windows box here to try again, but I'm afraid I'm not able to get any further down this road.
>
> dg
>
>> On Jan 4, 2017, at 1:07 PM, Cris Frusina <cr...@frusina.com> wrote:
>>
>> Hopefully David has some free time and can give it a try on his end yo see if he gets the same performance issues.
>
> --
> David G. Simmons
> (919) 534-5099
> Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
> /** Message digitally signed for security and authenticity.
> * If you cannot read the PGP.sig attachment, please go to
> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
> * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
> **/
> ♺ This email uses 100% recycled electrons. Don't blow it by printing!
>
> There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
>
>
Re: Compile and build time, any way to speed up?
Posted by Christopher Collins <cc...@apache.org>.
On Thu, Jan 05, 2017 at 08:34:04AM -0500, David G. Simmons wrote:
> Well, I spent the entire day yesterday working on this, and unfortunately, I don't have any real results.
>
> My only Windows setup is a VM, and it turns out that running a Docker instance on a Windows VM on top of Mac OS X is a fairly unrewarding experience in and of itself, and getting the mynewt stuff running in that setup is, to a first approximation, not easily achieved.
Ouch, yeah... I had a similar experience a while back, with a similar
amount of wasted time. Eventually a I concluded that it just isn't
possible to run VirtualBox in a Windows VM (the docker solution uses
VirtualBox).
Chris
Re: Compile and build time, any way to speed up?
Posted by "David G. Simmons" <sa...@mac.com>.
Well, I spent the entire day yesterday working on this, and unfortunately, I don't have any real results.
My only Windows setup is a VM, and it turns out that running a Docker instance on a Windows VM on top of Mac OS X is a fairly unrewarding experience in and of itself, and getting the mynewt stuff running in that setup is, to a first approximation, not easily achieved.
I'll see if I can resurrect a Windows box here to try again, but I'm afraid I'm not able to get any further down this road.
dg
> On Jan 4, 2017, at 1:07 PM, Cris Frusina <cr...@frusina.com> wrote:
>
> Hopefully David has some free time and can give it a try on his end yo see if he gets the same performance issues.
--
David G. Simmons
(919) 534-5099
Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
* http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
* Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!
There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
Re: Compile and build time, any way to speed up?
Posted by Cris Frusina <cr...@frusina.com>.
Hi Chris,
Wondering if this is a Windows issue with docker. Those are good additions, looking forward to the implementation!
Hopefully David has some free time and can give it a try on his end yo see if he gets the same performance issues.
I'm going to try setting everything up on another Windows machine as a sanity check.
Btw, thanks to everyone for actively helping out.
Cheers,
Cris
> On Jan 4, 2017, at 11:45 AM, Christopher Collins <cc...@apache.org> wrote:
>
> Hi Cris,
>
>> On Wed, Jan 04, 2017 at 09:32:37AM -0500, Cris Frusina wrote:
>> I'm building my code on top of the bleprph app. I haven't done too
>> much to it as I'm still just playing around with myNewt. I've added
>> the nffs dependency and some custom code, most of it is still the
>> same.
>>
>> It depends on the changes I make to the code, if I only change the
>> main.c file the compile, build and sign would be about 2 to 3 min for
>> any small change. This can go up to 5-7 min if the change requires to
>> recompile some other dependencies. A full compile and build is about
>> 20 to 30 min.
>
> Whoa, that's insane! A build definitely should not take that long. The
> docker solution on non-Linux machines will be slower than native, but
> not that much slower! It has been a while since I have used docker, so
> I don't recall the exact build times, but I wouldn't expect a full
> rebuild of bleprph to take longer than five minutes. If you're only
> changing one file, the build time definitely should not exceed 10
> seconds.
>
> There are two planned speed improvements for newt. I don't think these
> will fix the problem you're experiencing, but I wanted to mention them:
>
> 1. Multithreaded builds. This is analogous to make's "-j" option. This
> is an obvious improvement that should greatly reduce build times,
> particularly for full rebuilds.
>
> 2. Smarter YAML parsing. Newt has to do a lot of inefficient YAML file
> processing to determine which files need to be rebuilt and how to build
> them. This is especially noticeable in builds with a lot of syscfg
> settings (particularly bluetooth). I can think of a change that should
> give a pretty good speed improvement to this process. This should help
> most in cases where only one or a few files need to be rebuilt.
>
> Chris
>
Re: Compile and build time, any way to speed up?
Posted by Christopher Collins <cc...@apache.org>.
Hi Cris,
On Wed, Jan 04, 2017 at 09:32:37AM -0500, Cris Frusina wrote:
> I'm building my code on top of the bleprph app. I haven't done too
> much to it as I'm still just playing around with myNewt. I've added
> the nffs dependency and some custom code, most of it is still the
> same.
>
> It depends on the changes I make to the code, if I only change the
> main.c file the compile, build and sign would be about 2 to 3 min for
> any small change. This can go up to 5-7 min if the change requires to
> recompile some other dependencies. A full compile and build is about
> 20 to 30 min.
Whoa, that's insane! A build definitely should not take that long. The
docker solution on non-Linux machines will be slower than native, but
not that much slower! It has been a while since I have used docker, so
I don't recall the exact build times, but I wouldn't expect a full
rebuild of bleprph to take longer than five minutes. If you're only
changing one file, the build time definitely should not exceed 10
seconds.
There are two planned speed improvements for newt. I don't think these
will fix the problem you're experiencing, but I wanted to mention them:
1. Multithreaded builds. This is analogous to make's "-j" option. This
is an obvious improvement that should greatly reduce build times,
particularly for full rebuilds.
2. Smarter YAML parsing. Newt has to do a lot of inefficient YAML file
processing to determine which files need to be rebuilt and how to build
them. This is especially noticeable in builds with a lot of syscfg
settings (particularly bluetooth). I can think of a change that should
give a pretty good speed improvement to this process. This should help
most in cases where only one or a few files need to be rebuilt.
Chris
Re: Compile and build time, any way to speed up?
Posted by Cris Frusina <cr...@frusina.com>.
Hi David,
The 4 seconds sounds far more reasonable, that's what I was used to when working with Keil. I just got frustrated over the past week spending so much time in compile & build so I decided to ask the experts.
I'm using an XPS 15 9550 with Windows 10. So it's a SSD and decent amount of RAM, I installed everything as per the QuickStart. I also have 4 vCPU and 4 GB of RAM allowance to the VM running docker. Hope that helps.
Let me know if I can help in anyway.
Thanks
Cris
> On Jan 4, 2017, at 9:39 AM, David G. Simmons <sa...@mac.com> wrote:
>
> Hi Cris,
>
> Ok, that helps. Those times seem very high to me. I'm running natively on Mac OS Sierra and using the same bleprph code base -- modified a fair amount for an ADC sensor -- and any changes to main take about 4 seconds to compile, and more substantial changes to other parts of the code can take up to 20 seconds or so.
>
> I'll have to build a docker configuration to test all this out but it does sound like something's not right.
>
> dg
>
>> On Jan 4, 2017, at 9:32 AM, Cris Frusina <cr...@frusina.com> wrote:
>>
>> Hi David,
>>
>> I'm building my code on top of the bleprph app. I haven't done too much to it as I'm still just playing around with myNewt. I've added the nffs dependency and some custom code, most of it is still the same.
>>
>> It depends on the changes I make to the code, if I only change the main.c file the compile, build and sign would be about 2 to 3 min for any small change. This can go up to 5-7 min if the change requires to recompile some other dependencies. A full compile and build is about 20 to 30 min.
>>
>>
>> Just note: these times are estimates, I can get you real numbers later today by actually timing it if needed.
>>
>> Thanks
>> Cris
>
> --
> David G. Simmons
> (919) 534-5099
> Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
> /** Message digitally signed for security and authenticity.
> * If you cannot read the PGP.sig attachment, please go to
> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
> * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
> **/
> ♺ This email uses 100% recycled electrons. Don't blow it by printing!
>
> There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
>
>
Re: Compile and build time, any way to speed up?
Posted by "David G. Simmons" <sa...@mac.com>.
Hi Cris,
Ok, that helps. Those times seem very high to me. I'm running natively on Mac OS Sierra and using the same bleprph code base -- modified a fair amount for an ADC sensor -- and any changes to main take about 4 seconds to compile, and more substantial changes to other parts of the code can take up to 20 seconds or so.
I'll have to build a docker configuration to test all this out but it does sound like something's not right.
dg
> On Jan 4, 2017, at 9:32 AM, Cris Frusina <cr...@frusina.com> wrote:
>
> Hi David,
>
> I'm building my code on top of the bleprph app. I haven't done too much to it as I'm still just playing around with myNewt. I've added the nffs dependency and some custom code, most of it is still the same.
>
> It depends on the changes I make to the code, if I only change the main.c file the compile, build and sign would be about 2 to 3 min for any small change. This can go up to 5-7 min if the change requires to recompile some other dependencies. A full compile and build is about 20 to 30 min.
>
>
> Just note: these times are estimates, I can get you real numbers later today by actually timing it if needed.
>
> Thanks
> Cris
--
David G. Simmons
(919) 534-5099
Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
* http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
* Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!
There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
Re: Compile and build time, any way to speed up?
Posted by Cris Frusina <cr...@frusina.com>.
Hi David,
I'm building my code on top of the bleprph app. I haven't done too much to it as I'm still just playing around with myNewt. I've added the nffs dependency and some custom code, most of it is still the same.
It depends on the changes I make to the code, if I only change the main.c file the compile, build and sign would be about 2 to 3 min for any small change. This can go up to 5-7 min if the change requires to recompile some other dependencies. A full compile and build is about 20 to 30 min.
Just note: these times are estimates, I can get you real numbers later today by actually timing it if needed.
Thanks
Cris
> On Jan 4, 2017, at 9:04 AM, David G. Simmons <sa...@mac.com> wrote:
>
> Hi Cris,
>
> What sorts of compile times are you seeing? And for what apps?
>
> Thanks,
> dg
>
>> On Jan 4, 2017, at 8:53 AM, Cris Frusina <cr...@frusina.com> wrote:
>>
>> Hi All,
>>
>> I'm using the Docker method to run myNewt.
>>
>> I have tried giving the VM more resources doesn't seem to do anything in terms of speed.
>>
>> Is there any way to speed up the compile and build time?
>>
>> Would the native method be substantially faster?
>>
>> Thanks,
>> Cris
>
> --
> David G. Simmons
> (919) 534-5099
> Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
> /** Message digitally signed for security and authenticity.
> * If you cannot read the PGP.sig attachment, please go to
> * http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
> * Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
> **/
> ♺ This email uses 100% recycled electrons. Don't blow it by printing!
>
> There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.
>
>
Re: Compile and build time, any way to speed up?
Posted by "David G. Simmons" <sa...@mac.com>.
Hi Cris,
What sorts of compile times are you seeing? And for what apps?
Thanks,
dg
> On Jan 4, 2017, at 8:53 AM, Cris Frusina <cr...@frusina.com> wrote:
>
> Hi All,
>
> I'm using the Docker method to run myNewt.
>
> I have tried giving the VM more resources doesn't seem to do anything in terms of speed.
>
> Is there any way to speed up the compile and build time?
>
> Would the native method be substantially faster?
>
> Thanks,
> Cris
--
David G. Simmons
(919) 534-5099
Web <https://davidgs.com/> • Blog <https://davidgs.com/davidgs_blog> • Linkedin <http://linkedin.com/in/davidgsimmons> • Twitter <http://twitter.com/TechEvangelist1> • GitHub <http://github.com/davidgs>
/** Message digitally signed for security and authenticity.
* If you cannot read the PGP.sig attachment, please go to
* http://www.gnupg.com/ <http://www.gnupg.com/> Secure your email!!!
* Public key available at keyserver.pgp.com <http://keyserver.pgp.com/>
**/
♺ This email uses 100% recycled electrons. Don't blow it by printing!
There are only 2 hard things in computer science: Cache invalidation, naming things, and off-by-one errors.