Re: Cleaning hg binaries

[Jaroslav Tulach <ja...@oracle.com>]

On čtvrtek 14. září 2017 14:16:29 CEST Jan Lahoda wrote:
> On Thu, Sep 14, 2017 at 8:57 AM, Emilian Bold <em...@gmail.com>
> 
> wrote:
> > My feeling is that we could make a semi-automated script to find the
> > matching Maven dependency based on the MD5 we already have. The only
> > thing is to pick the authoritative dependency (ie. not a re-package to
> > another groupId).
> 
> I was thinking of that some time ago, attaching a script that I wrote for
> that. One of the issues is that not all of the libraries are on Maven.
> 
> Jarda, looking at the Maven support in external binaries, seems that if the
> binary is not in $HOME/.m2/repository, it is downloaded, but not cached
> anywhere, so next time it is needed, it is downloaded again. Is that the
> case? Can we cache the downloaded files somewhere, so we don't need to
> download them again?

Best to cache them in $HOME/.m2/repository, I assume. Sure that can be 
changed: download the file, and when done, atomically (well at least quickly) 
copy the file to appropriate place.

Major Péter wrote:
> assume converting the project to Maven/Gradle is out of question for a
> while.

We certainly need to find a way to upload Maven bits to Maven central 
repository[1], rather than maintaining our own repo.

However I believe that there are easier ways to achieve that than rewriting 
NetBeans build scripts.

> You may wish to consider using Ivy to resolve Maven dependencies. I

It is important to have incremental migration path - e.g. for now we need to 
be able to download both from Maven as well as hg.netbeans.org/binaries - that 
is easier to do with own DownloadBinaries task than Ivy, I guess.

-jt

[1] Right now there is 
http://bits.netbeans.org/nexus/content/groups/netbeans/
http://bits.netbeans.org/nexus/
and the binaries are needed by many projects out there - we wonder what Apache 
is going to do with that...


> > On Thu, Sep 14, 2017 at 8:43 AM, Sven Reimers <sv...@gmail.com>
> > 
> > wrote:
> > > +1 and +1 for the branch plan..
> > > 
> > > 
> > > Sven
> > > 
> > > On Thu, Sep 14, 2017 at 7:42 AM, Geertjan Wielenga <
> > > 
> > > geertjan.wielenga@googlemail.com> wrote:
> > >> Excellent. Should we do this on a branch, so that we can do one merge
> > >> of
> > >> all these changes at the same time into main?
> > >> 
> > >> Gj
> > >> 
> > >> On Thu, Sep 14, 2017 at 7:40 AM, Jaroslav Tulach <
> > >> jaroslav.tulach@oracle.com
> > >> 
> > >> > wrote:
> > >> > 
> > >> > Here is the plan to get rid of downloading binaries from
> > >> > hg.netbeans.org/binaries:
> > >> > 
> > >> > Let's download them from central Maven repository!
> > >> > 
> > >> > The support is already in the build harness and for example this
> > 
> > module
> > 
> > >> is
> > >> 
> > >> > already downloading the JAR from a Maven repo:
> > >> > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_apach
> > >> > e_incubator-2Dnetbeans_blob_&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5
> > >> > YTpkKY057SbK10&r=tnLz5-vaI8x9g_xBBlFGf_ldS0-TN1M3i9fMP1FPWgk&m=IUgEMs
> > >> > 5XaQXIgZJ7ylgOoN0HLTdUlxB-0w8ylq1aF9U&s=RjitTgNeZ9N_ZEevNSf1RyFSggKam
> > >> > mDyemwcZ3PCOwE&e= master/o.n.html.ko4j/external/binaries-list
> > >> > 
> > >> > We just need to go through all the binaries-list files and replace
> > >> > the
> > >> > current JAR references with the Maven syntax:
> > >> > 
> > >> > MD5Hash groupId:artifactId:version
> > >> > 
> > >> > That's the plan. Volunteers welcomed.
> > >> > -jt
> > >> > 
> > >> > 13. 9. 2017 v 21:10, Geertjan Wielenga <geertjan.wielenga@googlemail.
> > 
> > com
> > 
> > >> > > PS: These binaries are all listed in the SGA as being excluded from
> > 
> > the
> > 
> > >> > > donation and in each case, in the SGA, the related license is
> > 
> > mentioned
> > 
> > >> > too.
> > >> > 
> > >> > > On Wed, Sep 13, 2017 at 9:00 PM, Geertjan Wielenga <
> > >> > > 
> > >> > > geertjan.wielenga@googlemail.com> wrote:
> > >> > >>> On Wed, Sep 13, 2017 at 8:48 PM, Major Péter <
> > 
> > majorpetya@gmail.com>
> > 
> > >> > wrote:
> > >> > >>> I cloned the repo today and tried to run NetBeans (ant && ant
> > 
> > tryme)
> > 
> > >> on
> > >> 
> > >> > >>> OSX and I had various compilation related exceptions and NPEs
> > 
> > with a
> > 
> > >> > >>> Maven project. Almost every code completion request resulted in
> > >> > >>> an
> > >> > >>> exception popup..
> > >> > >>> Anyone had a similar experience with the donated code? Is this
> > >> 
> > >> NetBeans
> > >> 
> > >> > >>> version meant to be stable at this point?
> > >> > >> 
> > >> > >> More details please, step by step to reproduce this. I'm also on
> > 
> > OSX,
> > 
> > >> > >> happy to try to reproduce if you tell me specifically what,
> > >> > >> 
> > >> > >>> On a slightly different topic:
> > >> > >>> Ant downloaded various binary files from hg.netbeans.org/binaries
> > >> > 
> > >> > during
> > >> > 
> > >> > >>> the build. Are these also Apache licence friendly binaries, or
> > >> > >>> are
> > >> 
> > >> they
> > >> 
> > >> > >>> magic binaries that the build needs to bring in to actually get a
> > >> > >>> working NetBeans release. Are these binaries (well their source)
> > >> 
> > >> going
> > >> 
> > >> > >>> to be donated & built via Apache infrastracture in the long term?
> > >> > >> 
> > >> > >> No, they are not donated, they are needed by the build. We will be
> > >> > >> auditing them, will need to check whether we can use them under
> > 
> > Apache
> > 
> > >> > or
> > >> > 
> > >> > >> not, quite some work involved here.
> > >> > >> 
> > >> > >> Gj
> > > 
> > > --
> > > Sven Reimers
> > > 
> > > * Senior Expert Software Architect
> > > * Java Champion
> > > * NetBeans Dream Team Member:
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__dreamteam.netbeans.
> > > org&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=tnLz5-vaI8x
> > > 9g_xBBlFGf_ldS0-TN1M3i9fMP1FPWgk&m=IUgEMs5XaQXIgZJ7ylgOoN0HLTdUlxB-0w8yl
> > > q1aF9U&s=2BRS-A3SF3ErgHFKJa4UnB1cdMN68u9NYRzM1NJZCbk&e= * Community
> > > Leader  NetBeans:
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__community.java.net_
> > > netbeans&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=tnLz5-
> > > vaI8x9g_xBBlFGf_ldS0-TN1M3i9fMP1FPWgk&m=IUgEMs5XaQXIgZJ7ylgOoN0HLTdUlxB-
> > > 0w8ylq1aF9U&s=mqd8vxmQLgN1PEqvJmobg4ZszAcMToOQFA7ENlYZpBQ&e=> > 
> > >                               Desktop Java:
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__community.java.net_j
> > > avadesktop&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=tnLz
> > > 5-vaI8x9g_xBBlFGf_ldS0-TN1M3i9fMP1FPWgk&m=IUgEMs5XaQXIgZJ7ylgOoN0HLTdUlx
> > > B-0w8ylq1aF9U&s=3wQmTLMkUsGVIoMiZBATRbo9vuhsGVV3U3OKBh-lNyY&e= * JUG
> > > Leader JUG Bodensee:
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.jug-2Dbodensee.
> > > de&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=tnLz5-vaI8x9
> > > g_xBBlFGf_ldS0-TN1M3i9fMP1FPWgk&m=IUgEMs5XaQXIgZJ7ylgOoN0HLTdUlxB-0w8ylq
> > > 1aF9U&s=gmgMS57Q0YoQp6svOqnEiSk0lSVs-PMCvEWtu7VWoIk&e= * Duke's Choice
> > > Award Winner 2009
> > > 
> > > * XING:
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.xing.com_profi
> > > le_Sven-5FReimers8&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK1
> > > 0&r=tnLz5-vaI8x9g_xBBlFGf_ldS0-TN1M3i9fMP1FPWgk&m=IUgEMs5XaQXIgZJ7ylgOoN
> > > 0HLTdUlxB-0w8ylq1aF9U&s=Lm_cE84D6ogIrrX4N0XKTz04uIHR6KytYY1-LLZjFbo&e= *
> > > LinkedIn:
> > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www.linkedin.com_in
> > > _svenreimers&d=DwIFaQ&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=tn
> > > Lz5-vaI8x9g_xBBlFGf_ldS0-TN1M3i9fMP1FPWgk&m=IUgEMs5XaQXIgZJ7ylgOoN0HLTdU
> > > lxB-0w8ylq1aF9U&s=idxnmulTfUx29Y9uuh3y6WtW9MnBx2xxNYSnLEhz96g&e=