You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@apache.org by Antoine Toulme <an...@toulme.name> on 2019/04/17 17:23:36 UTC

Gradle publish from Jenkins slaves

Hello all,

I have managed to push to repository.apache.org a snapshot from my machine and want to automate this as part of CI.

It looks like Maven is leveraging the ~/.m2/settings.xml file which contains the credentials of the Jenkins user to push binaries to the snapshot repository. Is that correct?
If so, is there any way I can leverage that file? I would read the XML and extract the credentials, but I’m pretty sure you used a symmetric key to encrypt the LDAP password.

Are there other projects using Gradle to push binaries to nexus?

Cheers,

Antoine

Re: Gradle publish from Jenkins slaves

Posted by Gavin McDonald <ip...@gmail.com>.
Right, there is nothing to fix here. Any 'externally' managed nodes that
can have external users login to the nodes can not
and will not have any asf credentials placed on them. This includes all of
the 'Beam' nodes.

All the internally hosted and managed jenkins nodes all do not accept
logins from anyone and so we do deploy credentials
on those nodes.

And ftr and project that is not 'Beam' should not be using the beam label
anyway (same goes for Cassandra etc ...)

HTH

Gav...


On Mon, Apr 22, 2019 at 5:08 PM Kenneth Knowles <ke...@apache.org> wrote:

> And to be clear, I think there are two issues, and perhaps no bug nor "fix"
> per se. The Beam issue is https://issues.apache.org/jira/browse/BEAM-6919
> but
> then it sounds like having an easy way to identify which agents have the
> needed gradle.properties file is somewhat independent (but helpful for us).
>
> On Mon, Apr 22, 2019 at 9:02 AM Kenneth Knowles <ke...@apache.org> wrote:
>
> > It looks like your other thread has a part of the answer. There's no plan
> > to get the credentials onto external VMs as it used to, but the publish
> > command itself could run on ASF-managed instances.
> >
> > Kenn
> >
> > On Mon, Apr 22, 2019 at 8:37 AM Antoine Toulme <an...@toulme.name>
> > wrote:
> >
> >> Got it. Is there a ticket open to fix this?
> >>
> >> Alternatively, jenkins could store the credentials and inject them in
> the
> >> build.
> >>
> >> Where can I get more context?
> >>
> >> > On Apr 22, 2019, at 08:08, Kenneth Knowles <ke...@apache.org> wrote:
> >> >
> >> > Yes, this is because previously these boxes were managed by infra
> using
> >> > puppet. They published snapshots for a very long time succesfully.
> >> >
> >> > Infra has changed how they accept these workers, so now they are not
> >> > puppetized, hence do not have the private credentials. This is a very
> >> > recent change.
> >> >
> >> > Kenn
> >> >
> >> >> On Mon, Apr 22, 2019 at 12:14 AM Antoine Toulme <antoine@toulme.name
> >
> >> wrote:
> >> >>
> >> >> Hey folks,
> >> >>
> >> >> Actually your build fails just like mine:
> >> >>
> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console
> >> <
> >> >>
> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console
> >> >
> >> >>
> >> >> 10:05:31 > Failed to publish publication 'mavenJava' to repository
> >> 'maven'
> >> >> 10:05:31    > Could not write to resource '
> >> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> >> >> <
> >> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> >> >>> '.
> >> >> 10:05:31       > Could not PUT '
> >> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> >> >> <
> >> >>
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> >> >'.
> >> >> Received status code 401 from server: Unauthorized
> >> >>
> >> >> It looks like your last green build was Feb 29, on a slave that is
> >> >> disconnected now.
> >> >>
> >> >>
> >> >>> On Apr 21, 2019, at 10:46 PM, Antoine Toulme <an...@toulme.name>
> >> >> wrote:
> >> >>>
> >> >>> Thank you both. I’m stealing that code.
> >> >>>
> >> >>>> On Apr 21, 2019, at 8:00 PM, Kenneth Knowles <ke...@apache.org>
> >> wrote:
> >> >>>>
> >> >>>> This job uploads snapshots:
> >> >>>>
> >> >>
> >>
> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_Release_NightlySnapshot.groovy
> >> >>>>
> >> >>>> The place in Beam's Gradle build that sets up the credentials is:
> >> >>>>
> >> >>
> >>
> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L573
> >> >>>>
> >> >>>> Kenn
> >> >>>>
> >> >>>> On Sun, Apr 21, 2019 at 4:45 PM Antoine Toulme <
> antoine@toulme.name>
> >> >> wrote:
> >> >>>>
> >> >>>>> I’ll take any idea. I looked at the beam builds, nothing in there
> >> >> seems to
> >> >>>>> be uploading snapshots?
> >> >>>>>
> >> >>>>> It might be time to move to Maven to fit the mold.
> >> >>>>>
> >> >>>>>> On Apr 18, 2019, at 3:06 PM, Kenneth Knowles <ke...@apache.org>
> >> wrote:
> >> >>>>>>
> >> >>>>>> Beam pushed daily snapshots to nexus until the change where our
> >> >> (donated)
> >> >>>>>> Jenkins executors were moved outside of Infra's purview. Beam
> >> migrated
> >> >>>>> from
> >> >>>>>> Maven to Gradle while maintaining the functionality. I don't know
> >> if
> >> >>>>>> anything special had to be done. Added Yifan who I think knows
> the
> >> >> most
> >> >>>>>> about this.
> >> >>>>>>
> >> >>>>>> Kenn
> >> >>>>>>
> >> >>>>>> On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <
> >> antoine@toulme.name>
> >> >>>>> wrote:
> >> >>>>>>
> >> >>>>>>> Hello all,
> >> >>>>>>>
> >> >>>>>>> I have managed to push to repository.apache.org a snapshot from
> >> my
> >> >>>>>>> machine and want to automate this as part of CI.
> >> >>>>>>>
> >> >>>>>>> It looks like Maven is leveraging the ~/.m2/settings.xml file
> >> which
> >> >>>>>>> contains the credentials of the Jenkins user to push binaries to
> >> the
> >> >>>>>>> snapshot repository. Is that correct?
> >> >>>>>>> If so, is there any way I can leverage that file? I would read
> the
> >> >> XML
> >> >>>>> and
> >> >>>>>>> extract the credentials, but I’m pretty sure you used a
> symmetric
> >> >> key to
> >> >>>>>>> encrypt the LDAP password.
> >> >>>>>>>
> >> >>>>>>> Are there other projects using Gradle to push binaries to nexus?
> >> >>>>>>>
> >> >>>>>>> Cheers,
> >> >>>>>>>
> >> >>>>>>> Antoine
> >> >>>>>
> >> >>>>>
> >> >>>
> >> >>
> >> >>
> >>
> >>
>


-- 
Gav...

Re: Gradle publish from Jenkins slaves

Posted by Kenneth Knowles <ke...@apache.org>.
And to be clear, I think there are two issues, and perhaps no bug nor "fix"
per se. The Beam issue is https://issues.apache.org/jira/browse/BEAM-6919 but
then it sounds like having an easy way to identify which agents have the
needed gradle.properties file is somewhat independent (but helpful for us).

On Mon, Apr 22, 2019 at 9:02 AM Kenneth Knowles <ke...@apache.org> wrote:

> It looks like your other thread has a part of the answer. There's no plan
> to get the credentials onto external VMs as it used to, but the publish
> command itself could run on ASF-managed instances.
>
> Kenn
>
> On Mon, Apr 22, 2019 at 8:37 AM Antoine Toulme <an...@toulme.name>
> wrote:
>
>> Got it. Is there a ticket open to fix this?
>>
>> Alternatively, jenkins could store the credentials and inject them in the
>> build.
>>
>> Where can I get more context?
>>
>> > On Apr 22, 2019, at 08:08, Kenneth Knowles <ke...@apache.org> wrote:
>> >
>> > Yes, this is because previously these boxes were managed by infra using
>> > puppet. They published snapshots for a very long time succesfully.
>> >
>> > Infra has changed how they accept these workers, so now they are not
>> > puppetized, hence do not have the private credentials. This is a very
>> > recent change.
>> >
>> > Kenn
>> >
>> >> On Mon, Apr 22, 2019 at 12:14 AM Antoine Toulme <an...@toulme.name>
>> wrote:
>> >>
>> >> Hey folks,
>> >>
>> >> Actually your build fails just like mine:
>> >> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console
>> <
>> >> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console
>> >
>> >>
>> >> 10:05:31 > Failed to publish publication 'mavenJava' to repository
>> 'maven'
>> >> 10:05:31    > Could not write to resource '
>> >>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
>> >> <
>> >>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
>> >>> '.
>> >> 10:05:31       > Could not PUT '
>> >>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
>> >> <
>> >>
>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
>> >'.
>> >> Received status code 401 from server: Unauthorized
>> >>
>> >> It looks like your last green build was Feb 29, on a slave that is
>> >> disconnected now.
>> >>
>> >>
>> >>> On Apr 21, 2019, at 10:46 PM, Antoine Toulme <an...@toulme.name>
>> >> wrote:
>> >>>
>> >>> Thank you both. I’m stealing that code.
>> >>>
>> >>>> On Apr 21, 2019, at 8:00 PM, Kenneth Knowles <ke...@apache.org>
>> wrote:
>> >>>>
>> >>>> This job uploads snapshots:
>> >>>>
>> >>
>> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_Release_NightlySnapshot.groovy
>> >>>>
>> >>>> The place in Beam's Gradle build that sets up the credentials is:
>> >>>>
>> >>
>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L573
>> >>>>
>> >>>> Kenn
>> >>>>
>> >>>> On Sun, Apr 21, 2019 at 4:45 PM Antoine Toulme <an...@toulme.name>
>> >> wrote:
>> >>>>
>> >>>>> I’ll take any idea. I looked at the beam builds, nothing in there
>> >> seems to
>> >>>>> be uploading snapshots?
>> >>>>>
>> >>>>> It might be time to move to Maven to fit the mold.
>> >>>>>
>> >>>>>> On Apr 18, 2019, at 3:06 PM, Kenneth Knowles <ke...@apache.org>
>> wrote:
>> >>>>>>
>> >>>>>> Beam pushed daily snapshots to nexus until the change where our
>> >> (donated)
>> >>>>>> Jenkins executors were moved outside of Infra's purview. Beam
>> migrated
>> >>>>> from
>> >>>>>> Maven to Gradle while maintaining the functionality. I don't know
>> if
>> >>>>>> anything special had to be done. Added Yifan who I think knows the
>> >> most
>> >>>>>> about this.
>> >>>>>>
>> >>>>>> Kenn
>> >>>>>>
>> >>>>>> On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <
>> antoine@toulme.name>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>>> Hello all,
>> >>>>>>>
>> >>>>>>> I have managed to push to repository.apache.org a snapshot from
>> my
>> >>>>>>> machine and want to automate this as part of CI.
>> >>>>>>>
>> >>>>>>> It looks like Maven is leveraging the ~/.m2/settings.xml file
>> which
>> >>>>>>> contains the credentials of the Jenkins user to push binaries to
>> the
>> >>>>>>> snapshot repository. Is that correct?
>> >>>>>>> If so, is there any way I can leverage that file? I would read the
>> >> XML
>> >>>>> and
>> >>>>>>> extract the credentials, but I’m pretty sure you used a symmetric
>> >> key to
>> >>>>>>> encrypt the LDAP password.
>> >>>>>>>
>> >>>>>>> Are there other projects using Gradle to push binaries to nexus?
>> >>>>>>>
>> >>>>>>> Cheers,
>> >>>>>>>
>> >>>>>>> Antoine
>> >>>>>
>> >>>>>
>> >>>
>> >>
>> >>
>>
>>

Re: Gradle publish from Jenkins slaves

Posted by Kenneth Knowles <ke...@apache.org>.
It looks like your other thread has a part of the answer. There's no plan
to get the credentials onto external VMs as it used to, but the publish
command itself could run on ASF-managed instances.

Kenn

On Mon, Apr 22, 2019 at 8:37 AM Antoine Toulme <an...@toulme.name> wrote:

> Got it. Is there a ticket open to fix this?
>
> Alternatively, jenkins could store the credentials and inject them in the
> build.
>
> Where can I get more context?
>
> > On Apr 22, 2019, at 08:08, Kenneth Knowles <ke...@apache.org> wrote:
> >
> > Yes, this is because previously these boxes were managed by infra using
> > puppet. They published snapshots for a very long time succesfully.
> >
> > Infra has changed how they accept these workers, so now they are not
> > puppetized, hence do not have the private credentials. This is a very
> > recent change.
> >
> > Kenn
> >
> >> On Mon, Apr 22, 2019 at 12:14 AM Antoine Toulme <an...@toulme.name>
> wrote:
> >>
> >> Hey folks,
> >>
> >> Actually your build fails just like mine:
> >> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console
> <
> >> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console>
> >>
> >> 10:05:31 > Failed to publish publication 'mavenJava' to repository
> 'maven'
> >> 10:05:31    > Could not write to resource '
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> >> <
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> >>> '.
> >> 10:05:31       > Could not PUT '
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> >> <
> >>
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> >'.
> >> Received status code 401 from server: Unauthorized
> >>
> >> It looks like your last green build was Feb 29, on a slave that is
> >> disconnected now.
> >>
> >>
> >>> On Apr 21, 2019, at 10:46 PM, Antoine Toulme <an...@toulme.name>
> >> wrote:
> >>>
> >>> Thank you both. I’m stealing that code.
> >>>
> >>>> On Apr 21, 2019, at 8:00 PM, Kenneth Knowles <ke...@apache.org> wrote:
> >>>>
> >>>> This job uploads snapshots:
> >>>>
> >>
> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_Release_NightlySnapshot.groovy
> >>>>
> >>>> The place in Beam's Gradle build that sets up the credentials is:
> >>>>
> >>
> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L573
> >>>>
> >>>> Kenn
> >>>>
> >>>> On Sun, Apr 21, 2019 at 4:45 PM Antoine Toulme <an...@toulme.name>
> >> wrote:
> >>>>
> >>>>> I’ll take any idea. I looked at the beam builds, nothing in there
> >> seems to
> >>>>> be uploading snapshots?
> >>>>>
> >>>>> It might be time to move to Maven to fit the mold.
> >>>>>
> >>>>>> On Apr 18, 2019, at 3:06 PM, Kenneth Knowles <ke...@apache.org>
> wrote:
> >>>>>>
> >>>>>> Beam pushed daily snapshots to nexus until the change where our
> >> (donated)
> >>>>>> Jenkins executors were moved outside of Infra's purview. Beam
> migrated
> >>>>> from
> >>>>>> Maven to Gradle while maintaining the functionality. I don't know if
> >>>>>> anything special had to be done. Added Yifan who I think knows the
> >> most
> >>>>>> about this.
> >>>>>>
> >>>>>> Kenn
> >>>>>>
> >>>>>> On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <
> antoine@toulme.name>
> >>>>> wrote:
> >>>>>>
> >>>>>>> Hello all,
> >>>>>>>
> >>>>>>> I have managed to push to repository.apache.org a snapshot from my
> >>>>>>> machine and want to automate this as part of CI.
> >>>>>>>
> >>>>>>> It looks like Maven is leveraging the ~/.m2/settings.xml file which
> >>>>>>> contains the credentials of the Jenkins user to push binaries to
> the
> >>>>>>> snapshot repository. Is that correct?
> >>>>>>> If so, is there any way I can leverage that file? I would read the
> >> XML
> >>>>> and
> >>>>>>> extract the credentials, but I’m pretty sure you used a symmetric
> >> key to
> >>>>>>> encrypt the LDAP password.
> >>>>>>>
> >>>>>>> Are there other projects using Gradle to push binaries to nexus?
> >>>>>>>
> >>>>>>> Cheers,
> >>>>>>>
> >>>>>>> Antoine
> >>>>>
> >>>>>
> >>>
> >>
> >>
>
>

Re: Gradle publish from Jenkins slaves

Posted by Antoine Toulme <an...@toulme.name>.
Got it. Is there a ticket open to fix this?

Alternatively, jenkins could store the credentials and inject them in the build.

Where can I get more context?

> On Apr 22, 2019, at 08:08, Kenneth Knowles <ke...@apache.org> wrote:
> 
> Yes, this is because previously these boxes were managed by infra using
> puppet. They published snapshots for a very long time succesfully.
> 
> Infra has changed how they accept these workers, so now they are not
> puppetized, hence do not have the private credentials. This is a very
> recent change.
> 
> Kenn
> 
>> On Mon, Apr 22, 2019 at 12:14 AM Antoine Toulme <an...@toulme.name> wrote:
>> 
>> Hey folks,
>> 
>> Actually your build fails just like mine:
>> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console <
>> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console>
>> 
>> 10:05:31 > Failed to publish publication 'mavenJava' to repository 'maven'
>> 10:05:31    > Could not write to resource '
>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
>> <
>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
>>> '.
>> 10:05:31       > Could not PUT '
>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
>> <
>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar>'.
>> Received status code 401 from server: Unauthorized
>> 
>> It looks like your last green build was Feb 29, on a slave that is
>> disconnected now.
>> 
>> 
>>> On Apr 21, 2019, at 10:46 PM, Antoine Toulme <an...@toulme.name>
>> wrote:
>>> 
>>> Thank you both. I’m stealing that code.
>>> 
>>>> On Apr 21, 2019, at 8:00 PM, Kenneth Knowles <ke...@apache.org> wrote:
>>>> 
>>>> This job uploads snapshots:
>>>> 
>> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_Release_NightlySnapshot.groovy
>>>> 
>>>> The place in Beam's Gradle build that sets up the credentials is:
>>>> 
>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L573
>>>> 
>>>> Kenn
>>>> 
>>>> On Sun, Apr 21, 2019 at 4:45 PM Antoine Toulme <an...@toulme.name>
>> wrote:
>>>> 
>>>>> I’ll take any idea. I looked at the beam builds, nothing in there
>> seems to
>>>>> be uploading snapshots?
>>>>> 
>>>>> It might be time to move to Maven to fit the mold.
>>>>> 
>>>>>> On Apr 18, 2019, at 3:06 PM, Kenneth Knowles <ke...@apache.org> wrote:
>>>>>> 
>>>>>> Beam pushed daily snapshots to nexus until the change where our
>> (donated)
>>>>>> Jenkins executors were moved outside of Infra's purview. Beam migrated
>>>>> from
>>>>>> Maven to Gradle while maintaining the functionality. I don't know if
>>>>>> anything special had to be done. Added Yifan who I think knows the
>> most
>>>>>> about this.
>>>>>> 
>>>>>> Kenn
>>>>>> 
>>>>>> On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <an...@toulme.name>
>>>>> wrote:
>>>>>> 
>>>>>>> Hello all,
>>>>>>> 
>>>>>>> I have managed to push to repository.apache.org a snapshot from my
>>>>>>> machine and want to automate this as part of CI.
>>>>>>> 
>>>>>>> It looks like Maven is leveraging the ~/.m2/settings.xml file which
>>>>>>> contains the credentials of the Jenkins user to push binaries to the
>>>>>>> snapshot repository. Is that correct?
>>>>>>> If so, is there any way I can leverage that file? I would read the
>> XML
>>>>> and
>>>>>>> extract the credentials, but I’m pretty sure you used a symmetric
>> key to
>>>>>>> encrypt the LDAP password.
>>>>>>> 
>>>>>>> Are there other projects using Gradle to push binaries to nexus?
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> 
>>>>>>> Antoine
>>>>> 
>>>>> 
>>> 
>> 
>>

Re: Gradle publish from Jenkins slaves

Posted by Kenneth Knowles <ke...@apache.org>.
Yes, this is because previously these boxes were managed by infra using
puppet. They published snapshots for a very long time succesfully.

Infra has changed how they accept these workers, so now they are not
puppetized, hence do not have the private credentials. This is a very
recent change.

Kenn

On Mon, Apr 22, 2019 at 12:14 AM Antoine Toulme <an...@toulme.name> wrote:

> Hey folks,
>
> Actually your build fails just like mine:
> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console <
> https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console>
>
> 10:05:31 > Failed to publish publication 'mavenJava' to repository 'maven'
> 10:05:31    > Could not write to resource '
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> >'.
> 10:05:31       > Could not PUT '
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar
> <
> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar>'.
> Received status code 401 from server: Unauthorized
>
> It looks like your last green build was Feb 29, on a slave that is
> disconnected now.
>
>
> > On Apr 21, 2019, at 10:46 PM, Antoine Toulme <an...@toulme.name>
> wrote:
> >
> > Thank you both. I’m stealing that code.
> >
> >> On Apr 21, 2019, at 8:00 PM, Kenneth Knowles <ke...@apache.org> wrote:
> >>
> >> This job uploads snapshots:
> >>
> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_Release_NightlySnapshot.groovy
> >>
> >> The place in Beam's Gradle build that sets up the credentials is:
> >>
> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L573
> >>
> >> Kenn
> >>
> >> On Sun, Apr 21, 2019 at 4:45 PM Antoine Toulme <an...@toulme.name>
> wrote:
> >>
> >>> I’ll take any idea. I looked at the beam builds, nothing in there
> seems to
> >>> be uploading snapshots?
> >>>
> >>> It might be time to move to Maven to fit the mold.
> >>>
> >>>> On Apr 18, 2019, at 3:06 PM, Kenneth Knowles <ke...@apache.org> wrote:
> >>>>
> >>>> Beam pushed daily snapshots to nexus until the change where our
> (donated)
> >>>> Jenkins executors were moved outside of Infra's purview. Beam migrated
> >>> from
> >>>> Maven to Gradle while maintaining the functionality. I don't know if
> >>>> anything special had to be done. Added Yifan who I think knows the
> most
> >>>> about this.
> >>>>
> >>>> Kenn
> >>>>
> >>>> On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <an...@toulme.name>
> >>> wrote:
> >>>>
> >>>>> Hello all,
> >>>>>
> >>>>> I have managed to push to repository.apache.org a snapshot from my
> >>>>> machine and want to automate this as part of CI.
> >>>>>
> >>>>> It looks like Maven is leveraging the ~/.m2/settings.xml file which
> >>>>> contains the credentials of the Jenkins user to push binaries to the
> >>>>> snapshot repository. Is that correct?
> >>>>> If so, is there any way I can leverage that file? I would read the
> XML
> >>> and
> >>>>> extract the credentials, but I’m pretty sure you used a symmetric
> key to
> >>>>> encrypt the LDAP password.
> >>>>>
> >>>>> Are there other projects using Gradle to push binaries to nexus?
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>> Antoine
> >>>
> >>>
> >
>
>

Re: Gradle publish from Jenkins slaves

Posted by Antoine Toulme <an...@toulme.name>.
Hey folks,

Actually your build fails just like mine:
https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console <https://builds.apache.org/job/beam_Release_NightlySnapshot/415/console>

10:05:31 > Failed to publish publication 'mavenJava' to repository 'maven'
10:05:31    > Could not write to resource 'https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar <https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar>'.
10:05:31       > Could not PUT 'https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar <https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-vendor-sdks-java-extensions-protobuf/2.13.0-SNAPSHOT/beam-vendor-sdks-java-extensions-protobuf-2.13.0-20190421.100528-1.jar>'. Received status code 401 from server: Unauthorized

It looks like your last green build was Feb 29, on a slave that is disconnected now.


> On Apr 21, 2019, at 10:46 PM, Antoine Toulme <an...@toulme.name> wrote:
> 
> Thank you both. I’m stealing that code.
> 
>> On Apr 21, 2019, at 8:00 PM, Kenneth Knowles <ke...@apache.org> wrote:
>> 
>> This job uploads snapshots:
>> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_Release_NightlySnapshot.groovy
>> 
>> The place in Beam's Gradle build that sets up the credentials is:
>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L573
>> 
>> Kenn
>> 
>> On Sun, Apr 21, 2019 at 4:45 PM Antoine Toulme <an...@toulme.name> wrote:
>> 
>>> I’ll take any idea. I looked at the beam builds, nothing in there seems to
>>> be uploading snapshots?
>>> 
>>> It might be time to move to Maven to fit the mold.
>>> 
>>>> On Apr 18, 2019, at 3:06 PM, Kenneth Knowles <ke...@apache.org> wrote:
>>>> 
>>>> Beam pushed daily snapshots to nexus until the change where our (donated)
>>>> Jenkins executors were moved outside of Infra's purview. Beam migrated
>>> from
>>>> Maven to Gradle while maintaining the functionality. I don't know if
>>>> anything special had to be done. Added Yifan who I think knows the most
>>>> about this.
>>>> 
>>>> Kenn
>>>> 
>>>> On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <an...@toulme.name>
>>> wrote:
>>>> 
>>>>> Hello all,
>>>>> 
>>>>> I have managed to push to repository.apache.org a snapshot from my
>>>>> machine and want to automate this as part of CI.
>>>>> 
>>>>> It looks like Maven is leveraging the ~/.m2/settings.xml file which
>>>>> contains the credentials of the Jenkins user to push binaries to the
>>>>> snapshot repository. Is that correct?
>>>>> If so, is there any way I can leverage that file? I would read the XML
>>> and
>>>>> extract the credentials, but I’m pretty sure you used a symmetric key to
>>>>> encrypt the LDAP password.
>>>>> 
>>>>> Are there other projects using Gradle to push binaries to nexus?
>>>>> 
>>>>> Cheers,
>>>>> 
>>>>> Antoine
>>> 
>>> 
>

Re: Gradle publish from Jenkins slaves

Posted by Antoine Toulme <an...@toulme.name>.
Thank you both. I’m stealing that code.

> On Apr 21, 2019, at 8:00 PM, Kenneth Knowles <ke...@apache.org> wrote:
> 
> This job uploads snapshots:
> https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_Release_NightlySnapshot.groovy
> 
> The place in Beam's Gradle build that sets up the credentials is:
> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L573
> 
> Kenn
> 
> On Sun, Apr 21, 2019 at 4:45 PM Antoine Toulme <an...@toulme.name> wrote:
> 
>> I’ll take any idea. I looked at the beam builds, nothing in there seems to
>> be uploading snapshots?
>> 
>> It might be time to move to Maven to fit the mold.
>> 
>>> On Apr 18, 2019, at 3:06 PM, Kenneth Knowles <ke...@apache.org> wrote:
>>> 
>>> Beam pushed daily snapshots to nexus until the change where our (donated)
>>> Jenkins executors were moved outside of Infra's purview. Beam migrated
>> from
>>> Maven to Gradle while maintaining the functionality. I don't know if
>>> anything special had to be done. Added Yifan who I think knows the most
>>> about this.
>>> 
>>> Kenn
>>> 
>>> On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <an...@toulme.name>
>> wrote:
>>> 
>>>> Hello all,
>>>> 
>>>> I have managed to push to repository.apache.org a snapshot from my
>>>> machine and want to automate this as part of CI.
>>>> 
>>>> It looks like Maven is leveraging the ~/.m2/settings.xml file which
>>>> contains the credentials of the Jenkins user to push binaries to the
>>>> snapshot repository. Is that correct?
>>>> If so, is there any way I can leverage that file? I would read the XML
>> and
>>>> extract the credentials, but I’m pretty sure you used a symmetric key to
>>>> encrypt the LDAP password.
>>>> 
>>>> Are there other projects using Gradle to push binaries to nexus?
>>>> 
>>>> Cheers,
>>>> 
>>>> Antoine
>> 
>>

Re: Gradle publish from Jenkins slaves

Posted by Kenneth Knowles <ke...@apache.org>.
This job uploads snapshots:
https://github.com/apache/beam/blob/master/.test-infra/jenkins/job_Release_NightlySnapshot.groovy

The place in Beam's Gradle build that sets up the credentials is:
https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L573

Kenn

On Sun, Apr 21, 2019 at 4:45 PM Antoine Toulme <an...@toulme.name> wrote:

> I’ll take any idea. I looked at the beam builds, nothing in there seems to
> be uploading snapshots?
>
> It might be time to move to Maven to fit the mold.
>
> > On Apr 18, 2019, at 3:06 PM, Kenneth Knowles <ke...@apache.org> wrote:
> >
> > Beam pushed daily snapshots to nexus until the change where our (donated)
> > Jenkins executors were moved outside of Infra's purview. Beam migrated
> from
> > Maven to Gradle while maintaining the functionality. I don't know if
> > anything special had to be done. Added Yifan who I think knows the most
> > about this.
> >
> > Kenn
> >
> > On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <an...@toulme.name>
> wrote:
> >
> >> Hello all,
> >>
> >> I have managed to push to repository.apache.org a snapshot from my
> >> machine and want to automate this as part of CI.
> >>
> >> It looks like Maven is leveraging the ~/.m2/settings.xml file which
> >> contains the credentials of the Jenkins user to push binaries to the
> >> snapshot repository. Is that correct?
> >> If so, is there any way I can leverage that file? I would read the XML
> and
> >> extract the credentials, but I’m pretty sure you used a symmetric key to
> >> encrypt the LDAP password.
> >>
> >> Are there other projects using Gradle to push binaries to nexus?
> >>
> >> Cheers,
> >>
> >> Antoine
>
>

Re: Gradle publish from Jenkins slaves

Posted by Antoine Toulme <an...@toulme.name>.
I’ll take any idea. I looked at the beam builds, nothing in there seems to be uploading snapshots?

It might be time to move to Maven to fit the mold.

> On Apr 18, 2019, at 3:06 PM, Kenneth Knowles <ke...@apache.org> wrote:
> 
> Beam pushed daily snapshots to nexus until the change where our (donated)
> Jenkins executors were moved outside of Infra's purview. Beam migrated from
> Maven to Gradle while maintaining the functionality. I don't know if
> anything special had to be done. Added Yifan who I think knows the most
> about this.
> 
> Kenn
> 
> On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <an...@toulme.name> wrote:
> 
>> Hello all,
>> 
>> I have managed to push to repository.apache.org a snapshot from my
>> machine and want to automate this as part of CI.
>> 
>> It looks like Maven is leveraging the ~/.m2/settings.xml file which
>> contains the credentials of the Jenkins user to push binaries to the
>> snapshot repository. Is that correct?
>> If so, is there any way I can leverage that file? I would read the XML and
>> extract the credentials, but I’m pretty sure you used a symmetric key to
>> encrypt the LDAP password.
>> 
>> Are there other projects using Gradle to push binaries to nexus?
>> 
>> Cheers,
>> 
>> Antoine

Re: Gradle publish from Jenkins slaves

Posted by Kenneth Knowles <ke...@apache.org>.
Beam pushed daily snapshots to nexus until the change where our (donated)
Jenkins executors were moved outside of Infra's purview. Beam migrated from
Maven to Gradle while maintaining the functionality. I don't know if
anything special had to be done. Added Yifan who I think knows the most
about this.

Kenn

On Wed, Apr 17, 2019 at 10:23 AM Antoine Toulme <an...@toulme.name> wrote:

> Hello all,
>
> I have managed to push to repository.apache.org a snapshot from my
> machine and want to automate this as part of CI.
>
> It looks like Maven is leveraging the ~/.m2/settings.xml file which
> contains the credentials of the Jenkins user to push binaries to the
> snapshot repository. Is that correct?
> If so, is there any way I can leverage that file? I would read the XML and
> extract the credentials, but I’m pretty sure you used a symmetric key to
> encrypt the LDAP password.
>
> Are there other projects using Gradle to push binaries to nexus?
>
> Cheers,
>
> Antoine