YETUS-15 has been committed

[Allen Wittenauer <aw...@apache.org>]

[BCC: builds@apache.org since this may impact some folks there...]

Apologies for the wide distribution...  

YETUS-15 [https://issues.apache.org/jira/browse/YETUS-15] has been committed.  This patch switches Apache Yetus over to a Maven-based build system.  This has a few different impacts:

	* Any existing jobs that are tied specifically to master will almost certainly break.  I'm unsure if any of these still exist, with one exception ...

	* precommit-admin (the jobs that submits JIRAs to the various precommit-* jobs) has already been fixed to use the new layout.  Patch submission via that method should continue uninterrupted.

	* I'm currently working on getting the Yetus-specific jobs running again.  There are quite a few and testing the testing framework is always tricky. :)

	* Any patches sitting in the YETUS patch tree will obviously need to be rebased.

	* There are likely some rough spots in the new build and the new Maven plug-in for providing Yetus functionality.  Be sure to play with them, file JIRAs, and even better if patches come with it. :)

Thanks!


=== Current release note for YETUS-15 ===

Apache Yetus has been converted to use Apache Maven as a build tool. As a result, many changes have taken place that directly impacts the project. 

* Source directories have been re-arranged and re-named: 
  * All bash code is now in (feature)/src/main/shell 
  * All python code is now in (feature)/src/main/python 
* audience-annotations is mostly unchanged. 
* releasedocmaker and shelldocs are now available as Jython-built jars. 
* Introduction of the yetus-minimaven-plugin and yetus-maven-plugins. The yetus-minimaven-plugin is used to build Apache Yetus. yetus-maven-plugin is an end-user artifact that gives access some Apache Yetus features to Apache Maven and compatible build systems without needing any external help (e.g., yetus-wrapper) 
* Middleman is still used for creating the static website, however, it is now tied into the 'mvn site' command. 'mvn install' MUST be executed before running 'mvn site' as website generation depends upon the yetus-minimaven-plugin. 
* The content of yetus-project is now in the root of the source tree. 
* The new yetus-dist module handles the creation of a complete distribution. The artifacts are now in the yetus-dist/target directory. The artifact contents are largely unchanged. New yetus-assemblies module and various Apache Maven configuration files have been added to create distribution parity. 
* The website is also available as a tar.gz tarball in the yetus-dist artifact area. 
* The jdiff module is now always built. 
* Version handling has been modified in several different locations and the executables themselves. 

Also, other changes introduced: 

* start-build-env.sh has been added to create a Docker-ized development environment. In particular, this imports the .ssh and .gnupg directories and has all pre-requisites for building Apache Yetus and making releases. 
* A Dockerfile in root has been added for hub.docker.com and CI-system integration. 
* The old Dockerfile (previously located at precommit/test-patch-docker and now located at precommit/src/main/shell/test-patch-docker) has been changed to be able also to run releasedocmaker. 
* Some ruby dependencies for the website have been for security reasons. 
* JDK8 is now the minimum version of Java used to build the Apache Yetus Java components. 
* precommit's shellcheck.sh now recognizes src/main/shell as containing shell code to check. 
* releasedocmaker and shelldocs now explicitly call for python2

Re: YETUS-15 has been committed

[Allen Wittenauer <aw...@effectivemachines.com.INVALID>]

> On Nov 13, 2018, at 4:20 AM, Sean Busbey <bu...@apache.org> wrote:
> 
> Awesome! Do we have any kind of timeline to a release that includes this?
> I'd like to get one going sooner rather than later so that downstream folks
> can update with fewer new things that might be the cause of any
> frustrations, since project restructuring is high risk for causing problems.


	Not sure. Someone else will likely need to drive it. My Yetus time right now is majorly dedicated to:

	1: making it easier to use, especially for Not Jenkins.  This was a bunch of changes that I’ve pretty much got sitting in my fork now.  The robots rewrite makes Travis and Jenkins and lots of other systems now way smarter to the point that in some cases the # of command line flags is cut down to less than half (!) and some personalities aren’t particularly needed or useful anymore (!).

	2: getting it functional for my current work environment.  Right now, Yetus trunk isn’t anywhere close to working. The majority of the fixes I’ve been writing lately have been in support of that.  (Up next, a significant smack to docker.sh—core parts of which haven’t been touched in years.)

	On the positive side, after #2 is finished, I know I can say with confidence that (at least a form of) Yetus really is being used outside of the ASF and for non-Java projects. :)

	On the negative side though, while I’ve been trying to contribute the patches to the ASF, I need to work at a pace faster than the reviews are coming in. Thus the fork, where every day the code diverges just that much more. :(

Re: YETUS-15 has been committed

[Sean Busbey <bu...@apache.org>]

Awesome! Do we have any kind of timeline to a release that includes this?
I'd like to get one going sooner rather than later so that downstream folks
can update with fewer new things that might be the cause of any
frustrations, since project restructuring is high risk for causing problems.

On Sun, Nov 11, 2018, 16:58 Allen Wittenauer <aw@apache.org wrote:

> [BCC: builds@apache.org since this may impact some folks there...]
>
> Apologies for the wide distribution...
>
> YETUS-15 [https://issues.apache.org/jira/browse/YETUS-15] has been
> committed.  This patch switches Apache Yetus over to a Maven-based build
> system.  This has a few different impacts:
>
>         * Any existing jobs that are tied specifically to master will
> almost certainly break.  I'm unsure if any of these still exist, with one
> exception ...
>
>         * precommit-admin (the jobs that submits JIRAs to the various
> precommit-* jobs) has already been fixed to use the new layout.  Patch
> submission via that method should continue uninterrupted.
>
>         * I'm currently working on getting the Yetus-specific jobs running
> again.  There are quite a few and testing the testing framework is always
> tricky. :)
>
>         * Any patches sitting in the YETUS patch tree will obviously need
> to be rebased.
>
>         * There are likely some rough spots in the new build and the new
> Maven plug-in for providing Yetus functionality.  Be sure to play with
> them, file JIRAs, and even better if patches come with it. :)
>
> Thanks!
>
>
> === Current release note for YETUS-15 ===
>
> Apache Yetus has been converted to use Apache Maven as a build tool. As a
> result, many changes have taken place that directly impacts the project.
>
> * Source directories have been re-arranged and re-named:
>   * All bash code is now in (feature)/src/main/shell
>   * All python code is now in (feature)/src/main/python
> * audience-annotations is mostly unchanged.
> * releasedocmaker and shelldocs are now available as Jython-built jars.
> * Introduction of the yetus-minimaven-plugin and yetus-maven-plugins. The
> yetus-minimaven-plugin is used to build Apache Yetus. yetus-maven-plugin is
> an end-user artifact that gives access some Apache Yetus features to Apache
> Maven and compatible build systems without needing any external help (e.g.,
> yetus-wrapper)
> * Middleman is still used for creating the static website, however, it is
> now tied into the 'mvn site' command. 'mvn install' MUST be executed before
> running 'mvn site' as website generation depends upon the
> yetus-minimaven-plugin.
> * The content of yetus-project is now in the root of the source tree.
> * The new yetus-dist module handles the creation of a complete
> distribution. The artifacts are now in the yetus-dist/target directory. The
> artifact contents are largely unchanged. New yetus-assemblies module and
> various Apache Maven configuration files have been added to create
> distribution parity.
> * The website is also available as a tar.gz tarball in the yetus-dist
> artifact area.
> * The jdiff module is now always built.
> * Version handling has been modified in several different locations and
> the executables themselves.
>
> Also, other changes introduced:
>
> * start-build-env.sh has been added to create a Docker-ized development
> environment. In particular, this imports the .ssh and .gnupg directories
> and has all pre-requisites for building Apache Yetus and making releases.
> * A Dockerfile in root has been added for hub.docker.com and CI-system
> integration.
> * The old Dockerfile (previously located at precommit/test-patch-docker
> and now located at precommit/src/main/shell/test-patch-docker) has been
> changed to be able also to run releasedocmaker.
> * Some ruby dependencies for the website have been for security reasons.
> * JDK8 is now the minimum version of Java used to build the Apache Yetus
> Java components.
> * precommit's shellcheck.sh now recognizes src/main/shell as containing
> shell code to check.
> * releasedocmaker and shelldocs now explicitly call for python2
>
>