Yetus for Apache Hive

[Peter Vary <pv...@cloudera.com>]

Hi devs,

I have not seen an users list for yetus, so I write here.

At Apache Hive we are thinking of implementing precommit validation to help the reviews and ensure better quality. I have trying to implement Yetus for Hive - see the attached personality.

It went fairly straightforward most of the time, but faced some problems which I am not sure how best to solve:
Javac plugin: When I create a personality with only maven, and javac plugin enabled, then the maven install is run for pre, and post patch, but the patch generated java compile warnings are not highlighted. Am I wrong when expecting the javac plugin to calculate the results using the patch-mvninstall-<module>.txt and  branch-mvninstall-<module>.txt files? Where did I make the mistake in the configuration, why the result files are not compared?
Test4test plugin: We have different filename patterns for tests, so to use Yetus we have to either add a new plugin or has to update the existing plugin to have the pattern as a configurable parameter. I am willing to create a patch for it if the Yetus community thinks making test4test plugin configurable is the way to go. 
Findbugs plugin: Findbugs plugin expects the project structure to be the standard maven one (/src/main/java, /src/test/java). In Hive some of the modules have a different file layout. Like above either a new plugin is needed, or a parameter should be introduced to enable changes in checking.

I am attaching the hive-personality.sh, and the output of the test run with only maven and javac enabled.
If anyone can help me, or point me to the right direction, I would be very happy.

Thanks,
Peter

Re: Yetus for Apache Hive

[Allen Wittenauer <aw...@effectivemachines.com>]

> On Nov 9, 2016, at 7:15 AM, Peter Vary <pv...@cloudera.com> wrote:
> 
> Hi devs,
> 
> I have not seen an users list for yetus, so I write here.

	This is the correct place!

> 	• Javac plugin: When I create a personality with only maven, and javac plugin enabled, then the maven install is run for pre, and post patch, but the patch generated java compile warnings are not highlighted. Am I wrong when expecting the javac plugin to calculate the results using the patch-mvninstall-<module>.txt and  branch-mvninstall-<module>.txt files? Where did I make the mistake in the configuration, why the result files are not compared?

	javac does it's calculations based upon the compile phase's output. The mvn install phase is to just get the dependencies down into the repo cache. Looking at the out file, the compile phase and therefore javac didn't run.  compile can be finicky to configure.  This might also be related to a bug that stack filed regarding improper reading of the configured test types since it looks like it was shoved into test list.

> 	• Test4test plugin: We have different filename patterns for tests, so to use Yetus we have to either add a new plugin or has to update the existing plugin to have the pattern as a configurable parameter. I am willing to create a patch for it if the Yetus community thinks making test4test plugin configurable is the way to go.

	Yeah, this should probably be configurable. :)


>  	• Findbugs plugin: Findbugs plugin expects the project structure to be the standard maven one (/src/main/java, /src/test/java). In Hive some of the modules have a different file layout. Like above either a new plugin is needed, or a parameter should be introduced to enable changes in checking.

	IIRC, there are a lot of places where Yetus expects the maven file layout.  If you could expand on what Hive's layout looks like, we could probably provide better guidance on which way to go.  Off the cuff though, I'm hesitant towards making this a configurable since I'm generally not a fan of being a salmon swimming upstream.