[GitHub] [hadoop] aw-was-here edited a comment on pull request #2385: HADOOP-16649. hadoop_add_param function : change regexp test by iterative equality test

[GitBox <gi...@apache.org>]

aw-was-here edited a comment on pull request #2385:
URL: https://github.com/apache/hadoop/pull/2385#issuecomment-714249153


   Just to save everyone a lot of time and suffering:
   
   This approach will break a lot of things in very unexpected ways  (doing a search on everywhere hadoop_add_params is called should make this clear).  hadoop_add_param was specifically built for partial matches because the HADOOP_OPTS command line can't really do exact matches and this was a quick way to prevent duplicate options.  The unit test failure in  hadoop_finalize_hadoop_heap  was intended to provide a hint that "yar they be dragons here." I should have written better tests, but given it took like 2 years just to get most of this code in over the total @#$@#$ that was in hadoop 2.x ...
   
   When I wrote the code originally, we didn't have a need for exact matches anywhere (HADOOP_OPTIONAL_TOOLS wasn't written yet).  It was written and committed to 3.x. Then the HADOOP_OPTIONAL_TOOLS code was written but that would be the only place where an exact match would be useful and we didn't have any sooo... I just re-used hadoop_add_param with the (clearly faulty) assumption that people would test their code on Hadoop 3.x.   But since the azure team didn't bother to test with hadoop 3.x until it was too late...  At this point, I was getting tired of the Hadoop politics and bailed, leaving this furball hanging around.
   
   Anyway, the *real* fix for this is to convert HADOOP_OPTIONAL_TOOLS to an array and then do an exact match, looping over the array. I think there is code to do that now.  Might need some new helper code to do comma-delimited -> array but that shouldn't be hard.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org