Python Import Changes

[Jonathan Hurley <jh...@hortonworks.com>]

Hi Developers,

We’ve been seeing some issues in certain environments where our Python global imports aren’t working. On the surface, executing the imports in a clean Python shell works, but when the agent code gets executed, we’re hit with a bunch of NameErrors for the imports that Python can’t find.

Moving forward, please do not use import *. Instead, we should all be using explicit imports of each module/class that the Python script depends on.

One pattern that is especially bad is

from resource_management import *

When working on an existing file that includes a * import, please change this to explicit imports. Let’s be diligent in our code reviews as well. If you see a * import, flag it as an issue so that we can work toward eliminating this from the codebase.

Thanks,

- Jonathan

Re: Python Import Changes

[Jonathan Hurley <jh...@hortonworks.com>]

All,

I’d like to remind you to avoid “from resource_management import *” in the python code. “import *” seems to cause issues in some production environments where Ambari is deployed. Recently, the “import *” was removed from almost every params.py and a bad merge added it back in.

- When you encounter this import statement while working on python files, I recommend that you also remove it and add in the explicit imports
- If you add it back to a file that does not include it, you are actually un-doing work that was performed to eliminate this type of silent bug

Please be aware of this when performing code reviews as well.

> On Feb 18, 2015, at 1:17 PM, Alejandro Fernandez <af...@hortonworks.com> wrote:
> 
> I completely agree Jonathan. Thank you for starting this initiative.
> 
> Cheers,
> Alejandro
> 
> On 2/18/15, 9:19 AM, "Jonathan Hurley" <jh...@hortonworks.com> wrote:
> 
>> Hi Developers,
>> 
>> We¹ve been seeing some issues in certain environments where our Python
>> global imports aren¹t working. On the surface, executing the imports in a
>> clean Python shell works, but when the agent code gets executed, we¹re
>> hit with a bunch of NameErrors for the imports that Python can¹t find.
>> 
>> Moving forward, please do not use import *. Instead, we should all be
>> using explicit imports of each module/class that the Python script
>> depends on.
>> 
>> One pattern that is especially bad is
>> 
>> from resource_management import *
>> 
>> When working on an existing file that includes a * import, please change
>> this to explicit imports. Let¹s be diligent in our code reviews as well.
>> If you see a * import, flag it as an issue so that we can work toward
>> eliminating this from the codebase.
>> 
>> Thanks,
>> 
>> - Jonathan
> 

Re: Python Import Changes

[Alejandro Fernandez <af...@hortonworks.com>]

I completely agree Jonathan. Thank you for starting this initiative.

Cheers,
Alejandro

On 2/18/15, 9:19 AM, "Jonathan Hurley" <jh...@hortonworks.com> wrote:

>Hi Developers,
>
>We¹ve been seeing some issues in certain environments where our Python
>global imports aren¹t working. On the surface, executing the imports in a
>clean Python shell works, but when the agent code gets executed, we¹re
>hit with a bunch of NameErrors for the imports that Python can¹t find.
>
>Moving forward, please do not use import *. Instead, we should all be
>using explicit imports of each module/class that the Python script
>depends on.
>
>One pattern that is especially bad is
>
>from resource_management import *
>
>When working on an existing file that includes a * import, please change
>this to explicit imports. Let¹s be diligent in our code reviews as well.
>If you see a * import, flag it as an issue so that we can work toward
>eliminating this from the codebase.
>
>Thanks,
>
>- Jonathan