[jira] [Created] (AMBARI-24670) Directory creation should be atomic

["Doroszlai, Attila (JIRA)" <ji...@apache.org>]

Doroszlai, Attila created AMBARI-24670:
------------------------------------------

             Summary: Directory creation should be atomic
                 Key: AMBARI-24670
                 URL: https://issues.apache.org/jira/browse/AMBARI-24670
             Project: Ambari
          Issue Type: Improvement
          Components: ambari-agent
            Reporter: Doroszlai, Attila
            Assignee: Andrew Onischuk
             Fix For: 2.8.0


If parallel execution is enabled on Ambari Agent, concurrent directory creation may fail with:

{noformat:title=errors-62.txt}
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 35, in <module>
    BeforeAnyHook().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 375, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 31, in hook
    setup_hadoop_env()
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py", line 203, in setup_hadoop_env
    mode=01777
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 191, in action_create
    sudo.makedir(path, self.resource.mode or 0755)
  File "/usr/lib/ambari-agent/lib/resource_management/core/sudo.py", line 121, in makedir
    os.mkdir(path)
OSError: [Errno 17] File exists: '/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'
{noformat}

or

{noformat:title=errors-63.txt}
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 35, in <module>
    BeforeAnyHook().execute()
  File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 375, in execute
    method(env)
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 31, in hook
    setup_hadoop_env()
  File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py", line 203, in setup_hadoop_env
    mode=01777
  File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
    self.env.run()
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
    self.run_action(resource, action)
  File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
    provider_action()
  File "/usr/lib/ambari-agent/lib/resource_management/core/providers/system.py", line 179, in action_create
    path = sudo.readlink(path)
  File "/usr/lib/ambari-agent/lib/resource_management/core/sudo.py", line 161, in readlink
    return os.readlink(path)
OSError: [Errno 22] Invalid argument: '/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'
{noformat}

The failed tasks need to be retried to succeed, causing delays.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)