You are viewing a plain text version of this content. The canonical link for it is here.

Posted to issues@livy.apache.org by "Dylan Patterson (Jira)" <ji...@apache.org> on 2020/09/02 15:22:00 UTC

[jira] [Created] (LIVY-789) Spark data files are not available in interactive session

Dylan Patterson created LIVY-789:
------------------------------------

             Summary: Spark data files are not available in interactive session
                 Key: LIVY-789
                 URL: https://issues.apache.org/jira/browse/LIVY-789
             Project: Livy
          Issue Type: Bug
          Components: Interpreter
    Affects Versions: 0.7.0
            Reporter: Dylan Patterson


Files passed via the `files` parameter are not available from the main program in interactive mode. This is because livy runs the `fake_shell.py` interpreter in the root directory while yarn stages in a temp dir.  I have not checked whether batch programs suffer the same short coming.

*Potential Solutions*

# Set CWD of `fake_shell.py` to yarn staging directory
# Create temporary working directory and copy necessary files from the `files` parameter



--
This message was sent by Atlassian Jira
(v8.3.4#803005)