You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@samza.apache.org by "Chris Riccomini (JIRA)" <ji...@apache.org> on 2015/01/14 18:24:35 UTC
[jira] [Created] (SAMZA-508) Supporting integration test execution
on remote machines
Chris Riccomini created SAMZA-508:
-------------------------------------
Summary: Supporting integration test execution on remote machines
Key: SAMZA-508
URL: https://issues.apache.org/jira/browse/SAMZA-508
Project: Samza
Issue Type: Bug
Components: test
Reporter: Chris Riccomini
SAMZA-468 added support for single-node execution of integration tests. Multi-node integration tests don't work right now because of the way we're deploying job tarballs. We're using yarn.package.path=file:/... URI schemes, and SCP'ing the job tarballs to all machines. This is a problem when executing on multiple nodes because YARN stats the local job tarball when run-job.sh is run, and compares the tarball's [ctime/mtime/atime|http://www.linux-faqs.info/general/difference-between-mtime-ctime-and-atime] fields against the remote machines. They don't match since we're SCP'ing them, so YARN fails the job's deployment.
We'll have to either modify the remote timestamps, or use some other filesystem (HDFS, HTTP, etc) to deploy the job tarballs to YARN NMs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)