[jira] [Created] (BEAM-4413) Create infrastructure configuration (docker/k8s) for HCatalogIOIT

["Kasia Kucharczyk (JIRA)" <ji...@apache.org>]

Kasia Kucharczyk created BEAM-4413:
--------------------------------------

             Summary: Create infrastructure configuration (docker/k8s) for HCatalogIOIT
                 Key: BEAM-4413
                 URL: https://issues.apache.org/jira/browse/BEAM-4413
             Project: Beam
          Issue Type: Test
          Components: testing
            Reporter: Kasia Kucharczyk
            Assignee: Jason Kuster


Currently the HCatalogIOIT test is working +only+ with provided infrastructure (e.g. DataProc). It is caused by lack of docker image which would meet test requirements.
 * First of all some of Hive images are custom made and does not contain HCatalog.
 * Secondly some of images are problematic in terms of network configuration. It was impossible to make all components visible for each other (datanodes, namenodes, etc.), what was observed with following error:

{code:java}
org.apache.hadoop.ipc.RemoteException(java.io.IOException): File could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation.{code}

Here is related stackoverflow issue: [https://stackoverflow.com/questions/26743656/hadoop-2-5-0-failure-to-write-file-remotely]
For a moment there is no solution.

What seems to be a next reasonable point to check here is creating own docker image, publish it and test it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)