[jira] [Created] (FLINK-15527) can not control the number of container on yarn single job module

["chenchencc (Jira)" <ji...@apache.org>]

chenchencc created FLINK-15527:
----------------------------------

             Summary: can not control the number of container on yarn single job module
                 Key: FLINK-15527
                 URL: https://issues.apache.org/jira/browse/FLINK-15527
             Project: Flink
          Issue Type: Bug
          Components: Deployment / YARN
    Affects Versions: 1.10.0
            Reporter: chenchencc
             Fix For: 1.10.0


when run yarn single job run many container but paralism set 4

*scripts:*

./bin/flink run -m yarn-cluster -ys 3 -p 4 -yjm 1024m -ytm 4096m -yqu bi -c com.cc.test.HiveTest2 ./cc_jars/hive-1.0-SNAPSHOT.jar 11.txt test61

*result:*

the number of containers is 500+

 

*code:*

com.cc.test.HiveTest2

public static void main(String[] args) throws Exception {
 EnvironmentSettings settings = EnvironmentSettings.newInstance().useBlinkPlanner().inStreamingMode().build();
 StreamExecutionEnvironment settings2 = StreamExecutionEnvironment.getExecutionEnvironment();
 StreamTableEnvironment tableEnv = StreamTableEnvironment.create(settings2, settings);
 String name = "myhive";
 String defaultDatabase = "test";
 String hiveConfDir = "/etc/hive/conf";
 String version = "1.2.1"; // or 1.2.1 2.3.4
 HiveCatalog hive = new HiveCatalog(name, defaultDatabase, hiveConfDir, version);
 tableEnv.registerCatalog("myhive", hive);
 tableEnv.useCatalog("myhive");
 tableEnv.listTables();
 Table table = tableEnv.sqlQuery("select id from orderparent_test2 where id = 'A000021204170176'");
 tableEnv.toAppendStream(table, Row.class).print();
 tableEnv.toAppendStream(table, Row.class)
 .writeAsText("hdfs:///user/chenchao1/"+ args[0], FileSystem.WriteMode.OVERWRITE);
 tableEnv.execute(args[1]);
}

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)