IllegalArgumentException when trying to execute job

["Papadopoulos, Konstantinos" <Ko...@IRIworldwide.com>]

Hi all,

I am trying to execute a batch job that gets a list of IDs and perform a loop with a number of steps during each iteration including reading from a MS SQL Server DB.

A sample pseudo-code of our implementation is the following:

List<Long> ids = ...

ids.foreach(
     id -> executeIteration();
)

void executeIteration {
     // 1. Read from source DB
     // 2. Perform a number of transformations
     // 3. Write data to local file
}

When I am trying to execute the aforementioned job to a local Flink cluster, job fails retrieving the following exception:

java.lang.IllegalArgumentException: open() failed.The query processor could not start the necessary thread resources for parallel query execution.
     at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.open(JDBCInputFormat.java:241)
     at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:170)
     at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704)
     at java.lang.Thread.run(Unknown Source)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The query processor could not start the necessary thread resources for parallel query execution.
     at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216)
     at com.microsoft.sqlserver.jdbc.SQLServerResultSet$FetchBuffer.nextRow(SQLServerResultSet.java:4869)
     at com.microsoft.sqlserver.jdbc.SQLServerResultSet.fetchBufferNext(SQLServerResultSet.java:1781)
     at com.microsoft.sqlserver.jdbc.SQLServerResultSet.next(SQLServerResultSet.java:1034)
     at org.apache.flink.api.java.io.jdbc.JDBCInputFormat.open(JDBCInputFormat.java:239)
     ... 3 more

Does anyone have any idea why this is happening?

Thanks in advance,
Konstantinos