[GitHub] [doris] SaintBacchus commented on a diff in pull request #18059: [Feature-Wip](MySQL Load)Show load warnings for mysql load

["SaintBacchus (via GitHub)" <gi...@apache.org>]

SaintBacchus commented on code in PR #18059:
URL: https://github.com/apache/doris/pull/18059#discussion_r1148281026


##########
fe/fe-core/src/main/java/org/apache/doris/load/loadv2/MysqlLoadManager.java:
##########
@@ -124,20 +157,22 @@ public LoadJobRowResult executeMySqlLoadJobFromStmt(ConnectContext context, Load
             context.setExecTimeout(newTimeOut);
         }
         String token = tokenManager.acquireToken();
+        boolean clientLocal = dataDesc.isClientLocal();
+        MySqlLoadContext loadContext = new MySqlLoadContext();
+        loadContextMap.put(loadId, loadContext);
         LOG.info("execute MySqlLoadJob for id: {}.", loadId);
         try (final CloseableHttpClient httpclient = HttpClients.createDefault()) {
             for (String file : filePaths) {
-                InputStreamEntity entity = getInputStreamEntity(context, dataDesc.isClientLocal(), file, loadId);
+                InputStreamEntity entity = getInputStreamEntity(context, clientLocal, file, loadId);
                 HttpPut request = generateRequestForMySqlLoad(entity, dataDesc, database, table, token);
-                MySqlLoadContext loadContext = new MySqlLoadContext();
                 loadContext.setRequest(request);
-                loadContextMap.put(loadId, loadContext);
                 try (final CloseableHttpResponse response = httpclient.execute(request)) {
                     String body = EntityUtils.toString(response.getEntity());
                     JsonObject result = JsonParser.parseString(body).getAsJsonObject();
                     if (!result.get("Status").getAsString().equalsIgnoreCase("Success")) {
+                        failedRecords.offer(new MySqlLoadFailRecord(loadId, result.get("ErrorURL").getAsString()));
                         LOG.warn("Execute mysql data load failed with request: {} and response: {}", request, body);
-                        throw new LoadException(result.get("Message").getAsString());
+                        throw new LoadException(result.get("Message").getAsString() + " with load id " + loadId);

Review Comment:
   It's too long for user. A typical error url will have about 150 letters.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@doris.apache.org
For additional commands, e-mail: commits-help@doris.apache.org