You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Stamatis Zampetakis (Jira)" <ji...@apache.org> on 2021/10/20 21:05:00 UTC
[jira] [Created] (HIVE-25629) Drop support of multiple qfiles in
QTestUtil, output and result processors
Stamatis Zampetakis created HIVE-25629:
------------------------------------------
Summary: Drop support of multiple qfiles in QTestUtil, output and result processors
Key: HIVE-25629
URL: https://issues.apache.org/jira/browse/HIVE-25629
Project: Hive
Issue Type: Task
Components: Testing Infrastructure
Affects Versions: 4.0.0
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
The current implementation of [QTestUtil|https://github.com/apache/hive/blob/afeb0f8413b1fd777611e890e53925119a5e39f1/itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java], [QOutProcessor|https://github.com/apache/hive/blob/master/itests/util/src/main/java/org/apache/hadoop/hive/ql/QOutProcessor.java], and [QTestResultProcessor|https://github.com/apache/hive/blob/afeb0f8413b1fd777611e890e53925119a5e39f1/itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestResultProcessor.java], has some methods and fields (maps) for managing multiple input files. However, *all* clients of this API, such as [CoreCliDriver|https://github.com/apache/hive/blob/afeb0f8413b1fd777611e890e53925119a5e39f1/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreCliDriver.java], use these classes by processing one file per run.
+Example+
{code:java}
public void runTest(String testName, String fname, String fpath) {
...
qt.addFile(fpath);
qt.cliInit(new File(fpath));
...
try {
qt.executeClient(fname);
} catch (CommandProcessorException e) {
qt.failedQuery(e.getCause(), e.getResponseCode(), fname, QTestUtil.DEBUG_HINT);
}
...
}
{code}
Notice that {{qt.addFile}} will keep accumulating input files to memory (filename + content) while {{qt.executeClient}} (and other similar APIs) always operate on the last file added. Apart from wasting memory, the APIs for multiple files are harder to understand, and extend.
The goal of this JIRA is to simplify the aforementioned APIs by removing unused/redundant parts associated to multiple files to improve code readability, and reduce memory consumption.
+Historical note+
Before HIVE-25625 the functionality of multiple input files was used by the {{TestCompareCliDriver}} but it was still useless for all the other clients. With the removal of {{TestCompareCliDriver}} in HIVE-25625 keeping multiple files is completely redundant.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)