You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@iotdb.apache.org by sp...@apache.org on 2026/05/16 05:47:36 UTC
(iotdb) 05/06: update join
This is an automated email from the ASF dual-hosted git repository.
SpriCoder pushed a commit to branch fs/inner-view
in repository https://gitbox.apache.org/repos/asf/iotdb.git
commit eed096463f38890039105a1416a95906267e35dd
Author: spricoder <sp...@qq.com>
AuthorDate: Sat May 2 20:05:48 2026 +0800
update join
---
.../plans/2026-04-29-cli-filesystem-mode.md | 7 +-
.../specs/2026-04-29-cli-filesystem-mode-design.md | 16 +++-
.../org/apache/iotdb/cli/fs/FilesystemShell.java | 82 +++++++++++++++++-
.../iotdb/cli/fs/command/FilesystemCommand.java | 7 ++
.../cli/fs/command/FilesystemCommandParser.java | 61 ++++++++++++++
.../org/apache/iotdb/cli/utils/JlineUtils.java | 4 +-
.../apache/iotdb/cli/fs/FilesystemShellTest.java | 97 ++++++++++++++++++++++
.../fs/command/FilesystemCommandParserTest.java | 58 +++++++++++++
.../org/apache/iotdb/cli/utils/JlineUtilsTest.java | 7 ++
9 files changed, 330 insertions(+), 9 deletions(-)
diff --git a/docs/superpowers/plans/2026-04-29-cli-filesystem-mode.md b/docs/superpowers/plans/2026-04-29-cli-filesystem-mode.md
index a39d65c880c..f31646c213b 100644
--- a/docs/superpowers/plans/2026-04-29-cli-filesystem-mode.md
+++ b/docs/superpowers/plans/2026-04-29-cli-filesystem-mode.md
@@ -98,6 +98,8 @@ These notes capture follow-up implementation experience for quickly resuming thi
of the sidecar model and should resolve to `unknown` or a not-readable error.
- `paste /db/table1.csv /db/table2.csv` -> read each regular file as lines and join
corresponding lines with tabs, padding missing trailing lines with empty fields.
+ - `join -t, -1 2 -2 1 /db/table1.csv /db/table2.csv` -> Unix text join over regular file lines.
+ It is not SQL join, does not sort inputs, and treats headers as ordinary lines.
- Table-mode write boundaries are intentionally narrow and only active with
`--fs_write_mode enabled`:
- `mkdir /db` creates a database.
@@ -178,6 +180,7 @@ Design decisions already approved:
| `du <path>` | Print provider count and path; table sidecars use file-line counts. | `du /db/table.csv` |
| `cut -d<delimiter> -f<fields> <path>` | Delimiter-based Unix field selection; supports lists and closed ranges. | `cut -d, -f2,3 /db/table.csv` |
| `paste <path>...` | Print multiple regular files side by side, joining corresponding lines with tabs. | `paste /db/t1.csv /db/t2.csv` |
+| `join [-t delimiter] [-1 field] [-2 field] <path1> <path2>` | Inner join two readable files by 1-based text fields; inputs are expected to be sorted by the join key like Unix `join`. | `join -t, -1 2 -2 1 /db/t1.csv /db/t2.csv` |
| `tree [-L depth] [path]` | Print descendants with indentation and names only. | `tree -L 2 /db` |
| `mkdir <path>` | Write-gated; in table mode with writes enabled, creates a database. | `mkdir /newdb` |
| `rm <path>` | Write-gated; in table mode with writes enabled, only table CSV drop is allowed. | `rm /db/table.csv` |
@@ -203,8 +206,8 @@ Good subagent tasks for this branch:
- Reviewing docs for consistency after implementation changes.
- Assigning natural layers independently: `FsPath`, command parser, tree provider, table provider,
shell output, CLI dispatch, and docs.
-- Checking whether `ls`, `tree`, `cat`, `paste`, and `stat` still match the design document's Unix
- output semantics.
+- Checking whether `ls`, `tree`, `cat`, `paste`, `join`, and `stat` still match the design
+ document's Unix output semantics.
- Reviewing whether SQL mode remains the default and whether existing SQL CLI behavior is still
isolated from filesystem mode.
- Inspecting exception paths, especially interactive filesystem commands where one failed command
diff --git a/docs/superpowers/specs/2026-04-29-cli-filesystem-mode-design.md b/docs/superpowers/specs/2026-04-29-cli-filesystem-mode-design.md
index ed1d86742fd..8380042bf26 100644
--- a/docs/superpowers/specs/2026-04-29-cli-filesystem-mode-design.md
+++ b/docs/superpowers/specs/2026-04-29-cli-filesystem-mode-design.md
@@ -216,6 +216,7 @@ semantics wherever the same command exists. Provider support can still vary by d
| `du <path>` | Print provider count plus path. Table sidecars use file-line counts. | `du /db/table.csv` |
| `cut -d<delimiter> -f<fields> <path>` | Apply Unix delimiter-based field selection to each line. The delimiter must be one character. Field lists and closed ranges such as `2,3` and `1-2` are supported. | `cut -d, -f2,3 /db/table.csv` |
| `paste <path>...` | Read multiple regular files side by side and join corresponding lines with tabs. | `paste /db/t1.csv /db/t2.csv` |
+| `join [-t delimiter] [-1 field] [-2 field] <path1> <path2>` | Apply Unix inner join semantics to two readable files using 1-based fields. Default delimiter is whitespace; with `-t`, output uses the same delimiter. Inputs are expected to be sorted by join key. | `join -t, -1 2 -2 1 /db/t1.csv /db/t2.csv` |
| `tree [-L depth] [path]` | Recursively print descendants with indentation and names only. `-L` limits recursion depth. | `tree -L 2 /db` |
| `mkdir <path>` | Write-gated command. With table mode and `--fs_write_mode enabled`, `mkdir /db` creates a database. Otherwise it returns a read-only or unsupported error. | `mkdir /newdb` |
| `rm <path>` | Write-gated command. With table mode and `--fs_write_mode enabled`, only `rm /db/table.csv` is allowed and maps to table drop. | `rm /db/table.csv` |
@@ -254,6 +255,7 @@ Raw SQL should be run in the default SQL access mode.
| `cat /db/table.csv` | `SELECT * FROM db.table LIMIT <limit>`, formatted as CSV records. |
| `cut -d, -f2,3 /db/table.csv` | Delimiter-based text field projection over the CSV records. |
| `paste /db/t1.csv /db/t2.csv` | Read each regular file as CSV/text lines and join corresponding lines with tabs. |
+| `join -t, -1 2 -2 1 /db/t1.csv /db/t2.csv` | Read both regular files as text lines and perform Unix-style inner join on the selected fields. |
| `tee -a /db/table.csv` | Parse CSV input with Apache Commons CSV, validate columns and required `time`, then execute chunked `INSERT INTO db.table(...) VALUES ...`. |
| `cat /db/table.schema` | `DESC db.table DETAILS`, formatted as CSV with IoTDB result columns preserved. |
| `cat /db/table.meta` | `SHOW TABLES DETAILS FROM db`, filtered to the table and formatted as CSV with IoTDB result columns preserved. |
@@ -275,6 +277,9 @@ exposing internal implementation types or Java debug-style structures in normal
or introduce table-specific column-selection flags.
- `paste` prints multiple regular files side by side as tab-separated values. It does not select
database columns.
+- `join` performs the standard Unix text join over exactly two readable files. It is not SQL join,
+ does not sort inputs, and treats CSV headers as ordinary input lines. Default delimiter handling
+ follows Unix whitespace splitting; `-t,` is the CSV sidecar form.
- `less` and `more` are currently non-interactive read aliases with the default read limit.
- `stat` is the command that may expose typed metadata, because Unix `stat` is explicitly about
object metadata.
@@ -314,6 +319,10 @@ are unknown in the sidecar model and must not trigger table or column SQL reads.
corresponding lines with tabs. It must not become a database-specific `select` command or
`cat --columns` dialect.
+`join` also remains Unix-like: users pass exactly two regular file paths and optional text field
+selection flags. It must not become a SQL join alias or infer database relationships from table
+metadata.
+
For CSV-first table files, multi-column projection should prefer Unix `cut` syntax such as
`cut -d, -f2,3 /db/table.csv`. The implementation may later optimize this internally, but the
public interface must remain the standard `cut` form.
@@ -425,7 +434,7 @@ New code should live under `org.apache.iotdb.cli.fs`.
- `FilesystemShell`: filesystem-mode command loop and `-e` single-command execution.
- `command/*`: command parsing and command value objects for filesystem commands such as `pwd`,
- `ls`, `cd`, `stat`, `cat`, `cut`, `paste`, `tree`, and `help`.
+ `ls`, `cd`, `stat`, `cat`, `cut`, `paste`, `join`, `tree`, and `help`.
- `path/FsPath`: path normalization and resolution for absolute and relative paths.
- `node/FsNode`, `node/FsNodeType`, `node/FsNodeMetadata`: typed metadata model.
- `provider/FilesystemSchemaProvider`: schema and data read provider interface.
@@ -455,7 +464,8 @@ Filesystem mode separates reads from mutations. The schema provider owns read op
- `tail(FsPath path)` / `tailLines(FsPath path)` where the provider supports tail
- `count(FsPath path)` where the provider supports logical count
- `read(List<FsPath> paths)` remains available for providers that need optimized multi-path reads,
- but table mode sidecar `paste` is implemented as regular file line joining in the shell.
+ but table mode sidecar `paste` and `join` are implemented as regular file text operations in the
+ shell.
The mutation provider owns the current write-gated operations:
@@ -537,7 +547,7 @@ Unit tests should cover the behavior without needing a live IoTDB instance where
- `FsPath` tests for absolute paths, relative paths, `.`, `..`, empty input, and attempts to move
above root.
- Command parser tests for valid and invalid forms of all supported shell commands, including
- listing options, `head`/`tail` limits, `wc -l`, `find -name`, `cut`, `paste`, write-gated
+ listing options, `head`/`tail` limits, `wc -l`, `find -name`, `cut`, `paste`, `join`, write-gated
commands, and the parser-only `sql` form.
- Provider tests with a mocked `SqlExecutor`, verifying tree-mode path-to-SQL mapping.
- Provider tests with a mocked `SqlExecutor`, verifying table-mode path-to-SQL mapping, CSV
diff --git a/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/FilesystemShell.java b/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/FilesystemShell.java
index 051c96b5b35..407744fc03c 100644
--- a/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/FilesystemShell.java
+++ b/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/FilesystemShell.java
@@ -43,6 +43,7 @@ import java.nio.charset.StandardCharsets;
import java.sql.SQLException;
import java.util.ArrayList;
import java.util.Arrays;
+import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
@@ -53,8 +54,8 @@ public class FilesystemShell {
private static final List<String> COMMANDS =
Arrays.asList(
"pwd", "ls", "ll", "cd", "stat", "cat", "head", "tail", "wc", "grep", "find", "less",
- "more", "file", "du", "mkdir", "rm", "mv", "cut", "paste", "tree", "help", "exit", "quit",
- "tee");
+ "more", "file", "du", "mkdir", "rm", "mv", "cut", "paste", "join", "tree", "help", "exit",
+ "quit", "tee");
private final CliContext ctx;
private final FilesystemSchemaProvider provider;
@@ -138,6 +139,9 @@ public class FilesystemShell {
case PASTE:
printPaste(command.getPaths());
return true;
+ case JOIN:
+ printJoin(command.getPaths(), command.getOption(), command.getPattern());
+ return true;
case TEE:
append(command.getPath(), false);
return true;
@@ -360,6 +364,27 @@ public class FilesystemShell {
}
}
+ private void printJoin(List<String> paths, String delimiter, String fields) throws SQLException {
+ int[] joinFields = joinFields(fields);
+ List<String> leftLines = readableLines(resolve(paths.get(0)), DEFAULT_READ_LIMIT);
+ List<String> rightLines = readableLines(resolve(paths.get(1)), DEFAULT_READ_LIMIT);
+ Map<String, List<String[]>> rightRows = joinRowsByKey(rightLines, delimiter, joinFields[1]);
+
+ for (String leftLine : leftLines) {
+ String[] left = splitJoinFields(leftLine, delimiter);
+ if (!hasField(left, joinFields[0])) {
+ continue;
+ }
+ List<String[]> matches = rightRows.get(left[joinFields[0] - 1]);
+ if (matches == null) {
+ continue;
+ }
+ for (String[] right : matches) {
+ ctx.getPrinter().println(joinLine(left, right, joinFields[0], joinFields[1], delimiter));
+ }
+ }
+ }
+
private List<String> readableLines(FsPath path, int limit) throws SQLException {
if (isTextFile(path)) {
return provider.readLines(path, limit);
@@ -528,6 +553,7 @@ public class FilesystemShell {
ctx.getPrinter().println("mv <source> <target>");
ctx.getPrinter().println("cut -d<delimiter> -f<fields> <path>");
ctx.getPrinter().println("paste <path>...");
+ ctx.getPrinter().println("join [-t delimiter] [-1 field] [-2 field] <path1> <path2>");
ctx.getPrinter().println("tee -a <path>");
ctx.getPrinter().println("tree [-L depth] [path]");
ctx.getPrinter().println("exit");
@@ -596,6 +622,58 @@ public class FilesystemShell {
return builder.toString();
}
+ private static int[] joinFields(String fields) {
+ String[] values = fields.split(",", -1);
+ return new int[] {parsePositiveInt(values[0]), parsePositiveInt(values[1])};
+ }
+
+ private static Map<String, List<String[]>> joinRowsByKey(
+ List<String> lines, String delimiter, int keyField) {
+ Map<String, List<String[]>> rowsByKey = new LinkedHashMap<>();
+ for (String line : lines) {
+ String[] fields = splitJoinFields(line, delimiter);
+ if (!hasField(fields, keyField)) {
+ continue;
+ }
+ String key = fields[keyField - 1];
+ rowsByKey.computeIfAbsent(key, ignored -> new ArrayList<>()).add(fields);
+ }
+ return rowsByKey;
+ }
+
+ private static String[] splitJoinFields(String line, String delimiter) {
+ if (delimiter.isEmpty()) {
+ String trimmed = line.trim();
+ if (trimmed.isEmpty()) {
+ return new String[0];
+ }
+ return trimmed.split("\\s+");
+ }
+ return line.split(Pattern.quote(delimiter), -1);
+ }
+
+ private static boolean hasField(String[] fields, int fieldNumber) {
+ return fieldNumber > 0 && fieldNumber <= fields.length;
+ }
+
+ private static String joinLine(
+ String[] left, String[] right, int leftKeyField, int rightKeyField, String delimiter) {
+ String outputDelimiter = delimiter.isEmpty() ? " " : delimiter;
+ List<String> output = new ArrayList<>();
+ output.add(left[leftKeyField - 1]);
+ addNonKeyFields(output, left, leftKeyField);
+ addNonKeyFields(output, right, rightKeyField);
+ return String.join(outputDelimiter, output);
+ }
+
+ private static void addNonKeyFields(List<String> output, String[] fields, int keyField) {
+ for (int i = 0; i < fields.length; i++) {
+ if (i != keyField - 1) {
+ output.add(fields[i]);
+ }
+ }
+ }
+
private static boolean[] selectedFields(String fields, int fieldCount) {
boolean[] selected = new boolean[fieldCount];
for (String field : fields.split(",")) {
diff --git a/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/command/FilesystemCommand.java b/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/command/FilesystemCommand.java
index 2f77a787a54..7d379236a78 100644
--- a/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/command/FilesystemCommand.java
+++ b/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/command/FilesystemCommand.java
@@ -45,6 +45,7 @@ public class FilesystemCommand {
MV,
CUT,
PASTE,
+ JOIN,
TEE,
TREE,
SQL,
@@ -124,6 +125,12 @@ public class FilesystemCommand {
Type.CUT, path, Collections.singletonList(path), -1, -1, delimiter, fields, "", "");
}
+ public static FilesystemCommand join(String delimiter, String fields, List<String> paths) {
+ String path = paths.isEmpty() ? "" : paths.get(0);
+ return new FilesystemCommand(
+ Type.JOIN, path, Collections.unmodifiableList(paths), -1, -1, delimiter, fields, "", "");
+ }
+
public static FilesystemCommand tree(String path, int depth) {
return new FilesystemCommand(
Type.TREE, path, Collections.singletonList(path), depth, -1, "", "", "", "");
diff --git a/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/command/FilesystemCommandParser.java b/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/command/FilesystemCommandParser.java
index 75992cf0fc7..565b9bad58f 100644
--- a/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/command/FilesystemCommandParser.java
+++ b/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/fs/command/FilesystemCommandParser.java
@@ -27,6 +27,7 @@ public class FilesystemCommandParser {
private static final String DEFAULT_PATH = ".";
private static final String DEFAULT_CUT_DELIMITER = "\t";
+ private static final String DEFAULT_JOIN_DELIMITER = "";
private static final int DEFAULT_TREE_DEPTH = Integer.MAX_VALUE;
private static final int DEFAULT_HEAD_LIMIT = 10;
@@ -111,6 +112,9 @@ public class FilesystemCommandParser {
if ("paste".equals(command)) {
return parsePaste(tokens);
}
+ if ("join".equals(command)) {
+ return parseJoin(tokens);
+ }
if ("tee".equals(command)) {
return parseTee(tokens);
}
@@ -139,6 +143,63 @@ public class FilesystemCommandParser {
return FilesystemCommand.paths(FilesystemCommand.Type.PASTE, paths);
}
+ private static FilesystemCommand parseJoin(String[] tokens) {
+ String delimiter = DEFAULT_JOIN_DELIMITER;
+ int leftField = 1;
+ int rightField = 1;
+ List<String> paths = new ArrayList<>();
+
+ for (int i = 1; i < tokens.length; i++) {
+ String token = tokens[i];
+ if ("-t".equals(token)) {
+ if (i + 1 >= tokens.length) {
+ return invalidJoinUsage();
+ }
+ delimiter = tokens[++i];
+ } else if (token.startsWith("-t") && token.length() > 2) {
+ delimiter = token.substring(2);
+ } else if ("-1".equals(token)) {
+ if (i + 1 >= tokens.length) {
+ return invalidJoinUsage();
+ }
+ leftField = parseJoinField(tokens[++i]);
+ } else if ("-2".equals(token)) {
+ if (i + 1 >= tokens.length) {
+ return invalidJoinUsage();
+ }
+ rightField = parseJoinField(tokens[++i]);
+ } else if (token.startsWith("-")) {
+ return FilesystemCommand.invalid("Unsupported join option: " + token);
+ } else {
+ paths.add(token);
+ }
+ }
+
+ if (delimiter.length() > 1) {
+ return FilesystemCommand.invalid("Join delimiter must be a single character");
+ }
+ if (leftField <= 0 || rightField <= 0) {
+ return FilesystemCommand.invalid("Invalid join field");
+ }
+ if (paths.size() != 2) {
+ return invalidJoinUsage();
+ }
+ return FilesystemCommand.join(delimiter, leftField + "," + rightField, paths);
+ }
+
+ private static int parseJoinField(String value) {
+ try {
+ return Integer.parseInt(value);
+ } catch (NumberFormatException e) {
+ return -1;
+ }
+ }
+
+ private static FilesystemCommand invalidJoinUsage() {
+ return FilesystemCommand.invalid(
+ "Usage: join [-t delimiter] [-1 field] [-2 field] <path1> <path2>");
+ }
+
private static FilesystemCommand parseTee(String[] tokens) {
if (tokens.length != 3) {
return FilesystemCommand.invalid("Usage: tee -a <path>");
diff --git a/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/utils/JlineUtils.java b/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/utils/JlineUtils.java
index 704e10f6600..e773f7b521a 100644
--- a/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/utils/JlineUtils.java
+++ b/iotdb-client/cli/src/main/java/org/apache/iotdb/cli/utils/JlineUtils.java
@@ -153,8 +153,8 @@ public class JlineUtils {
if ("filesystem".equalsIgnoreCase(accessMode)) {
return new StringsCompleter(
"pwd", "ls", "ll", "cd", "stat", "cat", "head", "tail", "wc", "grep", "find", "less",
- "more", "file", "du", "mkdir", "rm", "mv", "cut", "paste", "tee", "tree", "help", "exit",
- "quit");
+ "more", "file", "du", "mkdir", "rm", "mv", "cut", "paste", "join", "tee", "tree", "help",
+ "exit", "quit");
}
return new StringsCompleter(SQL_KEYWORDS);
}
diff --git a/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/fs/FilesystemShellTest.java b/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/fs/FilesystemShellTest.java
index cc442be8be3..6cb435ede80 100644
--- a/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/fs/FilesystemShellTest.java
+++ b/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/fs/FilesystemShellTest.java
@@ -576,6 +576,96 @@ public class FilesystemShellTest {
verify(provider).readLines(FsPath.absolute("/db1/table2.csv"), 20);
}
+ @Test
+ public void executeJoinMatchesCsvRowsByFirstField() throws SQLException {
+ when(provider.readLines(FsPath.absolute("/db1/table1.csv"), 20))
+ .thenReturn(Arrays.asList("key,value", "a,10", "b,20", "c,30"));
+ when(provider.readLines(FsPath.absolute("/db1/table2.csv"), 20))
+ .thenReturn(Arrays.asList("key,status", "a,ok", "b,bad", "d,missing"));
+
+ assertTrue(shell.execute("join -t, /db1/table1.csv /db1/table2.csv"));
+
+ assertEquals(
+ "key,value,status"
+ + System.lineSeparator()
+ + "a,10,ok"
+ + System.lineSeparator()
+ + "b,20,bad"
+ + System.lineSeparator(),
+ out.toString());
+ verify(provider).readLines(FsPath.absolute("/db1/table1.csv"), 20);
+ verify(provider).readLines(FsPath.absolute("/db1/table2.csv"), 20);
+ }
+
+ @Test
+ public void executeJoinSupportsDifferentKeyFields() throws SQLException {
+ when(provider.readLines(FsPath.absolute("/db1/table1.csv"), 20))
+ .thenReturn(Arrays.asList("time,key,value", "1,a,10", "2,b,20"));
+ when(provider.readLines(FsPath.absolute("/db1/table2.csv"), 20))
+ .thenReturn(Arrays.asList("key,status", "a,ok", "b,bad"));
+
+ assertTrue(shell.execute("join -t, -1 2 -2 1 /db1/table1.csv /db1/table2.csv"));
+
+ assertEquals(
+ "key,time,value,status"
+ + System.lineSeparator()
+ + "a,1,10,ok"
+ + System.lineSeparator()
+ + "b,2,20,bad"
+ + System.lineSeparator(),
+ out.toString());
+ }
+
+ @Test
+ public void executeJoinPrintsCartesianMatchesForDuplicateKeys() throws SQLException {
+ when(provider.readLines(FsPath.absolute("/db1/table1.csv"), 20))
+ .thenReturn(Arrays.asList("a,10", "a,11"));
+ when(provider.readLines(FsPath.absolute("/db1/table2.csv"), 20))
+ .thenReturn(Arrays.asList("a,ok", "a,good"));
+
+ assertTrue(shell.execute("join -t, /db1/table1.csv /db1/table2.csv"));
+
+ assertEquals(
+ "a,10,ok"
+ + System.lineSeparator()
+ + "a,10,good"
+ + System.lineSeparator()
+ + "a,11,ok"
+ + System.lineSeparator()
+ + "a,11,good"
+ + System.lineSeparator(),
+ out.toString());
+ }
+
+ @Test
+ public void executeJoinSkipsRowsWithoutKeyOrMatch() throws SQLException {
+ when(provider.readLines(FsPath.absolute("/db1/table1.csv"), 20))
+ .thenReturn(Arrays.asList("a,10", "missing-key", "b,20"));
+ when(provider.readLines(FsPath.absolute("/db1/table2.csv"), 20))
+ .thenReturn(Arrays.asList("a,ok", "c,unused"));
+
+ assertTrue(shell.execute("join -t, -1 2 -2 1 /db1/table1.csv /db1/table2.csv"));
+
+ assertEquals("", out.toString());
+ }
+
+ @Test
+ public void executeJoinUsesReadableRowsForNonTextPath() throws SQLException {
+ when(provider.read(FsPath.absolute("/root/sg/d1/s1"), 20))
+ .thenReturn(
+ Arrays.asList(SqlRow.of("Time", "1", "s1", "10"), SqlRow.of("Time", "2", "s1", "20")));
+ when(provider.read(FsPath.absolute("/root/sg/d1/s2"), 20))
+ .thenReturn(
+ Arrays.asList(
+ SqlRow.of("Time", "1", "s2", "ok"), SqlRow.of("Time", "3", "s2", "skip")));
+
+ assertTrue(shell.execute("join /root/sg/d1/s1 /root/sg/d1/s2"));
+
+ assertEquals("1 10 ok" + System.lineSeparator(), out.toString());
+ verify(provider).read(FsPath.absolute("/root/sg/d1/s1"), 20);
+ verify(provider).read(FsPath.absolute("/root/sg/d1/s2"), 20);
+ }
+
@Test
public void executeCutSelectsCsvFieldsByNumber() throws SQLException {
when(provider.readLines(FsPath.absolute("/db1/table1.csv"), 20))
@@ -623,6 +713,13 @@ public class FilesystemShellTest {
verify(provider).list(FsPath.absolute("/"));
}
+ @Test
+ public void completerCompletesJoinCommand() {
+ List<String> values = complete(shell.createCompleter(), "j");
+
+ assertTrue(values.contains("join"));
+ }
+
private static List<String> complete(Completer completer, String line) {
ParsedLine parsedLine = new DefaultParser().parse(line, line.length());
List<Candidate> candidates = new ArrayList<>();
diff --git a/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/fs/command/FilesystemCommandParserTest.java b/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/fs/command/FilesystemCommandParserTest.java
index 31971bf6a1e..df0bf2ffdc0 100644
--- a/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/fs/command/FilesystemCommandParserTest.java
+++ b/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/fs/command/FilesystemCommandParserTest.java
@@ -168,6 +168,64 @@ public class FilesystemCommandParserTest {
assertEquals("/db1/table2.csv", command.getPaths().get(1));
}
+ @Test
+ public void parseJoinPathsUsesDefaultDelimiterAndFields() {
+ FilesystemCommand command =
+ FilesystemCommandParser.parse("join /db1/table1.csv /db1/table2.csv");
+
+ assertEquals(FilesystemCommand.Type.JOIN, command.getType());
+ assertEquals("", command.getOption());
+ assertEquals("1,1", command.getPattern());
+ assertEquals(2, command.getPaths().size());
+ assertEquals("/db1/table1.csv", command.getPaths().get(0));
+ assertEquals("/db1/table2.csv", command.getPaths().get(1));
+ }
+
+ @Test
+ public void parseJoinDelimiterAndFields() {
+ FilesystemCommand command =
+ FilesystemCommandParser.parse("join -t, -1 2 -2 1 /db1/table1.csv /db1/table2.csv");
+
+ assertEquals(FilesystemCommand.Type.JOIN, command.getType());
+ assertEquals(",", command.getOption());
+ assertEquals("2,1", command.getPattern());
+ assertEquals("/db1/table1.csv", command.getPaths().get(0));
+ assertEquals("/db1/table2.csv", command.getPaths().get(1));
+ }
+
+ @Test
+ public void parseJoinSeparatedDelimiter() {
+ FilesystemCommand command =
+ FilesystemCommandParser.parse("join -t , /db1/table1.csv /db1/table2.csv");
+
+ assertEquals(FilesystemCommand.Type.JOIN, command.getType());
+ assertEquals(",", command.getOption());
+ assertEquals("1,1", command.getPattern());
+ }
+
+ @Test
+ public void parseJoinRejectsInvalidArguments() {
+ assertEquals(FilesystemCommand.Type.INVALID, FilesystemCommandParser.parse("join").getType());
+ assertEquals(
+ FilesystemCommand.Type.INVALID,
+ FilesystemCommandParser.parse("join /db1/table1.csv").getType());
+ assertEquals(
+ FilesystemCommand.Type.INVALID,
+ FilesystemCommandParser.parse("join /db1/a.csv /db1/b.csv /db1/c.csv").getType());
+ assertEquals(
+ FilesystemCommand.Type.INVALID,
+ FilesystemCommandParser.parse("join -t /db1/a.csv /db1/b.csv").getType());
+ assertEquals(
+ FilesystemCommand.Type.INVALID,
+ FilesystemCommandParser.parse("join -t:: /db1/a.csv /db1/b.csv").getType());
+ assertEquals(
+ FilesystemCommand.Type.INVALID,
+ FilesystemCommandParser.parse("join -1 0 /db1/a.csv /db1/b.csv").getType());
+ assertEquals(
+ FilesystemCommand.Type.INVALID,
+ FilesystemCommandParser.parse("join -x /db1/a.csv /db1/b.csv").getType());
+ }
+
@Test
public void parseCutDelimiterFieldsAndPath() {
FilesystemCommand command = FilesystemCommandParser.parse("cut -d, -f2,3 /db1/table1.csv");
diff --git a/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/utils/JlineUtilsTest.java b/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/utils/JlineUtilsTest.java
index 6cc0fd9dd56..d8d2ac84da8 100644
--- a/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/utils/JlineUtilsTest.java
+++ b/iotdb-client/cli/src/test/java/org/apache/iotdb/cli/utils/JlineUtilsTest.java
@@ -51,6 +51,13 @@ public class JlineUtilsTest {
assertFalse(values.contains("CREATE"));
}
+ @Test
+ public void filesystemCompleterIncludesJoinCommand() {
+ List<String> values = complete(JlineUtils.createCompleter("filesystem"), "j");
+
+ assertTrue(values.contains("join"));
+ }
+
private static List<String> complete(Completer completer, String line) {
ParsedLine parsedLine = new DefaultParser().parse(line, line.length());
List<Candidate> candidates = new ArrayList<>();