[GitHub] [spark] MaxGekk commented on pull request #30358: [SPARK-33394][SQL] Throw `NoSuchNamespaceException` for not existing namespace in `InMemoryTableCatalog.listTables()`

[GitBox <gi...@apache.org>]

MaxGekk commented on pull request #30358:
URL: https://github.com/apache/spark/pull/30358#issuecomment-726577886


   The failed test is some kind of corner case. This guy
   https://github.com/apache/spark/blob/fcf8aa59b5025dde9b4af36953146894659967e2/sql/core/src/test/scala/org/apache/spark/sql/connector/SupportsCatalogOptionsSuite.scala#L64
   assumes that if it passes `Array.empty` to `listTables()`, the function should return all tables.
   
   This works fine if `allNamespaces`:
   https://github.com/apache/spark/blob/0bd7a3dfab41336dba2788a3d1fa3cf5b9f410d3/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/InMemoryTableCatalog.scala#L127
   is not empty. But when `allNamespaces` is empty, there is no element with prefix `Array.empty`, and `namespaceExists` returns false:
   ```scala
     override def namespaceExists(namespace: Array[String]): Boolean = {
       allNamespaces.exists(_.startsWith(namespace))
     }
   ```
   
   As a possible solution, I would handle `Array.empty` in `listTables()`, and return all tables like:
   ```scala
     override def listTables(namespace: Array[String]): Array[Identifier] = {
       if (namespace.isEmpty || namespaceExists(namespace)) {
         super.listTables(namespace)
       } else {
         throw new NoSuchNamespaceException(namespace)
       }
     }
   ```


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscribe@spark.apache.org
For additional commands, e-mail: reviews-help@spark.apache.org