You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@drill.apache.org by "egasimov (via GitHub)" <gi...@apache.org> on 2024/09/10 11:52:14 UTC

[I] DATA_WRITE ERROR: No FileSystem for scheme "null" (drill)

egasimov opened a new issue, #2940:
URL: https://github.com/apache/drill/issues/2940

   Hello Drill community,
   Recently, we have encountered with the following issue(` DATA_WRITE ERROR: No FileSystem for scheme "null"`), when writing query result into S3 through Apache drill.
   
   We have checked two ways.
   1. Write data thorugh s3 storage plugin configured from Apache drill web UI.
   2. Mounted to VM's local directory via s3fs protocol.
   and getting the above exception: `<No FileSystem for scheme "null">`
   
   P:S We are using `Flashblade//S3` as hardware.
   Workaround solution for now, to write the data to local VM's `/tmp` directory, and later move them to mounted(`through s3fs`) directory.
   
   **Executed queries:**
   ```
   CREATE TABLE dfs.root.`/datas3/joined-data/join_data.parquet` 
   AS 
   (
       SELECT *
       FROM dfs.root.`/datas3/customers/*` d
   );
   
   CREATE TABLE s3.root.`/datas3/joined-data/join_data.parquet` 
   AS 
   (
       SELECT *
       FROM dfs.root.`/datas3/customers/*` d
   );
   
   ```
   
   **Error details:**
   ```
   -------------------------------------------------------------------------------------------------------
   org.apache.drill.common.exceptions.UserRemoteException: DATA_WRITE ERROR: No FileSystem for scheme "null"
   
   Failure when writing the batch
   Fragment: 0:0
   
   [Error Id: bbecb92a-9fad-4774-b3c6-9d940150883a on 8a288c59e4a7:31010]
   ------------------------------------------------------------------------
   
   ```
     
   **Drill version**
   Apache drill version: 1.21.2
   
   **Additional context**
   Test data
   
   ```
   [
       {
       "customer_id": 1000001,
       "customer_name": "John Doe",
       "purchased_items": [
           {
               "item_id": 2000001,
               "item_class": "A",
               "product_id": 777,
               "created_at": "2024-06-12T11:36:37.751Z"
           },
           {
               "item_id": 2000002,
               "item_class": "B",
               "product_id": 888,
               "created_at": "2024-06-12T08:46:37.751Z"
           },
           {
               "item_id": 2000003,  # Corrected item_id for uniqueness
               "item_class": "C",
               "product_id": 999,
               "created_at": "2024-06-12T11:56:00.751Z"
           }
       ]
       },
       {
       "customer_id": 1000002,
       "customer_name": "Black Smith",
       "purchased_items": [
           {
               "item_id": 2000004,
               "item_class": "A",
               "product_id": 777,
               "created_at": "2024-06-12T11:36:33.751Z"
           },
           {
               "item_id": 2000006,  # Corrected item_id for uniqueness
               "item_class": "C",
               "product_id": 999,
               "created_at": "2024-08-12T11:56:37.751Z"
           }
       ]
       },
           {
       "customer_id": 1000003,
       "customer_name": "Alice Doe",
       "purchased_items": [
           {
               "item_id": 2000010,
               "item_class": "A",
               "product_id": 777,
               "created_at": "2024-01-12T11:36:37.751Z"
           },
           {
               "item_id": 2000011,  # Corrected item_id for uniqueness
               "item_class": "C",
               "product_id": 888,
               "created_at": "2024-04-12T11:56:37.751Z"
           }
       ]
       }
   ]
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DATA_WRITE ERROR: No FileSystem for scheme "null" (drill)

Posted by "egasimov (via GitHub)" <gi...@apache.org>.
egasimov closed issue #2940:  DATA_WRITE ERROR: No FileSystem for scheme "null"
URL: https://github.com/apache/drill/issues/2940


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DATA_WRITE ERROR: No FileSystem for scheme "null" (drill)

Posted by "egasimov (via GitHub)" <gi...@apache.org>.
egasimov commented on issue #2940:
URL: https://github.com/apache/drill/issues/2940#issuecomment-2340662406

   In s3 storage plugin, after making `"writable"` param from `/` to `/any-subdirectory`, it worked well.
   ```
     "workspaces": {
       "root": {
         "location": "/save",
         "writable": true,
         "defaultInputFormat": null,
         "allowAccessOutsideWorkspace": false
       }
     },
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DATA_WRITE ERROR: No FileSystem for scheme "null" (drill)

Posted by "cgivre (via GitHub)" <gi...@apache.org>.
cgivre commented on issue #2940:
URL: https://github.com/apache/drill/issues/2940#issuecomment-2340785892

   Have you tried 
   ```sql
   CREATE TABLE s3.root.`/datas3/joined-data/join_data`
   ... 
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DATA_WRITE ERROR: No FileSystem for scheme "null" (drill)

Posted by "egasimov (via GitHub)" <gi...@apache.org>.
egasimov commented on issue #2940:
URL: https://github.com/apache/drill/issues/2940#issuecomment-2340878456

   @cgivre yes, we have checked it as well.
   
   In `s3` storage plugin, after making "writable" param from `/` to `/any-subdirectory`, and then it worked well.
   ```
     "workspaces": {
       "root": {
         "location": "/save",
         "writable": true,
         "defaultInputFormat": null,
         "allowAccessOutsideWorkspace": false
       }
     },
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] DATA_WRITE ERROR: No FileSystem for scheme "null" (drill)

Posted by "egasimov (via GitHub)" <gi...@apache.org>.
egasimov closed issue #2940:  DATA_WRITE ERROR: No FileSystem for scheme "null"
URL: https://github.com/apache/drill/issues/2940


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@drill.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org