You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@iotdb.apache.org by GitBox <gi...@apache.org> on 2021/03/25 08:28:34 UTC
[GitHub] [iotdb] HeimingZ opened a new issue #2905: Cannot write data into HDFS
HeimingZ opened a new issue #2905:
URL: https://github.com/apache/iotdb/issues/2905
**Describe the bug**
IoTDB cannot write data into HDFS because org.apache.iotdb.hadoop.fileSystem.HDFSFile doesn't implement toPath() method.
2021-03-25 16:12:59,934 [pool-1-IoTDB-Flush-ServerServiceImpl-thread-1] ERROR org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:873 - root.ln/0 meet error when flush FileMetadata to hdfs://localhost:9000/data/data/sequence/root.ln/0/0/1616659978569-1-0.tsfile, change system mode to read-only
java.lang.UnsupportedOperationException: Unsupported operation.
at org.apache.iotdb.hadoop.fileSystem.HDFSFile.toPath(HDFSFile.java:433)
at org.apache.iotdb.db.engine.storagegroup.TsFileResource.serialize(TsFileResource.java:257)
at org.apache.iotdb.db.engine.storagegroup.TsFileProcessor.endFile(TsFileProcessor.java:939)
at org.apache.iotdb.db.engine.storagegroup.TsFileProcessor.flushOneMemTable(TsFileProcessor.java:868)
at org.apache.iotdb.db.engine.flush.FlushManager$FlushThread.runMayThrow(FlushManager.java:102)
at org.apache.iotdb.db.concurrent.WrappedRunnable.run(WrappedRunnable.java:32)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2021-03-25 16:12:59,949 [pool-1-IoTDB-Flush-ServerServiceImpl-thread-1] ERROR org.apache.iotdb.db.engine.storagegroup.TsFileProcessor:888 - root.ln/0: 1616659978569-1-0.tsfile marking or ending file meet error
java.lang.UnsupportedOperationException: Unsupported operation.
at org.apache.iotdb.hadoop.fileSystem.HDFSFile.toPath(HDFSFile.java:433)
at org.apache.iotdb.db.engine.storagegroup.TsFileResource.serialize(TsFileResource.java:257)
at org.apache.iotdb.db.engine.storagegroup.TsFileProcessor.endFile(TsFileProcessor.java:939)
at org.apache.iotdb.db.engine.storagegroup.TsFileProcessor.flushOneMemTable(TsFileProcessor.java:868)
at org.apache.iotdb.db.engine.flush.FlushManager$FlushThread.runMayThrow(FlushManager.java:102)
at org.apache.iotdb.db.concurrent.WrappedRunnable.run(WrappedRunnable.java:32)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
**To Reproduce**
Steps to reproduce the behavior:
1. use HDFS to store data
2. write data
set storage group to root.ln
create timeseries root.ln.wf01.wt01.status with datatype=BOOLEAN,encoding=PLAIN
insert into root.ln.wf01.wt01(timestamp,status) values(1509465600000,true)
flush
3. see the log_error.log
**Expected behavior**
succeed in writing data
**Screenshots**
use java.nio package like org.apache.iotdb.db.engine.storagegroup.TsFileResource.serialize() will cause failure in hadoop
<img width="908" alt="截屏2021-03-25 16 24 21" src="https://user-images.githubusercontent.com/43991780/112442303-a945f380-8d86-11eb-9d8f-fe48863d0c25.png">
which is caused by
<img width="1025" alt="截屏2021-03-25 16 20 19" src="https://user-images.githubusercontent.com/43991780/112442352-b367f200-8d86-11eb-82b5-f53c6d89aedf.png">
**Possible solution**
Implements Path, FileSystem, FileSystemProvider like https://github.com/damiencarol/jsr203-hadoop or just reuse it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HTHou closed issue #2905: Cannot write data into HDFS
Posted by GitBox <gi...@apache.org>.
HTHou closed issue #2905:
URL: https://github.com/apache/iotdb/issues/2905
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] samperson1997 commented on issue #2905: Cannot write data into HDFS
Posted by GitBox <gi...@apache.org>.
samperson1997 commented on issue #2905:
URL: https://github.com/apache/iotdb/issues/2905#issuecomment-806465061
Hi, thanks for bringing up this issue. I’ll fix it ASAP: )
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] github-actions[bot] commented on issue #2905: Cannot write data into HDFS
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #2905:
URL: https://github.com/apache/iotdb/issues/2905#issuecomment-806634163
Hi, this is your first issue in IoTDB project. Thanks for your report. Welcome to join the community!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] samperson1997 commented on issue #2905: Cannot write data into HDFS
Posted by GitBox <gi...@apache.org>.
samperson1997 commented on issue #2905:
URL: https://github.com/apache/iotdb/issues/2905#issuecomment-808311182
Hi @HeimingZ , would you like to try and review the [PR which fixes this issue](https://github.com/apache/iotdb/pull/2912)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [iotdb] HeimingZ commented on issue #2905: Cannot write data into HDFS
Posted by GitBox <gi...@apache.org>.
HeimingZ commented on issue #2905:
URL: https://github.com/apache/iotdb/issues/2905#issuecomment-808623153
This PR fixes this issue, thx :) @samperson1997
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org