You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@calcite.apache.org by 勾王敏浩 <gw...@mails.tsinghua.edu.cn> on 2019/03/08 02:08:52 UTC
using calcite to read csv files from hdfs.
Hello, everyone.
This is Wangminhao Gou. I have an urgent need now. I want to read CSV files from remote HDFS and parse them using calcite. Now that I can use a file stream to get this CSV file, and understand how to use calcite to make a relational table of the local csv, what should I do next?
Looking forward to your reply.
Best,
Wangminhao Gou
Re: using calcite to read csv files from hdfs.
Posted by Andrei Sereda <an...@sereda.cc>.
Hi Gou,
CsvSchema uses java.io.File API to scan a directory and doesn't speak HDFS
directly (nor does it accept generic java.io.Reader to construct it).
Is there a way for you to mount HDFS using FUSE ? Example MountableHDFS
<https://wiki.apache.org/hadoop/MountableHDFS>.
Regards,
Andrei.
On Thu, Mar 7, 2019 at 9:10 PM 勾王敏浩 <gw...@mails.tsinghua.edu.cn> wrote:
> Hello, everyone.
> This is Wangminhao Gou. I have an urgent need now. I want to read CSV
> files from remote HDFS and parse them using calcite. Now that I can use a
> file stream to get this CSV file, and understand how to use calcite to make
> a relational table of the local csv, what should I do next?
> Looking forward to your reply.
> Best,
> Wangminhao Gou
Re: using calcite to read csv files from hdfs.
Posted by Albert <zi...@gmail.com>.
I would say calcite csv example is a good start point.
On Fri, Mar 8, 2019 at 10:10 AM 勾王敏浩 <gw...@mails.tsinghua.edu.cn> wrote:
> Hello, everyone.
> This is Wangminhao Gou. I have an urgent need now. I want to read CSV
> files from remote HDFS and parse them using calcite. Now that I can use a
> file stream to get this CSV file, and understand how to use calcite to make
> a relational table of the local csv, what should I do next?
> Looking forward to your reply.
> Best,
> Wangminhao Gou
--
~~~~~~~~~~~~~~~
no mistakes
~~~~~~~~~~~~~~~~~~
Re: using calcite to read csv files from hdfs.
Posted by YuZhao Chan <yu...@gmail.com>.
I think you need to define a AbstractShema and CsvTableFactory by your self. I might fire a JIRA and make the Csv to support HDFS.
Best,
YuZhao Chen
在 2019年3月8日 +0800 AM10:10，dev@calcite.apache.org，写道：
>
> CsvSchema