Amazon EFS and EC2 based Drillbits

[Stefán Baxter <st...@activitystream.com>]

Hi,

Soon the Elastic File System will be available by Amazon that promises fast
access to "thousands of concurrent EC2 nodes" based on the NFSv4 protocol.

I know some people here have been using S3 as storage and the way I
understand it it's working so-so and hase the following ddrawbacks:


   - Does not take advantage of the columnar format of the Parquet files
   - S3 Being blob bases

   - Involves quite a bit of latency (file loading etc.)

Have you looked at this new offering and what do you think about it?

Do you think it stands a change of becoming a hdfs alternative for a
Parquet+Drill setup?

Any instance-type preferences for Drill on EC2?

We were looking at fronting S3 with Tachyon as an alternative but this
seems like a service that we should explore further.

Regards,
 -Stefan