You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@accumulo.apache.org by "Roberts, Geoffry [USA]" <Ro...@bah.com> on 2021/04/21 14:56:47 UTC
Running Apache Accumulo on Amazon EMR
All,
I am confronted with running Accumulo on AWS. I found this: Running Apache Accumulo on Amazon EMR<https://aws.amazon.com/blogs/big-data/running-apache-accumulo-on-amazon-emr/>, which should be just what I need but it is five years old. It gives a recipe that implies that Accumulo can be installed from a staged place within AWS. Does anyone know if one could simply update the version numbers make a go of it?
I need Accumulo 2.0 and java 13+ these days. I find AWS to be quite silent lately wrt Accumulo, which raises a concern that the aforementioned recipe is now stale.
Thanks
Re: [External] Running Apache Accumulo on Amazon EMR
Posted by Brian Loss <br...@gmail.com>.
If you don’t have to use EMR specifically and can use EC2, then Muchos (https://github.com/apache/fluo-muchos) might help you.
> On Apr 21, 2021, at 3:27 PM, Roberts, Geoffry [USA] <Ro...@bah.com> wrote:
>
> Thanks Christopher,
>
> I was looking for an easy way to spin up an Accumulo instance(s), oh well. I'll take your advice and do on AWS what I did on Azure--roll my own HAZoo.
> I need at least jdk 13 because I *must* interop with Julia. Actually, I only need my java access library, which uses the Accumulo client to be at jdk 13 or better.
> I'll try running the DB at jdk 11 and the client at 13 and see how it goes.
>
> Cheers
>
> On 4/21/21, 12:56 PM, "Christopher" <ct...@apache.org> wrote:
>
> That article, and these questions, seem to be about AWS support of
> Accumulo versions using Amazon's EMR. It might be best to ask Amazon
> directly about what they do and don't support, since it's their recipe
> and their features. As I understand it, though, EMR is just a
> specialized distribution of Hadoop. You don't necessarily have to use
> EMR to run Hadoop in AWS. So, even if Amazon doesn't support newer
> Accumulo versions on EMR using the recipe you found, you could just
> try to run your own Hadoop instance on EC2.
>
> Regarding running Accumulo 2.0 and Java 13+: while we have tried to
> address many issues running Java 11 with Accumulo 2.0, it's possible
> there are some issues that weren't backported from 2.1. One issue, for
> example, is that 2.0 still defaults to using CMS instead of G1 for the
> Java garbage collector. CMS was deprecated and then removed in more
> recent versions of Java. 2.1 should have really good support for newer
> Java versions, though when it is released. Although it will require
> Java 11, I've tried to do development using the latest Java versions
> (currently using Java 16), to try to detect any new issues that might
> arise. Please let us know if you find any issues running newer Java
> versions. I definitely want to try to fix those if we can.
>
> On Wed, Apr 21, 2021 at 10:57 AM Roberts, Geoffry [USA]
> <Ro...@bah.com> wrote:
>>
>> All,
>>
>>
>>
>> I am confronted with running Accumulo on AWS. I found this: Running Apache Accumulo on Amazon EMR, which should be just what I need but it is five years old. It gives a recipe that implies that Accumulo can be installed from a staged place within AWS. Does anyone know if one could simply update the version numbers make a go of it?
>>
>>
>>
>> I need Accumulo 2.0 and java 13+ these days. I find AWS to be quite silent lately wrt Accumulo, which raises a concern that the aforementioned recipe is now stale.
>>
>>
>>
>> Thanks
>
Re: [External] Re: Running Apache Accumulo on Amazon EMR
Posted by "Roberts, Geoffry [USA]" <Ro...@bah.com>.
Thanks Christopher,
I was looking for an easy way to spin up an Accumulo instance(s), oh well. I'll take your advice and do on AWS what I did on Azure--roll my own HAZoo.
I need at least jdk 13 because I *must* interop with Julia. Actually, I only need my java access library, which uses the Accumulo client to be at jdk 13 or better.
I'll try running the DB at jdk 11 and the client at 13 and see how it goes.
Cheers
On 4/21/21, 12:56 PM, "Christopher" <ct...@apache.org> wrote:
That article, and these questions, seem to be about AWS support of
Accumulo versions using Amazon's EMR. It might be best to ask Amazon
directly about what they do and don't support, since it's their recipe
and their features. As I understand it, though, EMR is just a
specialized distribution of Hadoop. You don't necessarily have to use
EMR to run Hadoop in AWS. So, even if Amazon doesn't support newer
Accumulo versions on EMR using the recipe you found, you could just
try to run your own Hadoop instance on EC2.
Regarding running Accumulo 2.0 and Java 13+: while we have tried to
address many issues running Java 11 with Accumulo 2.0, it's possible
there are some issues that weren't backported from 2.1. One issue, for
example, is that 2.0 still defaults to using CMS instead of G1 for the
Java garbage collector. CMS was deprecated and then removed in more
recent versions of Java. 2.1 should have really good support for newer
Java versions, though when it is released. Although it will require
Java 11, I've tried to do development using the latest Java versions
(currently using Java 16), to try to detect any new issues that might
arise. Please let us know if you find any issues running newer Java
versions. I definitely want to try to fix those if we can.
On Wed, Apr 21, 2021 at 10:57 AM Roberts, Geoffry [USA]
<Ro...@bah.com> wrote:
>
> All,
>
>
>
> I am confronted with running Accumulo on AWS. I found this: Running Apache Accumulo on Amazon EMR, which should be just what I need but it is five years old. It gives a recipe that implies that Accumulo can be installed from a staged place within AWS. Does anyone know if one could simply update the version numbers make a go of it?
>
>
>
> I need Accumulo 2.0 and java 13+ these days. I find AWS to be quite silent lately wrt Accumulo, which raises a concern that the aforementioned recipe is now stale.
>
>
>
> Thanks
Re: Running Apache Accumulo on Amazon EMR
Posted by Christopher <ct...@apache.org>.
That article, and these questions, seem to be about AWS support of
Accumulo versions using Amazon's EMR. It might be best to ask Amazon
directly about what they do and don't support, since it's their recipe
and their features. As I understand it, though, EMR is just a
specialized distribution of Hadoop. You don't necessarily have to use
EMR to run Hadoop in AWS. So, even if Amazon doesn't support newer
Accumulo versions on EMR using the recipe you found, you could just
try to run your own Hadoop instance on EC2.
Regarding running Accumulo 2.0 and Java 13+: while we have tried to
address many issues running Java 11 with Accumulo 2.0, it's possible
there are some issues that weren't backported from 2.1. One issue, for
example, is that 2.0 still defaults to using CMS instead of G1 for the
Java garbage collector. CMS was deprecated and then removed in more
recent versions of Java. 2.1 should have really good support for newer
Java versions, though when it is released. Although it will require
Java 11, I've tried to do development using the latest Java versions
(currently using Java 16), to try to detect any new issues that might
arise. Please let us know if you find any issues running newer Java
versions. I definitely want to try to fix those if we can.
On Wed, Apr 21, 2021 at 10:57 AM Roberts, Geoffry [USA]
<Ro...@bah.com> wrote:
>
> All,
>
>
>
> I am confronted with running Accumulo on AWS. I found this: Running Apache Accumulo on Amazon EMR, which should be just what I need but it is five years old. It gives a recipe that implies that Accumulo can be installed from a staged place within AWS. Does anyone know if one could simply update the version numbers make a go of it?
>
>
>
> I need Accumulo 2.0 and java 13+ these days. I find AWS to be quite silent lately wrt Accumulo, which raises a concern that the aforementioned recipe is now stale.
>
>
>
> Thanks