You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Vova Shelgunov <vv...@gmail.com> on 2016/09/01 17:47:44 UTC
cassandra schema initialization in docker
Hi,
I wonder if anyone can suggest a way how to initialize application schema
to cassandra inside docker container (e.g. by executing cql file). Is there
a way?
Thanks,
Uladzimir
Re: cassandra schema initialization in docker
Posted by Patrick McFadin <pm...@gmail.com>.
Actually, I was advocating keeping the data directory outside of the
container and using docker volumes to mount the existing files. You would
add something like this to your docker run:
-v <local_host_dir>/data:/var/cassandra/data
As explained here: https://docs.docker.com/engine/tutorials/dockervolumes/
Create a "gold" copy of your data directory. Script the startup of the
container with the pre condition being the creation of the target data
directory on the host. One the container starts and c* is online, you will
have a clean set of data online and ready to go.
Patrick
On Thu, Sep 1, 2016 at 2:49 PM, Vova Shelgunov <vv...@gmail.com> wrote:
> Thank you. Maybe it does make sense to put these sstables inside docker
> image, what will I lose in this case?
>
> On Sep 2, 2016 12:28 AM, "Patrick McFadin" <pm...@gmail.com> wrote:
>
>> You can create a custom endpoint script that you will need to add with a
>> docker build. In that script you'll have to do do a wait loop for when c*
>> is up and running.
>>
>> Being a little more creative, you could pre-bake some sstables and then
>> use docker volumes to mount /data on a local file system. As the node
>> starts, it will index the files like you were restoring from a backup.
>>
>> Patrick
>>
>>
>>
>> On Thu, Sep 1, 2016 at 12:32 PM, Vova Shelgunov <vv...@gmail.com> wrote:
>>
>>> Thanks.
>>> As for production I will do it in another way, it is just for
>>> development purposes to provide an easy way to run application on local
>>> machine.
>>>
>>> On Sep 1, 2016 10:26 PM, "Jonathan Haddad" <jo...@jonhaddad.com> wrote:
>>>
>>>> I wouldn't recommend blindly executing CQL statements on startup in a
>>>> production cluster, IF NOT EXISTS or not. Those statements don't use LWT
>>>> or anything
>>>>
>>>> See https://issues.apache.org/jira/browse/CASSANDRA-8387
>>>>
>>>> On Thu, Sep 1, 2016 at 12:22 PM Michael Mior <mm...@uwaterloo.ca>
>>>> wrote:
>>>>
>>>>> This is really more of a Docker question than a Cassandra question but
>>>>> if you include the CQL file in your Docker image, you could just change the
>>>>> CMD line in your Dockerfile to run the script after starting Cassandra. You
>>>>> would probably need to add a delay and some retries to ensure the server
>>>>> has finished starting.
>>>>>
>>>>> --
>>>>> Michael Mior
>>>>> michael.mior@gmail.com
>>>>>
>>>>> 2016-09-01 14:45 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>>
>>>>>> I accept both cases. Second will work because I use create if not
>>>>>> exists.
>>>>>>
>>>>>> 2016-09-01 21:02 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>>>>
>>>>>>> I'm not sure I understand what you're trying to do. Do you want this
>>>>>>> to be executed once when the container is built or every time the container
>>>>>>> is started?
>>>>>>>
>>>>>>> --
>>>>>>> Michael Mior
>>>>>>> michael.mior@gmail.com
>>>>>>>
>>>>>>> 2016-09-01 13:57 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>>>>
>>>>>>>> Sorry, I did not specify, that I need to execute cql right
>>>>>>>> after cassandra container start.
>>>>>>>>
>>>>>>>> 2016-09-01 20:52 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>>>>>>
>>>>>>>>> You should just be able to connect to the Cassandra instance and
>>>>>>>>> execute CQL as you would against any other Cassandra installation. Any
>>>>>>>>> applications wishing to use the Cassandra instance inside the container
>>>>>>>>> will require the port to be exposed somehow anyway.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Michael Mior
>>>>>>>>> michael.mior@gmail.com
>>>>>>>>>
>>>>>>>>> 2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I wonder if anyone can suggest a way how to initialize
>>>>>>>>>> application schema to cassandra inside docker container (e.g. by executing
>>>>>>>>>> cql file). Is there a way?
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> Uladzimir
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>
Re: cassandra schema initialization in docker
Posted by Vova Shelgunov <vv...@gmail.com>.
Thank you. Maybe it does make sense to put these sstables inside docker
image, what will I lose in this case?
On Sep 2, 2016 12:28 AM, "Patrick McFadin" <pm...@gmail.com> wrote:
> You can create a custom endpoint script that you will need to add with a
> docker build. In that script you'll have to do do a wait loop for when c*
> is up and running.
>
> Being a little more creative, you could pre-bake some sstables and then
> use docker volumes to mount /data on a local file system. As the node
> starts, it will index the files like you were restoring from a backup.
>
> Patrick
>
>
>
> On Thu, Sep 1, 2016 at 12:32 PM, Vova Shelgunov <vv...@gmail.com> wrote:
>
>> Thanks.
>> As for production I will do it in another way, it is just for development
>> purposes to provide an easy way to run application on local machine.
>>
>> On Sep 1, 2016 10:26 PM, "Jonathan Haddad" <jo...@jonhaddad.com> wrote:
>>
>>> I wouldn't recommend blindly executing CQL statements on startup in a
>>> production cluster, IF NOT EXISTS or not. Those statements don't use LWT
>>> or anything
>>>
>>> See https://issues.apache.org/jira/browse/CASSANDRA-8387
>>>
>>> On Thu, Sep 1, 2016 at 12:22 PM Michael Mior <mm...@uwaterloo.ca> wrote:
>>>
>>>> This is really more of a Docker question than a Cassandra question but
>>>> if you include the CQL file in your Docker image, you could just change the
>>>> CMD line in your Dockerfile to run the script after starting Cassandra. You
>>>> would probably need to add a delay and some retries to ensure the server
>>>> has finished starting.
>>>>
>>>> --
>>>> Michael Mior
>>>> michael.mior@gmail.com
>>>>
>>>> 2016-09-01 14:45 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>
>>>>> I accept both cases. Second will work because I use create if not
>>>>> exists.
>>>>>
>>>>> 2016-09-01 21:02 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>>>
>>>>>> I'm not sure I understand what you're trying to do. Do you want this
>>>>>> to be executed once when the container is built or every time the container
>>>>>> is started?
>>>>>>
>>>>>> --
>>>>>> Michael Mior
>>>>>> michael.mior@gmail.com
>>>>>>
>>>>>> 2016-09-01 13:57 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>>>
>>>>>>> Sorry, I did not specify, that I need to execute cql right
>>>>>>> after cassandra container start.
>>>>>>>
>>>>>>> 2016-09-01 20:52 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>>>>>
>>>>>>>> You should just be able to connect to the Cassandra instance and
>>>>>>>> execute CQL as you would against any other Cassandra installation. Any
>>>>>>>> applications wishing to use the Cassandra instance inside the container
>>>>>>>> will require the port to be exposed somehow anyway.
>>>>>>>>
>>>>>>>> --
>>>>>>>> Michael Mior
>>>>>>>> michael.mior@gmail.com
>>>>>>>>
>>>>>>>> 2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I wonder if anyone can suggest a way how to initialize application
>>>>>>>>> schema to cassandra inside docker container (e.g. by executing cql file).
>>>>>>>>> Is there a way?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Uladzimir
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>
Re: cassandra schema initialization in docker
Posted by Patrick McFadin <pm...@gmail.com>.
You can create a custom endpoint script that you will need to add with a
docker build. In that script you'll have to do do a wait loop for when c*
is up and running.
Being a little more creative, you could pre-bake some sstables and then use
docker volumes to mount /data on a local file system. As the node starts,
it will index the files like you were restoring from a backup.
Patrick
On Thu, Sep 1, 2016 at 12:32 PM, Vova Shelgunov <vv...@gmail.com> wrote:
> Thanks.
> As for production I will do it in another way, it is just for development
> purposes to provide an easy way to run application on local machine.
>
> On Sep 1, 2016 10:26 PM, "Jonathan Haddad" <jo...@jonhaddad.com> wrote:
>
>> I wouldn't recommend blindly executing CQL statements on startup in a
>> production cluster, IF NOT EXISTS or not. Those statements don't use LWT
>> or anything
>>
>> See https://issues.apache.org/jira/browse/CASSANDRA-8387
>>
>> On Thu, Sep 1, 2016 at 12:22 PM Michael Mior <mm...@uwaterloo.ca> wrote:
>>
>>> This is really more of a Docker question than a Cassandra question but
>>> if you include the CQL file in your Docker image, you could just change the
>>> CMD line in your Dockerfile to run the script after starting Cassandra. You
>>> would probably need to add a delay and some retries to ensure the server
>>> has finished starting.
>>>
>>> --
>>> Michael Mior
>>> michael.mior@gmail.com
>>>
>>> 2016-09-01 14:45 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>
>>>> I accept both cases. Second will work because I use create if not
>>>> exists.
>>>>
>>>> 2016-09-01 21:02 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>>
>>>>> I'm not sure I understand what you're trying to do. Do you want this
>>>>> to be executed once when the container is built or every time the container
>>>>> is started?
>>>>>
>>>>> --
>>>>> Michael Mior
>>>>> michael.mior@gmail.com
>>>>>
>>>>> 2016-09-01 13:57 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>>
>>>>>> Sorry, I did not specify, that I need to execute cql right
>>>>>> after cassandra container start.
>>>>>>
>>>>>> 2016-09-01 20:52 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>>>>
>>>>>>> You should just be able to connect to the Cassandra instance and
>>>>>>> execute CQL as you would against any other Cassandra installation. Any
>>>>>>> applications wishing to use the Cassandra instance inside the container
>>>>>>> will require the port to be exposed somehow anyway.
>>>>>>>
>>>>>>> --
>>>>>>> Michael Mior
>>>>>>> michael.mior@gmail.com
>>>>>>>
>>>>>>> 2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I wonder if anyone can suggest a way how to initialize application
>>>>>>>> schema to cassandra inside docker container (e.g. by executing cql file).
>>>>>>>> Is there a way?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Uladzimir
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
Re: cassandra schema initialization in docker
Posted by Vova Shelgunov <vv...@gmail.com>.
Thanks.
As for production I will do it in another way, it is just for development
purposes to provide an easy way to run application on local machine.
On Sep 1, 2016 10:26 PM, "Jonathan Haddad" <jo...@jonhaddad.com> wrote:
> I wouldn't recommend blindly executing CQL statements on startup in a
> production cluster, IF NOT EXISTS or not. Those statements don't use LWT
> or anything
>
> See https://issues.apache.org/jira/browse/CASSANDRA-8387
>
> On Thu, Sep 1, 2016 at 12:22 PM Michael Mior <mm...@uwaterloo.ca> wrote:
>
>> This is really more of a Docker question than a Cassandra question but if
>> you include the CQL file in your Docker image, you could just change the
>> CMD line in your Dockerfile to run the script after starting Cassandra. You
>> would probably need to add a delay and some retries to ensure the server
>> has finished starting.
>>
>> --
>> Michael Mior
>> michael.mior@gmail.com
>>
>> 2016-09-01 14:45 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>
>>> I accept both cases. Second will work because I use create if not exists.
>>>
>>> 2016-09-01 21:02 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>
>>>> I'm not sure I understand what you're trying to do. Do you want this to
>>>> be executed once when the container is built or every time the container is
>>>> started?
>>>>
>>>> --
>>>> Michael Mior
>>>> michael.mior@gmail.com
>>>>
>>>> 2016-09-01 13:57 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>
>>>>> Sorry, I did not specify, that I need to execute cql right
>>>>> after cassandra container start.
>>>>>
>>>>> 2016-09-01 20:52 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>>>
>>>>>> You should just be able to connect to the Cassandra instance and
>>>>>> execute CQL as you would against any other Cassandra installation. Any
>>>>>> applications wishing to use the Cassandra instance inside the container
>>>>>> will require the port to be exposed somehow anyway.
>>>>>>
>>>>>> --
>>>>>> Michael Mior
>>>>>> michael.mior@gmail.com
>>>>>>
>>>>>> 2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I wonder if anyone can suggest a way how to initialize application
>>>>>>> schema to cassandra inside docker container (e.g. by executing cql file).
>>>>>>> Is there a way?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Uladzimir
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Re: cassandra schema initialization in docker
Posted by Jonathan Haddad <jo...@jonhaddad.com>.
I wouldn't recommend blindly executing CQL statements on startup in a
production cluster, IF NOT EXISTS or not. Those statements don't use LWT
or anything
See https://issues.apache.org/jira/browse/CASSANDRA-8387
On Thu, Sep 1, 2016 at 12:22 PM Michael Mior <mm...@uwaterloo.ca> wrote:
> This is really more of a Docker question than a Cassandra question but if
> you include the CQL file in your Docker image, you could just change the
> CMD line in your Dockerfile to run the script after starting Cassandra. You
> would probably need to add a delay and some retries to ensure the server
> has finished starting.
>
> --
> Michael Mior
> michael.mior@gmail.com
>
> 2016-09-01 14:45 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>
>> I accept both cases. Second will work because I use create if not exists.
>>
>> 2016-09-01 21:02 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>
>>> I'm not sure I understand what you're trying to do. Do you want this to
>>> be executed once when the container is built or every time the container is
>>> started?
>>>
>>> --
>>> Michael Mior
>>> michael.mior@gmail.com
>>>
>>> 2016-09-01 13:57 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>
>>>> Sorry, I did not specify, that I need to execute cql right
>>>> after cassandra container start.
>>>>
>>>> 2016-09-01 20:52 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>>
>>>>> You should just be able to connect to the Cassandra instance and
>>>>> execute CQL as you would against any other Cassandra installation. Any
>>>>> applications wishing to use the Cassandra instance inside the container
>>>>> will require the port to be exposed somehow anyway.
>>>>>
>>>>> --
>>>>> Michael Mior
>>>>> michael.mior@gmail.com
>>>>>
>>>>> 2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I wonder if anyone can suggest a way how to initialize application
>>>>>> schema to cassandra inside docker container (e.g. by executing cql file).
>>>>>> Is there a way?
>>>>>>
>>>>>> Thanks,
>>>>>> Uladzimir
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: cassandra schema initialization in docker
Posted by Michael Mior <mm...@uwaterloo.ca>.
This is really more of a Docker question than a Cassandra question but if
you include the CQL file in your Docker image, you could just change the
CMD line in your Dockerfile to run the script after starting Cassandra. You
would probably need to add a delay and some retries to ensure the server
has finished starting.
--
Michael Mior
michael.mior@gmail.com
2016-09-01 14:45 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
> I accept both cases. Second will work because I use create if not exists.
>
> 2016-09-01 21:02 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>
>> I'm not sure I understand what you're trying to do. Do you want this to
>> be executed once when the container is built or every time the container is
>> started?
>>
>> --
>> Michael Mior
>> michael.mior@gmail.com
>>
>> 2016-09-01 13:57 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>
>>> Sorry, I did not specify, that I need to execute cql right
>>> after cassandra container start.
>>>
>>> 2016-09-01 20:52 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>>
>>>> You should just be able to connect to the Cassandra instance and
>>>> execute CQL as you would against any other Cassandra installation. Any
>>>> applications wishing to use the Cassandra instance inside the container
>>>> will require the port to be exposed somehow anyway.
>>>>
>>>> --
>>>> Michael Mior
>>>> michael.mior@gmail.com
>>>>
>>>> 2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>>
>>>>> Hi,
>>>>>
>>>>> I wonder if anyone can suggest a way how to initialize application
>>>>> schema to cassandra inside docker container (e.g. by executing cql file).
>>>>> Is there a way?
>>>>>
>>>>> Thanks,
>>>>> Uladzimir
>>>>>
>>>>
>>>>
>>>
>>
>
Re: cassandra schema initialization in docker
Posted by Vova Shelgunov <vv...@gmail.com>.
I accept both cases. Second will work because I use create if not exists.
2016-09-01 21:02 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
> I'm not sure I understand what you're trying to do. Do you want this to be
> executed once when the container is built or every time the container is
> started?
>
> --
> Michael Mior
> michael.mior@gmail.com
>
> 2016-09-01 13:57 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>
>> Sorry, I did not specify, that I need to execute cql right
>> after cassandra container start.
>>
>> 2016-09-01 20:52 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>>
>>> You should just be able to connect to the Cassandra instance and execute
>>> CQL as you would against any other Cassandra installation. Any applications
>>> wishing to use the Cassandra instance inside the container will require the
>>> port to be exposed somehow anyway.
>>>
>>> --
>>> Michael Mior
>>> michael.mior@gmail.com
>>>
>>> 2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>>
>>>> Hi,
>>>>
>>>> I wonder if anyone can suggest a way how to initialize application
>>>> schema to cassandra inside docker container (e.g. by executing cql file).
>>>> Is there a way?
>>>>
>>>> Thanks,
>>>> Uladzimir
>>>>
>>>
>>>
>>
>
Re: cassandra schema initialization in docker
Posted by Michael Mior <mm...@uwaterloo.ca>.
I'm not sure I understand what you're trying to do. Do you want this to be
executed once when the container is built or every time the container is
started?
--
Michael Mior
michael.mior@gmail.com
2016-09-01 13:57 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
> Sorry, I did not specify, that I need to execute cql right
> after cassandra container start.
>
> 2016-09-01 20:52 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
>
>> You should just be able to connect to the Cassandra instance and execute
>> CQL as you would against any other Cassandra installation. Any applications
>> wishing to use the Cassandra instance inside the container will require the
>> port to be exposed somehow anyway.
>>
>> --
>> Michael Mior
>> michael.mior@gmail.com
>>
>> 2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>>
>>> Hi,
>>>
>>> I wonder if anyone can suggest a way how to initialize application
>>> schema to cassandra inside docker container (e.g. by executing cql file).
>>> Is there a way?
>>>
>>> Thanks,
>>> Uladzimir
>>>
>>
>>
>
Re: cassandra schema initialization in docker
Posted by Vova Shelgunov <vv...@gmail.com>.
Sorry, I did not specify, that I need to execute cql right
after cassandra container start.
2016-09-01 20:52 GMT+03:00 Michael Mior <mm...@uwaterloo.ca>:
> You should just be able to connect to the Cassandra instance and execute
> CQL as you would against any other Cassandra installation. Any applications
> wishing to use the Cassandra instance inside the container will require the
> port to be exposed somehow anyway.
>
> --
> Michael Mior
> michael.mior@gmail.com
>
> 2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
>
>> Hi,
>>
>> I wonder if anyone can suggest a way how to initialize application schema
>> to cassandra inside docker container (e.g. by executing cql file). Is there
>> a way?
>>
>> Thanks,
>> Uladzimir
>>
>
>
Re: cassandra schema initialization in docker
Posted by Michael Mior <mm...@uwaterloo.ca>.
You should just be able to connect to the Cassandra instance and execute
CQL as you would against any other Cassandra installation. Any applications
wishing to use the Cassandra instance inside the container will require the
port to be exposed somehow anyway.
--
Michael Mior
michael.mior@gmail.com
2016-09-01 13:47 GMT-04:00 Vova Shelgunov <vv...@gmail.com>:
> Hi,
>
> I wonder if anyone can suggest a way how to initialize application schema
> to cassandra inside docker container (e.g. by executing cql file). Is there
> a way?
>
> Thanks,
> Uladzimir
>