You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@druid.apache.org by Clint Wylie <cw...@apache.org> on 2021/05/14 19:42:26 UTC
[VOTE] Release Apache Druid 0.21.1 [RC1]
Hi all,
I have created a build for Apache Druid 0.21.1, release
candidate 1.
Thanks to everyone who has helped contribute to the release! You can read
the proposed release notes here:
https://github.com/apache/druid/issues/11249
The release candidate has been tagged in GitHub as
druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
available here:
https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
The artifacts to be voted on are located here:
https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
A staged Maven repository is available for review at:
https://repository.apache.org/content/repositories/orgapachedruid-1024/
Staged druid.apache.org website documentation is available here:
https://druid.staged.apache.org/docs/0.21.1/design/index.html
A Docker image containing the binary of the release candidate can be
retrieved via:
docker pull apache/druid:0.21.1-rc1
artifact checksums
src:
8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
bin:
170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
docker: 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/cwylie.asc
This key and the key of other committers can also be found in the project's
KEYS file here:
https://dist.apache.org/repos/dist/release/druid/KEYS
(If you are a committer, please feel free to add your own key to that file
by following the instructions in the file's header.)
Verify checksums:
diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
cut -d ' ' -f1) \
<(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
Verify signatures:
gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
apache-druid-0.21.1-src.tar.gz
gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
apache-druid-0.21.1-bin.tar.gz
Please review the proposed artifacts and vote. Note that Apache has
specific requirements that must be met before +1 binding votes can be cast
by PMC members. Please refer to the policy at
http://www.apache.org/legal/release-policy.html#policy for more details.
As part of the validation process, the release artifacts can be generated
from source by running:
mvn clean install -Papache-release,dist -Dgpg.skip
The RAT license check can be run from source by:
mvn apache-rat:check -Prat
This vote will be open for at least 72 hours. The vote will pass if a
majority of at least three +1 PMC votes are cast.
[ ] +1 Release this package as Apache Druid 0.21.1
[ ] 0 I don't feel strongly about it, but I'm okay with the release
[ ] -1 Do not release this package because...
Thanks!
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Clint Wylie <cw...@apache.org>.
Thanks for the votes, though I think since
https://github.com/apache/druid/pull/11299 appears to fix the issue
described in this thread that I should go ahead and cancel this vote and do
an RC2. I'll try to get new artifacts prepared and a vote up soon.
On Fri, May 28, 2021 at 7:49 AM Lucas Capistrant <ca...@gmail.com>
wrote:
> +1 (binding)
>
> src
> - Proper NOTICE and LICENSE files present
> - Verified .asc signatures and .sha512 checksums
> - git.version file is present and correct
> - mvn build passed with unit tests
> - mvn rat checks passed
> - ran cluster with built binaries and executed native batch ingestion
> followed by basic queries and api issued kill task on data. Tested with
> HDFS deep store.
>
> bin
> - Proper NOTICE and LICENSE files present
> - Verified .asc signatures and .sha512 checksums
> - ran cluster with pre-packaged binaries and executed native batch
> ingestion followed by basic queries and api issued kill task on data.
> Tested with HDFS deep store.
>
> On Fri, May 14, 2021 at 2:42 PM Clint Wylie <cw...@apache.org> wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Druid 0.21.1, release
> > candidate 1.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/11249
> >
> > The release candidate has been tagged in GitHub as
> > druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1024/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/0.21.1/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:0.21.1-rc1
> >
> > artifact checksums
> > src:
> >
> >
> 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> > bin:
> >
> >
> 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> > docker: 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/cwylie.asc
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> > apache-druid-0.21.1-src.tar.gz
> >
> > gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> > apache-druid-0.21.1-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 0.21.1
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Lucas Capistrant <ca...@gmail.com>.
+1 (binding)
src
- Proper NOTICE and LICENSE files present
- Verified .asc signatures and .sha512 checksums
- git.version file is present and correct
- mvn build passed with unit tests
- mvn rat checks passed
- ran cluster with built binaries and executed native batch ingestion
followed by basic queries and api issued kill task on data. Tested with
HDFS deep store.
bin
- Proper NOTICE and LICENSE files present
- Verified .asc signatures and .sha512 checksums
- ran cluster with pre-packaged binaries and executed native batch
ingestion followed by basic queries and api issued kill task on data.
Tested with HDFS deep store.
On Fri, May 14, 2021 at 2:42 PM Clint Wylie <cw...@apache.org> wrote:
> Hi all,
>
> I have created a build for Apache Druid 0.21.1, release
> candidate 1.
>
> Thanks to everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/11249
>
> The release candidate has been tagged in GitHub as
> druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> available here:
> https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachedruid-1024/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/0.21.1/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:0.21.1-rc1
>
> artifact checksums
> src:
>
> 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> bin:
>
> 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> docker: 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/cwylie.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> apache-druid-0.21.1-src.tar.gz
>
> gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> apache-druid-0.21.1-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Druid 0.21.1
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Xavier Léauté <xa...@confluent.io.INVALID>.
>
> Let's collect all the platforms that have reported having problems, create
> a table for it at GitHub, and check whether they have fixed or not?
>
I agree we should file individual docker issues *with detailed environment
and steps to reproduce*.
However, I don't think this should be discussed in the release thread, and
merits its own thread.
The goal of the bugfix release should not be to fix every possible docker
bug.
We should focus on fixing regressions that prevent users from upgrading.
If there are bugs that were already present in 0.20 or earlier, we should
not block the release on those.
> I have a question, does the problem reported by Clint could be reproduced
> > when executing the k8s integration test ?
>
I'm not sure, Clint could you look into this?
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi Frank,
Let's collect all the platforms that have reported having problems, create
a table for it at GitHub, and check whether they have fixed or not?
Kind Regards,
Furkan KAMACI
On Tue, May 25, 2021 at 2:07 PM frank chen <fr...@apache.org> wrote:
> Hi Xavier,
>
> I have a question, does the problem reported by Clint could be reproduced
> when executing the k8s integration test ?
>
> Thanks.
>
> Xavier Léauté <xa...@confluent.io.invalid> 于2021年5月21日周五 下午11:41写道:
>
> > > I agree with you that the root cause is lack of some automated tests
> > > against released docker images.
> > >
> >
> > Since https://github.com/apache/druid/pull/11187 we are now running k8s
> > integration tests against the same image we release.
> > So this should at least catch the kind of issues people would encounter
> > with a typical k8s deployment.
> >
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by frank chen <fr...@apache.org>.
Hi Xavier,
I have a question, does the problem reported by Clint could be reproduced
when executing the k8s integration test ?
Thanks.
Xavier Léauté <xa...@confluent.io.invalid> 于2021年5月21日周五 下午11:41写道:
> > I agree with you that the root cause is lack of some automated tests
> > against released docker images.
> >
>
> Since https://github.com/apache/druid/pull/11187 we are now running k8s
> integration tests against the same image we release.
> So this should at least catch the kind of issues people would encounter
> with a typical k8s deployment.
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by frank chen <fr...@apache.org>.
Hi all,
Whether or not we release the RC1, we have to resolve the issues ASAP. I
have submitted a PR https://github.com/apache/druid/pull/11299 , and I'm
still testing these changes on different platforms(macOS and centos). But
since the reported problems are platform-related, any help to verify that
change is appreciated.
Thanks.
frank chen <fr...@apache.org> 于2021年5月25日周二 下午6:47写道:
>
> Do we see Unable to use AWS Fargate bind mounts with Dockerfile · Issue
> #11278 · apache/druid (github.com)
> <https://github.com/apache/druid/issues/11278> as a regression test
> failure for https://github.com/apache/druid/pull/11167
>
>
> Furkan KAMACI <fu...@gmail.com> 于2021年5月25日周二 下午3:31写道:
>
>> Hi,
>>
>> Here is my opinion:
>>
>> Druid has had problems with docker for a while now. We use Druid on a
>> dockerized environment and manually solve this problem via setting
>> permissions. It is not a blocker issue, and we can release the artifact.
>>
>> However, this is a bugfix release that aims to cover Docker volume issues
>> (whether directly related to the discussed problem or not); we should
>> solve
>> this problem and then start a new RC spin. Otherwise, it seems that this
>> conversation will take more days to finalize (it has been 11 days since
>> the
>> vote thread has started), and we will not be able to release it soon.
>> Solving that issue may result in faster release.
>>
>> So, I am -0.
>>
>> Kind Regards,
>> Furkan KAMACI
>>
>> On Tue, May 25, 2021 at 10:06 AM Clint Wylie <cw...@apache.org> wrote:
>>
>> > I agree that this probably isn't a blocker, but also agree based on the
>> > number of reports that we should consider either fixing so that the
>> > docker-compose quickstart instructions work as is, or adjust the docs so
>> > that the instructions at least include something about setting the
>> > permissions so that things can work correctly. Given that I still don't
>> > have enough PMC votes yet to release 0.21.1, and I don't really have a
>> > strong preference of which thing we do (nor am quite sure on what the
>> > correct docker fix is), anyone have opinions on the best path forward?
>> >
>> > On Sat, May 22, 2021 at 3:56 PM Jihoon Son <ji...@apache.org>
>> wrote:
>> >
>> > > Xavier, that's good to know! Glad we already have some.
>> > >
>> > > On Fri, May 21, 2021 at 8:41 AM Xavier Léauté
>> > > <xa...@confluent.io.invalid> wrote:
>> > > >
>> > > > > I agree with you that the root cause is lack of some automated
>> tests
>> > > > > against released docker images.
>> > > > >
>> > > >
>> > > > Since https://github.com/apache/druid/pull/11187 we are now running
>> > k8s
>> > > > integration tests against the same image we release.
>> > > > So this should at least catch the kind of issues people would
>> encounter
>> > > > with a typical k8s deployment.
>> > >
>> > > ---------------------------------------------------------------------
>> > > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
>> > > For additional commands, e-mail: dev-help@druid.apache.org
>> > >
>> > >
>> >
>>
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by frank chen <fr...@apache.org>.
Do we see Unable to use AWS Fargate bind mounts with Dockerfile · Issue
#11278 · apache/druid (github.com)
<https://github.com/apache/druid/issues/11278> as a regression test
failure for https://github.com/apache/druid/pull/11167
Furkan KAMACI <fu...@gmail.com> 于2021年5月25日周二 下午3:31写道:
> Hi,
>
> Here is my opinion:
>
> Druid has had problems with docker for a while now. We use Druid on a
> dockerized environment and manually solve this problem via setting
> permissions. It is not a blocker issue, and we can release the artifact.
>
> However, this is a bugfix release that aims to cover Docker volume issues
> (whether directly related to the discussed problem or not); we should solve
> this problem and then start a new RC spin. Otherwise, it seems that this
> conversation will take more days to finalize (it has been 11 days since the
> vote thread has started), and we will not be able to release it soon.
> Solving that issue may result in faster release.
>
> So, I am -0.
>
> Kind Regards,
> Furkan KAMACI
>
> On Tue, May 25, 2021 at 10:06 AM Clint Wylie <cw...@apache.org> wrote:
>
> > I agree that this probably isn't a blocker, but also agree based on the
> > number of reports that we should consider either fixing so that the
> > docker-compose quickstart instructions work as is, or adjust the docs so
> > that the instructions at least include something about setting the
> > permissions so that things can work correctly. Given that I still don't
> > have enough PMC votes yet to release 0.21.1, and I don't really have a
> > strong preference of which thing we do (nor am quite sure on what the
> > correct docker fix is), anyone have opinions on the best path forward?
> >
> > On Sat, May 22, 2021 at 3:56 PM Jihoon Son <ji...@apache.org> wrote:
> >
> > > Xavier, that's good to know! Glad we already have some.
> > >
> > > On Fri, May 21, 2021 at 8:41 AM Xavier Léauté
> > > <xa...@confluent.io.invalid> wrote:
> > > >
> > > > > I agree with you that the root cause is lack of some automated
> tests
> > > > > against released docker images.
> > > > >
> > > >
> > > > Since https://github.com/apache/druid/pull/11187 we are now running
> > k8s
> > > > integration tests against the same image we release.
> > > > So this should at least catch the kind of issues people would
> encounter
> > > > with a typical k8s deployment.
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > For additional commands, e-mail: dev-help@druid.apache.org
> > >
> > >
> >
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Furkan KAMACI <fu...@gmail.com>.
Hi,
Here is my opinion:
Druid has had problems with docker for a while now. We use Druid on a
dockerized environment and manually solve this problem via setting
permissions. It is not a blocker issue, and we can release the artifact.
However, this is a bugfix release that aims to cover Docker volume issues
(whether directly related to the discussed problem or not); we should solve
this problem and then start a new RC spin. Otherwise, it seems that this
conversation will take more days to finalize (it has been 11 days since the
vote thread has started), and we will not be able to release it soon.
Solving that issue may result in faster release.
So, I am -0.
Kind Regards,
Furkan KAMACI
On Tue, May 25, 2021 at 10:06 AM Clint Wylie <cw...@apache.org> wrote:
> I agree that this probably isn't a blocker, but also agree based on the
> number of reports that we should consider either fixing so that the
> docker-compose quickstart instructions work as is, or adjust the docs so
> that the instructions at least include something about setting the
> permissions so that things can work correctly. Given that I still don't
> have enough PMC votes yet to release 0.21.1, and I don't really have a
> strong preference of which thing we do (nor am quite sure on what the
> correct docker fix is), anyone have opinions on the best path forward?
>
> On Sat, May 22, 2021 at 3:56 PM Jihoon Son <ji...@apache.org> wrote:
>
> > Xavier, that's good to know! Glad we already have some.
> >
> > On Fri, May 21, 2021 at 8:41 AM Xavier Léauté
> > <xa...@confluent.io.invalid> wrote:
> > >
> > > > I agree with you that the root cause is lack of some automated tests
> > > > against released docker images.
> > > >
> > >
> > > Since https://github.com/apache/druid/pull/11187 we are now running
> k8s
> > > integration tests against the same image we release.
> > > So this should at least catch the kind of issues people would encounter
> > > with a typical k8s deployment.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > For additional commands, e-mail: dev-help@druid.apache.org
> >
> >
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Clint Wylie <cw...@apache.org>.
I agree that this probably isn't a blocker, but also agree based on the
number of reports that we should consider either fixing so that the
docker-compose quickstart instructions work as is, or adjust the docs so
that the instructions at least include something about setting the
permissions so that things can work correctly. Given that I still don't
have enough PMC votes yet to release 0.21.1, and I don't really have a
strong preference of which thing we do (nor am quite sure on what the
correct docker fix is), anyone have opinions on the best path forward?
On Sat, May 22, 2021 at 3:56 PM Jihoon Son <ji...@apache.org> wrote:
> Xavier, that's good to know! Glad we already have some.
>
> On Fri, May 21, 2021 at 8:41 AM Xavier Léauté
> <xa...@confluent.io.invalid> wrote:
> >
> > > I agree with you that the root cause is lack of some automated tests
> > > against released docker images.
> > >
> >
> > Since https://github.com/apache/druid/pull/11187 we are now running k8s
> > integration tests against the same image we release.
> > So this should at least catch the kind of issues people would encounter
> > with a typical k8s deployment.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> For additional commands, e-mail: dev-help@druid.apache.org
>
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Jihoon Son <ji...@apache.org>.
Xavier, that's good to know! Glad we already have some.
On Fri, May 21, 2021 at 8:41 AM Xavier Léauté
<xa...@confluent.io.invalid> wrote:
>
> > I agree with you that the root cause is lack of some automated tests
> > against released docker images.
> >
>
> Since https://github.com/apache/druid/pull/11187 we are now running k8s
> integration tests against the same image we release.
> So this should at least catch the kind of issues people would encounter
> with a typical k8s deployment.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
For additional commands, e-mail: dev-help@druid.apache.org
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Xavier Léauté <xa...@confluent.io.INVALID>.
> I agree with you that the root cause is lack of some automated tests
> against released docker images.
>
Since https://github.com/apache/druid/pull/11187 we are now running k8s
integration tests against the same image we release.
So this should at least catch the kind of issues people would encounter
with a typical k8s deployment.
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by frank chen <fr...@apache.org>.
Hi Jihoon,
I agree with you that the root cause is lack of some automated tests
against released docker images.
For now, because of permission of docker volume behaves differently on
macOS and linux, following steps are suggested to verify released docker
images:
1. execute command [docker volume prune] to clear all previous volumes to
make sure the host environment is clean before starting druid cluster by
docker-compose
2. running some basic tests(e.g. native ingestion) both on macOS and Linux
Jihoon Son <ji...@apache.org> 于2021年5月21日周五 上午10:42写道:
> Hey Frank, thank you for looking into this problem! I still haven't
> had a chance to look at it closely, but your suggestion seems valid to
> fix the bug.
>
> However, I don't think this should be a blocker for 0.21.1 because
> 1) If you set to use non-local deep storage, the docker cluster
> properly spins up and ingestion succeeds. I tested this by myself
> using s3 deep storage. If you want to use the local deep storage, you
> can still work around the bug by setting the permission manually.
> 2) We cannot fix all bugs in one release because it's not possible. We
> don't know how many bugs there are, and thus trying to fix all bugs
> can delay the release forever. This is why we don't treat
> non-regression bugs as the release blocker.
> https://github.com/apache/druid/pull/11167 is a regression that was
> first introduced in 0.21.0. This is why we are doing 0.21.1 to fix the
> bug. But for non-regression bugs, I think our policy is reasonable to
> move forward without holding ourselves back and spending a lot of time
> for release.
>
> The real problem I see here is the lack of tests for docker builds.
> None of our tests currently use the docker image that we release. We
> use docker for integration tests, but the image we use for it is
> different. For a longer term, we should use the same docker image that
> we release for integration tests, but this will be pretty tough work.
> So we can start with a simpler test that verifies the docker image
> only as a short term solution, such as building a docker image and
> spinning up the cluster using it. I would suggest adding such tests as
> well as fixing the bugs reported in the docker image in 0.22.0 instead
> of further delaying 0.21.1. I believe we will start discussion for
> 0.22.0 release soon.
>
> Jihoon
>
> On Thu, May 20, 2021 at 12:13 AM frank chen <fr...@apache.org> wrote:
> >
> > Hi Clint, Jihoon,
> >
> > I checked this problem, it's caused by the VOLUME problem too.
> >
> > /opt/data directory is mounted as a volume in docker-compose.yml ,
> however
> > it's not created in Dockerfile. That means the directory will be created
> as
> > a user of root by docker during the starting of docker process.
> > This problem does not exist on macOS because of different mechanism of
> > osxfs(macos - Docker on MacOSX does not translate file ownership
> correctly
> > in volumes - Stack Overflow
> > <
> https://stackoverflow.com/questions/43097341/docker-on-macosx-does-not-translate-file-ownership-correctly-in-volumes
> >
> > )
> >
> > I also checked old issues on GitHub, actually, there is an issue that
> > reported this problem: https://github.com/apache/druid/issues/9779 .
> > This problem has been there for a long time.
> >
> > IMO, we SHOULD NOT release this candidate unless we fix this problem
> > because:
> > 1. ingestion has no chance to succeed on linux
> > 2. users have experienced enough docker related problems we had better
> > resolve them all in a single release
> >
> > I'm going to fix this problem by adding following instructions in
> > Dockerfile:
> >
> > RUN mkdir /opt/data \
> > && chown druid:druid /opt/data \
> > && chmod 775 /opt/data
> >
> > Thanks.
> >
> >
> > Jihoon Son <ji...@apache.org> 于2021年5月20日周四 下午12:07写道:
> >
> > > I could reproduce the same issue on my linux as what Clint saw.
> > > I didn't look into it closely, but agree that this doesn't seem like a
> > > blocker for the 0.21.1 release since the issue is about the permission
> > > on the local deep storage. However, for the users who want to try
> > > Druid out using docker, we should call it out in the release notes.
> > >
> > > So, my vote is +1.
> > >
> > > src package:
> > > - verified signature/checksum
> > > - LICENSE/NOTICE present
> > > - compiled, ran the license check/unit tests
> > > - built binary distribution, ran both batch and kafka ingestion using
> > > tutorial, ran some queries
> > >
> > > binary package:
> > > - verified signature/checksum
> > > - LICENSE/NOTICE present
> > > - ran both batch and kafka ingestion using tutorial, ran some queries
> > >
> > > docker:
> > > - verified checksum
> > > - started cluster with docker-compose with the modified permission on
> > > the local deep storage, ingested some data and ran some
> > > queries
> > >
> > > On Wed, May 19, 2021 at 2:21 AM frank chen <fr...@apache.org>
> wrote:
> > > >
> > > > Hi Clint,
> > > >
> > > > According to the docker instruction `COPY --chown=druid:druid
> > > > --from=builder /opt /opt` in docker file, the user `druid` has
> permission
> > > > to create directories under /opt.
> > > >
> > > >
> > > > I don't know what happened. I will check RC1 tomorrow on a fresh
> linux
> > > > environment to see if there's such problem.
> > > >
> > > >
> > > > Clint Wylie <cw...@apache.org> 于2021年5月19日周三 上午6:23写道:
> > > >
> > > > > Hmm, I'm actually still seeing a permissions related issue with
> the rc1
> > > > > Docker image and Linux (Docker on Mac OS seems to work ok, which is
> > > where I
> > > > > initially tested).
> > > > >
> > > > > middlemanager | 2021-05-18T21:20:39,239 INFO
> [forking-task-runner-0]
> > > > > org.apache.druid.indexing.overlord.ForkingTaskRunner - Exception
> caught
> > > > > during execution
> > > > > middlemanager | org.apache.druid.java.util.common.IOE: Unable to
> > > create
> > > > > task log dir[/opt/data/indexing-logs]
> > > > > middlemanager | at
> > > > >
> > > > >
> > >
> org.apache.druid.indexing.common.tasklogs.FileTaskLogs.pushTaskLog(FileTaskLogs.java:59)
> > > > > ~[druid-indexing-service-0.21.1.jar:0.21.1]
> > > > > middlemanager | at
> > > > >
> > > > >
> > >
> org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:386)
> > > > > [druid-indexing-service-0.21.1.jar:0.21.1]
> > > > > middlemanager | at
> > > > >
> > > > >
> > >
> org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:137)
> > > > > [druid-indexing-service-0.21.1.jar:0.21.1]
> > > > > middlemanager | at
> > > > > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [?:1.8.0_275]
> > > > > middlemanager | at
> > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > > [?:1.8.0_275]
> > > > > middlemanager | at
> > > > >
> > > > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > > [?:1.8.0_275]
> > > > > middlemanager | at java.lang.Thread.run(Thread.java:748)
> > > > > [?:1.8.0_275]
> > > > > middlemanager | 2021-05-18T21:20:39,247 INFO
> [forking-task-runner-0]
> > > > > org.apache.druid.indexing.overlord.ForkingTaskRunner - Removing
> task
> > > > > directory:
> > > > >
> > >
> var/druid/task/index_parallel_wikipedia_gmkopeph_2021-05-18T21:20:10.899Z
> > > > >
> > > > > the 'storage' directory that is used for tasklogs and deep storage
> in
> > > the
> > > > > docker-compose file seems to belong to root. If I change ownership
> to
> > > the
> > > > > user it works.
> > > > >
> > > > > I am unsure if this is necessarily a blocker because the containers
> > > > > themselves do seem to actually work ok, and this seems to be an
> issue
> > > with
> > > > > the locally attached volume deep storage used by the example
> compose
> > > file
> > > > > (which I also forgot to update the version to 0.21.1 in this
> release).
> > > > > Presumably everything would work ok if using something like s3 for
> deep
> > > > > storage, though I haven't had the chance to confirm this yet.
> > > > >
> > > > >
> > > > > On Fri, May 14, 2021 at 12:42 PM Clint Wylie <cw...@apache.org>
> > > wrote:
> > > > >
> > > > > > Hi all,
> > > > > >
> > > > > > I have created a build for Apache Druid 0.21.1, release
> > > > > > candidate 1.
> > > > > >
> > > > > > Thanks to everyone who has helped contribute to the release! You
> can
> > > read
> > > > > > the proposed release notes here:
> > > > > > https://github.com/apache/druid/issues/11249
> > > > > >
> > > > > > The release candidate has been tagged in GitHub as
> > > > > > druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> > > > > > available here:
> > > > > > https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
> > > > > >
> > > > > > The artifacts to be voted on are located here:
> > > > > > https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
> > > > > >
> > > > > > A staged Maven repository is available for review at:
> > > > > >
> > >
> https://repository.apache.org/content/repositories/orgapachedruid-1024/
> > > > > >
> > > > > > Staged druid.apache.org website documentation is available here:
> > > > > > https://druid.staged.apache.org/docs/0.21.1/design/index.html
> > > > > >
> > > > > > A Docker image containing the binary of the release candidate
> can be
> > > > > > retrieved via:
> > > > > > docker pull apache/druid:0.21.1-rc1
> > > > > >
> > > > > > artifact checksums
> > > > > > src:
> > > > > >
> > > > > >
> > > > >
> > >
> 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> > > > > > bin:
> > > > > >
> > > > > >
> > > > >
> > >
> 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> > > > > > docker:
> > > 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
> > > > > >
> > > > > > Release artifacts are signed with the following key:
> > > > > > https://people.apache.org/keys/committer/cwylie.asc
> > > > > >
> > > > > > This key and the key of other committers can also be found in the
> > > > > project's
> > > > > > KEYS file here:
> > > > > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > > > > >
> > > > > > (If you are a committer, please feel free to add your own key to
> that
> > > > > file
> > > > > > by following the instructions in the file's header.)
> > > > > >
> > > > > >
> > > > > > Verify checksums:
> > > > > > diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> > > > > > cut -d ' ' -f1) \
> > > > > > <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
> > > > > >
> > > > > > diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> > > > > > cut -d ' ' -f1) \
> > > > > > <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
> > > > > >
> > > > > > Verify signatures:
> > > > > > gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> > > > > > apache-druid-0.21.1-src.tar.gz
> > > > > >
> > > > > > gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> > > > > > apache-druid-0.21.1-bin.tar.gz
> > > > > >
> > > > > > Please review the proposed artifacts and vote. Note that Apache
> has
> > > > > > specific requirements that must be met before +1 binding votes
> can be
> > > > > cast
> > > > > > by PMC members. Please refer to the policy at
> > > > > > http://www.apache.org/legal/release-policy.html#policy for more
> > > details.
> > > > > >
> > > > > > As part of the validation process, the release artifacts can be
> > > generated
> > > > > > from source by running:
> > > > > > mvn clean install -Papache-release,dist -Dgpg.skip
> > > > > >
> > > > > > The RAT license check can be run from source by:
> > > > > > mvn apache-rat:check -Prat
> > > > > >
> > > > > > This vote will be open for at least 72 hours. The vote will pass
> if a
> > > > > > majority of at least three +1 PMC votes are cast.
> > > > > >
> > > > > > [ ] +1 Release this package as Apache Druid 0.21.1
> > > > > > [ ] 0 I don't feel strongly about it, but I'm okay with the
> release
> > > > > > [ ] -1 Do not release this package because...
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > > For additional commands, e-mail: dev-help@druid.apache.org
> > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> For additional commands, e-mail: dev-help@druid.apache.org
>
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Jihoon Son <ji...@apache.org>.
Hey Frank, thank you for looking into this problem! I still haven't
had a chance to look at it closely, but your suggestion seems valid to
fix the bug.
However, I don't think this should be a blocker for 0.21.1 because
1) If you set to use non-local deep storage, the docker cluster
properly spins up and ingestion succeeds. I tested this by myself
using s3 deep storage. If you want to use the local deep storage, you
can still work around the bug by setting the permission manually.
2) We cannot fix all bugs in one release because it's not possible. We
don't know how many bugs there are, and thus trying to fix all bugs
can delay the release forever. This is why we don't treat
non-regression bugs as the release blocker.
https://github.com/apache/druid/pull/11167 is a regression that was
first introduced in 0.21.0. This is why we are doing 0.21.1 to fix the
bug. But for non-regression bugs, I think our policy is reasonable to
move forward without holding ourselves back and spending a lot of time
for release.
The real problem I see here is the lack of tests for docker builds.
None of our tests currently use the docker image that we release. We
use docker for integration tests, but the image we use for it is
different. For a longer term, we should use the same docker image that
we release for integration tests, but this will be pretty tough work.
So we can start with a simpler test that verifies the docker image
only as a short term solution, such as building a docker image and
spinning up the cluster using it. I would suggest adding such tests as
well as fixing the bugs reported in the docker image in 0.22.0 instead
of further delaying 0.21.1. I believe we will start discussion for
0.22.0 release soon.
Jihoon
On Thu, May 20, 2021 at 12:13 AM frank chen <fr...@apache.org> wrote:
>
> Hi Clint, Jihoon,
>
> I checked this problem, it's caused by the VOLUME problem too.
>
> /opt/data directory is mounted as a volume in docker-compose.yml , however
> it's not created in Dockerfile. That means the directory will be created as
> a user of root by docker during the starting of docker process.
> This problem does not exist on macOS because of different mechanism of
> osxfs(macos - Docker on MacOSX does not translate file ownership correctly
> in volumes - Stack Overflow
> <https://stackoverflow.com/questions/43097341/docker-on-macosx-does-not-translate-file-ownership-correctly-in-volumes>
> )
>
> I also checked old issues on GitHub, actually, there is an issue that
> reported this problem: https://github.com/apache/druid/issues/9779 .
> This problem has been there for a long time.
>
> IMO, we SHOULD NOT release this candidate unless we fix this problem
> because:
> 1. ingestion has no chance to succeed on linux
> 2. users have experienced enough docker related problems we had better
> resolve them all in a single release
>
> I'm going to fix this problem by adding following instructions in
> Dockerfile:
>
> RUN mkdir /opt/data \
> && chown druid:druid /opt/data \
> && chmod 775 /opt/data
>
> Thanks.
>
>
> Jihoon Son <ji...@apache.org> 于2021年5月20日周四 下午12:07写道:
>
> > I could reproduce the same issue on my linux as what Clint saw.
> > I didn't look into it closely, but agree that this doesn't seem like a
> > blocker for the 0.21.1 release since the issue is about the permission
> > on the local deep storage. However, for the users who want to try
> > Druid out using docker, we should call it out in the release notes.
> >
> > So, my vote is +1.
> >
> > src package:
> > - verified signature/checksum
> > - LICENSE/NOTICE present
> > - compiled, ran the license check/unit tests
> > - built binary distribution, ran both batch and kafka ingestion using
> > tutorial, ran some queries
> >
> > binary package:
> > - verified signature/checksum
> > - LICENSE/NOTICE present
> > - ran both batch and kafka ingestion using tutorial, ran some queries
> >
> > docker:
> > - verified checksum
> > - started cluster with docker-compose with the modified permission on
> > the local deep storage, ingested some data and ran some
> > queries
> >
> > On Wed, May 19, 2021 at 2:21 AM frank chen <fr...@apache.org> wrote:
> > >
> > > Hi Clint,
> > >
> > > According to the docker instruction `COPY --chown=druid:druid
> > > --from=builder /opt /opt` in docker file, the user `druid` has permission
> > > to create directories under /opt.
> > >
> > >
> > > I don't know what happened. I will check RC1 tomorrow on a fresh linux
> > > environment to see if there's such problem.
> > >
> > >
> > > Clint Wylie <cw...@apache.org> 于2021年5月19日周三 上午6:23写道:
> > >
> > > > Hmm, I'm actually still seeing a permissions related issue with the rc1
> > > > Docker image and Linux (Docker on Mac OS seems to work ok, which is
> > where I
> > > > initially tested).
> > > >
> > > > middlemanager | 2021-05-18T21:20:39,239 INFO [forking-task-runner-0]
> > > > org.apache.druid.indexing.overlord.ForkingTaskRunner - Exception caught
> > > > during execution
> > > > middlemanager | org.apache.druid.java.util.common.IOE: Unable to
> > create
> > > > task log dir[/opt/data/indexing-logs]
> > > > middlemanager | at
> > > >
> > > >
> > org.apache.druid.indexing.common.tasklogs.FileTaskLogs.pushTaskLog(FileTaskLogs.java:59)
> > > > ~[druid-indexing-service-0.21.1.jar:0.21.1]
> > > > middlemanager | at
> > > >
> > > >
> > org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:386)
> > > > [druid-indexing-service-0.21.1.jar:0.21.1]
> > > > middlemanager | at
> > > >
> > > >
> > org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:137)
> > > > [druid-indexing-service-0.21.1.jar:0.21.1]
> > > > middlemanager | at
> > > > java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
> > > > middlemanager | at
> > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > > [?:1.8.0_275]
> > > > middlemanager | at
> > > >
> > > >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > > [?:1.8.0_275]
> > > > middlemanager | at java.lang.Thread.run(Thread.java:748)
> > > > [?:1.8.0_275]
> > > > middlemanager | 2021-05-18T21:20:39,247 INFO [forking-task-runner-0]
> > > > org.apache.druid.indexing.overlord.ForkingTaskRunner - Removing task
> > > > directory:
> > > >
> > var/druid/task/index_parallel_wikipedia_gmkopeph_2021-05-18T21:20:10.899Z
> > > >
> > > > the 'storage' directory that is used for tasklogs and deep storage in
> > the
> > > > docker-compose file seems to belong to root. If I change ownership to
> > the
> > > > user it works.
> > > >
> > > > I am unsure if this is necessarily a blocker because the containers
> > > > themselves do seem to actually work ok, and this seems to be an issue
> > with
> > > > the locally attached volume deep storage used by the example compose
> > file
> > > > (which I also forgot to update the version to 0.21.1 in this release).
> > > > Presumably everything would work ok if using something like s3 for deep
> > > > storage, though I haven't had the chance to confirm this yet.
> > > >
> > > >
> > > > On Fri, May 14, 2021 at 12:42 PM Clint Wylie <cw...@apache.org>
> > wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > I have created a build for Apache Druid 0.21.1, release
> > > > > candidate 1.
> > > > >
> > > > > Thanks to everyone who has helped contribute to the release! You can
> > read
> > > > > the proposed release notes here:
> > > > > https://github.com/apache/druid/issues/11249
> > > > >
> > > > > The release candidate has been tagged in GitHub as
> > > > > druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> > > > > available here:
> > > > > https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
> > > > >
> > > > > The artifacts to be voted on are located here:
> > > > > https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
> > > > >
> > > > > A staged Maven repository is available for review at:
> > > > >
> > https://repository.apache.org/content/repositories/orgapachedruid-1024/
> > > > >
> > > > > Staged druid.apache.org website documentation is available here:
> > > > > https://druid.staged.apache.org/docs/0.21.1/design/index.html
> > > > >
> > > > > A Docker image containing the binary of the release candidate can be
> > > > > retrieved via:
> > > > > docker pull apache/druid:0.21.1-rc1
> > > > >
> > > > > artifact checksums
> > > > > src:
> > > > >
> > > > >
> > > >
> > 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> > > > > bin:
> > > > >
> > > > >
> > > >
> > 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> > > > > docker:
> > 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
> > > > >
> > > > > Release artifacts are signed with the following key:
> > > > > https://people.apache.org/keys/committer/cwylie.asc
> > > > >
> > > > > This key and the key of other committers can also be found in the
> > > > project's
> > > > > KEYS file here:
> > > > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > > > >
> > > > > (If you are a committer, please feel free to add your own key to that
> > > > file
> > > > > by following the instructions in the file's header.)
> > > > >
> > > > >
> > > > > Verify checksums:
> > > > > diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> > > > > cut -d ' ' -f1) \
> > > > > <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
> > > > >
> > > > > diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> > > > > cut -d ' ' -f1) \
> > > > > <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
> > > > >
> > > > > Verify signatures:
> > > > > gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> > > > > apache-druid-0.21.1-src.tar.gz
> > > > >
> > > > > gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> > > > > apache-druid-0.21.1-bin.tar.gz
> > > > >
> > > > > Please review the proposed artifacts and vote. Note that Apache has
> > > > > specific requirements that must be met before +1 binding votes can be
> > > > cast
> > > > > by PMC members. Please refer to the policy at
> > > > > http://www.apache.org/legal/release-policy.html#policy for more
> > details.
> > > > >
> > > > > As part of the validation process, the release artifacts can be
> > generated
> > > > > from source by running:
> > > > > mvn clean install -Papache-release,dist -Dgpg.skip
> > > > >
> > > > > The RAT license check can be run from source by:
> > > > > mvn apache-rat:check -Prat
> > > > >
> > > > > This vote will be open for at least 72 hours. The vote will pass if a
> > > > > majority of at least three +1 PMC votes are cast.
> > > > >
> > > > > [ ] +1 Release this package as Apache Druid 0.21.1
> > > > > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > > > > [ ] -1 Do not release this package because...
> > > > >
> > > > > Thanks!
> > > > >
> > > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> > For additional commands, e-mail: dev-help@druid.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
For additional commands, e-mail: dev-help@druid.apache.org
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by frank chen <fr...@apache.org>.
Hi Clint, Jihoon,
I checked this problem, it's caused by the VOLUME problem too.
/opt/data directory is mounted as a volume in docker-compose.yml , however
it's not created in Dockerfile. That means the directory will be created as
a user of root by docker during the starting of docker process.
This problem does not exist on macOS because of different mechanism of
osxfs(macos - Docker on MacOSX does not translate file ownership correctly
in volumes - Stack Overflow
<https://stackoverflow.com/questions/43097341/docker-on-macosx-does-not-translate-file-ownership-correctly-in-volumes>
)
I also checked old issues on GitHub, actually, there is an issue that
reported this problem: https://github.com/apache/druid/issues/9779 .
This problem has been there for a long time.
IMO, we SHOULD NOT release this candidate unless we fix this problem
because:
1. ingestion has no chance to succeed on linux
2. users have experienced enough docker related problems we had better
resolve them all in a single release
I'm going to fix this problem by adding following instructions in
Dockerfile:
RUN mkdir /opt/data \
&& chown druid:druid /opt/data \
&& chmod 775 /opt/data
Thanks.
Jihoon Son <ji...@apache.org> 于2021年5月20日周四 下午12:07写道:
> I could reproduce the same issue on my linux as what Clint saw.
> I didn't look into it closely, but agree that this doesn't seem like a
> blocker for the 0.21.1 release since the issue is about the permission
> on the local deep storage. However, for the users who want to try
> Druid out using docker, we should call it out in the release notes.
>
> So, my vote is +1.
>
> src package:
> - verified signature/checksum
> - LICENSE/NOTICE present
> - compiled, ran the license check/unit tests
> - built binary distribution, ran both batch and kafka ingestion using
> tutorial, ran some queries
>
> binary package:
> - verified signature/checksum
> - LICENSE/NOTICE present
> - ran both batch and kafka ingestion using tutorial, ran some queries
>
> docker:
> - verified checksum
> - started cluster with docker-compose with the modified permission on
> the local deep storage, ingested some data and ran some
> queries
>
> On Wed, May 19, 2021 at 2:21 AM frank chen <fr...@apache.org> wrote:
> >
> > Hi Clint,
> >
> > According to the docker instruction `COPY --chown=druid:druid
> > --from=builder /opt /opt` in docker file, the user `druid` has permission
> > to create directories under /opt.
> >
> >
> > I don't know what happened. I will check RC1 tomorrow on a fresh linux
> > environment to see if there's such problem.
> >
> >
> > Clint Wylie <cw...@apache.org> 于2021年5月19日周三 上午6:23写道:
> >
> > > Hmm, I'm actually still seeing a permissions related issue with the rc1
> > > Docker image and Linux (Docker on Mac OS seems to work ok, which is
> where I
> > > initially tested).
> > >
> > > middlemanager | 2021-05-18T21:20:39,239 INFO [forking-task-runner-0]
> > > org.apache.druid.indexing.overlord.ForkingTaskRunner - Exception caught
> > > during execution
> > > middlemanager | org.apache.druid.java.util.common.IOE: Unable to
> create
> > > task log dir[/opt/data/indexing-logs]
> > > middlemanager | at
> > >
> > >
> org.apache.druid.indexing.common.tasklogs.FileTaskLogs.pushTaskLog(FileTaskLogs.java:59)
> > > ~[druid-indexing-service-0.21.1.jar:0.21.1]
> > > middlemanager | at
> > >
> > >
> org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:386)
> > > [druid-indexing-service-0.21.1.jar:0.21.1]
> > > middlemanager | at
> > >
> > >
> org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:137)
> > > [druid-indexing-service-0.21.1.jar:0.21.1]
> > > middlemanager | at
> > > java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
> > > middlemanager | at
> > >
> > >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > > [?:1.8.0_275]
> > > middlemanager | at
> > >
> > >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > > [?:1.8.0_275]
> > > middlemanager | at java.lang.Thread.run(Thread.java:748)
> > > [?:1.8.0_275]
> > > middlemanager | 2021-05-18T21:20:39,247 INFO [forking-task-runner-0]
> > > org.apache.druid.indexing.overlord.ForkingTaskRunner - Removing task
> > > directory:
> > >
> var/druid/task/index_parallel_wikipedia_gmkopeph_2021-05-18T21:20:10.899Z
> > >
> > > the 'storage' directory that is used for tasklogs and deep storage in
> the
> > > docker-compose file seems to belong to root. If I change ownership to
> the
> > > user it works.
> > >
> > > I am unsure if this is necessarily a blocker because the containers
> > > themselves do seem to actually work ok, and this seems to be an issue
> with
> > > the locally attached volume deep storage used by the example compose
> file
> > > (which I also forgot to update the version to 0.21.1 in this release).
> > > Presumably everything would work ok if using something like s3 for deep
> > > storage, though I haven't had the chance to confirm this yet.
> > >
> > >
> > > On Fri, May 14, 2021 at 12:42 PM Clint Wylie <cw...@apache.org>
> wrote:
> > >
> > > > Hi all,
> > > >
> > > > I have created a build for Apache Druid 0.21.1, release
> > > > candidate 1.
> > > >
> > > > Thanks to everyone who has helped contribute to the release! You can
> read
> > > > the proposed release notes here:
> > > > https://github.com/apache/druid/issues/11249
> > > >
> > > > The release candidate has been tagged in GitHub as
> > > > druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> > > > available here:
> > > > https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
> > > >
> > > > The artifacts to be voted on are located here:
> > > > https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
> > > >
> > > > A staged Maven repository is available for review at:
> > > >
> https://repository.apache.org/content/repositories/orgapachedruid-1024/
> > > >
> > > > Staged druid.apache.org website documentation is available here:
> > > > https://druid.staged.apache.org/docs/0.21.1/design/index.html
> > > >
> > > > A Docker image containing the binary of the release candidate can be
> > > > retrieved via:
> > > > docker pull apache/druid:0.21.1-rc1
> > > >
> > > > artifact checksums
> > > > src:
> > > >
> > > >
> > >
> 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> > > > bin:
> > > >
> > > >
> > >
> 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> > > > docker:
> 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
> > > >
> > > > Release artifacts are signed with the following key:
> > > > https://people.apache.org/keys/committer/cwylie.asc
> > > >
> > > > This key and the key of other committers can also be found in the
> > > project's
> > > > KEYS file here:
> > > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > > >
> > > > (If you are a committer, please feel free to add your own key to that
> > > file
> > > > by following the instructions in the file's header.)
> > > >
> > > >
> > > > Verify checksums:
> > > > diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> > > > cut -d ' ' -f1) \
> > > > <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
> > > >
> > > > diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> > > > cut -d ' ' -f1) \
> > > > <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
> > > >
> > > > Verify signatures:
> > > > gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> > > > apache-druid-0.21.1-src.tar.gz
> > > >
> > > > gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> > > > apache-druid-0.21.1-bin.tar.gz
> > > >
> > > > Please review the proposed artifacts and vote. Note that Apache has
> > > > specific requirements that must be met before +1 binding votes can be
> > > cast
> > > > by PMC members. Please refer to the policy at
> > > > http://www.apache.org/legal/release-policy.html#policy for more
> details.
> > > >
> > > > As part of the validation process, the release artifacts can be
> generated
> > > > from source by running:
> > > > mvn clean install -Papache-release,dist -Dgpg.skip
> > > >
> > > > The RAT license check can be run from source by:
> > > > mvn apache-rat:check -Prat
> > > >
> > > > This vote will be open for at least 72 hours. The vote will pass if a
> > > > majority of at least three +1 PMC votes are cast.
> > > >
> > > > [ ] +1 Release this package as Apache Druid 0.21.1
> > > > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > > > [ ] -1 Do not release this package because...
> > > >
> > > > Thanks!
> > > >
> > >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
> For additional commands, e-mail: dev-help@druid.apache.org
>
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Jihoon Son <ji...@apache.org>.
I could reproduce the same issue on my linux as what Clint saw.
I didn't look into it closely, but agree that this doesn't seem like a
blocker for the 0.21.1 release since the issue is about the permission
on the local deep storage. However, for the users who want to try
Druid out using docker, we should call it out in the release notes.
So, my vote is +1.
src package:
- verified signature/checksum
- LICENSE/NOTICE present
- compiled, ran the license check/unit tests
- built binary distribution, ran both batch and kafka ingestion using
tutorial, ran some queries
binary package:
- verified signature/checksum
- LICENSE/NOTICE present
- ran both batch and kafka ingestion using tutorial, ran some queries
docker:
- verified checksum
- started cluster with docker-compose with the modified permission on
the local deep storage, ingested some data and ran some
queries
On Wed, May 19, 2021 at 2:21 AM frank chen <fr...@apache.org> wrote:
>
> Hi Clint,
>
> According to the docker instruction `COPY --chown=druid:druid
> --from=builder /opt /opt` in docker file, the user `druid` has permission
> to create directories under /opt.
>
>
> I don't know what happened. I will check RC1 tomorrow on a fresh linux
> environment to see if there's such problem.
>
>
> Clint Wylie <cw...@apache.org> 于2021年5月19日周三 上午6:23写道:
>
> > Hmm, I'm actually still seeing a permissions related issue with the rc1
> > Docker image and Linux (Docker on Mac OS seems to work ok, which is where I
> > initially tested).
> >
> > middlemanager | 2021-05-18T21:20:39,239 INFO [forking-task-runner-0]
> > org.apache.druid.indexing.overlord.ForkingTaskRunner - Exception caught
> > during execution
> > middlemanager | org.apache.druid.java.util.common.IOE: Unable to create
> > task log dir[/opt/data/indexing-logs]
> > middlemanager | at
> >
> > org.apache.druid.indexing.common.tasklogs.FileTaskLogs.pushTaskLog(FileTaskLogs.java:59)
> > ~[druid-indexing-service-0.21.1.jar:0.21.1]
> > middlemanager | at
> >
> > org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:386)
> > [druid-indexing-service-0.21.1.jar:0.21.1]
> > middlemanager | at
> >
> > org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:137)
> > [druid-indexing-service-0.21.1.jar:0.21.1]
> > middlemanager | at
> > java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
> > middlemanager | at
> >
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > [?:1.8.0_275]
> > middlemanager | at
> >
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > [?:1.8.0_275]
> > middlemanager | at java.lang.Thread.run(Thread.java:748)
> > [?:1.8.0_275]
> > middlemanager | 2021-05-18T21:20:39,247 INFO [forking-task-runner-0]
> > org.apache.druid.indexing.overlord.ForkingTaskRunner - Removing task
> > directory:
> > var/druid/task/index_parallel_wikipedia_gmkopeph_2021-05-18T21:20:10.899Z
> >
> > the 'storage' directory that is used for tasklogs and deep storage in the
> > docker-compose file seems to belong to root. If I change ownership to the
> > user it works.
> >
> > I am unsure if this is necessarily a blocker because the containers
> > themselves do seem to actually work ok, and this seems to be an issue with
> > the locally attached volume deep storage used by the example compose file
> > (which I also forgot to update the version to 0.21.1 in this release).
> > Presumably everything would work ok if using something like s3 for deep
> > storage, though I haven't had the chance to confirm this yet.
> >
> >
> > On Fri, May 14, 2021 at 12:42 PM Clint Wylie <cw...@apache.org> wrote:
> >
> > > Hi all,
> > >
> > > I have created a build for Apache Druid 0.21.1, release
> > > candidate 1.
> > >
> > > Thanks to everyone who has helped contribute to the release! You can read
> > > the proposed release notes here:
> > > https://github.com/apache/druid/issues/11249
> > >
> > > The release candidate has been tagged in GitHub as
> > > druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> > > available here:
> > > https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
> > >
> > > The artifacts to be voted on are located here:
> > > https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
> > >
> > > A staged Maven repository is available for review at:
> > > https://repository.apache.org/content/repositories/orgapachedruid-1024/
> > >
> > > Staged druid.apache.org website documentation is available here:
> > > https://druid.staged.apache.org/docs/0.21.1/design/index.html
> > >
> > > A Docker image containing the binary of the release candidate can be
> > > retrieved via:
> > > docker pull apache/druid:0.21.1-rc1
> > >
> > > artifact checksums
> > > src:
> > >
> > >
> > 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> > > bin:
> > >
> > >
> > 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> > > docker: 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/cwylie.asc
> > >
> > > This key and the key of other committers can also be found in the
> > project's
> > > KEYS file here:
> > > https://dist.apache.org/repos/dist/release/druid/KEYS
> > >
> > > (If you are a committer, please feel free to add your own key to that
> > file
> > > by following the instructions in the file's header.)
> > >
> > >
> > > Verify checksums:
> > > diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
> > >
> > > diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> > > cut -d ' ' -f1) \
> > > <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
> > >
> > > Verify signatures:
> > > gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> > > apache-druid-0.21.1-src.tar.gz
> > >
> > > gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> > > apache-druid-0.21.1-bin.tar.gz
> > >
> > > Please review the proposed artifacts and vote. Note that Apache has
> > > specific requirements that must be met before +1 binding votes can be
> > cast
> > > by PMC members. Please refer to the policy at
> > > http://www.apache.org/legal/release-policy.html#policy for more details.
> > >
> > > As part of the validation process, the release artifacts can be generated
> > > from source by running:
> > > mvn clean install -Papache-release,dist -Dgpg.skip
> > >
> > > The RAT license check can be run from source by:
> > > mvn apache-rat:check -Prat
> > >
> > > This vote will be open for at least 72 hours. The vote will pass if a
> > > majority of at least three +1 PMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Druid 0.21.1
> > > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > > [ ] -1 Do not release this package because...
> > >
> > > Thanks!
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@druid.apache.org
For additional commands, e-mail: dev-help@druid.apache.org
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by frank chen <fr...@apache.org>.
Hi Clint,
According to the docker instruction `COPY --chown=druid:druid
--from=builder /opt /opt` in docker file, the user `druid` has permission
to create directories under /opt.
I don't know what happened. I will check RC1 tomorrow on a fresh linux
environment to see if there's such problem.
Clint Wylie <cw...@apache.org> 于2021年5月19日周三 上午6:23写道:
> Hmm, I'm actually still seeing a permissions related issue with the rc1
> Docker image and Linux (Docker on Mac OS seems to work ok, which is where I
> initially tested).
>
> middlemanager | 2021-05-18T21:20:39,239 INFO [forking-task-runner-0]
> org.apache.druid.indexing.overlord.ForkingTaskRunner - Exception caught
> during execution
> middlemanager | org.apache.druid.java.util.common.IOE: Unable to create
> task log dir[/opt/data/indexing-logs]
> middlemanager | at
>
> org.apache.druid.indexing.common.tasklogs.FileTaskLogs.pushTaskLog(FileTaskLogs.java:59)
> ~[druid-indexing-service-0.21.1.jar:0.21.1]
> middlemanager | at
>
> org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:386)
> [druid-indexing-service-0.21.1.jar:0.21.1]
> middlemanager | at
>
> org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:137)
> [druid-indexing-service-0.21.1.jar:0.21.1]
> middlemanager | at
> java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
> middlemanager | at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [?:1.8.0_275]
> middlemanager | at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [?:1.8.0_275]
> middlemanager | at java.lang.Thread.run(Thread.java:748)
> [?:1.8.0_275]
> middlemanager | 2021-05-18T21:20:39,247 INFO [forking-task-runner-0]
> org.apache.druid.indexing.overlord.ForkingTaskRunner - Removing task
> directory:
> var/druid/task/index_parallel_wikipedia_gmkopeph_2021-05-18T21:20:10.899Z
>
> the 'storage' directory that is used for tasklogs and deep storage in the
> docker-compose file seems to belong to root. If I change ownership to the
> user it works.
>
> I am unsure if this is necessarily a blocker because the containers
> themselves do seem to actually work ok, and this seems to be an issue with
> the locally attached volume deep storage used by the example compose file
> (which I also forgot to update the version to 0.21.1 in this release).
> Presumably everything would work ok if using something like s3 for deep
> storage, though I haven't had the chance to confirm this yet.
>
>
> On Fri, May 14, 2021 at 12:42 PM Clint Wylie <cw...@apache.org> wrote:
>
> > Hi all,
> >
> > I have created a build for Apache Druid 0.21.1, release
> > candidate 1.
> >
> > Thanks to everyone who has helped contribute to the release! You can read
> > the proposed release notes here:
> > https://github.com/apache/druid/issues/11249
> >
> > The release candidate has been tagged in GitHub as
> > druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> > available here:
> > https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
> >
> > The artifacts to be voted on are located here:
> > https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
> >
> > A staged Maven repository is available for review at:
> > https://repository.apache.org/content/repositories/orgapachedruid-1024/
> >
> > Staged druid.apache.org website documentation is available here:
> > https://druid.staged.apache.org/docs/0.21.1/design/index.html
> >
> > A Docker image containing the binary of the release candidate can be
> > retrieved via:
> > docker pull apache/druid:0.21.1-rc1
> >
> > artifact checksums
> > src:
> >
> >
> 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> > bin:
> >
> >
> 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> > docker: 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/cwylie.asc
> >
> > This key and the key of other committers can also be found in the
> project's
> > KEYS file here:
> > https://dist.apache.org/repos/dist/release/druid/KEYS
> >
> > (If you are a committer, please feel free to add your own key to that
> file
> > by following the instructions in the file's header.)
> >
> >
> > Verify checksums:
> > diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
> >
> > diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> > cut -d ' ' -f1) \
> > <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
> >
> > Verify signatures:
> > gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> > apache-druid-0.21.1-src.tar.gz
> >
> > gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> > apache-druid-0.21.1-bin.tar.gz
> >
> > Please review the proposed artifacts and vote. Note that Apache has
> > specific requirements that must be met before +1 binding votes can be
> cast
> > by PMC members. Please refer to the policy at
> > http://www.apache.org/legal/release-policy.html#policy for more details.
> >
> > As part of the validation process, the release artifacts can be generated
> > from source by running:
> > mvn clean install -Papache-release,dist -Dgpg.skip
> >
> > The RAT license check can be run from source by:
> > mvn apache-rat:check -Prat
> >
> > This vote will be open for at least 72 hours. The vote will pass if a
> > majority of at least three +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Druid 0.21.1
> > [ ] 0 I don't feel strongly about it, but I'm okay with the release
> > [ ] -1 Do not release this package because...
> >
> > Thanks!
> >
>
Re: [VOTE] Release Apache Druid 0.21.1 [RC1]
Posted by Clint Wylie <cw...@apache.org>.
Hmm, I'm actually still seeing a permissions related issue with the rc1
Docker image and Linux (Docker on Mac OS seems to work ok, which is where I
initially tested).
middlemanager | 2021-05-18T21:20:39,239 INFO [forking-task-runner-0]
org.apache.druid.indexing.overlord.ForkingTaskRunner - Exception caught
during execution
middlemanager | org.apache.druid.java.util.common.IOE: Unable to create
task log dir[/opt/data/indexing-logs]
middlemanager | at
org.apache.druid.indexing.common.tasklogs.FileTaskLogs.pushTaskLog(FileTaskLogs.java:59)
~[druid-indexing-service-0.21.1.jar:0.21.1]
middlemanager | at
org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:386)
[druid-indexing-service-0.21.1.jar:0.21.1]
middlemanager | at
org.apache.druid.indexing.overlord.ForkingTaskRunner$1.call(ForkingTaskRunner.java:137)
[druid-indexing-service-0.21.1.jar:0.21.1]
middlemanager | at
java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_275]
middlemanager | at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[?:1.8.0_275]
middlemanager | at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[?:1.8.0_275]
middlemanager | at java.lang.Thread.run(Thread.java:748)
[?:1.8.0_275]
middlemanager | 2021-05-18T21:20:39,247 INFO [forking-task-runner-0]
org.apache.druid.indexing.overlord.ForkingTaskRunner - Removing task
directory:
var/druid/task/index_parallel_wikipedia_gmkopeph_2021-05-18T21:20:10.899Z
the 'storage' directory that is used for tasklogs and deep storage in the
docker-compose file seems to belong to root. If I change ownership to the
user it works.
I am unsure if this is necessarily a blocker because the containers
themselves do seem to actually work ok, and this seems to be an issue with
the locally attached volume deep storage used by the example compose file
(which I also forgot to update the version to 0.21.1 in this release).
Presumably everything would work ok if using something like s3 for deep
storage, though I haven't had the chance to confirm this yet.
On Fri, May 14, 2021 at 12:42 PM Clint Wylie <cw...@apache.org> wrote:
> Hi all,
>
> I have created a build for Apache Druid 0.21.1, release
> candidate 1.
>
> Thanks to everyone who has helped contribute to the release! You can read
> the proposed release notes here:
> https://github.com/apache/druid/issues/11249
>
> The release candidate has been tagged in GitHub as
> druid-0.21.1-rc3 (9d142a2f19cceef38173e8d463a8cc1dfe1cb7ec),
> available here:
> https://github.com/apache/druid/releases/tag/druid-0.21.1-rc1
>
> The artifacts to be voted on are located here:
> https://dist.apache.org/repos/dist/dev/druid/0.21.1-rc1/
>
> A staged Maven repository is available for review at:
> https://repository.apache.org/content/repositories/orgapachedruid-1024/
>
> Staged druid.apache.org website documentation is available here:
> https://druid.staged.apache.org/docs/0.21.1/design/index.html
>
> A Docker image containing the binary of the release candidate can be
> retrieved via:
> docker pull apache/druid:0.21.1-rc1
>
> artifact checksums
> src:
>
> 8ac8267e9fa8aebbd7e2aa7bf30cfa598083eea4e862856f2e28521443a74c8b4f3d7ede5ba9837ee802028913995567d8f10e991f2aac069fea0155b30083f9
> bin:
>
> 170a8861c2bd00078689316013b32d16e110c021be87bf183898b4eb3ffc1c06ca56e3c7b92ca38f9732ba3cf4ff4c1de22a68d7a4b93e6dc271b3c994416053
> docker: 16b47d4d03b41aa6c5784b6f356d51bb7d3c50198cb5f8a289d0e9c4d425bb26
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/cwylie.asc
>
> This key and the key of other committers can also be found in the project's
> KEYS file here:
> https://dist.apache.org/repos/dist/release/druid/KEYS
>
> (If you are a committer, please feel free to add your own key to that file
> by following the instructions in the file's header.)
>
>
> Verify checksums:
> diff <(shasum -a512 apache-druid-0.21.1-src.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.21.1-src.tar.gz.sha512 ; echo)
>
> diff <(shasum -a512 apache-druid-0.21.1-bin.tar.gz | \
> cut -d ' ' -f1) \
> <(cat apache-druid-0.21.1-bin.tar.gz.sha512 ; echo)
>
> Verify signatures:
> gpg --verify apache-druid-0.21.1-src.tar.gz.asc \
> apache-druid-0.21.1-src.tar.gz
>
> gpg --verify apache-druid-0.21.1-bin.tar.gz.asc \
> apache-druid-0.21.1-bin.tar.gz
>
> Please review the proposed artifacts and vote. Note that Apache has
> specific requirements that must be met before +1 binding votes can be cast
> by PMC members. Please refer to the policy at
> http://www.apache.org/legal/release-policy.html#policy for more details.
>
> As part of the validation process, the release artifacts can be generated
> from source by running:
> mvn clean install -Papache-release,dist -Dgpg.skip
>
> The RAT license check can be run from source by:
> mvn apache-rat:check -Prat
>
> This vote will be open for at least 72 hours. The vote will pass if a
> majority of at least three +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Druid 0.21.1
> [ ] 0 I don't feel strongly about it, but I'm okay with the release
> [ ] -1 Do not release this package because...
>
> Thanks!
>