You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by "Rajendraprasad B. Hosamani" <ra...@gmail.com> on 2014/03/13 17:29:13 UTC
Trouble downloading kafka during bin/grid bootstrap
Hi All,
I'm new to the samza project and this list. Really like what i see about
samza so far! Great work... keep it up :)
I'm trying to setup a test system and run the hello-samza project. During
the execution of bin/grid bootstrap, the process exits out with the
following error
BUILD SUCCESSFUL
Total time: 5 mins 14.093 secs
~/misc/hello-samza
Downloading zookeeper-3.4.3...
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 15.4M 100 15.4M 0 0 61849 0 0:04:21 0:04:21 --:--:--
364k
Downloading hadoop-2.2.0...
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 104M 100 104M 0 0 743k 0 0:02:23 0:02:23 --:--:--
364k
Downloading kafka_2.8.0-0.8.0...
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
107 323 107 323 0 0 1507 0 --:--:-- --:--:-- --:--:--
3364
tar: This does not look like a tar archive
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
I think this is because the kafka mirror used in the grid script is broken.
DOWNLOAD_KAFKA=
http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
Looks like the kafka download location
http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz gives
a 404 error.
This location used to work a few days back.
Perhaps change the download location to a different mirror?
-Raj
Re: Trouble downloading kafka during bin/grid bootstrap
Posted by "Rajendraprasad B. Hosamani" <ra...@gmail.com>.
Hey Chris,
Sent the pull request. https://github.com/linkedin/hello-samza/pull/34
Let me know if everything is in order.
-Raj
On Thu, Mar 13, 2014 at 3:56 PM, Chris Riccomini <cr...@linkedin.com>wrote:
> Hey Raj,
>
> Good catch. Yeah, let's do both.
>
> Cheers,
> Chris
>
> On 3/13/14 12:47 PM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
> wrote:
>
> >Thanks Chris and Joe for the quick response.
> >
> >By the same token, I see that the YARN is also pointing to mirrors.sonic.
> >Should this be changed to say something like
> >
> https://archive.apache.org/dist/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.ta
> >r.gz
> >
> >Will send a pull request once i have heard back from you folks regarding
> >YARN.
> >
> >-Raj
> >
> >
> >On Thu, Mar 13, 2014 at 12:50 PM, Joe Stein <jo...@stealth.ly> wrote:
> >
> >> Yup.
> >>
> >> That is an Apache change requirement now.
> >>
> >> Only the latest build should be in front of the mirrors (and linked to
> >>the
> >> mirrors in download pages). Everything always on archive.
> >>
> >> It is 30) a) in
> >> https://cwiki.apache.org/confluence/display/KAFKA/Release+Process just
> >>svn
> >> add and svn del to the Apache dist repo.
> >>
> >> /*******************************************
> >> Joe Stein
> >> Founder, Principal Consultant
> >> Big Data Open Source Security LLC
> >> http://www.stealth.ly
> >> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> >> ********************************************/
> >>
> >>
> >> On Thu, Mar 13, 2014 at 12:40 PM, Chris Riccomini
> >> <cr...@linkedin.com>wrote:
> >>
> >> > Hey Raj,
> >> >
> >> > Looks like this broke when Kafka released 0.8.1. They moved 0.8 to
> >>the
> >> > archives Apache server. You should be able to replace this URL in the
> >> > bin/grid script:
> >> >
> >> >
> https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> >> >
> >> > If you send a pull request for hello-samza, we'll merge it in.
> >> >
> >> > Cheers,
> >> > Chris
> >> >
> >> > On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
> >> > wrote:
> >> >
> >> > >Hi All,
> >> > > I'm new to the samza project and this list. Really like what i see
> >> about
> >> > >samza so far! Great work... keep it up :)
> >> > >
> >> > > I'm trying to setup a test system and run the hello-samza project.
> >> > >During
> >> > >the execution of bin/grid bootstrap, the process exits out with the
> >> > >following error
> >> > >BUILD SUCCESSFUL
> >> > >
> >> > >Total time: 5 mins 14.093 secs
> >> > >~/misc/hello-samza
> >> > >Downloading zookeeper-3.4.3...
> >> > > % Total % Received % Xferd Average Speed Time Time
> >>Time
> >> > >Current
> >> > > Dload Upload Total Spent
> >>Left
> >> > >Speed
> >> > >100 15.4M 100 15.4M 0 0 61849 0 0:04:21 0:04:21
> >>--:--:--
> >> > >364k
> >> > >Downloading hadoop-2.2.0...
> >> > > % Total % Received % Xferd Average Speed Time Time
> >>Time
> >> > >Current
> >> > > Dload Upload Total Spent
> >>Left
> >> > >Speed
> >> > >100 104M 100 104M 0 0 743k 0 0:02:23 0:02:23
> >>--:--:--
> >> > >364k
> >> > >Downloading kafka_2.8.0-0.8.0...
> >> > > % Total % Received % Xferd Average Speed Time Time
> >>Time
> >> > >Current
> >> > > Dload Upload Total Spent
> >>Left
> >> > >Speed
> >> > >107 323 107 323 0 0 1507 0 --:--:-- --:--:--
> >>--:--:--
> >> > >3364
> >> > >tar: This does not look like a tar archive
> >> > >
> >> > >gzip: stdin: not in gzip format
> >> > >tar: Child returned status 1
> >> > >tar: Error is not recoverable: exiting now
> >> > >
> >> > >I think this is because the kafka mirror used in the grid script is
> >> > >broken.
> >> > >
> >> > >DOWNLOAD_KAFKA=
> >> > >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> >> > >
> >> > >Looks like the kafka download location
> >> > >
> >>
> >>
> http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gzgives
> >> > >a 404 error.
> >> > >
> >> > >This location used to work a few days back.
> >> > >
> >> > >Perhaps change the download location to a different mirror?
> >> > >
> >> > >-Raj
> >> >
> >> >
> >>
>
>
Re: Trouble downloading kafka during bin/grid bootstrap
Posted by Chris Riccomini <cr...@linkedin.com>.
Hey Raj,
Good catch. Yeah, let's do both.
Cheers,
Chris
On 3/13/14 12:47 PM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
wrote:
>Thanks Chris and Joe for the quick response.
>
>By the same token, I see that the YARN is also pointing to mirrors.sonic.
>Should this be changed to say something like
>https://archive.apache.org/dist/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.ta
>r.gz
>
>Will send a pull request once i have heard back from you folks regarding
>YARN.
>
>-Raj
>
>
>On Thu, Mar 13, 2014 at 12:50 PM, Joe Stein <jo...@stealth.ly> wrote:
>
>> Yup.
>>
>> That is an Apache change requirement now.
>>
>> Only the latest build should be in front of the mirrors (and linked to
>>the
>> mirrors in download pages). Everything always on archive.
>>
>> It is 30) a) in
>> https://cwiki.apache.org/confluence/display/KAFKA/Release+Process just
>>svn
>> add and svn del to the Apache dist repo.
>>
>> /*******************************************
>> Joe Stein
>> Founder, Principal Consultant
>> Big Data Open Source Security LLC
>> http://www.stealth.ly
>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> ********************************************/
>>
>>
>> On Thu, Mar 13, 2014 at 12:40 PM, Chris Riccomini
>> <cr...@linkedin.com>wrote:
>>
>> > Hey Raj,
>> >
>> > Looks like this broke when Kafka released 0.8.1. They moved 0.8 to
>>the
>> > archives Apache server. You should be able to replace this URL in the
>> > bin/grid script:
>> >
>> > https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
>> >
>> > If you send a pull request for hello-samza, we'll merge it in.
>> >
>> > Cheers,
>> > Chris
>> >
>> > On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
>> > wrote:
>> >
>> > >Hi All,
>> > > I'm new to the samza project and this list. Really like what i see
>> about
>> > >samza so far! Great work... keep it up :)
>> > >
>> > > I'm trying to setup a test system and run the hello-samza project.
>> > >During
>> > >the execution of bin/grid bootstrap, the process exits out with the
>> > >following error
>> > >BUILD SUCCESSFUL
>> > >
>> > >Total time: 5 mins 14.093 secs
>> > >~/misc/hello-samza
>> > >Downloading zookeeper-3.4.3...
>> > > % Total % Received % Xferd Average Speed Time Time
>>Time
>> > >Current
>> > > Dload Upload Total Spent
>>Left
>> > >Speed
>> > >100 15.4M 100 15.4M 0 0 61849 0 0:04:21 0:04:21
>>--:--:--
>> > >364k
>> > >Downloading hadoop-2.2.0...
>> > > % Total % Received % Xferd Average Speed Time Time
>>Time
>> > >Current
>> > > Dload Upload Total Spent
>>Left
>> > >Speed
>> > >100 104M 100 104M 0 0 743k 0 0:02:23 0:02:23
>>--:--:--
>> > >364k
>> > >Downloading kafka_2.8.0-0.8.0...
>> > > % Total % Received % Xferd Average Speed Time Time
>>Time
>> > >Current
>> > > Dload Upload Total Spent
>>Left
>> > >Speed
>> > >107 323 107 323 0 0 1507 0 --:--:-- --:--:--
>>--:--:--
>> > >3364
>> > >tar: This does not look like a tar archive
>> > >
>> > >gzip: stdin: not in gzip format
>> > >tar: Child returned status 1
>> > >tar: Error is not recoverable: exiting now
>> > >
>> > >I think this is because the kafka mirror used in the grid script is
>> > >broken.
>> > >
>> > >DOWNLOAD_KAFKA=
>> > >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
>> > >
>> > >Looks like the kafka download location
>> > >
>>
>>http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gzgives
>> > >a 404 error.
>> > >
>> > >This location used to work a few days back.
>> > >
>> > >Perhaps change the download location to a different mirror?
>> > >
>> > >-Raj
>> >
>> >
>>
Re: Trouble downloading kafka during bin/grid bootstrap
Posted by "Rajendraprasad B. Hosamani" <ra...@gmail.com>.
Thanks Chris and Joe for the quick response.
By the same token, I see that the YARN is also pointing to mirrors.sonic.
Should this be changed to say something like
https://archive.apache.org/dist/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.tar.gz
Will send a pull request once i have heard back from you folks regarding
YARN.
-Raj
On Thu, Mar 13, 2014 at 12:50 PM, Joe Stein <jo...@stealth.ly> wrote:
> Yup.
>
> That is an Apache change requirement now.
>
> Only the latest build should be in front of the mirrors (and linked to the
> mirrors in download pages). Everything always on archive.
>
> It is 30) a) in
> https://cwiki.apache.org/confluence/display/KAFKA/Release+Process just svn
> add and svn del to the Apache dist repo.
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
>
> On Thu, Mar 13, 2014 at 12:40 PM, Chris Riccomini
> <cr...@linkedin.com>wrote:
>
> > Hey Raj,
> >
> > Looks like this broke when Kafka released 0.8.1. They moved 0.8 to the
> > archives Apache server. You should be able to replace this URL in the
> > bin/grid script:
> >
> > https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> >
> > If you send a pull request for hello-samza, we'll merge it in.
> >
> > Cheers,
> > Chris
> >
> > On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
> > wrote:
> >
> > >Hi All,
> > > I'm new to the samza project and this list. Really like what i see
> about
> > >samza so far! Great work... keep it up :)
> > >
> > > I'm trying to setup a test system and run the hello-samza project.
> > >During
> > >the execution of bin/grid bootstrap, the process exits out with the
> > >following error
> > >BUILD SUCCESSFUL
> > >
> > >Total time: 5 mins 14.093 secs
> > >~/misc/hello-samza
> > >Downloading zookeeper-3.4.3...
> > > % Total % Received % Xferd Average Speed Time Time Time
> > >Current
> > > Dload Upload Total Spent Left
> > >Speed
> > >100 15.4M 100 15.4M 0 0 61849 0 0:04:21 0:04:21 --:--:--
> > >364k
> > >Downloading hadoop-2.2.0...
> > > % Total % Received % Xferd Average Speed Time Time Time
> > >Current
> > > Dload Upload Total Spent Left
> > >Speed
> > >100 104M 100 104M 0 0 743k 0 0:02:23 0:02:23 --:--:--
> > >364k
> > >Downloading kafka_2.8.0-0.8.0...
> > > % Total % Received % Xferd Average Speed Time Time Time
> > >Current
> > > Dload Upload Total Spent Left
> > >Speed
> > >107 323 107 323 0 0 1507 0 --:--:-- --:--:-- --:--:--
> > >3364
> > >tar: This does not look like a tar archive
> > >
> > >gzip: stdin: not in gzip format
> > >tar: Child returned status 1
> > >tar: Error is not recoverable: exiting now
> > >
> > >I think this is because the kafka mirror used in the grid script is
> > >broken.
> > >
> > >DOWNLOAD_KAFKA=
> > >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> > >
> > >Looks like the kafka download location
> > >
> http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gzgives
> > >a 404 error.
> > >
> > >This location used to work a few days back.
> > >
> > >Perhaps change the download location to a different mirror?
> > >
> > >-Raj
> >
> >
>
Re: Trouble downloading kafka during bin/grid bootstrap
Posted by Joe Stein <jo...@stealth.ly>.
Yup.
That is an Apache change requirement now.
Only the latest build should be in front of the mirrors (and linked to the
mirrors in download pages). Everything always on archive.
It is 30) a) in
https://cwiki.apache.org/confluence/display/KAFKA/Release+Process just svn
add and svn del to the Apache dist repo.
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Thu, Mar 13, 2014 at 12:40 PM, Chris Riccomini
<cr...@linkedin.com>wrote:
> Hey Raj,
>
> Looks like this broke when Kafka released 0.8.1. They moved 0.8 to the
> archives Apache server. You should be able to replace this URL in the
> bin/grid script:
>
> https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
>
> If you send a pull request for hello-samza, we'll merge it in.
>
> Cheers,
> Chris
>
> On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
> wrote:
>
> >Hi All,
> > I'm new to the samza project and this list. Really like what i see about
> >samza so far! Great work... keep it up :)
> >
> > I'm trying to setup a test system and run the hello-samza project.
> >During
> >the execution of bin/grid bootstrap, the process exits out with the
> >following error
> >BUILD SUCCESSFUL
> >
> >Total time: 5 mins 14.093 secs
> >~/misc/hello-samza
> >Downloading zookeeper-3.4.3...
> > % Total % Received % Xferd Average Speed Time Time Time
> >Current
> > Dload Upload Total Spent Left
> >Speed
> >100 15.4M 100 15.4M 0 0 61849 0 0:04:21 0:04:21 --:--:--
> >364k
> >Downloading hadoop-2.2.0...
> > % Total % Received % Xferd Average Speed Time Time Time
> >Current
> > Dload Upload Total Spent Left
> >Speed
> >100 104M 100 104M 0 0 743k 0 0:02:23 0:02:23 --:--:--
> >364k
> >Downloading kafka_2.8.0-0.8.0...
> > % Total % Received % Xferd Average Speed Time Time Time
> >Current
> > Dload Upload Total Spent Left
> >Speed
> >107 323 107 323 0 0 1507 0 --:--:-- --:--:-- --:--:--
> >3364
> >tar: This does not look like a tar archive
> >
> >gzip: stdin: not in gzip format
> >tar: Child returned status 1
> >tar: Error is not recoverable: exiting now
> >
> >I think this is because the kafka mirror used in the grid script is
> >broken.
> >
> >DOWNLOAD_KAFKA=
> >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> >
> >Looks like the kafka download location
> >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gzgives
> >a 404 error.
> >
> >This location used to work a few days back.
> >
> >Perhaps change the download location to a different mirror?
> >
> >-Raj
>
>
Re: Trouble downloading kafka during bin/grid bootstrap
Posted by Chris Riccomini <cr...@linkedin.com>.
Hey Raj,
Looks like this broke when Kafka released 0.8.1. They moved 0.8 to the
archives Apache server. You should be able to replace this URL in the
bin/grid script:
https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
If you send a pull request for hello-samza, we'll merge it in.
Cheers,
Chris
On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com> wrote:
>Hi All,
> I'm new to the samza project and this list. Really like what i see about
>samza so far! Great work... keep it up :)
>
> I'm trying to setup a test system and run the hello-samza project.
>During
>the execution of bin/grid bootstrap, the process exits out with the
>following error
>BUILD SUCCESSFUL
>
>Total time: 5 mins 14.093 secs
>~/misc/hello-samza
>Downloading zookeeper-3.4.3...
> % Total % Received % Xferd Average Speed Time Time Time
>Current
> Dload Upload Total Spent Left
>Speed
>100 15.4M 100 15.4M 0 0 61849 0 0:04:21 0:04:21 --:--:--
>364k
>Downloading hadoop-2.2.0...
> % Total % Received % Xferd Average Speed Time Time Time
>Current
> Dload Upload Total Spent Left
>Speed
>100 104M 100 104M 0 0 743k 0 0:02:23 0:02:23 --:--:--
>364k
>Downloading kafka_2.8.0-0.8.0...
> % Total % Received % Xferd Average Speed Time Time Time
>Current
> Dload Upload Total Spent Left
>Speed
>107 323 107 323 0 0 1507 0 --:--:-- --:--:-- --:--:--
>3364
>tar: This does not look like a tar archive
>
>gzip: stdin: not in gzip format
>tar: Child returned status 1
>tar: Error is not recoverable: exiting now
>
>I think this is because the kafka mirror used in the grid script is
>broken.
>
>DOWNLOAD_KAFKA=
>http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
>
>Looks like the kafka download location
>http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz gives
>a 404 error.
>
>This location used to work a few days back.
>
>Perhaps change the download location to a different mirror?
>
>-Raj