You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by "Rajendraprasad B. Hosamani" <ra...@gmail.com> on 2014/03/13 17:29:13 UTC

Trouble downloading kafka during bin/grid bootstrap

Hi All,
  I'm new to the samza project and this list. Really like what i see about
samza so far! Great work... keep it up :)

  I'm trying to setup a test system and run the hello-samza project. During
the execution of bin/grid bootstrap, the process exits out with the
following error
BUILD SUCCESSFUL

Total time: 5 mins 14.093 secs
~/misc/hello-samza
Downloading zookeeper-3.4.3...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time
Current
                                 Dload  Upload   Total   Spent    Left
Speed
100 15.4M  100 15.4M    0     0  61849      0  0:04:21  0:04:21 --:--:--
364k
Downloading hadoop-2.2.0...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time
Current
                                 Dload  Upload   Total   Spent    Left
Speed
100  104M  100  104M    0     0   743k      0  0:02:23  0:02:23 --:--:--
364k
Downloading kafka_2.8.0-0.8.0...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time
Current
                                 Dload  Upload   Total   Spent    Left
Speed
107   323  107   323    0     0   1507      0 --:--:-- --:--:-- --:--:--
3364
tar: This does not look like a tar archive

gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now

I think this is because the kafka mirror used in the grid script is broken.

DOWNLOAD_KAFKA=
http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz

Looks like the kafka download location
http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz gives
a 404 error.

This location used to work a few days back.

Perhaps change the download location to a different mirror?

-Raj

Re: Trouble downloading kafka during bin/grid bootstrap

Posted by "Rajendraprasad B. Hosamani" <ra...@gmail.com>.
Hey Chris,
  Sent the pull request. https://github.com/linkedin/hello-samza/pull/34

Let me know if everything is in order.

-Raj


On Thu, Mar 13, 2014 at 3:56 PM, Chris Riccomini <cr...@linkedin.com>wrote:

> Hey Raj,
>
> Good catch. Yeah, let's do both.
>
> Cheers,
> Chris
>
> On 3/13/14 12:47 PM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
> wrote:
>
> >Thanks Chris and Joe for the quick response.
> >
> >By the same token, I see that the YARN is also pointing to mirrors.sonic.
> >Should this be changed to say something like
> >
> https://archive.apache.org/dist/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.ta
> >r.gz
> >
> >Will send a pull request once i have heard back from you folks regarding
> >YARN.
> >
> >-Raj
> >
> >
> >On Thu, Mar 13, 2014 at 12:50 PM, Joe Stein <jo...@stealth.ly> wrote:
> >
> >> Yup.
> >>
> >> That is an Apache change requirement now.
> >>
> >> Only the latest build should be in front of the mirrors (and linked to
> >>the
> >> mirrors in download pages).  Everything always on archive.
> >>
> >> It is 30) a) in
> >> https://cwiki.apache.org/confluence/display/KAFKA/Release+Process just
> >>svn
> >> add and svn del to the Apache dist repo.
> >>
> >> /*******************************************
> >>  Joe Stein
> >>  Founder, Principal Consultant
> >>  Big Data Open Source Security LLC
> >>  http://www.stealth.ly
> >>  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> >> ********************************************/
> >>
> >>
> >> On Thu, Mar 13, 2014 at 12:40 PM, Chris Riccomini
> >> <cr...@linkedin.com>wrote:
> >>
> >> > Hey Raj,
> >> >
> >> > Looks like this broke when Kafka released 0.8.1. They  moved 0.8 to
> >>the
> >> > archives Apache server. You should be able to replace this URL in the
> >> > bin/grid script:
> >> >
> >> >
> https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> >> >
> >> > If you send a pull request for hello-samza, we'll merge it in.
> >> >
> >> > Cheers,
> >> > Chris
> >> >
> >> > On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
> >> > wrote:
> >> >
> >> > >Hi All,
> >> > >  I'm new to the samza project and this list. Really like what i see
> >> about
> >> > >samza so far! Great work... keep it up :)
> >> > >
> >> > >  I'm trying to setup a test system and run the hello-samza project.
> >> > >During
> >> > >the execution of bin/grid bootstrap, the process exits out with the
> >> > >following error
> >> > >BUILD SUCCESSFUL
> >> > >
> >> > >Total time: 5 mins 14.093 secs
> >> > >~/misc/hello-samza
> >> > >Downloading zookeeper-3.4.3...
> >> > >  % Total    % Received % Xferd  Average Speed   Time    Time
> >>Time
> >> > >Current
> >> > >                                 Dload  Upload   Total   Spent
> >>Left
> >> > >Speed
> >> > >100 15.4M  100 15.4M    0     0  61849      0  0:04:21  0:04:21
> >>--:--:--
> >> > >364k
> >> > >Downloading hadoop-2.2.0...
> >> > >  % Total    % Received % Xferd  Average Speed   Time    Time
> >>Time
> >> > >Current
> >> > >                                 Dload  Upload   Total   Spent
> >>Left
> >> > >Speed
> >> > >100  104M  100  104M    0     0   743k      0  0:02:23  0:02:23
> >>--:--:--
> >> > >364k
> >> > >Downloading kafka_2.8.0-0.8.0...
> >> > >  % Total    % Received % Xferd  Average Speed   Time    Time
> >>Time
> >> > >Current
> >> > >                                 Dload  Upload   Total   Spent
> >>Left
> >> > >Speed
> >> > >107   323  107   323    0     0   1507      0 --:--:-- --:--:--
> >>--:--:--
> >> > >3364
> >> > >tar: This does not look like a tar archive
> >> > >
> >> > >gzip: stdin: not in gzip format
> >> > >tar: Child returned status 1
> >> > >tar: Error is not recoverable: exiting now
> >> > >
> >> > >I think this is because the kafka mirror used in the grid script is
> >> > >broken.
> >> > >
> >> > >DOWNLOAD_KAFKA=
> >> > >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> >> > >
> >> > >Looks like the kafka download location
> >> > >
> >>
> >>
> http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gzgives
> >> > >a 404 error.
> >> > >
> >> > >This location used to work a few days back.
> >> > >
> >> > >Perhaps change the download location to a different mirror?
> >> > >
> >> > >-Raj
> >> >
> >> >
> >>
>
>

Re: Trouble downloading kafka during bin/grid bootstrap

Posted by Chris Riccomini <cr...@linkedin.com>.
Hey Raj,

Good catch. Yeah, let's do both.

Cheers,
Chris

On 3/13/14 12:47 PM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
wrote:

>Thanks Chris and Joe for the quick response.
>
>By the same token, I see that the YARN is also pointing to mirrors.sonic.
>Should this be changed to say something like
>https://archive.apache.org/dist/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.ta
>r.gz
>
>Will send a pull request once i have heard back from you folks regarding
>YARN.
>
>-Raj
>
>
>On Thu, Mar 13, 2014 at 12:50 PM, Joe Stein <jo...@stealth.ly> wrote:
>
>> Yup.
>>
>> That is an Apache change requirement now.
>>
>> Only the latest build should be in front of the mirrors (and linked to
>>the
>> mirrors in download pages).  Everything always on archive.
>>
>> It is 30) a) in
>> https://cwiki.apache.org/confluence/display/KAFKA/Release+Process just
>>svn
>> add and svn del to the Apache dist repo.
>>
>> /*******************************************
>>  Joe Stein
>>  Founder, Principal Consultant
>>  Big Data Open Source Security LLC
>>  http://www.stealth.ly
>>  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> ********************************************/
>>
>>
>> On Thu, Mar 13, 2014 at 12:40 PM, Chris Riccomini
>> <cr...@linkedin.com>wrote:
>>
>> > Hey Raj,
>> >
>> > Looks like this broke when Kafka released 0.8.1. They  moved 0.8 to
>>the
>> > archives Apache server. You should be able to replace this URL in the
>> > bin/grid script:
>> >
>> >   https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
>> >
>> > If you send a pull request for hello-samza, we'll merge it in.
>> >
>> > Cheers,
>> > Chris
>> >
>> > On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
>> > wrote:
>> >
>> > >Hi All,
>> > >  I'm new to the samza project and this list. Really like what i see
>> about
>> > >samza so far! Great work... keep it up :)
>> > >
>> > >  I'm trying to setup a test system and run the hello-samza project.
>> > >During
>> > >the execution of bin/grid bootstrap, the process exits out with the
>> > >following error
>> > >BUILD SUCCESSFUL
>> > >
>> > >Total time: 5 mins 14.093 secs
>> > >~/misc/hello-samza
>> > >Downloading zookeeper-3.4.3...
>> > >  % Total    % Received % Xferd  Average Speed   Time    Time
>>Time
>> > >Current
>> > >                                 Dload  Upload   Total   Spent
>>Left
>> > >Speed
>> > >100 15.4M  100 15.4M    0     0  61849      0  0:04:21  0:04:21
>>--:--:--
>> > >364k
>> > >Downloading hadoop-2.2.0...
>> > >  % Total    % Received % Xferd  Average Speed   Time    Time
>>Time
>> > >Current
>> > >                                 Dload  Upload   Total   Spent
>>Left
>> > >Speed
>> > >100  104M  100  104M    0     0   743k      0  0:02:23  0:02:23
>>--:--:--
>> > >364k
>> > >Downloading kafka_2.8.0-0.8.0...
>> > >  % Total    % Received % Xferd  Average Speed   Time    Time
>>Time
>> > >Current
>> > >                                 Dload  Upload   Total   Spent
>>Left
>> > >Speed
>> > >107   323  107   323    0     0   1507      0 --:--:-- --:--:--
>>--:--:--
>> > >3364
>> > >tar: This does not look like a tar archive
>> > >
>> > >gzip: stdin: not in gzip format
>> > >tar: Child returned status 1
>> > >tar: Error is not recoverable: exiting now
>> > >
>> > >I think this is because the kafka mirror used in the grid script is
>> > >broken.
>> > >
>> > >DOWNLOAD_KAFKA=
>> > >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
>> > >
>> > >Looks like the kafka download location
>> > >
>> 
>>http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gzgives
>> > >a 404 error.
>> > >
>> > >This location used to work a few days back.
>> > >
>> > >Perhaps change the download location to a different mirror?
>> > >
>> > >-Raj
>> >
>> >
>>


Re: Trouble downloading kafka during bin/grid bootstrap

Posted by "Rajendraprasad B. Hosamani" <ra...@gmail.com>.
Thanks Chris and Joe for the quick response.

By the same token, I see that the YARN is also pointing to mirrors.sonic.
Should this be changed to say something like
https://archive.apache.org/dist/hadoop/common/hadoop-2.2.0/hadoop-2.2.0.tar.gz

Will send a pull request once i have heard back from you folks regarding
YARN.

-Raj


On Thu, Mar 13, 2014 at 12:50 PM, Joe Stein <jo...@stealth.ly> wrote:

> Yup.
>
> That is an Apache change requirement now.
>
> Only the latest build should be in front of the mirrors (and linked to the
> mirrors in download pages).  Everything always on archive.
>
> It is 30) a) in
> https://cwiki.apache.org/confluence/display/KAFKA/Release+Process just svn
> add and svn del to the Apache dist repo.
>
> /*******************************************
>  Joe Stein
>  Founder, Principal Consultant
>  Big Data Open Source Security LLC
>  http://www.stealth.ly
>  Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
>
> On Thu, Mar 13, 2014 at 12:40 PM, Chris Riccomini
> <cr...@linkedin.com>wrote:
>
> > Hey Raj,
> >
> > Looks like this broke when Kafka released 0.8.1. They  moved 0.8 to the
> > archives Apache server. You should be able to replace this URL in the
> > bin/grid script:
> >
> >   https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> >
> > If you send a pull request for hello-samza, we'll merge it in.
> >
> > Cheers,
> > Chris
> >
> > On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
> > wrote:
> >
> > >Hi All,
> > >  I'm new to the samza project and this list. Really like what i see
> about
> > >samza so far! Great work... keep it up :)
> > >
> > >  I'm trying to setup a test system and run the hello-samza project.
> > >During
> > >the execution of bin/grid bootstrap, the process exits out with the
> > >following error
> > >BUILD SUCCESSFUL
> > >
> > >Total time: 5 mins 14.093 secs
> > >~/misc/hello-samza
> > >Downloading zookeeper-3.4.3...
> > >  % Total    % Received % Xferd  Average Speed   Time    Time     Time
> > >Current
> > >                                 Dload  Upload   Total   Spent    Left
> > >Speed
> > >100 15.4M  100 15.4M    0     0  61849      0  0:04:21  0:04:21 --:--:--
> > >364k
> > >Downloading hadoop-2.2.0...
> > >  % Total    % Received % Xferd  Average Speed   Time    Time     Time
> > >Current
> > >                                 Dload  Upload   Total   Spent    Left
> > >Speed
> > >100  104M  100  104M    0     0   743k      0  0:02:23  0:02:23 --:--:--
> > >364k
> > >Downloading kafka_2.8.0-0.8.0...
> > >  % Total    % Received % Xferd  Average Speed   Time    Time     Time
> > >Current
> > >                                 Dload  Upload   Total   Spent    Left
> > >Speed
> > >107   323  107   323    0     0   1507      0 --:--:-- --:--:-- --:--:--
> > >3364
> > >tar: This does not look like a tar archive
> > >
> > >gzip: stdin: not in gzip format
> > >tar: Child returned status 1
> > >tar: Error is not recoverable: exiting now
> > >
> > >I think this is because the kafka mirror used in the grid script is
> > >broken.
> > >
> > >DOWNLOAD_KAFKA=
> > >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> > >
> > >Looks like the kafka download location
> > >
> http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gzgives
> > >a 404 error.
> > >
> > >This location used to work a few days back.
> > >
> > >Perhaps change the download location to a different mirror?
> > >
> > >-Raj
> >
> >
>

Re: Trouble downloading kafka during bin/grid bootstrap

Posted by Joe Stein <jo...@stealth.ly>.
Yup.

That is an Apache change requirement now.

Only the latest build should be in front of the mirrors (and linked to the
mirrors in download pages).  Everything always on archive.

It is 30) a) in
https://cwiki.apache.org/confluence/display/KAFKA/Release+Process just svn
add and svn del to the Apache dist repo.

/*******************************************
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/


On Thu, Mar 13, 2014 at 12:40 PM, Chris Riccomini
<cr...@linkedin.com>wrote:

> Hey Raj,
>
> Looks like this broke when Kafka released 0.8.1. They  moved 0.8 to the
> archives Apache server. You should be able to replace this URL in the
> bin/grid script:
>
>   https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
>
> If you send a pull request for hello-samza, we'll merge it in.
>
> Cheers,
> Chris
>
> On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com>
> wrote:
>
> >Hi All,
> >  I'm new to the samza project and this list. Really like what i see about
> >samza so far! Great work... keep it up :)
> >
> >  I'm trying to setup a test system and run the hello-samza project.
> >During
> >the execution of bin/grid bootstrap, the process exits out with the
> >following error
> >BUILD SUCCESSFUL
> >
> >Total time: 5 mins 14.093 secs
> >~/misc/hello-samza
> >Downloading zookeeper-3.4.3...
> >  % Total    % Received % Xferd  Average Speed   Time    Time     Time
> >Current
> >                                 Dload  Upload   Total   Spent    Left
> >Speed
> >100 15.4M  100 15.4M    0     0  61849      0  0:04:21  0:04:21 --:--:--
> >364k
> >Downloading hadoop-2.2.0...
> >  % Total    % Received % Xferd  Average Speed   Time    Time     Time
> >Current
> >                                 Dload  Upload   Total   Spent    Left
> >Speed
> >100  104M  100  104M    0     0   743k      0  0:02:23  0:02:23 --:--:--
> >364k
> >Downloading kafka_2.8.0-0.8.0...
> >  % Total    % Received % Xferd  Average Speed   Time    Time     Time
> >Current
> >                                 Dload  Upload   Total   Spent    Left
> >Speed
> >107   323  107   323    0     0   1507      0 --:--:-- --:--:-- --:--:--
> >3364
> >tar: This does not look like a tar archive
> >
> >gzip: stdin: not in gzip format
> >tar: Child returned status 1
> >tar: Error is not recoverable: exiting now
> >
> >I think this is because the kafka mirror used in the grid script is
> >broken.
> >
> >DOWNLOAD_KAFKA=
> >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
> >
> >Looks like the kafka download location
> >http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gzgives
> >a 404 error.
> >
> >This location used to work a few days back.
> >
> >Perhaps change the download location to a different mirror?
> >
> >-Raj
>
>

Re: Trouble downloading kafka during bin/grid bootstrap

Posted by Chris Riccomini <cr...@linkedin.com>.
Hey Raj,

Looks like this broke when Kafka released 0.8.1. They  moved 0.8 to the
archives Apache server. You should be able to replace this URL in the
bin/grid script:

  https://archive.apache.org/dist/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz

If you send a pull request for hello-samza, we'll merge it in.

Cheers,
Chris

On 3/13/14 9:29 AM, "Rajendraprasad B. Hosamani" <ra...@gmail.com> wrote:

>Hi All,
>  I'm new to the samza project and this list. Really like what i see about
>samza so far! Great work... keep it up :)
>
>  I'm trying to setup a test system and run the hello-samza project.
>During
>the execution of bin/grid bootstrap, the process exits out with the
>following error
>BUILD SUCCESSFUL
>
>Total time: 5 mins 14.093 secs
>~/misc/hello-samza
>Downloading zookeeper-3.4.3...
>  % Total    % Received % Xferd  Average Speed   Time    Time     Time
>Current
>                                 Dload  Upload   Total   Spent    Left
>Speed
>100 15.4M  100 15.4M    0     0  61849      0  0:04:21  0:04:21 --:--:--
>364k
>Downloading hadoop-2.2.0...
>  % Total    % Received % Xferd  Average Speed   Time    Time     Time
>Current
>                                 Dload  Upload   Total   Spent    Left
>Speed
>100  104M  100  104M    0     0   743k      0  0:02:23  0:02:23 --:--:--
>364k
>Downloading kafka_2.8.0-0.8.0...
>  % Total    % Received % Xferd  Average Speed   Time    Time     Time
>Current
>                                 Dload  Upload   Total   Spent    Left
>Speed
>107   323  107   323    0     0   1507      0 --:--:-- --:--:-- --:--:--
>3364
>tar: This does not look like a tar archive
>
>gzip: stdin: not in gzip format
>tar: Child returned status 1
>tar: Error is not recoverable: exiting now
>
>I think this is because the kafka mirror used in the grid script is
>broken.
>
>DOWNLOAD_KAFKA=
>http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz
>
>Looks like the kafka download location
>http://mirrors.sonic.net/apache/kafka/0.8.0/kafka_2.8.0-0.8.0.tar.gz gives
>a 404 error.
>
>This location used to work a few days back.
>
>Perhaps change the download location to a different mirror?
>
>-Raj