You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@giraph.apache.org by Roman Shaposhnik <ro...@shaposhnik.org> on 2014/11/01 23:49:47 UTC
Re: [VOTE] Apache Giraph 1.1.0 RC1
Ping! Any progress on testing the current RC?
Thanks,
Roman.
On Fri, Oct 31, 2014 at 9:00 AM, Claudio Martella
<cl...@gmail.com> wrote:
> Oh, thanks for the info!
>
> On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik <ro...@shaposhnik.org>
> wrote:
>>
>> On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella
>> <cl...@gmail.com> wrote:
>> > Hi Roman,
>> >
>> > thanks again for this. I have had a look at the staging site so far (our
>> > cluster has been down whole week... universities...), and I was
>> > wondering if
>> > you have an insight why some of the docs are missing, e.g. gora and
>> > rexster
>> > documentation.
>>
>> None of them are missing. The links moved to a User Docs -> Modules
>> though:
>> http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/gora.html
>> http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/rexster.html
>> and so forth.
>>
>> Thanks,
>> Roman.
>
>
>
>
> --
> Claudio Martella
>
Re: [VOTE] Apache Giraph 1.1.0 RC1
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
Hi!
as per prior suggestion I've recut RC2 with the only
extra fix included:
https://issues.apache.org/jira/browse/GIRAPH-961
Given the scope of the fix, I firmly believe that the
testing we've conducted so far should be 100%
applicable to the new RC.
Still, I started a new vote thread to keep things clean.
Claudio, can you please recast your vote in that
other new thread?
Maja, Eugene, we really need you guys to vote. Please
do so. We're sooo close to finally push 1.1.0 out.
Thanks,
Roman.
On Mon, Nov 10, 2014 at 4:01 AM, Claudio Martella
<cl...@gmail.com> wrote:
> Yes,
>
> I did re-run the build this weekend, and it built succesfully for the
> default profile and the hadoop_2 one.
> I ran a couple of examples on the cluster, and it ran succesfully.
>
> I'm +1.
>
> On Tue, Nov 4, 2014 at 8:10 PM, Roman Shaposhnik <ro...@shaposhnik.org>
> wrote:
>>
>> On Tue, Nov 4, 2014 at 5:47 AM, Claudio Martella
>> <cl...@gmail.com> wrote:
>> > I am indeed having some problems. mvn install will fail because the test
>> > is
>> > opening too many files:
>>
>> [snip]
>>
>> > I have to investigate why this happens. I'm not using a different ulimit
>> > than what I have on my Mac OS X by default. Where are you building
>> > yours?
>>
>> This is really weird. I have not issues whatsoever on Mac OS X with
>> the following setup:
>> $ uname -a
>> Darwin usxxshaporm1.corp.emc.com 12.4.1 Darwin Kernel Version
>> 12.4.1: Tue May 21 17:04:50 PDT 2013;
>> root:xnu-2050.40.51~1/RELEASE_X86_64 x86_64
>> $ ulimit -a
>> core file size (blocks, -c) 0
>> data seg size (kbytes, -d) unlimited
>> file size (blocks, -f) unlimited
>> max locked memory (kbytes, -l) unlimited
>> max memory size (kbytes, -m) unlimited
>> open files (-n) 2560
>> pipe size (512 bytes, -p) 1
>> stack size (kbytes, -s) 8192
>> cpu time (seconds, -t) unlimited
>> max user processes (-u) 709
>> virtual memory (kbytes, -v) unlimited
>> $ mvn --version
>> Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4;
>> 2014-08-11T13:58:10-07:00)
>> Maven home: /Users/shapor/dist/apache-maven-3.2.3
>> Java version: 1.7.0_51, vendor: Oracle Corporation
>> Java home:
>> /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre
>> Default locale: en_US, platform encoding: UTF-8
>> OS name: "mac os x", version: "10.8.4", arch: "x86_64", family: "mac"
>>
>>
>> Thanks,
>> Roman.
>
>
>
>
> --
> Claudio Martella
>
Re: [VOTE] Apache Giraph 1.1.0 RC1
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
Hi!
as per prior suggestion I've recut RC2 with the only
extra fix included:
https://issues.apache.org/jira/browse/GIRAPH-961
Given the scope of the fix, I firmly believe that the
testing we've conducted so far should be 100%
applicable to the new RC.
Still, I started a new vote thread to keep things clean.
Claudio, can you please recast your vote in that
other new thread?
Maja, Eugene, we really need you guys to vote. Please
do so. We're sooo close to finally push 1.1.0 out.
Thanks,
Roman.
On Mon, Nov 10, 2014 at 4:01 AM, Claudio Martella
<cl...@gmail.com> wrote:
> Yes,
>
> I did re-run the build this weekend, and it built succesfully for the
> default profile and the hadoop_2 one.
> I ran a couple of examples on the cluster, and it ran succesfully.
>
> I'm +1.
>
> On Tue, Nov 4, 2014 at 8:10 PM, Roman Shaposhnik <ro...@shaposhnik.org>
> wrote:
>>
>> On Tue, Nov 4, 2014 at 5:47 AM, Claudio Martella
>> <cl...@gmail.com> wrote:
>> > I am indeed having some problems. mvn install will fail because the test
>> > is
>> > opening too many files:
>>
>> [snip]
>>
>> > I have to investigate why this happens. I'm not using a different ulimit
>> > than what I have on my Mac OS X by default. Where are you building
>> > yours?
>>
>> This is really weird. I have not issues whatsoever on Mac OS X with
>> the following setup:
>> $ uname -a
>> Darwin usxxshaporm1.corp.emc.com 12.4.1 Darwin Kernel Version
>> 12.4.1: Tue May 21 17:04:50 PDT 2013;
>> root:xnu-2050.40.51~1/RELEASE_X86_64 x86_64
>> $ ulimit -a
>> core file size (blocks, -c) 0
>> data seg size (kbytes, -d) unlimited
>> file size (blocks, -f) unlimited
>> max locked memory (kbytes, -l) unlimited
>> max memory size (kbytes, -m) unlimited
>> open files (-n) 2560
>> pipe size (512 bytes, -p) 1
>> stack size (kbytes, -s) 8192
>> cpu time (seconds, -t) unlimited
>> max user processes (-u) 709
>> virtual memory (kbytes, -v) unlimited
>> $ mvn --version
>> Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4;
>> 2014-08-11T13:58:10-07:00)
>> Maven home: /Users/shapor/dist/apache-maven-3.2.3
>> Java version: 1.7.0_51, vendor: Oracle Corporation
>> Java home:
>> /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre
>> Default locale: en_US, platform encoding: UTF-8
>> OS name: "mac os x", version: "10.8.4", arch: "x86_64", family: "mac"
>>
>>
>> Thanks,
>> Roman.
>
>
>
>
> --
> Claudio Martella
>
Re: [VOTE] Apache Giraph 1.1.0 RC1
Posted by Claudio Martella <cl...@gmail.com>.
Yes,
I did re-run the build this weekend, and it built succesfully for the
default profile and the hadoop_2 one.
I ran a couple of examples on the cluster, and it ran succesfully.
I'm +1.
On Tue, Nov 4, 2014 at 8:10 PM, Roman Shaposhnik <ro...@shaposhnik.org>
wrote:
> On Tue, Nov 4, 2014 at 5:47 AM, Claudio Martella
> <cl...@gmail.com> wrote:
> > I am indeed having some problems. mvn install will fail because the test
> is
> > opening too many files:
>
> [snip]
>
> > I have to investigate why this happens. I'm not using a different ulimit
> > than what I have on my Mac OS X by default. Where are you building yours?
>
> This is really weird. I have not issues whatsoever on Mac OS X with
> the following setup:
> $ uname -a
> Darwin usxxshaporm1.corp.emc.com 12.4.1 Darwin Kernel Version
> 12.4.1: Tue May 21 17:04:50 PDT 2013;
> root:xnu-2050.40.51~1/RELEASE_X86_64 x86_64
> $ ulimit -a
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> file size (blocks, -f) unlimited
> max locked memory (kbytes, -l) unlimited
> max memory size (kbytes, -m) unlimited
> open files (-n) 2560
> pipe size (512 bytes, -p) 1
> stack size (kbytes, -s) 8192
> cpu time (seconds, -t) unlimited
> max user processes (-u) 709
> virtual memory (kbytes, -v) unlimited
> $ mvn --version
> Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4;
> 2014-08-11T13:58:10-07:00)
> Maven home: /Users/shapor/dist/apache-maven-3.2.3
> Java version: 1.7.0_51, vendor: Oracle Corporation
> Java home:
> /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "mac os x", version: "10.8.4", arch: "x86_64", family: "mac"
>
>
> Thanks,
> Roman.
>
--
Claudio Martella
Re: [VOTE] Apache Giraph 1.1.0 RC1
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Tue, Nov 4, 2014 at 5:47 AM, Claudio Martella
<cl...@gmail.com> wrote:
> I am indeed having some problems. mvn install will fail because the test is
> opening too many files:
[snip]
> I have to investigate why this happens. I'm not using a different ulimit
> than what I have on my Mac OS X by default. Where are you building yours?
This is really weird. I have not issues whatsoever on Mac OS X with
the following setup:
$ uname -a
Darwin usxxshaporm1.corp.emc.com 12.4.1 Darwin Kernel Version
12.4.1: Tue May 21 17:04:50 PDT 2013;
root:xnu-2050.40.51~1/RELEASE_X86_64 x86_64
$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 2560
pipe size (512 bytes, -p) 1
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 709
virtual memory (kbytes, -v) unlimited
$ mvn --version
Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4;
2014-08-11T13:58:10-07:00)
Maven home: /Users/shapor/dist/apache-maven-3.2.3
Java version: 1.7.0_51, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.8.4", arch: "x86_64", family: "mac"
Thanks,
Roman.
Re: [VOTE] Apache Giraph 1.1.0 RC1
Posted by Claudio Martella <cl...@gmail.com>.
I am indeed having some problems. mvn install will fail because the test is
opening too many files:
Caused by: java.io.FileNotFoundException:
/private/var/folders/5b/8yx5dbyn40nbt_70syjs86ch0000gp/T/giraph-hive-1415098102276/metastore_db/seg0/c90.dat
(Too many open files in system)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
at org.apache.derby.impl.io.DirRandomAccessFile.<init>(Unknown Source)
at org.apache.derby.impl.io.DirRandomAccessFile4.<init>(Unknown Source)
at org.apache.derby.impl.io.DirFile4.getRandomAccessFile(Unknown Source)
at org.apache.derby.impl.store.raw.data.RAFContainer.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at
org.apache.derby.impl.store.raw.data.RAFContainer.createContainer(Unknown
Source)
at
org.apache.derby.impl.store.raw.data.RAFContainer4.createContainer(Unknown
Source)
at org.apache.derby.impl.store.raw.data.FileContainer.createIdent(Unknown
Source)
at org.apache.derby.impl.store.raw.data.RAFContainer.createIdentity(Unknown
Source)
at org.apache.derby.impl.services.cache.ConcurrentCache.create(Unknown
Source)
at
org.apache.derby.impl.store.raw.data.BaseDataFileFactory.addContainer(Unknown
Source)
at org.apache.derby.impl.store.raw.xact.Xact.addContainer(Unknown Source)
at org.apache.derby.impl.store.access.heap.Heap.create(Unknown Source)
at
org.apache.derby.impl.store.access.heap.HeapConglomerateFactory.createConglomerate(Unknown
Source)
at
org.apache.derby.impl.store.access.RAMTransaction.createConglomerate(Unknown
Source)
at
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.createConglomerate(Unknown
Source)
at
org.apache.derby.impl.sql.catalog.DataDictionaryImpl.createDictionaryTables(Unknown
Source)
at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.boot(Unknown Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown
Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown
Source)
at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown
Source)
at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown
Source)
at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown
Source)
at
org.apache.derby.impl.services.monitor.BaseMonitor.createPersistentService(Unknown
Source)
at
org.apache.derby.iapi.services.monitor.Monitor.createPersistentService(Unknown
Source)
... 96 more
I have to investigate why this happens. I'm not using a different ulimit
than what I have on my Mac OS X by default. Where are you building yours?
On Sat, Nov 1, 2014 at 11:49 PM, Roman Shaposhnik <ro...@shaposhnik.org>
wrote:
> Ping! Any progress on testing the current RC?
>
> Thanks,
> Roman.
>
> On Fri, Oct 31, 2014 at 9:00 AM, Claudio Martella
> <cl...@gmail.com> wrote:
> > Oh, thanks for the info!
> >
> > On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik <ro...@shaposhnik.org>
> > wrote:
> >>
> >> On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella
> >> <cl...@gmail.com> wrote:
> >> > Hi Roman,
> >> >
> >> > thanks again for this. I have had a look at the staging site so far
> (our
> >> > cluster has been down whole week... universities...), and I was
> >> > wondering if
> >> > you have an insight why some of the docs are missing, e.g. gora and
> >> > rexster
> >> > documentation.
> >>
> >> None of them are missing. The links moved to a User Docs -> Modules
> >> though:
> >> http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/gora.html
> >> http://people.apache.org/~rvs/giraph-1.1.0-RC1/site/rexster.html
> >> and so forth.
> >>
> >> Thanks,
> >> Roman.
> >
> >
> >
> >
> > --
> > Claudio Martella
> >
>
--
Claudio Martella
Re: [VOTE] Apache Giraph 1.1.0 RC1
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Mon, Nov 3, 2014 at 4:51 PM, Maja Kabiljo <ma...@fb.com> wrote:
> We¹ve been running code which is the same as release candidate plus fix on
> GIRAPH-961 in production for 5 days now, no problems. This is
> hadoop_facebook profile, using only hive-io from all io modules.
Great! This tells me that once I cut RC2 with GIRAPH-961 you guys will be
ready to vote!
Thanks,
Roman.
Re: [VOTE] Apache Giraph 1.1.0 RC1
Posted by Roman Shaposhnik <ro...@shaposhnik.org>.
On Mon, Nov 3, 2014 at 4:51 PM, Maja Kabiljo <ma...@fb.com> wrote:
> We¹ve been running code which is the same as release candidate plus fix on
> GIRAPH-961 in production for 5 days now, no problems. This is
> hadoop_facebook profile, using only hive-io from all io modules.
Great! This tells me that once I cut RC2 with GIRAPH-961 you guys will be
ready to vote!
Thanks,
Roman.
Re: [VOTE] Apache Giraph 1.1.0 RC1
Posted by Maja Kabiljo <ma...@fb.com>.
We¹ve been running code which is the same as release candidate plus fix on
GIRAPH-961 in production for 5 days now, no problems. This is
hadoop_facebook profile, using only hive-io from all io modules.
On 11/1/14, 3:49 PM, "Roman Shaposhnik" <ro...@shaposhnik.org> wrote:
>Ping! Any progress on testing the current RC?
>
>Thanks,
>Roman.
>
>On Fri, Oct 31, 2014 at 9:00 AM, Claudio Martella
><cl...@gmail.com> wrote:
>> Oh, thanks for the info!
>>
>> On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik <ro...@shaposhnik.org>
>> wrote:
>>>
>>> On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella
>>> <cl...@gmail.com> wrote:
>>> > Hi Roman,
>>> >
>>> > thanks again for this. I have had a look at the staging site so far
>>>(our
>>> > cluster has been down whole week... universities...), and I was
>>> > wondering if
>>> > you have an insight why some of the docs are missing, e.g. gora and
>>> > rexster
>>> > documentation.
>>>
>>> None of them are missing. The links moved to a User Docs -> Modules
>>> though:
>>>
>>>https://urldefense.proofpoint.com/v1/url?u=http://people.apache.org/~rvs
>>>/giraph-1.1.0-RC1/site/gora.html&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=RGg
>>>8bUFUf%2FM2K95hnYD1RGWK1CQ%2BbcclArMcjzJodKY%3D%0A&m=8PzjCy0QzsbRm9lbAnj
>>>1Sreanb81jw%2FnRRX1Zju8ZvM%3D%0A&s=aabb0575b0830bb2c1b05645279b426e8789e
>>>fca3a6049073b214e2fbf832ec7
>>>
>>>https://urldefense.proofpoint.com/v1/url?u=http://people.apache.org/~rvs
>>>/giraph-1.1.0-RC1/site/rexster.html&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=
>>>RGg8bUFUf%2FM2K95hnYD1RGWK1CQ%2BbcclArMcjzJodKY%3D%0A&m=8PzjCy0QzsbRm9lb
>>>Anj1Sreanb81jw%2FnRRX1Zju8ZvM%3D%0A&s=08f4a813900872e6085eea6d3569bf7db0
>>>078c050d4aac784c0d61ba8f70504d
>>> and so forth.
>>>
>>> Thanks,
>>> Roman.
>>
>>
>>
>>
>> --
>> Claudio Martella
>>
Re: [VOTE] Apache Giraph 1.1.0 RC1
Posted by Maja Kabiljo <ma...@fb.com>.
We¹ve been running code which is the same as release candidate plus fix on
GIRAPH-961 in production for 5 days now, no problems. This is
hadoop_facebook profile, using only hive-io from all io modules.
On 11/1/14, 3:49 PM, "Roman Shaposhnik" <ro...@shaposhnik.org> wrote:
>Ping! Any progress on testing the current RC?
>
>Thanks,
>Roman.
>
>On Fri, Oct 31, 2014 at 9:00 AM, Claudio Martella
><cl...@gmail.com> wrote:
>> Oh, thanks for the info!
>>
>> On Fri, Oct 31, 2014 at 3:06 PM, Roman Shaposhnik <ro...@shaposhnik.org>
>> wrote:
>>>
>>> On Fri, Oct 31, 2014 at 3:26 AM, Claudio Martella
>>> <cl...@gmail.com> wrote:
>>> > Hi Roman,
>>> >
>>> > thanks again for this. I have had a look at the staging site so far
>>>(our
>>> > cluster has been down whole week... universities...), and I was
>>> > wondering if
>>> > you have an insight why some of the docs are missing, e.g. gora and
>>> > rexster
>>> > documentation.
>>>
>>> None of them are missing. The links moved to a User Docs -> Modules
>>> though:
>>>
>>>https://urldefense.proofpoint.com/v1/url?u=http://people.apache.org/~rvs
>>>/giraph-1.1.0-RC1/site/gora.html&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=RGg
>>>8bUFUf%2FM2K95hnYD1RGWK1CQ%2BbcclArMcjzJodKY%3D%0A&m=8PzjCy0QzsbRm9lbAnj
>>>1Sreanb81jw%2FnRRX1Zju8ZvM%3D%0A&s=aabb0575b0830bb2c1b05645279b426e8789e
>>>fca3a6049073b214e2fbf832ec7
>>>
>>>https://urldefense.proofpoint.com/v1/url?u=http://people.apache.org/~rvs
>>>/giraph-1.1.0-RC1/site/rexster.html&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=
>>>RGg8bUFUf%2FM2K95hnYD1RGWK1CQ%2BbcclArMcjzJodKY%3D%0A&m=8PzjCy0QzsbRm9lb
>>>Anj1Sreanb81jw%2FnRRX1Zju8ZvM%3D%0A&s=08f4a813900872e6085eea6d3569bf7db0
>>>078c050d4aac784c0d61ba8f70504d
>>> and so forth.
>>>
>>> Thanks,
>>> Roman.
>>
>>
>>
>>
>> --
>> Claudio Martella
>>