You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@aurora.apache.org by Benjamin Mahler <be...@gmail.com> on 2015/05/02 00:27:03 UTC

End to end tests broken?

I seem to have issues running the end to end tests on master. Seems to get
stuck in a ZooKeeper related loop detecting no active schedulers:

...

BUILD SUCCESSFUL

Total time: 19.711 secs
I0501 22:12:27.352841  4901 leveldb.cpp:176] Opened db in 1.990798ms
I0501 22:12:27.354225  4901 leveldb.cpp:183] Compacted db in 1.04968ms
I0501 22:12:27.354480  4901 leveldb.cpp:198] Created db iterator in 16569ns
I0501 22:12:27.354573  4901 leveldb.cpp:204] Seeked to beginning of db in
80366ns
I0501 22:12:27.354867  4901 leveldb.cpp:273] Iterated through 10 keys in
the db in 112016ns
I0501 22:12:27.355093  4901 replica.cpp:744] Replica recovered with log
positions 754 -> 762 with 0 holes and 0 unlearned
The log is not empty
Replicated log initialization failed with code 1 (likely already
initialized).
aurora-scheduler stop/waiting
aurora-scheduler start/running, process 4917
+ test_version
++ aurora --version
++ cat /vagrant/.auroraversion
+ [[ 0.7.1-SNAPSHOT = 0.7.1-SNAPSHOT ]]
+ test_http_example devcluster vagrant test http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
+ local _cluster=devcluster _role=vagrant _env=test _job=http_example
+ local
_base_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
_updated_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
+ local _jobkey=devcluster/vagrant/test/http_example
+ test_config
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
devcluster/vagrant/test/http_example
+ local
_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
_jobkey=devcluster/vagrant/test/http_example
++ tr -dc '[[:print:]]'
++ aurora config list
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ joblist='jobs=[devcluster/vagrant/test/http_example]'
+ [[ jobs=[devcluster/vagrant/test/http_example] =
\j\o\b\s\=\[\d\e\v\c\l\u\s\t\e\r\/\v\a\g\r\a\n\t\/\t\e\s\t\/\h\t\t\p\_\e\x\a\m\p\l\e\]
]]
+ test_inspect devcluster/vagrant/test/http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ local _jobkey=devcluster/vagrant/test/http_example
_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ aurora job inspect devcluster/vagrant/test/http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
Job level information
  name:       'http_example'
  role:       'vagrant'
  contact:    'vagrant@localhost'
  cluster:    'devcluster'
  instances:  '2'
  constraints:
    'host': 'limit:2'
  service:    True
  production: False

Task level information
  name: 'http_example'
  constraints:
  stage_server < run_server

Process 'stage_server':
  cmdline:
    cp /vagrant/src/test/sh/org/apache/aurora/e2e/http_example.py .

Process 'run_server':
  cmdline:
    python http_example.py {{thermos.ports[http]}}

+ test_create devcluster/vagrant/test/http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ local _jobkey=devcluster/vagrant/test/http_example
_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
+ aurora job create devcluster/vagrant/test/http_example
/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
 INFO] Creating job http_example
 INFO] Connecting to 192.168.33.7:2181
 INFO] Sending request(xid=None): Connect(protocol_version=0,
last_zxid_seen=0, time_out=10000, session_id=0,
passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
read_only=None)
 INFO] Zookeeper connection established, state: CONNECTED
 INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
watcher=<function get_watch at 0x7f5a8fc28578>)
 INFO] Received response(xid=1): []
 INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
watcher=None)
 INFO] Received response(xid=2): []
 WARN] Could not connect to scheduler: No schedulers detected in devcluster!
 INFO] Connecting to 192.168.33.7:2181
 INFO] Sending request(xid=None): Connect(protocol_version=0,
last_zxid_seen=0, time_out=10000, session_id=0,
passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
read_only=None)
 INFO] Zookeeper connection established, state: CONNECTED
 INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
watcher=<function get_watch at 0x7f5a8fc28cf8>)
 INFO] Received response(xid=1): []
 INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
watcher=None)
 INFO] Received response(xid=2): []
 WARN] Could not connect to scheduler: No schedulers detected in devcluster!
 INFO] Connecting to 192.168.33.7:2181
 INFO] Sending request(xid=None): Connect(protocol_version=0,
last_zxid_seen=0, time_out=10000, session_id=0,
passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
read_only=None)
 INFO] Zookeeper connection established, state: CONNECTED
 INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
watcher=<function get_watch at 0x7f5a8fc8c398>)
 INFO] Received response(xid=1): []
 INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
watcher=None)
 INFO] Received response(xid=2): []
 WARN] Could not connect to scheduler: No schedulers detected in devcluster!
 INFO] Connecting to 192.168.33.7:2181
 INFO] Sending request(xid=None): Connect(protocol_version=0,
last_zxid_seen=0, time_out=10000, session_id=0,
passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
read_only=None)
...

This last part continues on forever as far as I can tell.

Is this a known issue?

Ben

Re: End to end tests broken?

Posted by Kevin Sweeney <ke...@apache.org>.
This is possibly related to
https://issues.apache.org/jira/browse/AURORA-1304

On Mon, May 4, 2015 at 2:44 PM, Benjamin Mahler <be...@gmail.com>
wrote:

> Thanks, looks good now.
>
> On Sat, May 2, 2015 at 4:31 PM, Bill Farner <wf...@apache.org> wrote:
>
> > Try destroying the VM (vagrant destroy -f) and running again.  The
> upstart
> > scripts are not currently synced with the repo, so they can drift.  I
> > suspect my recent scheduler arg removal is biting you.
> >
> > On Friday, May 1, 2015, Benjamin Mahler <be...@gmail.com>
> wrote:
> >
> > > I seem to have issues running the end to end tests on master. Seems to
> > get
> > > stuck in a ZooKeeper related loop detecting no active schedulers:
> > >
> > > ...
> > >
> > > BUILD SUCCESSFUL
> > >
> > > Total time: 19.711 secs
> > > I0501 22:12:27.352841 4901 leveldb.cpp:176] Opened db in 1.990798ms
> > > I0501 22:12:27.354225  4901 leveldb.cpp:183] Compacted db in 1.04968ms
> > > I0501 22:12:27.354480  4901 leveldb.cpp:198] Created db iterator in
> > 16569ns
> > > I0501 22:12:27.354573  4901 leveldb.cpp:204] Seeked to beginning of db
> in
> > > 80366ns
> > > I0501 22:12:27.354867  4901 leveldb.cpp:273] Iterated through 10 keys
> in
> > > the db in 112016ns
> > > I0501 22:12:27.355093  4901 replica.cpp:744] Replica recovered with log
> > > positions 754 -> 762 with 0 holes and 0 unlearned
> > > The log is not empty
> > > Replicated log initialization failed with code 1 (likely already
> > > initialized).
> > > aurora-scheduler stop/waiting
> > > aurora-scheduler start/running, process 4917
> > > + test_version
> > > ++ aurora --version
> > > ++ cat /vagrant/.auroraversion
> > > + [[ 0.7.1-SNAPSHOT = 0.7.1-SNAPSHOT ]]
> > > + test_http_example devcluster vagrant test http_example
> > > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > >
> >
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
> > > + local _cluster=devcluster _role=vagrant _env=test _job=http_example
> > > + local
> > >
> > >
> >
> _base_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > >
> > >
> >
> _updated_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
> > > + local _jobkey=devcluster/vagrant/test/http_example
> > > + test_config
> > > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > > devcluster/vagrant/test/http_example
> > > + local
> > >
> >
> _config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > > _jobkey=devcluster/vagrant/test/http_example
> > > ++ tr -dc '[[:print:]]'
> > > ++ aurora config list
> > > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > > + joblist='jobs=[devcluster/vagrant/test/http_example]'
> > > + [[ jobs=[devcluster/vagrant/test/http_example] =
> > >
> > >
> >
> \j\o\b\s\=\[\d\e\v\c\l\u\s\t\e\r\/\v\a\g\r\a\n\t\/\t\e\s\t\/\h\t\t\p\_\e\x\a\m\p\l\e\]
> > > ]]
> > > + test_inspect devcluster/vagrant/test/http_example
> > > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > > + local _jobkey=devcluster/vagrant/test/http_example
> > >
> >
> _config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > > + aurora job inspect devcluster/vagrant/test/http_example
> > > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > > Job level information
> > >   name:       'http_example'
> > >   role:       'vagrant'
> > >   contact:    'vagrant@localhost'
> > >   cluster:    'devcluster'
> > >   instances:  '2'
> > >   constraints:
> > >     'host': 'limit:2'
> > >   service:    True
> > >   production: False
> > >
> > > Task level information
> > >   name: 'http_example'
> > >   constraints:
> > >   stage_server < run_server
> > >
> > > Process 'stage_server':
> > >   cmdline:
> > >     cp /vagrant/src/test/sh/org/apache/aurora/e2e/http_example.py .
> > >
> > > Process 'run_server':
> > >   cmdline:
> > >     python http_example.py {{thermos.ports[http]}}
> > >
> > > + test_create devcluster/vagrant/test/http_example
> > > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > > + local _jobkey=devcluster/vagrant/test/http_example
> > >
> >
> _config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > > + aurora job create devcluster/vagrant/test/http_example
> > > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > >  INFO] Creating job http_example
> > >  INFO] Connecting to 192.168.33.7:2181
> > >  INFO] Sending request(xid=None): Connect(protocol_version=0,
> > > last_zxid_seen=0, time_out=10000, session_id=0,
> > >
> >
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> > > read_only=None)
> > >  INFO] Zookeeper connection established, state: CONNECTED
> > >  INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
> > > watcher=<function get_watch at 0x7f5a8fc28578>)
> > >  INFO] Received response(xid=1): []
> > >  INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
> > > watcher=None)
> > >  INFO] Received response(xid=2): []
> > >  WARN] Could not connect to scheduler: No schedulers detected in
> > > devcluster!
> > >  INFO] Connecting to 192.168.33.7:2181
> > >  INFO] Sending request(xid=None): Connect(protocol_version=0,
> > > last_zxid_seen=0, time_out=10000, session_id=0,
> > >
> >
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> > > read_only=None)
> > >  INFO] Zookeeper connection established, state: CONNECTED
> > >  INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
> > > watcher=<function get_watch at 0x7f5a8fc28cf8>)
> > >  INFO] Received response(xid=1): []
> > >  INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
> > > watcher=None)
> > >  INFO] Received response(xid=2): []
> > >  WARN] Could not connect to scheduler: No schedulers detected in
> > > devcluster!
> > >  INFO] Connecting to 192.168.33.7:2181
> > >  INFO] Sending request(xid=None): Connect(protocol_version=0,
> > > last_zxid_seen=0, time_out=10000, session_id=0,
> > >
> >
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> > > read_only=None)
> > >  INFO] Zookeeper connection established, state: CONNECTED
> > >  INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
> > > watcher=<function get_watch at 0x7f5a8fc8c398>)
> > >  INFO] Received response(xid=1): []
> > >  INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
> > > watcher=None)
> > >  INFO] Received response(xid=2): []
> > >  WARN] Could not connect to scheduler: No schedulers detected in
> > > devcluster!
> > >  INFO] Connecting to 192.168.33.7:2181
> > >  INFO] Sending request(xid=None): Connect(protocol_version=0,
> > > last_zxid_seen=0, time_out=10000, session_id=0,
> > >
> >
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> > > read_only=None)
> > > ...
> > >
> > > This last part continues on forever as far as I can tell.
> > >
> > > Is this a known issue?
> > >
> > > Ben
> > >
> >
> >
> > --
> > -=Bill
> >
>

Re: End to end tests broken?

Posted by Benjamin Mahler <be...@gmail.com>.
Thanks, looks good now.

On Sat, May 2, 2015 at 4:31 PM, Bill Farner <wf...@apache.org> wrote:

> Try destroying the VM (vagrant destroy -f) and running again.  The upstart
> scripts are not currently synced with the repo, so they can drift.  I
> suspect my recent scheduler arg removal is biting you.
>
> On Friday, May 1, 2015, Benjamin Mahler <be...@gmail.com> wrote:
>
> > I seem to have issues running the end to end tests on master. Seems to
> get
> > stuck in a ZooKeeper related loop detecting no active schedulers:
> >
> > ...
> >
> > BUILD SUCCESSFUL
> >
> > Total time: 19.711 secs
> > I0501 22:12:27.352841 4901 leveldb.cpp:176] Opened db in 1.990798ms
> > I0501 22:12:27.354225  4901 leveldb.cpp:183] Compacted db in 1.04968ms
> > I0501 22:12:27.354480  4901 leveldb.cpp:198] Created db iterator in
> 16569ns
> > I0501 22:12:27.354573  4901 leveldb.cpp:204] Seeked to beginning of db in
> > 80366ns
> > I0501 22:12:27.354867  4901 leveldb.cpp:273] Iterated through 10 keys in
> > the db in 112016ns
> > I0501 22:12:27.355093  4901 replica.cpp:744] Replica recovered with log
> > positions 754 -> 762 with 0 holes and 0 unlearned
> > The log is not empty
> > Replicated log initialization failed with code 1 (likely already
> > initialized).
> > aurora-scheduler stop/waiting
> > aurora-scheduler start/running, process 4917
> > + test_version
> > ++ aurora --version
> > ++ cat /vagrant/.auroraversion
> > + [[ 0.7.1-SNAPSHOT = 0.7.1-SNAPSHOT ]]
> > + test_http_example devcluster vagrant test http_example
> > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> >
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
> > + local _cluster=devcluster _role=vagrant _env=test _job=http_example
> > + local
> >
> >
> _base_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> >
> >
> _updated_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
> > + local _jobkey=devcluster/vagrant/test/http_example
> > + test_config
> > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > devcluster/vagrant/test/http_example
> > + local
> >
> _config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > _jobkey=devcluster/vagrant/test/http_example
> > ++ tr -dc '[[:print:]]'
> > ++ aurora config list
> > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > + joblist='jobs=[devcluster/vagrant/test/http_example]'
> > + [[ jobs=[devcluster/vagrant/test/http_example] =
> >
> >
> \j\o\b\s\=\[\d\e\v\c\l\u\s\t\e\r\/\v\a\g\r\a\n\t\/\t\e\s\t\/\h\t\t\p\_\e\x\a\m\p\l\e\]
> > ]]
> > + test_inspect devcluster/vagrant/test/http_example
> > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > + local _jobkey=devcluster/vagrant/test/http_example
> >
> _config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > + aurora job inspect devcluster/vagrant/test/http_example
> > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > Job level information
> >   name:       'http_example'
> >   role:       'vagrant'
> >   contact:    'vagrant@localhost'
> >   cluster:    'devcluster'
> >   instances:  '2'
> >   constraints:
> >     'host': 'limit:2'
> >   service:    True
> >   production: False
> >
> > Task level information
> >   name: 'http_example'
> >   constraints:
> >   stage_server < run_server
> >
> > Process 'stage_server':
> >   cmdline:
> >     cp /vagrant/src/test/sh/org/apache/aurora/e2e/http_example.py .
> >
> > Process 'run_server':
> >   cmdline:
> >     python http_example.py {{thermos.ports[http]}}
> >
> > + test_create devcluster/vagrant/test/http_example
> > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > + local _jobkey=devcluster/vagrant/test/http_example
> >
> _config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> > + aurora job create devcluster/vagrant/test/http_example
> > /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> >  INFO] Creating job http_example
> >  INFO] Connecting to 192.168.33.7:2181
> >  INFO] Sending request(xid=None): Connect(protocol_version=0,
> > last_zxid_seen=0, time_out=10000, session_id=0,
> >
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> > read_only=None)
> >  INFO] Zookeeper connection established, state: CONNECTED
> >  INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
> > watcher=<function get_watch at 0x7f5a8fc28578>)
> >  INFO] Received response(xid=1): []
> >  INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
> > watcher=None)
> >  INFO] Received response(xid=2): []
> >  WARN] Could not connect to scheduler: No schedulers detected in
> > devcluster!
> >  INFO] Connecting to 192.168.33.7:2181
> >  INFO] Sending request(xid=None): Connect(protocol_version=0,
> > last_zxid_seen=0, time_out=10000, session_id=0,
> >
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> > read_only=None)
> >  INFO] Zookeeper connection established, state: CONNECTED
> >  INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
> > watcher=<function get_watch at 0x7f5a8fc28cf8>)
> >  INFO] Received response(xid=1): []
> >  INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
> > watcher=None)
> >  INFO] Received response(xid=2): []
> >  WARN] Could not connect to scheduler: No schedulers detected in
> > devcluster!
> >  INFO] Connecting to 192.168.33.7:2181
> >  INFO] Sending request(xid=None): Connect(protocol_version=0,
> > last_zxid_seen=0, time_out=10000, session_id=0,
> >
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> > read_only=None)
> >  INFO] Zookeeper connection established, state: CONNECTED
> >  INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
> > watcher=<function get_watch at 0x7f5a8fc8c398>)
> >  INFO] Received response(xid=1): []
> >  INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
> > watcher=None)
> >  INFO] Received response(xid=2): []
> >  WARN] Could not connect to scheduler: No schedulers detected in
> > devcluster!
> >  INFO] Connecting to 192.168.33.7:2181
> >  INFO] Sending request(xid=None): Connect(protocol_version=0,
> > last_zxid_seen=0, time_out=10000, session_id=0,
> >
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> > read_only=None)
> > ...
> >
> > This last part continues on forever as far as I can tell.
> >
> > Is this a known issue?
> >
> > Ben
> >
>
>
> --
> -=Bill
>

Re: End to end tests broken?

Posted by Bill Farner <wf...@apache.org>.
Try destroying the VM (vagrant destroy -f) and running again.  The upstart
scripts are not currently synced with the repo, so they can drift.  I
suspect my recent scheduler arg removal is biting you.

On Friday, May 1, 2015, Benjamin Mahler <be...@gmail.com> wrote:

> I seem to have issues running the end to end tests on master. Seems to get
> stuck in a ZooKeeper related loop detecting no active schedulers:
>
> ...
>
> BUILD SUCCESSFUL
>
> Total time: 19.711 secs
> I0501 22:12:27.352841  4901 leveldb.cpp:176] Opened db in 1.990798ms
> I0501 22:12:27.354225  4901 leveldb.cpp:183] Compacted db in 1.04968ms
> I0501 22:12:27.354480  4901 leveldb.cpp:198] Created db iterator in 16569ns
> I0501 22:12:27.354573  4901 leveldb.cpp:204] Seeked to beginning of db in
> 80366ns
> I0501 22:12:27.354867  4901 leveldb.cpp:273] Iterated through 10 keys in
> the db in 112016ns
> I0501 22:12:27.355093  4901 replica.cpp:744] Replica recovered with log
> positions 754 -> 762 with 0 holes and 0 unlearned
> The log is not empty
> Replicated log initialization failed with code 1 (likely already
> initialized).
> aurora-scheduler stop/waiting
> aurora-scheduler start/running, process 4917
> + test_version
> ++ aurora --version
> ++ cat /vagrant/.auroraversion
> + [[ 0.7.1-SNAPSHOT = 0.7.1-SNAPSHOT ]]
> + test_http_example devcluster vagrant test http_example
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
> + local _cluster=devcluster _role=vagrant _env=test _job=http_example
> + local
>
> _base_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
>
> _updated_config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example_updated.aurora
> + local _jobkey=devcluster/vagrant/test/http_example
> + test_config
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> devcluster/vagrant/test/http_example
> + local
> _config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> _jobkey=devcluster/vagrant/test/http_example
> ++ tr -dc '[[:print:]]'
> ++ aurora config list
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> + joblist='jobs=[devcluster/vagrant/test/http_example]'
> + [[ jobs=[devcluster/vagrant/test/http_example] =
>
> \j\o\b\s\=\[\d\e\v\c\l\u\s\t\e\r\/\v\a\g\r\a\n\t\/\t\e\s\t\/\h\t\t\p\_\e\x\a\m\p\l\e\]
> ]]
> + test_inspect devcluster/vagrant/test/http_example
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> + local _jobkey=devcluster/vagrant/test/http_example
> _config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> + aurora job inspect devcluster/vagrant/test/http_example
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> Job level information
>   name:       'http_example'
>   role:       'vagrant'
>   contact:    'vagrant@localhost'
>   cluster:    'devcluster'
>   instances:  '2'
>   constraints:
>     'host': 'limit:2'
>   service:    True
>   production: False
>
> Task level information
>   name: 'http_example'
>   constraints:
>   stage_server < run_server
>
> Process 'stage_server':
>   cmdline:
>     cp /vagrant/src/test/sh/org/apache/aurora/e2e/http_example.py .
>
> Process 'run_server':
>   cmdline:
>     python http_example.py {{thermos.ports[http]}}
>
> + test_create devcluster/vagrant/test/http_example
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> + local _jobkey=devcluster/vagrant/test/http_example
> _config=/vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
> + aurora job create devcluster/vagrant/test/http_example
> /vagrant/src/test/sh/org/apache/aurora/e2e/http/http_example.aurora
>  INFO] Creating job http_example
>  INFO] Connecting to 192.168.33.7:2181
>  INFO] Sending request(xid=None): Connect(protocol_version=0,
> last_zxid_seen=0, time_out=10000, session_id=0,
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> read_only=None)
>  INFO] Zookeeper connection established, state: CONNECTED
>  INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
> watcher=<function get_watch at 0x7f5a8fc28578>)
>  INFO] Received response(xid=1): []
>  INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
> watcher=None)
>  INFO] Received response(xid=2): []
>  WARN] Could not connect to scheduler: No schedulers detected in
> devcluster!
>  INFO] Connecting to 192.168.33.7:2181
>  INFO] Sending request(xid=None): Connect(protocol_version=0,
> last_zxid_seen=0, time_out=10000, session_id=0,
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> read_only=None)
>  INFO] Zookeeper connection established, state: CONNECTED
>  INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
> watcher=<function get_watch at 0x7f5a8fc28cf8>)
>  INFO] Received response(xid=1): []
>  INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
> watcher=None)
>  INFO] Received response(xid=2): []
>  WARN] Could not connect to scheduler: No schedulers detected in
> devcluster!
>  INFO] Connecting to 192.168.33.7:2181
>  INFO] Sending request(xid=None): Connect(protocol_version=0,
> last_zxid_seen=0, time_out=10000, session_id=0,
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> read_only=None)
>  INFO] Zookeeper connection established, state: CONNECTED
>  INFO] Sending request(xid=1): GetChildren(path=u'/aurora/scheduler',
> watcher=<function get_watch at 0x7f5a8fc8c398>)
>  INFO] Received response(xid=1): []
>  INFO] Sending request(xid=2): GetChildren(path=u'/aurora/scheduler',
> watcher=None)
>  INFO] Received response(xid=2): []
>  WARN] Could not connect to scheduler: No schedulers detected in
> devcluster!
>  INFO] Connecting to 192.168.33.7:2181
>  INFO] Sending request(xid=None): Connect(protocol_version=0,
> last_zxid_seen=0, time_out=10000, session_id=0,
> passwd='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',
> read_only=None)
> ...
>
> This last part continues on forever as far as I can tell.
>
> Is this a known issue?
>
> Ben
>


-- 
-=Bill