You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Markus Weimer <ma...@weimo.de> on 2017/11/01 18:15:10 UTC
Is IJobSubmissionResult.DriverUrl working?
Hi,
`IJobSubmissionResult.DriverUrl` doesn't seem to be working for me. I
changed lines 74ff of `HelloREEF.cs` to:
```
IJobSubmissionResult jobSubmissionResult =
_reefClient.SubmitAndGetJobStatus(helloJobRequest);
Console.WriteLine("Driver URL: " + jobSubmissionResult.DriverUrl);
```
The resulting printout on the console is:
```
Driver URL:
```
So, the URL is empty.
Is this known to (not) work?
Thanks,
Markus
Re: Is IJobSubmissionResult.DriverUrl working?
Posted by Markus Weimer <ma...@weimo.de>.
As this is blocking my work on REEF-1942 to determine the status of a
Driver from the .NET Client (which in turn blocks REEF-1271, an overhaul of
our test infra), I filled a JIRA for it:
https://issues.apache.org/jira/browse/REEF-1943
Let's move the discussion there.
Markus
On Wed, Nov 1, 2017 at 11:15 AM, Markus Weimer <ma...@weimo.de> wrote:
> Hi,
>
> `IJobSubmissionResult.DriverUrl` doesn't seem to be working for me. I
> changed lines 74ff of `HelloREEF.cs` to:
>
> ```
> IJobSubmissionResult jobSubmissionResult = _reefClient.
> SubmitAndGetJobStatus(helloJobRequest);
> Console.WriteLine("Driver URL: " + jobSubmissionResult.DriverUrl);
> ```
>
> The resulting printout on the console is:
>
> ```
> Driver URL:
> ```
>
> So, the URL is empty.
>
> Is this known to (not) work?
>
> Thanks,
>
> Markus
>
Re: Is IJobSubmissionResult.DriverUrl working?
Posted by Markus Weimer <ma...@weimo.de>.
On Wed, Nov 1, 2017 at 2:21 PM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com.invalid> wrote:
> For any code that inject YarnREEFClient, when it calls YarnREEFClient.SubmitAndGetJobStatus(),
> YarnJobSubmissionResult will be created and DriverUrl will be pulled out
> and logged.
>
> I always run HelloReefYarn in cluster, I am able to see driverUrl logged
> on the console.
>
OK, I believe I have traced this back to a change I made in my work on
REEF-1942. Will close REEF-1943 once I confirm that. Sorry about the mail
storm!
Markus
RE: Is IJobSubmissionResult.DriverUrl working?
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com.INVALID>.
For any code that inject YarnREEFClient, when it calls YarnREEFClient.SubmitAndGetJobStatus(), YarnJobSubmissionResult will be created and DriverUrl will be pulled out and logged.
I always run HelloReefYarn in cluster, I am able to see driverUrl logged on the console.
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, November 1, 2017 2:02 PM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: Is IJobSubmissionResult.DriverUrl working?
On Wed, Nov 1, 2017 at 1:03 PM, Markus Weimer <ma...@weimo.de> wrote:
> On Wed, Nov 1, 2017 at 12:47 PM, Julia Wang (QIUHE) <
> Qiuhe.Wang@microsoft.com.invalid> wrote:
>
>> Yes, it can be added there. Then for client, it will become blocking
>> call until the method is returned. Also we might want allow client to
>> pass retry number, etc
>
>
> Thanks, I'll give that a try.
>
Turns out that the code already does that. See LocalJobSubmissionResult.cs line 47. Hence, this is simply not working for me. Do we have a test that confirms this working for others?
Thanks,
Markus
Re: Is IJobSubmissionResult.DriverUrl working?
Posted by Markus Weimer <ma...@weimo.de>.
On Wed, Nov 1, 2017 at 1:03 PM, Markus Weimer <ma...@weimo.de> wrote:
> On Wed, Nov 1, 2017 at 12:47 PM, Julia Wang (QIUHE) <
> Qiuhe.Wang@microsoft.com.invalid> wrote:
>
>> Yes, it can be added there. Then for client, it will become blocking call
>> until the method is returned. Also we might want allow client to pass retry
>> number, etc
>
>
> Thanks, I'll give that a try.
>
Turns out that the code already does that. See LocalJobSubmissionResult.cs
line 47. Hence, this is simply not working for me. Do we have a test that
confirms this working for others?
Thanks,
Markus
Re: Is IJobSubmissionResult.DriverUrl working?
Posted by Markus Weimer <ma...@weimo.de>.
On Wed, Nov 1, 2017 at 12:47 PM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com.invalid> wrote:
> Yes, it can be added there. Then for client, it will become blocking call
> until the method is returned. Also we might want allow client to pass retry
> number, etc
Thanks, I'll give that a try.
That being said, this seems pretty brittle, but probably the best we can do
until we get around to implementing a proper client for REEF.NET. I'll mark
this as Unstable in all the right places for nobody to take a production
dependency on it :)
Thanks,
Markus
>
RE: Is IJobSubmissionResult.DriverUrl working?
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com.INVALID>.
Yes, it can be added there. Then for client, it will become blocking call until the method is returned. Also we might want allow client to pass retry number, etc.
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, November 1, 2017 12:43 PM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: Is IJobSubmissionResult.DriverUrl working?
On Wed, Nov 1, 2017 at 11:50 AM, Julia Wang (QIUHE) < Qiuhe.Wang@microsoft.com.invalid> wrote:
> In client code, like HelloReef, the client may also need some retry
> logic to get it.
>
OK. And we'd put that retry logic in the implementation of IJobSubmissionResult, right?
Markus
Re: Is IJobSubmissionResult.DriverUrl working?
Posted by Markus Weimer <ma...@weimo.de>.
On Wed, Nov 1, 2017 at 11:50 AM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com.invalid> wrote:
> In client code, like HelloReef, the client may also need some retry logic
> to get it.
>
OK. And we'd put that retry logic in the implementation of
IJobSubmissionResult, right?
Markus
RE: Is IJobSubmissionResult.DriverUrl working?
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com.INVALID>.
In Java REEF client, we have re-try logic to get the driver url and then write it to a file for client. In client code, like HelloReef, the client may also need some retry logic to get it.
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, November 1, 2017 11:33 AM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: Is IJobSubmissionResult.DriverUrl working?
On Wed, Nov 1, 2017 at 11:26 AM, Julia Wang (QIUHE) < Qiuhe.Wang@microsoft.com.invalid> wrote:
> It takes some time for DriverUrl to be available. Usually after submit
> job, you won't get driver url right way.
>
What is a safe way to use the API, then?
Thanks,
Markus
Re: Is IJobSubmissionResult.DriverUrl working?
Posted by Markus Weimer <ma...@weimo.de>.
On Wed, Nov 1, 2017 at 11:26 AM, Julia Wang (QIUHE) <
Qiuhe.Wang@microsoft.com.invalid> wrote:
> It takes some time for DriverUrl to be available. Usually after submit
> job, you won't get driver url right way.
>
What is a safe way to use the API, then?
Thanks,
Markus
RE: Is IJobSubmissionResult.DriverUrl working?
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com.INVALID>.
It takes some time for DriverUrl to be available. Usually after submit job, you won't get driver url right way.
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, November 1, 2017 11:15 AM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Is IJobSubmissionResult.DriverUrl working?
Hi,
`IJobSubmissionResult.DriverUrl` doesn't seem to be working for me. I changed lines 74ff of `HelloREEF.cs` to:
```
IJobSubmissionResult jobSubmissionResult = _reefClient.SubmitAndGetJobStatus(helloJobRequest);
Console.WriteLine("Driver URL: " + jobSubmissionResult.DriverUrl); ```
The resulting printout on the console is:
```
Driver URL:
```
So, the URL is empty.
Is this known to (not) work?
Thanks,
Markus