.Net Core Compatibility

[Shravan Matthur Narayanamurthy <sh...@gmail.com>]

Hello All,

As you might have noticed, there is work in REEF-334
<https://issues.apache.org/jira/browse/REEF-334> that is trying to enable
use of REEF.Net on linux by making the REEF.Net code CoreCLR compliant.

So far we have fixed a few issues by replacing certain usages and patterns
in REEF.Net with usages that are both core and framework compliant.
However, there is one issue with this process. Compliance achieved so far
can regress due to other commits which are not geared towards core
compliance.

I want to pose this problem to the members of the group to get a good
handle on this situation. One idea that has been floating around is to make
the .Net Core portability analyzer a part of our build infrastructure and
check for regressions. Another is to create a new solution targeted for
CoreCLR and including the files that have been deemed compliant and make
building this solution a step in our build infrastructure.

Will really appreciate your help with other principled solutions to this
problem

Thanks,
--Shravan

Re: .Net Core Compatibility

[Markus Weimer <ma...@weimo.de>]

On Wed, Sep 7, 2016 at 11:17 AM, Mariia Mykhailova
<ma...@microsoft.com> wrote:
>
> Alternatively we could have a dedicated build job in AppVeyor which focuses on running portability analyzer and reports the number of violations, and this number must not increase for each PR.


That is what I was thinking. Right now, we have neither one issue
completely resolved nor one project which builds on CoreCLR. Hence,
committing to a monotonic decrease in the overall violation count
would be most feasible. We can also maintain the current number of
violations in the repository and decrease it with every commit that
aims to fix issues. That way, we make our bound on CoreCLR
incompatibilities ever tighter.

WDYT?

Markus

RE: .Net Core Compatibility

[Mariia Mykhailova <ma...@microsoft.com>]

When we worked on Checkstyle/Findbugs issues, we enabled checks which we've already fixed in Checkstyle configuration, and they became build breaks for existing build process, while other violations were still allowed. 

Ideally we should have something similar here, a step of the build which runs a subset of checks and fails if any check which is already fixed is violated. Does CoreCLR portability analyzer support such use? Such approach would also require fixing issues not per-project but per-issue (or configuring different sets of checks for different projects, but I think this might be more complicated, we did it for Checkstyle only for one final check)

Alternatively we could have a dedicated build job in AppVeyor which focuses on running portability analyzer and reports the number of violations, and this number must not increase for each PR.

-Mariia

-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de] 
Sent: Wednesday, September 7, 2016 8:33 AM
To: dev@reef.apache.org
Subject: Re: .Net Core Compatibility

On 2016-09-02 1:12 PM, Shravan Matthur Narayanamurthy wrote:
> As you might have noticed, there is work in REEF-334 that is trying to 
> enable use of REEF.Net on linux by making the REEF.Net code CoreCLR 
> compliant.

Much appreciated!

> One idea that has been floating around is to make the .Net Core 
> portability analyzer a part of our build infrastructure and check for 
> regressions.

This should be doable. Does the analyzer need to be installed or can we e.g. make a MBSBuild task to download the analyzer to our build machines and apply it?

> Another is to create a new solution targeted for CoreCLR and including 
> the files that have been deemed compliant and make building this 
> solution a step in our build infrastructure.

This should, in principle, be better. But maintaining multiple build files could be more of a challenge than is worth it. Maybe we do that once full sub-projects compile on CoreCLR? I believe for this to work, all REEF devs will have to also install CoreCLR on their machines, right?

Markus

Re: .Net Core Compatibility

[Markus Weimer <ma...@weimo.de>]

On 2016-09-02 1:12 PM, Shravan Matthur Narayanamurthy wrote:
> As you might have noticed, there is work in REEF-334 that is trying
> to enable use of REEF.Net on linux by making the REEF.Net code
> CoreCLR compliant.

Much appreciated!

> One idea that has been floating around is to make the .Net Core
> portability analyzer a part of our build infrastructure and check
> for regressions.

This should be doable. Does the analyzer need to be installed or can we
e.g. make a MBSBuild task to download the analyzer to our build machines
and apply it?

> Another is to create a new solution targeted for CoreCLR and
> including the files that have been deemed compliant and make
> building this solution a step in our build infrastructure.

This should, in principle, be better. But maintaining multiple build 
files could be more of a challenge than is worth it. Maybe we do that 
once full sub-projects compile on CoreCLR? I believe for this to work, 
all REEF devs will have to also install CoreCLR on their machines, right?

Markus

Re: .Net Core Compatibility

[Markus Weimer <ma...@weimo.de>]

On 2016-09-07 8:28 AM, Byung-Gon Chun wrote:
> Hi Shravan,

He's on a well deserved vacation, let me chime in :)

> Any update on this topic?
> Does CoreCLR include all the functionalities we need for REEF .Net?

So far, it looks like it. Changes fall into three categories:

(1) Trivial changes where we have to call a different API available in 
both CoreCLR and .NET 4.5 (the version we depend on right now)

(2) Changes where we have to call a different API available in both 
CoreCLR and .NET *4.6*. We can attack those once our current users give 
the green light on their clusters being upgraded to that .NET Version.

(3) Hairy changes where .NET and CoreCLR differ. The bridge is one of 
them, but there have been others. We have to either re-architect or 
enter the world of conditional compilation.

So far, work has been done in category (1), moving to (2) as soon as we 
run out of issues there.

Another major obstacle are our dependencies and their availability on 
CoreCLR, hence the push to update Newtonsoft JSON and drop Avro.

Markus

Re: .Net Core Compatibility

[Byung-Gon Chun <bg...@gmail.com>]

Hi Shravan,

Any update on this topic?
Does CoreCLR include all the functionalities we need for REEF .Net?

Thanks.
-Gon

On Fri, Sep 2, 2016 at 1:12 PM, Shravan Matthur Narayanamurthy <
shravanmn@gmail.com> wrote:

> Hello All,
>
> As you might have noticed, there is work in REEF-334
> <https://issues.apache.org/jira/browse/REEF-334> that is trying to enable
> use of REEF.Net on linux by making the REEF.Net code CoreCLR compliant.
>
> So far we have fixed a few issues by replacing certain usages and patterns
> in REEF.Net with usages that are both core and framework compliant.
> However, there is one issue with this process. Compliance achieved so far
> can regress due to other commits which are not geared towards core
> compliance.
>
> I want to pose this problem to the members of the group to get a good
> handle on this situation. One idea that has been floating around is to make
> the .Net Core portability analyzer a part of our build infrastructure and
> check for regressions. Another is to create a new solution targeted for
> CoreCLR and including the files that have been deemed compliant and make
> building this solution a step in our build infrastructure.
>
> Will really appreciate your help with other principled solutions to this
> problem
>
> Thanks,
> --Shravan
>



-- 
Byung-Gon Chun