Should we continue to support Windows?

[Nigel Daley <nd...@yahoo-inc.com>]

I'm moving the comment I made on
https://issues.apache.org/jira/browse/HADOOP-5114
to core-dev so we can discuss this.

The Jira indicates that there are currently 30 test failures on  
Windows.  If this is true, and no one has report Windows test failures  
before now (we haven't run Windows tests in a while), then should we  
continue to support Windows?

If we *do* continue to support Windows, is anyone motivated to fix  
HADOOP-5114?

Nige

Re: Should we continue to support Windows?

["Tsz Wo (Nicholas), Sze" <s2...@yahoo.com>]

> If this is true, and no one has report Windows test failures
before now (we haven't run Windows tests in a while), then should we
continue to support Windows? 

Before answering whether we should continue to support Windows, we
better make sure that "this is true". I ran first 4 tests listed in the
description, TestQueueManager did success in my machine.

Why the tests reported in HADOOP-5114 failed in Windows?  Do these tests only fail in Windows?  We may have codes that not general enough for running in all platforms but it is unlikely that our codes specifically not working in Windows.  I am afraid that similar problem may be observed in other platforms as well.

Nicholas Sze

RE: Should we continue to support Windows?

["Jim Kellerman (POWERSET)" <Ji...@microsoft.com>]

I do all my development on Windows, but only run tests in a single
process (using MiniDFSCluster, MiniMRCluster, MiniHBaseCluster, etc.)
For big tests, HBase uses a small Linux cluster.

However, given my employer, I would say that supporting Windows
is a good thing.

---
Jim Kellerman, Powerset (Live Search, Microsoft Corporation)


> -----Original Message-----
> From: Konstantin Shvachko [mailto:shv@yahoo-inc.com]
> Sent: Friday, January 23, 2009 11:41 AM
> To: core-dev@hadoop.apache.org
> Subject: Re: Should we continue to support Windows?
> 
> Of course we should support Windows. It's 90% of the market :-)
> Seriously, there is a lot of students out there running one-two node
> clusters.
> 
> HADOOP-5114. The problem seems to be in ConnectException.
> The rest is just how it is revealed by different servers/clients.
> 
> --Konstantin
> 
> Nigel Daley wrote:
> > I'm moving the comment I made on
> > https://issues.apache.org/jira/browse/HADOOP-5114
> > to core-dev so we can discuss this.
> >
> > The Jira indicates that there are currently 30 test failures on
> > Windows.  If this is true, and no one has report Windows test failures
> > before now (we haven't run Windows tests in a while), then should we
> > continue to support Windows?
> >
> > If we *do* continue to support Windows, is anyone motivated to fix
> > HADOOP-5114?
> >
> > Nige
> >

Re: Should we continue to support Windows?

[Steve Loughran <st...@apache.org>]

Doug Cutting wrote:
> Steve Loughran wrote:
>> 4. It looks like nobody bothers to test the release regularly on windows.
> 
> Um, we test releases before they are released.  We don't need to test 
> them any more regularly than that.  It would be good to test trunk and 
> perhaps active branches more regularly so that we're less surprised come 
> releases.  Is that what you meant?
> 
> Doug

I think a CI server should somehow do the build and test on windows, 
even if nobody goes to the extent of bringing up a windows cluster and 
trying things out there.

For Ant, we stopped supporting Win9x when we ran out of machines to test 
it on, and got fed up with fielding the main source of platform-specific 
support calls: the batch files. It was just too much hassle, and without 
the machines, no way to be sure that you hadn't regressed and put a 
windows-nt specific bit into the .BAT.

Re: Should we continue to support Windows?

[Doug Cutting <cu...@apache.org>]

Steve Loughran wrote:
> 4. It looks like nobody bothers to test the release regularly on 
> windows.

Um, we test releases before they are released.  We don't need to test 
them any more regularly than that.  It would be good to test trunk and 
perhaps active branches more regularly so that we're less surprised come 
releases.  Is that what you meant?

Doug

Re: Should we continue to support Windows?

[Steve Loughran <st...@apache.org>]

Doug Cutting wrote:
> Konstantin Shvachko wrote:
>> HADOOP-5114. The problem seems to be in ConnectException.
>> The rest is just how it is revealed by different servers/clients.
> 
> If this is indeed just a single bug that's causing 30 tests to fail, I'm 
> not sure it's yet time to abandon Windows compatibility.  It would 
> surely be nice to, but I suspect there are large numbers of folks who 
> use Hadoop on Windows but who are not well represented among Hadoop 
> developers (i.e., who regularly use trunk) and who would be dismayed if 
> we stopped supporting Windows.
> 
> If we really want to pursue this, we should perhaps first conduct a poll 
> on core-users.
> 
> Doug


1. I do think windows as client is important
2. I also think that windows standalone is useful, for people that 
havent yet made the jump to unix.
3. I dont know of anyone who runs a production cluster on windows: and 
if they do, they havent been reporting the problems which are bound to arise
4. It looks like nobody bothers to test the release regularly on 
windows. I think at some point this is something I could set up with our 
Hudson server, though I dont like VMWare-image tests as things are more 
brittle than raw machine (timing issues, usually). First I need to move 
my patches to Git so that hudson can see what I've been doing.

A very large percentage of people's client machines is windows; its only 
recently I've moved my laptops over to Linux, and I do think that client 
stuff is very important. For OSS applications, that means running the 
tests matters, otherwise nobody who uses windows will be able to send in 
patches that you will trust, because all the tests will fail.

The other thing is that failing tests are good, they show that the tests 
are detecting trouble.

Re: Should we continue to support Windows?

[Doug Cutting <cu...@apache.org>]

Konstantin Shvachko wrote:
> HADOOP-5114. The problem seems to be in ConnectException.
> The rest is just how it is revealed by different servers/clients.

If this is indeed just a single bug that's causing 30 tests to fail, I'm 
not sure it's yet time to abandon Windows compatibility.  It would 
surely be nice to, but I suspect there are large numbers of folks who 
use Hadoop on Windows but who are not well represented among Hadoop 
developers (i.e., who regularly use trunk) and who would be dismayed if 
we stopped supporting Windows.

If we really want to pursue this, we should perhaps first conduct a poll 
on core-users.

Doug

Re: Should we continue to support Windows?

[Raghu Angadi <ra...@yahoo-inc.com>]

Nigel Daley wrote:
> We either support it or we don't.  "Unofficial" support is no support.  
> Defining "badly break" will be impossible.

Right. It is vague and subjective. I think any policy that that does not 
go as far saying "we don't support Windows" is good enough for now.

Raghu.

> If we support it, we must have all unit tests passing on Windows before 
> releases.
> 
> If we don't support it, contributors are welcome to submit patches to 
> fix issues for Windows as they find them.
> 
> Nige
> 
> 
> On Jan 23, 2009, at 11:54 AM, Raghu Angadi wrote:
> 
>>
>> I think we should at least "unoffically" support windows. I am not 
>> sure what exactly it should mean. It could imply: no official tests or 
>> performance tests before a release. A patch should not badly break 
>> windows (if someone reports the problem), in most cases.
>>
>> I don't run Hadoop on windows myself but surely think it is very 
>> important, especially for beginners. as an added bonus they learn 
>> about goodies like cygwin, ssh :-).
>>
>> There are multiple examples where the implementation of a feature 
>> improved because of different considerations for Windows.
>>
>> Raghu.
>>
>> Konstantin Shvachko wrote:
>>> Of course we should support Windows. It's 90% of the market :-)
>>> Seriously, there is a lot of students out there running one-two node 
>>> clusters.
>>> HADOOP-5114. The problem seems to be in ConnectException.
>>> The rest is just how it is revealed by different servers/clients.
>>> --Konstantin
>>> Nigel Daley wrote:
>>>> I'm moving the comment I made on
>>>> https://issues.apache.org/jira/browse/HADOOP-5114
>>>> to core-dev so we can discuss this.
>>>>
>>>> The Jira indicates that there are currently 30 test failures on 
>>>> Windows.  If this is true, and no one has report Windows test 
>>>> failures before now (we haven't run Windows tests in a while), then 
>>>> should we continue to support Windows?
>>>>
>>>> If we *do* continue to support Windows, is anyone motivated to fix 
>>>> HADOOP-5114?
>>>>
>>>> Nige
>>>>
>>
> 

Re: Should we continue to support Windows?

[Nigel Daley <nd...@yahoo-inc.com>]

We either support it or we don't.  "Unofficial" support is no  
support.  Defining "badly break" will be impossible.

If we support it, we must have all unit tests passing on Windows  
before releases.

If we don't support it, contributors are welcome to submit patches to  
fix issues for Windows as they find them.

Nige


On Jan 23, 2009, at 11:54 AM, Raghu Angadi wrote:

>
> I think we should at least "unoffically" support windows. I am not  
> sure what exactly it should mean. It could imply: no official tests  
> or performance tests before a release. A patch should not badly  
> break windows (if someone reports the problem), in most cases.
>
> I don't run Hadoop on windows myself but surely think it is very  
> important, especially for beginners. as an added bonus they learn  
> about goodies like cygwin, ssh :-).
>
> There are multiple examples where the implementation of a feature  
> improved because of different considerations for Windows.
>
> Raghu.
>
> Konstantin Shvachko wrote:
>> Of course we should support Windows. It's 90% of the market :-)
>> Seriously, there is a lot of students out there running one-two  
>> node clusters.
>> HADOOP-5114. The problem seems to be in ConnectException.
>> The rest is just how it is revealed by different servers/clients.
>> --Konstantin
>> Nigel Daley wrote:
>>> I'm moving the comment I made on
>>> https://issues.apache.org/jira/browse/HADOOP-5114
>>> to core-dev so we can discuss this.
>>>
>>> The Jira indicates that there are currently 30 test failures on  
>>> Windows.  If this is true, and no one has report Windows test  
>>> failures before now (we haven't run Windows tests in a while),  
>>> then should we continue to support Windows?
>>>
>>> If we *do* continue to support Windows, is anyone motivated to fix  
>>> HADOOP-5114?
>>>
>>> Nige
>>>
>

Re: Should we continue to support Windows?

[Raghu Angadi <ra...@yahoo-inc.com>]

I think we should at least "unoffically" support windows. I am not sure 
what exactly it should mean. It could imply: no official tests or 
performance tests before a release. A patch should not badly break 
windows (if someone reports the problem), in most cases.

I don't run Hadoop on windows myself but surely think it is very 
important, especially for beginners. as an added bonus they learn about 
goodies like cygwin, ssh :-).

There are multiple examples where the implementation of a feature 
improved because of different considerations for Windows.

Raghu.

Konstantin Shvachko wrote:
> Of course we should support Windows. It's 90% of the market :-)
> Seriously, there is a lot of students out there running one-two node 
> clusters.
> 
> HADOOP-5114. The problem seems to be in ConnectException.
> The rest is just how it is revealed by different servers/clients.
> 
> --Konstantin
> 
> Nigel Daley wrote:
>> I'm moving the comment I made on
>> https://issues.apache.org/jira/browse/HADOOP-5114
>> to core-dev so we can discuss this.
>>
>> The Jira indicates that there are currently 30 test failures on 
>> Windows.  If this is true, and no one has report Windows test failures 
>> before now (we haven't run Windows tests in a while), then should we 
>> continue to support Windows?
>>
>> If we *do* continue to support Windows, is anyone motivated to fix 
>> HADOOP-5114?
>>
>> Nige
>>

Re: Should we continue to support Windows?

[Konstantin Shvachko <sh...@yahoo-inc.com>]

Of course we should support Windows. It's 90% of the market :-)
Seriously, there is a lot of students out there running one-two node clusters.

HADOOP-5114. The problem seems to be in ConnectException.
The rest is just how it is revealed by different servers/clients.

--Konstantin

Nigel Daley wrote:
> I'm moving the comment I made on
> https://issues.apache.org/jira/browse/HADOOP-5114
> to core-dev so we can discuss this.
> 
> The Jira indicates that there are currently 30 test failures on 
> Windows.  If this is true, and no one has report Windows test failures 
> before now (we haven't run Windows tests in a while), then should we 
> continue to support Windows?
> 
> If we *do* continue to support Windows, is anyone motivated to fix 
> HADOOP-5114?
> 
> Nige
> 

Re: Should we continue to support Windows?

[Chris K Wensel <ch...@wensel.net>]

fwiw, we require students to bring Linux/Solaris/OS X laptops or a vm  
with linux/solaris to our classes.

allowing Windows users to run MR apps in local mode (in an IDE for  
example) would be good to support.

otherwise I'm wondering if officially not supporting Windows for  
deployment will simplify things and free Hadoop developers in some  
fashion.

ckw

On Jan 23, 2009, at 10:49 AM, Owen O'Malley wrote:

> I agree that if Windows is a priority for someone, they need to step  
> up
> supporting it as a platform. Personally, I use Linux in production and
> develop on Mac OSX. I've filed and fixed a reasonable portion of the  
> jiras
> for those platforms. *smile*
>
> -- Owen

--
Chris K Wensel
chris@wensel.net
http://www.cascading.org/
http://www.scaleunlimited.com/

Re: Should we continue to support Windows?

[Owen O'Malley <om...@apache.org>]

I agree that if Windows is a priority for someone, they need to step up
supporting it as a platform. Personally, I use Linux in production and
develop on Mac OSX. I've filed and fixed a reasonable portion of the jiras
for those platforms. *smile*

-- Owen