You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ctakes.apache.org by "Chen, Pei" <Pe...@childrens.harvard.edu> on 2013/06/03 21:35:06 UTC
RE: Aren't the *-res dependencies backwards?
Has anyone tried maven's <runtime/> scoped dependency?
The primary requirement is not to require developers/users to download large models (*-res projects) if they just want to use the cTAKES code.
After some tinkering, I think what the ideal solution is not to delete them from the pom's, but rather scope the *-res dependencies to "runtime"- wanted to ping those who may have had some experience with this- does maven still download the runtime scoped libs if you just need the code?
Example from ctakes-core/pom.xml would be:
<dependency>
<groupId>org.apache.ctakes</groupId>
<artifactId>ctakes-core-res</artifactId>
<scope>runtime</scope>
</dependency>
> -----Original Message-----
> From: Steven Bethard [mailto:steven.bethard@Colorado.EDU]
> Sent: Thursday, May 09, 2013 2:46 AM
> To: dev@ctakes.apache.org
> Subject: Re: Aren't the *-res dependencies backwards?
>
> On May 8, 2013, at 9:03 PM, "Chen, Pei" <Pe...@childrens.harvard.edu>
> wrote:
> > I finally got a chance to look into this again, but noticed 2 things:
> > 1) The various TypeSystem.xml files doesn't/shouldn't need to be in the
> resource/models projects. I think that was an oversight on my part last time.
>
> Yeah, I agree that The TypeSystem should be in the main project, not the -
> res project.
>
> > 2) I noticed that some of the projects have drivers or main classes which I
> assume a lot of folks are able to and plan to run pipelines directly from
> individual projects. If this is true, then I think it is actually easier to leave it as
> is and default to have the corresponding -res dependency in the pom.xml.
> Since it's maven controlled now, an external app can always add <exclude
> ctakes-drug-ner-res/>. This is just based on observation-- otherwise, each
> project will need to think about *-res during runtime if it's not in the
> pom.xml.
>
> I think we absolutely want each project to think about *-res. Isn't that the
> whole point of the *-res decomposition?
>
> My experience with <exclude> is that it usually comes back to bite you. For
> example, imagine that project A depends on ctakes-lvg, but excludes the -
> res dependency, and then project B depends on project A but wants ctakes-
> lvg plus the -res dependency. My experience in the past was that in
> scenarios like this, the <exclude> messed things up for me. Maybe Maven's
> gotten better about this (or maybe I know how to handle this better now),
> but my experience is that <exclude> and transitive dependencies get
> complicated very quickly.
>
> I would argue that instead, anything that actually uses the models should be
> in the -res project. That way, there's no confusion. Are you just using the
> code? Then use the regular project. Are you using models? Then use the -res
> project (which would depend on the regular project).
>
> If we assume that most people want the models, then maybe instead of
> adding "-res" to the projects that contain the models, we should add "-base"
> (or whatever) to the projects that don't contain the models?
>
> Steve
>
>
> >
> > Updated in r.1480409
> >
> >> -----Original Message-----
> >> From: Chen, Pei [mailto:Pei.Chen@childrens.harvard.edu]
> >> Sent: Thursday, April 11, 2013 8:59 PM
> >> To: dev@ctakes.apache.org
> >> Subject: RE: Aren't the *-res dependencies backwards?
> >>
> >> Steve,
> >> I think that would make sense... should be fairly straightforward
> >> and transparent change. I can take a closer look next week with a clear
> mind.
> >> ________________________________________
> >> From: Steven Bethard [steven.bethard@Colorado.EDU]
> >> Sent: Thursday, April 11, 2013 10:46 AM
> >> To: dev@ctakes.apache.org
> >> Subject: Aren't the *-res dependencies backwards?
> >>
> >> On Apr 11, 2013, at 8:05 AM, "Masanz, James J."
> >> <Ma...@mayo.edu> wrote:
> >>>
> >>> And ctakes-drug-ner-res is listed within the Maven Dependencies for
> >> ctakes-drug-ner
> >>
> >> Shouldn't it be exactly the opposite of this? Shouldn't
> >> ctakes-drug-ner-res depend on ctakes-drug-ner? Otherwise, Maven's
> >> always going to pull in ctakes-drug-ner-res whenever you use
> >> ctakes-drug-ner. I.e. with the current setup, it's impossible to depend on
> just the code, not the models.
> >>
> >> Steve
> >
RE: Aren't the *-res dependencies backwards?
Posted by "Chen, Pei" <Pe...@childrens.harvard.edu>.
Yes, maven 'optional' or 'test' scoped for models/resources would probably be the way to go.
But while working on this, we need to fix/make consistent a slew of issues that came up related to how resources are loading (InputStream vs File; some classes require physical files, etc.)... I'll commit this change afterwards.
--Pei
> -----Original Message-----
> From: Steven Bethard [mailto:steven.bethard@Colorado.EDU]
> Sent: Tuesday, June 04, 2013 8:33 PM
> To: dev@ctakes.apache.org
> Subject: Re: Aren't the *-res dependencies backwards?
>
>
> On Jun 3, 2013, at 1:35 PM, "Chen, Pei" <Pe...@childrens.harvard.edu>
> wrote:
>
> > Has anyone tried maven's <runtime/> scoped dependency?
> > The primary requirement is not to require developers/users to download
> large models (*-res projects) if they just want to use the cTAKES code.
> > After some tinkering, I think what the ideal solution is not to delete them
> from the pom's, but rather scope the *-res dependencies to "runtime"-
> wanted to ping those who may have had some experience with this- does
> maven still download the runtime scoped libs if you just need the code?
> > Example from ctakes-core/pom.xml would be:
> > <dependency>
> > <groupId>org.apache.ctakes</groupId>
> > <artifactId>ctakes-core-res</artifactId>
> > <scope>runtime</scope>
> > </dependency>
>
>
> I've never declared something as "runtime" scope but the Maven
> documentation suggests that only "provided" dependencies are not
> transitive:
>
> http://maven.apache.org/guides/introduction/introduction-to-dependency-
> mechanism.html#Dependency_Scope
>
> Looking around a bit, I think maybe we should be using optional
> dependencies here:
>
> "Optional dependencies are used when it's not really possible (for whatever
> reason) to split a project up into sub-modules. The idea is that some of the
> dependencies are only used for certain features in the project, and will not
> be needed if that feature isn't used. Ideally, such a feature would be split
> into a sub-module that depended on the core functionality project...this new
> subproject would have only non-optional dependencies, since you'd need
> them all if you decided to use the subproject's functionality.
> However, since the project cannot be split up (again, for whatever reason),
> these dependencies are declared optional."
> http://maven.apache.org/guides/introduction/introduction-to-optional-and-
> excludes-dependencies.html
>
> This sounds like our situation: ideally (from a Maven point of view), the code
> that depended on the full models would be split into a separate sub-module.
> But if we don't want to do that, we can declare the dependency as optional.
>
> I believe if we declare the *-res dependencies as optional then:
>
> (1) XXX can depend on XXX-res and compile and run its tests as usual
> (2) If YYY depends on XXX, it won't transitively include the XXX-res
> dependency. (If the XXX-res dependency is needed in YYY, it will have to be
> declared explicitly.)
>
> I think that sound about like what we want, right?
>
> Steve
>
>
> >
> >
> >> -----Original Message-----
> >> From: Steven Bethard [mailto:steven.bethard@Colorado.EDU]
> >> Sent: Thursday, May 09, 2013 2:46 AM
> >> To: dev@ctakes.apache.org
> >> Subject: Re: Aren't the *-res dependencies backwards?
> >>
> >> On May 8, 2013, at 9:03 PM, "Chen, Pei"
> >> <Pe...@childrens.harvard.edu>
> >> wrote:
> >>> I finally got a chance to look into this again, but noticed 2 things:
> >>> 1) The various TypeSystem.xml files doesn't/shouldn't need to be in
> >>> the
> >> resource/models projects. I think that was an oversight on my part last
> time.
> >>
> >> Yeah, I agree that The TypeSystem should be in the main project, not
> >> the - res project.
> >>
> >>> 2) I noticed that some of the projects have drivers or main classes
> >>> which I
> >> assume a lot of folks are able to and plan to run pipelines directly
> >> from individual projects. If this is true, then I think it is
> >> actually easier to leave it as is and default to have the corresponding -res
> dependency in the pom.xml.
> >> Since it's maven controlled now, an external app can always add
> >> <exclude ctakes-drug-ner-res/>. This is just based on observation--
> >> otherwise, each project will need to think about *-res during runtime
> >> if it's not in the pom.xml.
> >>
> >> I think we absolutely want each project to think about *-res. Isn't
> >> that the whole point of the *-res decomposition?
> >>
> >> My experience with <exclude> is that it usually comes back to bite
> >> you. For example, imagine that project A depends on ctakes-lvg, but
> >> excludes the - res dependency, and then project B depends on project
> >> A but wants ctakes- lvg plus the -res dependency. My experience in
> >> the past was that in scenarios like this, the <exclude> messed things
> >> up for me. Maybe Maven's gotten better about this (or maybe I know
> >> how to handle this better now), but my experience is that <exclude>
> >> and transitive dependencies get complicated very quickly.
> >>
> >> I would argue that instead, anything that actually uses the models
> >> should be in the -res project. That way, there's no confusion. Are
> >> you just using the code? Then use the regular project. Are you using
> >> models? Then use the -res project (which would depend on the regular
> project).
> >>
> >> If we assume that most people want the models, then maybe instead of
> >> adding "-res" to the projects that contain the models, we should add "-
> base"
> >> (or whatever) to the projects that don't contain the models?
> >>
> >> Steve
> >>
> >>
> >>>
> >>> Updated in r.1480409
> >>>
> >>>> -----Original Message-----
> >>>> From: Chen, Pei [mailto:Pei.Chen@childrens.harvard.edu]
> >>>> Sent: Thursday, April 11, 2013 8:59 PM
> >>>> To: dev@ctakes.apache.org
> >>>> Subject: RE: Aren't the *-res dependencies backwards?
> >>>>
> >>>> Steve,
> >>>> I think that would make sense... should be fairly straightforward
> >>>> and transparent change. I can take a closer look next week with a
> >>>> clear
> >> mind.
> >>>> ________________________________________
> >>>> From: Steven Bethard [steven.bethard@Colorado.EDU]
> >>>> Sent: Thursday, April 11, 2013 10:46 AM
> >>>> To: dev@ctakes.apache.org
> >>>> Subject: Aren't the *-res dependencies backwards?
> >>>>
> >>>> On Apr 11, 2013, at 8:05 AM, "Masanz, James J."
> >>>> <Ma...@mayo.edu> wrote:
> >>>>>
> >>>>> And ctakes-drug-ner-res is listed within the Maven Dependencies
> >>>>> for
> >>>> ctakes-drug-ner
> >>>>
> >>>> Shouldn't it be exactly the opposite of this? Shouldn't
> >>>> ctakes-drug-ner-res depend on ctakes-drug-ner? Otherwise, Maven's
> >>>> always going to pull in ctakes-drug-ner-res whenever you use
> >>>> ctakes-drug-ner. I.e. with the current setup, it's impossible to
> >>>> depend on
> >> just the code, not the models.
> >>>>
> >>>> Steve
> >>>
> >
Re: Aren't the *-res dependencies backwards?
Posted by Steven Bethard <st...@Colorado.EDU>.
On Jun 3, 2013, at 1:35 PM, "Chen, Pei" <Pe...@childrens.harvard.edu> wrote:
> Has anyone tried maven's <runtime/> scoped dependency?
> The primary requirement is not to require developers/users to download large models (*-res projects) if they just want to use the cTAKES code.
> After some tinkering, I think what the ideal solution is not to delete them from the pom's, but rather scope the *-res dependencies to "runtime"- wanted to ping those who may have had some experience with this- does maven still download the runtime scoped libs if you just need the code?
> Example from ctakes-core/pom.xml would be:
> <dependency>
> <groupId>org.apache.ctakes</groupId>
> <artifactId>ctakes-core-res</artifactId>
> <scope>runtime</scope>
> </dependency>
I've never declared something as "runtime" scope but the Maven documentation suggests that only "provided" dependencies are not transitive:
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope
Looking around a bit, I think maybe we should be using optional dependencies here:
"Optional dependencies are used when it's not really possible (for whatever reason) to split a project up into sub-modules. The idea is that some of the dependencies are only used for certain features in the project, and will not be needed if that feature isn't used. Ideally, such a feature would be split into a sub-module that depended on the core functionality project...this new subproject would have only non-optional dependencies, since you'd need them all if you decided to use the subproject's functionality.
However, since the project cannot be split up (again, for whatever reason), these dependencies are declared optional."
http://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html
This sounds like our situation: ideally (from a Maven point of view), the code that depended on the full models would be split into a separate sub-module. But if we don't want to do that, we can declare the dependency as optional.
I believe if we declare the *-res dependencies as optional then:
(1) XXX can depend on XXX-res and compile and run its tests as usual
(2) If YYY depends on XXX, it won't transitively include the XXX-res dependency. (If the XXX-res dependency is needed in YYY, it will have to be declared explicitly.)
I think that sound about like what we want, right?
Steve
>
>
>> -----Original Message-----
>> From: Steven Bethard [mailto:steven.bethard@Colorado.EDU]
>> Sent: Thursday, May 09, 2013 2:46 AM
>> To: dev@ctakes.apache.org
>> Subject: Re: Aren't the *-res dependencies backwards?
>>
>> On May 8, 2013, at 9:03 PM, "Chen, Pei" <Pe...@childrens.harvard.edu>
>> wrote:
>>> I finally got a chance to look into this again, but noticed 2 things:
>>> 1) The various TypeSystem.xml files doesn't/shouldn't need to be in the
>> resource/models projects. I think that was an oversight on my part last time.
>>
>> Yeah, I agree that The TypeSystem should be in the main project, not the -
>> res project.
>>
>>> 2) I noticed that some of the projects have drivers or main classes which I
>> assume a lot of folks are able to and plan to run pipelines directly from
>> individual projects. If this is true, then I think it is actually easier to leave it as
>> is and default to have the corresponding -res dependency in the pom.xml.
>> Since it's maven controlled now, an external app can always add <exclude
>> ctakes-drug-ner-res/>. This is just based on observation-- otherwise, each
>> project will need to think about *-res during runtime if it's not in the
>> pom.xml.
>>
>> I think we absolutely want each project to think about *-res. Isn't that the
>> whole point of the *-res decomposition?
>>
>> My experience with <exclude> is that it usually comes back to bite you. For
>> example, imagine that project A depends on ctakes-lvg, but excludes the -
>> res dependency, and then project B depends on project A but wants ctakes-
>> lvg plus the -res dependency. My experience in the past was that in
>> scenarios like this, the <exclude> messed things up for me. Maybe Maven's
>> gotten better about this (or maybe I know how to handle this better now),
>> but my experience is that <exclude> and transitive dependencies get
>> complicated very quickly.
>>
>> I would argue that instead, anything that actually uses the models should be
>> in the -res project. That way, there's no confusion. Are you just using the
>> code? Then use the regular project. Are you using models? Then use the -res
>> project (which would depend on the regular project).
>>
>> If we assume that most people want the models, then maybe instead of
>> adding "-res" to the projects that contain the models, we should add "-base"
>> (or whatever) to the projects that don't contain the models?
>>
>> Steve
>>
>>
>>>
>>> Updated in r.1480409
>>>
>>>> -----Original Message-----
>>>> From: Chen, Pei [mailto:Pei.Chen@childrens.harvard.edu]
>>>> Sent: Thursday, April 11, 2013 8:59 PM
>>>> To: dev@ctakes.apache.org
>>>> Subject: RE: Aren't the *-res dependencies backwards?
>>>>
>>>> Steve,
>>>> I think that would make sense... should be fairly straightforward
>>>> and transparent change. I can take a closer look next week with a clear
>> mind.
>>>> ________________________________________
>>>> From: Steven Bethard [steven.bethard@Colorado.EDU]
>>>> Sent: Thursday, April 11, 2013 10:46 AM
>>>> To: dev@ctakes.apache.org
>>>> Subject: Aren't the *-res dependencies backwards?
>>>>
>>>> On Apr 11, 2013, at 8:05 AM, "Masanz, James J."
>>>> <Ma...@mayo.edu> wrote:
>>>>>
>>>>> And ctakes-drug-ner-res is listed within the Maven Dependencies for
>>>> ctakes-drug-ner
>>>>
>>>> Shouldn't it be exactly the opposite of this? Shouldn't
>>>> ctakes-drug-ner-res depend on ctakes-drug-ner? Otherwise, Maven's
>>>> always going to pull in ctakes-drug-ner-res whenever you use
>>>> ctakes-drug-ner. I.e. with the current setup, it's impossible to depend on
>> just the code, not the models.
>>>>
>>>> Steve
>>>
>