You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Will Stevens <ws...@cloudops.com> on 2014/05/30 21:44:11 UTC
[DOCS] Problem generating 4.4 API docs
Hey All,
Paul Angus and I have both tested this and this is what we are seeing.
When we compile the the 'master' branch, the docs in
'tools/apidoc/target/xmldoc/html', but they appear to be the wrong docs.
Yes, we know that the versions that appear in the output is hardcoded in
the XSL files, but that is not what we are using as our reference.
So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I have
also confirmed that when a build is done, the 'tools/apidoc/log/@AGENTLOG@'
shows that the 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
directory is being referenced.
However, when I check the 'tools/apidoc/target/commands.xml', it does not
include API calls which were added in 4.3 (I can verify with the published
4.3 API docs). Also, the docs that are generated in the
'tools/apidoc/target/xmldoc/html' directory also does not have the API
calls that were added in 4.3.
I am stumped as to how this is happening. It is almost like the
4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how that
would be possible.
If someone who understands this piece of the software can have a look and
verify what we are seeing, we would appreciate the insight...
Thanks,
Will
Re: [DOCS] Problem generating 4.4 API docs
Posted by sebgoa <ru...@gmail.com>.
On Jun 2, 2014, at 9:16 PM, Will Stevens <ws...@cloudops.com> wrote:
> I started playing with the 'javasphinx' project to see if it would work for generating the API docs.
>
> So here is the basic idea of what it produces.
>
> I created docs for the following cloudstack directory: 'api/src/org/apache/cloudstack/api/'
>
> I then took the RST files and generated the html docs from it. I got a lot of errors like the following when trying to build the html docs with 'sphinx-build':
>
> ERROR: Unknown directive type "java:package"
> ERROR: Unknown directive type "java:import
> ERROR: Unknown directive type "java:type"
> ERROR: Unknown directive type "java:constructor"
> ERROR: Unknown directive type "java:method"
> etc...
>
> I am not sure if this is a long term issue, but we will probably have to solve other bigger problems first.
>
> For example, here is the RST output for the CreateNetwork command for an Admin:
>
> .. java:import:: org.apache.log4j Logger
>
> .. java:import:: org.apache.cloudstack.api APICommand
>
> .. java:import:: org.apache.cloudstack.api ApiConstants
>
> .. java:import:: org.apache.cloudstack.api ApiErrorCode
>
> .. java:import:: org.apache.cloudstack.api Parameter
>
> .. java:import:: org.apache.cloudstack.api ResponseObject.ResponseView
>
> .. java:import:: org.apache.cloudstack.api ServerApiException
>
> .. java:import:: org.apache.cloudstack.api.command.user.network CreateNetworkCmd
>
> .. java:import:: org.apache.cloudstack.api.response NetworkResponse
>
> .. java:import:: com.cloud.exception ConcurrentOperationException
>
> .. java:import:: com.cloud.exception InsufficientCapacityException
>
> .. java:import:: com.cloud.exception ResourceAllocationException
>
> .. java:import:: com.cloud.network Network
>
> CreateNetworkCmdByAdmin
> =======================
>
> .. java:package:: org.apache.cloudstack.api.command.admin.network
> :noindex:
>
> .. java:type:: @APICommand public class CreateNetworkCmdByAdmin extends CreateNetworkCmd
>
> Fields
> ------
> s_logger
> ^^^^^^^^
>
> .. java:field:: public static final Logger s_logger
> :outertype: CreateNetworkCmdByAdmin
>
> Methods
> -------
> execute
> ^^^^^^^
>
> .. java:method:: @Override public void execute() throws InsufficientCapacityException, ConcurrentOperationException, ResourceAllocationException
> :outertype: CreateNetworkCmdByAdmin
>
> getVlan
> ^^^^^^^
>
> .. java:method:: public String getVlan()
> :outertype: CreateNetworkCmdByAdmin
>
>
> I have also attached the themed output for this doc...
>
> Obviously, this is not enough detail for us to use for actual API documentation.
>
> If we wanted to go with this sort of documentation, we would need to really look at how the details are being discovered...
>
> Do any of you have ideas or suggestions on this front?
>
I got to the same stage as you,
Might not be worth pursuing. I will try to contact some of the javasphinx-developer and see what they say.
-sebastien
> Cheers,
>
> Will
>
>
>
> On Fri, May 30, 2014 at 4:33 PM, Sebastien Goasguen <ru...@gmail.com> wrote:
>
> On May 30, 2014, at 4:25 PM, Will Stevens <ws...@cloudops.com> wrote:
>
> > By clean install you mean starting from scratch, not 'mvn clean install'
> > right? I have been doing mvn clean installs…
> >
>
> Will, quickly stated I think that piece of code is a nightmare + folks have complained about our apidocs.
>
> Since you are familiar with sphinx now, maybe you can give a try with:
> http://bronto.github.io/javasphinx/
>
> Create some brand new api docs …that we can build automatically like the regular docs.
>
> 2 cts
>
> > Will
> >
> >
> > On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org> wrote:
> >
> >> Hi Will,
> >>
> >> Based on my last memories of the apidocs tool and maven poms, I think it
> >> used to scan built jar artifacts and reference them against something like
> >> a properties file (commands.properties?) and internally scans bunch of
> >> annotations in available class files to find apis and create apidocs. The
> >> ApiDiscovery plugin uses the same approach to discover available apis but
> >> during load time instead of build time.
> >>
> >> I would also recommend a clean install in case there are any caching
> >> issues. See if this helps.
> >>
> >> Regards.
> >>
> >>
> >> On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com>
> >> wrote:
> >>
> >>> Hey All,
> >>> Paul Angus and I have both tested this and this is what we are seeing.
> >>>
> >>> When we compile the the 'master' branch, the docs in
> >>> 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong docs.
> >>> Yes, we know that the versions that appear in the output is hardcoded in
> >>> the XSL files, but that is not what we are using as our reference.
> >>>
> >>> So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I
> >> have
> >>> also confirmed that when a build is done, the
> >> 'tools/apidoc/log/@AGENTLOG@
> >>> '
> >>> shows that the
> >> 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
> >>> directory is being referenced.
> >>>
> >>> However, when I check the 'tools/apidoc/target/commands.xml', it does not
> >>> include API calls which were added in 4.3 (I can verify with the
> >> published
> >>> 4.3 API docs). Also, the docs that are generated in the
> >>> 'tools/apidoc/target/xmldoc/html' directory also does not have the API
> >>> calls that were added in 4.3.
> >>>
> >>> I am stumped as to how this is happening. It is almost like the
> >>> 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how that
> >>> would be possible.
> >>>
> >>> If someone who understands this piece of the software can have a look and
> >>> verify what we are seeing, we would appreciate the insight...
> >>>
> >>> Thanks,
> >>>
> >>> Will
> >>>
> >>
>
>
> <api_docs.png>
Re: [DOCS] Problem generating 4.4 API docs
Posted by Will Stevens <ws...@cloudops.com>.
I started playing with the 'javasphinx' project to see if it would work for
generating the API docs.
So here is the basic idea of what it produces.
I created docs for the following cloudstack directory:
'api/src/org/apache/cloudstack/api/'
I then took the RST files and generated the html docs from it. I got a lot
of errors like the following when trying to build the html docs with
'sphinx-build':
ERROR: Unknown directive type "java:package"
ERROR: Unknown directive type "java:import
ERROR: Unknown directive type "java:type"
ERROR: Unknown directive type "java:constructor"
ERROR: Unknown directive type "java:method"
etc...
I am not sure if this is a long term issue, but we will probably have to
solve other bigger problems first.
For example, here is the RST output for the CreateNetwork command for an
Admin:
.. java:import:: org.apache.log4j Logger
.. java:import:: org.apache.cloudstack.api APICommand
.. java:import:: org.apache.cloudstack.api ApiConstants
.. java:import:: org.apache.cloudstack.api ApiErrorCode
.. java:import:: org.apache.cloudstack.api Parameter
.. java:import:: org.apache.cloudstack.api ResponseObject.ResponseView
.. java:import:: org.apache.cloudstack.api ServerApiException
.. java:import:: org.apache.cloudstack.api.command.user.network
CreateNetworkCmd
.. java:import:: org.apache.cloudstack.api.response NetworkResponse
.. java:import:: com.cloud.exception ConcurrentOperationException
.. java:import:: com.cloud.exception InsufficientCapacityException
.. java:import:: com.cloud.exception ResourceAllocationException
.. java:import:: com.cloud.network Network
CreateNetworkCmdByAdmin
=======================
.. java:package:: org.apache.cloudstack.api.command.admin.network
:noindex:
.. java:type:: @APICommand public class CreateNetworkCmdByAdmin extends
CreateNetworkCmd
Fields
------
s_logger
^^^^^^^^
.. java:field:: public static final Logger s_logger
:outertype: CreateNetworkCmdByAdmin
Methods
-------
execute
^^^^^^^
.. java:method:: @Override public void execute() throws
InsufficientCapacityException, ConcurrentOperationException,
ResourceAllocationException
:outertype: CreateNetworkCmdByAdmin
getVlan
^^^^^^^
.. java:method:: public String getVlan()
:outertype: CreateNetworkCmdByAdmin
I have also attached the themed output for this doc...
Obviously, this is not enough detail for us to use for actual API
documentation.
If we wanted to go with this sort of documentation, we would need to really
look at how the details are being discovered...
Do any of you have ideas or suggestions on this front?
Cheers,
Will
On Fri, May 30, 2014 at 4:33 PM, Sebastien Goasguen <ru...@gmail.com>
wrote:
>
> On May 30, 2014, at 4:25 PM, Will Stevens <ws...@cloudops.com> wrote:
>
> > By clean install you mean starting from scratch, not 'mvn clean install'
> > right? I have been doing mvn clean installs…
> >
>
> Will, quickly stated I think that piece of code is a nightmare + folks
> have complained about our apidocs.
>
> Since you are familiar with sphinx now, maybe you can give a try with:
> http://bronto.github.io/javasphinx/
>
> Create some brand new api docs …that we can build automatically like the
> regular docs.
>
> 2 cts
>
> > Will
> >
> >
> > On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org>
> wrote:
> >
> >> Hi Will,
> >>
> >> Based on my last memories of the apidocs tool and maven poms, I think it
> >> used to scan built jar artifacts and reference them against something
> like
> >> a properties file (commands.properties?) and internally scans bunch of
> >> annotations in available class files to find apis and create apidocs.
> The
> >> ApiDiscovery plugin uses the same approach to discover available apis
> but
> >> during load time instead of build time.
> >>
> >> I would also recommend a clean install in case there are any caching
> >> issues. See if this helps.
> >>
> >> Regards.
> >>
> >>
> >> On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com>
> >> wrote:
> >>
> >>> Hey All,
> >>> Paul Angus and I have both tested this and this is what we are seeing.
> >>>
> >>> When we compile the the 'master' branch, the docs in
> >>> 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong
> docs.
> >>> Yes, we know that the versions that appear in the output is hardcoded
> in
> >>> the XSL files, but that is not what we are using as our reference.
> >>>
> >>> So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I
> >> have
> >>> also confirmed that when a build is done, the
> >> 'tools/apidoc/log/@AGENTLOG@
> >>> '
> >>> shows that the
> >> 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
> >>> directory is being referenced.
> >>>
> >>> However, when I check the 'tools/apidoc/target/commands.xml', it does
> not
> >>> include API calls which were added in 4.3 (I can verify with the
> >> published
> >>> 4.3 API docs). Also, the docs that are generated in the
> >>> 'tools/apidoc/target/xmldoc/html' directory also does not have the API
> >>> calls that were added in 4.3.
> >>>
> >>> I am stumped as to how this is happening. It is almost like the
> >>> 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how
> that
> >>> would be possible.
> >>>
> >>> If someone who understands this piece of the software can have a look
> and
> >>> verify what we are seeing, we would appreciate the insight...
> >>>
> >>> Thanks,
> >>>
> >>> Will
> >>>
> >>
>
>
Re: [DOCS] Problem generating 4.4 API docs
Posted by Will Stevens <ws...@cloudops.com>.
I can look into that Seb. I know the docs are a point of complaint. I
actually build the docs for my own personal use already, but I have been
doing it based on the published API docs. I have attached the 4.3 docs
that I build for my own use.
I will see what is involved in using javasphinx to build the docs. In the
mean time it would be good if this did work. :)
Will
On Fri, May 30, 2014 at 4:33 PM, Sebastien Goasguen <ru...@gmail.com>
wrote:
>
> On May 30, 2014, at 4:25 PM, Will Stevens <ws...@cloudops.com> wrote:
>
> > By clean install you mean starting from scratch, not 'mvn clean install'
> > right? I have been doing mvn clean installs…
> >
>
> Will, quickly stated I think that piece of code is a nightmare + folks
> have complained about our apidocs.
>
> Since you are familiar with sphinx now, maybe you can give a try with:
> http://bronto.github.io/javasphinx/
>
> Create some brand new api docs …that we can build automatically like the
> regular docs.
>
> 2 cts
>
> > Will
> >
> >
> > On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org>
> wrote:
> >
> >> Hi Will,
> >>
> >> Based on my last memories of the apidocs tool and maven poms, I think it
> >> used to scan built jar artifacts and reference them against something
> like
> >> a properties file (commands.properties?) and internally scans bunch of
> >> annotations in available class files to find apis and create apidocs.
> The
> >> ApiDiscovery plugin uses the same approach to discover available apis
> but
> >> during load time instead of build time.
> >>
> >> I would also recommend a clean install in case there are any caching
> >> issues. See if this helps.
> >>
> >> Regards.
> >>
> >>
> >> On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com>
> >> wrote:
> >>
> >>> Hey All,
> >>> Paul Angus and I have both tested this and this is what we are seeing.
> >>>
> >>> When we compile the the 'master' branch, the docs in
> >>> 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong
> docs.
> >>> Yes, we know that the versions that appear in the output is hardcoded
> in
> >>> the XSL files, but that is not what we are using as our reference.
> >>>
> >>> So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I
> >> have
> >>> also confirmed that when a build is done, the
> >> 'tools/apidoc/log/@AGENTLOG@
> >>> '
> >>> shows that the
> >> 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
> >>> directory is being referenced.
> >>>
> >>> However, when I check the 'tools/apidoc/target/commands.xml', it does
> not
> >>> include API calls which were added in 4.3 (I can verify with the
> >> published
> >>> 4.3 API docs). Also, the docs that are generated in the
> >>> 'tools/apidoc/target/xmldoc/html' directory also does not have the API
> >>> calls that were added in 4.3.
> >>>
> >>> I am stumped as to how this is happening. It is almost like the
> >>> 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how
> that
> >>> would be possible.
> >>>
> >>> If someone who understands this piece of the software can have a look
> and
> >>> verify what we are seeing, we would appreciate the insight...
> >>>
> >>> Thanks,
> >>>
> >>> Will
> >>>
> >>
>
>
Re: [DOCS] Problem generating 4.4 API docs
Posted by Sebastien Goasguen <ru...@gmail.com>.
On May 30, 2014, at 4:25 PM, Will Stevens <ws...@cloudops.com> wrote:
> By clean install you mean starting from scratch, not 'mvn clean install'
> right? I have been doing mvn clean installs…
>
Will, quickly stated I think that piece of code is a nightmare + folks have complained about our apidocs.
Since you are familiar with sphinx now, maybe you can give a try with:
http://bronto.github.io/javasphinx/
Create some brand new api docs …that we can build automatically like the regular docs.
2 cts
> Will
>
>
> On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org> wrote:
>
>> Hi Will,
>>
>> Based on my last memories of the apidocs tool and maven poms, I think it
>> used to scan built jar artifacts and reference them against something like
>> a properties file (commands.properties?) and internally scans bunch of
>> annotations in available class files to find apis and create apidocs. The
>> ApiDiscovery plugin uses the same approach to discover available apis but
>> during load time instead of build time.
>>
>> I would also recommend a clean install in case there are any caching
>> issues. See if this helps.
>>
>> Regards.
>>
>>
>> On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com>
>> wrote:
>>
>>> Hey All,
>>> Paul Angus and I have both tested this and this is what we are seeing.
>>>
>>> When we compile the the 'master' branch, the docs in
>>> 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong docs.
>>> Yes, we know that the versions that appear in the output is hardcoded in
>>> the XSL files, but that is not what we are using as our reference.
>>>
>>> So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I
>> have
>>> also confirmed that when a build is done, the
>> 'tools/apidoc/log/@AGENTLOG@
>>> '
>>> shows that the
>> 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
>>> directory is being referenced.
>>>
>>> However, when I check the 'tools/apidoc/target/commands.xml', it does not
>>> include API calls which were added in 4.3 (I can verify with the
>> published
>>> 4.3 API docs). Also, the docs that are generated in the
>>> 'tools/apidoc/target/xmldoc/html' directory also does not have the API
>>> calls that were added in 4.3.
>>>
>>> I am stumped as to how this is happening. It is almost like the
>>> 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how that
>>> would be possible.
>>>
>>> If someone who understands this piece of the software can have a look and
>>> verify what we are seeing, we would appreciate the insight...
>>>
>>> Thanks,
>>>
>>> Will
>>>
>>
Re: [DOCS] Problem generating 4.4 API docs
Posted by Rohit Yadav <bh...@apache.org>.
Hi Will,
I don't know why it was not updated, one reason, I'm anticipating, could be
that we moved to Spring for many things which uses bunch of xmls for some
configuration, any not this properties file anymore?
Probably Alex or Kelven could comment if this is the case. I think you can
add the class names and ACL number/id for APIs you want in to work for now,
if it needs fixing send a patch or we'll fix it eventually. Thanks for
reporting this.
Cheers.
On May 31, 2014 4:03 AM, "Will Stevens" <ws...@cloudops.com> wrote:
> I have completely started from scratch with a new clone of the CloudStack
> repo.
>
> Master is now pointing to 4.5, so I checked out the 4.4 branch.
>
> I put all the deps in place as well as the vhd-util file.
>
> I completely removed my ~/.m2 directory so I am starting from completely
> clean.
>
> I checked the directory structure for 'commands.*' before building and the
> only one was the following:
>
> client/tomcatconf/commands.properties.in
>
> It does not have all of the API calls in it (the UCS section for example is
> not up to date).
>
> I built the project successfully and then did the same check to see what
> 'commands.*' files were available.
>
> Now I have:
>
>
> client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/classes/commands.properties
> client/target/conf/commands.properties
> client/target/generated-webapp/WEB-INF/classes/commands.properties
> client/tomcatconf/commands.properties.in
> tools/apidoc/target/commands.xml
>
> I checked the 'client/target/conf/commands.properties' and verified it does
> not have the updated UCS api calls.
>
> I checked the generated docs and they are still pre-4.3.
>
> How do we go about getting commands.properties updated? Why is the
> commands.properties two releases behind? Its not even up to date for 4.3.
>
> Cheers,
>
> Will
>
>
> On Fri, May 30, 2014 at 4:59 PM, Ove Ewerlid <Ov...@oracle.com>
> wrote:
>
> > On 05/30/2014 10:40 PM, Will Stevens wrote:
> >
> >> Thanks Rohit, I will do all those steps.
> >>
> >> I know the git branch is correct, but I will do a clean anyway.
> >>
> >> Removing the maven repositories is a good idea.
> >>
> >
> > NB; for this issue, e.g., fear of old cloudstack stuff being used rather
> > then newly built bits, removing by;
> > rm -rf ~/.m2/repository/org/apache/cloudstack
> > is a good idea.
> > Removing all of ~/.m2 will cause the entire cache, including non
> > cloudstack artifacts, to need to be re-downloaded. That takes time, and
> the
> > same artifact dependencies will be downloaded again hence a time
> consuming
> > noop.
> >
> > To mitigate ~/.m2 artifact cache download time during build from scratch,
> > and the purpose is not to repopulate ~/.m2 cache from scratch, this
> > directory minus cloudstack artifacts, can be saved and preloaded prior to
> > the build start. If there are new artifact updates, maven will detect
> this
> > and only download the new stuff.
> >
> > /Ove
> >
> >
> > Thx,
> >>
> >> Will
> >>
> >>
> >> On Fri, May 30, 2014 at 4:35 PM, Rohit Yadav <bh...@apache.org>
> wrote:
> >>
> >> Check if git branch or source code in question is correct, git clean
> -df
> >>> (note: this will remove files not tracked by git), check that
> >>> the commands.properties has the API you were searching in apidocs,
> remove
> >>> ~/.m2 and do mvn clean install, if that does not work check if the
> >>> API/plugin in question implements a valid CloudStack API (annotations
> >>> etc.). Lastly, if nothing worked probably apidocs needs fixing.
> >>>
> >>> Regards.
> >>>
> >>>
> >>> On Sat, May 31, 2014 at 1:55 AM, Will Stevens <ws...@cloudops.com>
> >>> wrote:
> >>>
> >>> By clean install you mean starting from scratch, not 'mvn clean
> install'
> >>>> right? I have been doing mvn clean installs...
> >>>>
> >>>> Will
> >>>>
> >>>>
> >>>> On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org>
> >>>>
> >>> wrote:
> >>>
> >>>>
> >>>> Hi Will,
> >>>>>
> >>>>> Based on my last memories of the apidocs tool and maven poms, I think
> >>>>>
> >>>> it
> >>>
> >>>> used to scan built jar artifacts and reference them against something
> >>>>>
> >>>> like
> >>>>
> >>>>> a properties file (commands.properties?) and internally scans bunch
> of
> >>>>> annotations in available class files to find apis and create apidocs.
> >>>>>
> >>>> The
> >>>
> >>>> ApiDiscovery plugin uses the same approach to discover available apis
> >>>>>
> >>>> but
> >>>
> >>>> during load time instead of build time.
> >>>>>
> >>>>> I would also recommend a clean install in case there are any caching
> >>>>> issues. See if this helps.
> >>>>>
> >>>>> Regards.
> >>>>>
> >>>>>
> >>>>> On Sat, May 31, 2014 at 1:14 AM, Will Stevens <wstevens@cloudops.com
> >
> >>>>> wrote:
> >>>>>
> >>>>> Hey All,
> >>>>>> Paul Angus and I have both tested this and this is what we are
> >>>>>>
> >>>>> seeing.
> >>>
> >>>>
> >>>>>> When we compile the the 'master' branch, the docs in
> >>>>>> 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong
> >>>>>>
> >>>>> docs.
> >>>>
> >>>>> Yes, we know that the versions that appear in the output is
> >>>>>>
> >>>>> hardcoded
> >>>
> >>>> in
> >>>>
> >>>>> the XSL files, but that is not what we are using as our reference.
> >>>>>>
> >>>>>> So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced.
> I
> >>>>>>
> >>>>> have
> >>>>>
> >>>>>> also confirmed that when a build is done, the
> >>>>>>
> >>>>> 'tools/apidoc/log/@AGENTLOG@
> >>>>>
> >>>>>> '
> >>>>>> shows that the
> >>>>>>
> >>>>> 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
> >>>>>
> >>>>>> directory is being referenced.
> >>>>>>
> >>>>>> However, when I check the 'tools/apidoc/target/commands.xml', it
> does
> >>>>>>
> >>>>> not
> >>>>
> >>>>> include API calls which were added in 4.3 (I can verify with the
> >>>>>>
> >>>>> published
> >>>>>
> >>>>>> 4.3 API docs). Also, the docs that are generated in the
> >>>>>> 'tools/apidoc/target/xmldoc/html' directory also does not have the
> >>>>>>
> >>>>> API
> >>>
> >>>> calls that were added in 4.3.
> >>>>>>
> >>>>>> I am stumped as to how this is happening. It is almost like the
> >>>>>> 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how
> >>>>>>
> >>>>> that
> >>>>
> >>>>> would be possible.
> >>>>>>
> >>>>>> If someone who understands this piece of the software can have a
> look
> >>>>>>
> >>>>> and
> >>>>
> >>>>> verify what we are seeing, we would appreciate the insight...
> >>>>>>
> >>>>>> Thanks,
> >>>>>>
> >>>>>> Will
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
> > --
> > Ove Everlid
> > System Administrator / Architect / SDN- & Automation- & Linux-hacker
> > Mobile: +46706668199 (dedicated work mobile)
> > Country: Sweden, timezone; Middle Europan Time (MET or GMT+1)
> >
>
Re: [DOCS] Problem generating 4.4 API docs
Posted by Will Stevens <ws...@cloudops.com>.
I have completely started from scratch with a new clone of the CloudStack
repo.
Master is now pointing to 4.5, so I checked out the 4.4 branch.
I put all the deps in place as well as the vhd-util file.
I completely removed my ~/.m2 directory so I am starting from completely
clean.
I checked the directory structure for 'commands.*' before building and the
only one was the following:
client/tomcatconf/commands.properties.in
It does not have all of the API calls in it (the UCS section for example is
not up to date).
I built the project successfully and then did the same check to see what
'commands.*' files were available.
Now I have:
client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/classes/commands.properties
client/target/conf/commands.properties
client/target/generated-webapp/WEB-INF/classes/commands.properties
client/tomcatconf/commands.properties.in
tools/apidoc/target/commands.xml
I checked the 'client/target/conf/commands.properties' and verified it does
not have the updated UCS api calls.
I checked the generated docs and they are still pre-4.3.
How do we go about getting commands.properties updated? Why is the
commands.properties two releases behind? Its not even up to date for 4.3.
Cheers,
Will
On Fri, May 30, 2014 at 4:59 PM, Ove Ewerlid <Ov...@oracle.com> wrote:
> On 05/30/2014 10:40 PM, Will Stevens wrote:
>
>> Thanks Rohit, I will do all those steps.
>>
>> I know the git branch is correct, but I will do a clean anyway.
>>
>> Removing the maven repositories is a good idea.
>>
>
> NB; for this issue, e.g., fear of old cloudstack stuff being used rather
> then newly built bits, removing by;
> rm -rf ~/.m2/repository/org/apache/cloudstack
> is a good idea.
> Removing all of ~/.m2 will cause the entire cache, including non
> cloudstack artifacts, to need to be re-downloaded. That takes time, and the
> same artifact dependencies will be downloaded again hence a time consuming
> noop.
>
> To mitigate ~/.m2 artifact cache download time during build from scratch,
> and the purpose is not to repopulate ~/.m2 cache from scratch, this
> directory minus cloudstack artifacts, can be saved and preloaded prior to
> the build start. If there are new artifact updates, maven will detect this
> and only download the new stuff.
>
> /Ove
>
>
> Thx,
>>
>> Will
>>
>>
>> On Fri, May 30, 2014 at 4:35 PM, Rohit Yadav <bh...@apache.org> wrote:
>>
>> Check if git branch or source code in question is correct, git clean -df
>>> (note: this will remove files not tracked by git), check that
>>> the commands.properties has the API you were searching in apidocs, remove
>>> ~/.m2 and do mvn clean install, if that does not work check if the
>>> API/plugin in question implements a valid CloudStack API (annotations
>>> etc.). Lastly, if nothing worked probably apidocs needs fixing.
>>>
>>> Regards.
>>>
>>>
>>> On Sat, May 31, 2014 at 1:55 AM, Will Stevens <ws...@cloudops.com>
>>> wrote:
>>>
>>> By clean install you mean starting from scratch, not 'mvn clean install'
>>>> right? I have been doing mvn clean installs...
>>>>
>>>> Will
>>>>
>>>>
>>>> On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org>
>>>>
>>> wrote:
>>>
>>>>
>>>> Hi Will,
>>>>>
>>>>> Based on my last memories of the apidocs tool and maven poms, I think
>>>>>
>>>> it
>>>
>>>> used to scan built jar artifacts and reference them against something
>>>>>
>>>> like
>>>>
>>>>> a properties file (commands.properties?) and internally scans bunch of
>>>>> annotations in available class files to find apis and create apidocs.
>>>>>
>>>> The
>>>
>>>> ApiDiscovery plugin uses the same approach to discover available apis
>>>>>
>>>> but
>>>
>>>> during load time instead of build time.
>>>>>
>>>>> I would also recommend a clean install in case there are any caching
>>>>> issues. See if this helps.
>>>>>
>>>>> Regards.
>>>>>
>>>>>
>>>>> On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com>
>>>>> wrote:
>>>>>
>>>>> Hey All,
>>>>>> Paul Angus and I have both tested this and this is what we are
>>>>>>
>>>>> seeing.
>>>
>>>>
>>>>>> When we compile the the 'master' branch, the docs in
>>>>>> 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong
>>>>>>
>>>>> docs.
>>>>
>>>>> Yes, we know that the versions that appear in the output is
>>>>>>
>>>>> hardcoded
>>>
>>>> in
>>>>
>>>>> the XSL files, but that is not what we are using as our reference.
>>>>>>
>>>>>> So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I
>>>>>>
>>>>> have
>>>>>
>>>>>> also confirmed that when a build is done, the
>>>>>>
>>>>> 'tools/apidoc/log/@AGENTLOG@
>>>>>
>>>>>> '
>>>>>> shows that the
>>>>>>
>>>>> 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
>>>>>
>>>>>> directory is being referenced.
>>>>>>
>>>>>> However, when I check the 'tools/apidoc/target/commands.xml', it does
>>>>>>
>>>>> not
>>>>
>>>>> include API calls which were added in 4.3 (I can verify with the
>>>>>>
>>>>> published
>>>>>
>>>>>> 4.3 API docs). Also, the docs that are generated in the
>>>>>> 'tools/apidoc/target/xmldoc/html' directory also does not have the
>>>>>>
>>>>> API
>>>
>>>> calls that were added in 4.3.
>>>>>>
>>>>>> I am stumped as to how this is happening. It is almost like the
>>>>>> 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how
>>>>>>
>>>>> that
>>>>
>>>>> would be possible.
>>>>>>
>>>>>> If someone who understands this piece of the software can have a look
>>>>>>
>>>>> and
>>>>
>>>>> verify what we are seeing, we would appreciate the insight...
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Will
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
> --
> Ove Everlid
> System Administrator / Architect / SDN- & Automation- & Linux-hacker
> Mobile: +46706668199 (dedicated work mobile)
> Country: Sweden, timezone; Middle Europan Time (MET or GMT+1)
>
Re: [DOCS] Problem generating 4.4 API docs
Posted by Ove Ewerlid <Ov...@oracle.com>.
On 05/30/2014 10:40 PM, Will Stevens wrote:
> Thanks Rohit, I will do all those steps.
>
> I know the git branch is correct, but I will do a clean anyway.
>
> Removing the maven repositories is a good idea.
NB; for this issue, e.g., fear of old cloudstack stuff being used rather
then newly built bits, removing by;
rm -rf ~/.m2/repository/org/apache/cloudstack
is a good idea.
Removing all of ~/.m2 will cause the entire cache, including non
cloudstack artifacts, to need to be re-downloaded. That takes time, and
the same artifact dependencies will be downloaded again hence a time
consuming noop.
To mitigate ~/.m2 artifact cache download time during build from
scratch, and the purpose is not to repopulate ~/.m2 cache from scratch,
this directory minus cloudstack artifacts, can be saved and preloaded
prior to the build start. If there are new artifact updates, maven will
detect this and only download the new stuff.
/Ove
> Thx,
>
> Will
>
>
> On Fri, May 30, 2014 at 4:35 PM, Rohit Yadav <bh...@apache.org> wrote:
>
>> Check if git branch or source code in question is correct, git clean -df
>> (note: this will remove files not tracked by git), check that
>> the commands.properties has the API you were searching in apidocs, remove
>> ~/.m2 and do mvn clean install, if that does not work check if the
>> API/plugin in question implements a valid CloudStack API (annotations
>> etc.). Lastly, if nothing worked probably apidocs needs fixing.
>>
>> Regards.
>>
>>
>> On Sat, May 31, 2014 at 1:55 AM, Will Stevens <ws...@cloudops.com>
>> wrote:
>>
>>> By clean install you mean starting from scratch, not 'mvn clean install'
>>> right? I have been doing mvn clean installs...
>>>
>>> Will
>>>
>>>
>>> On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org>
>> wrote:
>>>
>>>> Hi Will,
>>>>
>>>> Based on my last memories of the apidocs tool and maven poms, I think
>> it
>>>> used to scan built jar artifacts and reference them against something
>>> like
>>>> a properties file (commands.properties?) and internally scans bunch of
>>>> annotations in available class files to find apis and create apidocs.
>> The
>>>> ApiDiscovery plugin uses the same approach to discover available apis
>> but
>>>> during load time instead of build time.
>>>>
>>>> I would also recommend a clean install in case there are any caching
>>>> issues. See if this helps.
>>>>
>>>> Regards.
>>>>
>>>>
>>>> On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com>
>>>> wrote:
>>>>
>>>>> Hey All,
>>>>> Paul Angus and I have both tested this and this is what we are
>> seeing.
>>>>>
>>>>> When we compile the the 'master' branch, the docs in
>>>>> 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong
>>> docs.
>>>>> Yes, we know that the versions that appear in the output is
>> hardcoded
>>> in
>>>>> the XSL files, but that is not what we are using as our reference.
>>>>>
>>>>> So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I
>>>> have
>>>>> also confirmed that when a build is done, the
>>>> 'tools/apidoc/log/@AGENTLOG@
>>>>> '
>>>>> shows that the
>>>> 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
>>>>> directory is being referenced.
>>>>>
>>>>> However, when I check the 'tools/apidoc/target/commands.xml', it does
>>> not
>>>>> include API calls which were added in 4.3 (I can verify with the
>>>> published
>>>>> 4.3 API docs). Also, the docs that are generated in the
>>>>> 'tools/apidoc/target/xmldoc/html' directory also does not have the
>> API
>>>>> calls that were added in 4.3.
>>>>>
>>>>> I am stumped as to how this is happening. It is almost like the
>>>>> 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how
>>> that
>>>>> would be possible.
>>>>>
>>>>> If someone who understands this piece of the software can have a look
>>> and
>>>>> verify what we are seeing, we would appreciate the insight...
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Will
>>>>>
>>>>
>>>
>>
>
--
Ove Everlid
System Administrator / Architect / SDN- & Automation- & Linux-hacker
Mobile: +46706668199 (dedicated work mobile)
Country: Sweden, timezone; Middle Europan Time (MET or GMT+1)
Re: [DOCS] Problem generating 4.4 API docs
Posted by Will Stevens <ws...@cloudops.com>.
Thanks Rohit, I will do all those steps.
I know the git branch is correct, but I will do a clean anyway.
Removing the maven repositories is a good idea.
Thx,
Will
On Fri, May 30, 2014 at 4:35 PM, Rohit Yadav <bh...@apache.org> wrote:
> Check if git branch or source code in question is correct, git clean -df
> (note: this will remove files not tracked by git), check that
> the commands.properties has the API you were searching in apidocs, remove
> ~/.m2 and do mvn clean install, if that does not work check if the
> API/plugin in question implements a valid CloudStack API (annotations
> etc.). Lastly, if nothing worked probably apidocs needs fixing.
>
> Regards.
>
>
> On Sat, May 31, 2014 at 1:55 AM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > By clean install you mean starting from scratch, not 'mvn clean install'
> > right? I have been doing mvn clean installs...
> >
> > Will
> >
> >
> > On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org>
> wrote:
> >
> > > Hi Will,
> > >
> > > Based on my last memories of the apidocs tool and maven poms, I think
> it
> > > used to scan built jar artifacts and reference them against something
> > like
> > > a properties file (commands.properties?) and internally scans bunch of
> > > annotations in available class files to find apis and create apidocs.
> The
> > > ApiDiscovery plugin uses the same approach to discover available apis
> but
> > > during load time instead of build time.
> > >
> > > I would also recommend a clean install in case there are any caching
> > > issues. See if this helps.
> > >
> > > Regards.
> > >
> > >
> > > On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com>
> > > wrote:
> > >
> > > > Hey All,
> > > > Paul Angus and I have both tested this and this is what we are
> seeing.
> > > >
> > > > When we compile the the 'master' branch, the docs in
> > > > 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong
> > docs.
> > > > Yes, we know that the versions that appear in the output is
> hardcoded
> > in
> > > > the XSL files, but that is not what we are using as our reference.
> > > >
> > > > So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I
> > > have
> > > > also confirmed that when a build is done, the
> > > 'tools/apidoc/log/@AGENTLOG@
> > > > '
> > > > shows that the
> > > 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
> > > > directory is being referenced.
> > > >
> > > > However, when I check the 'tools/apidoc/target/commands.xml', it does
> > not
> > > > include API calls which were added in 4.3 (I can verify with the
> > > published
> > > > 4.3 API docs). Also, the docs that are generated in the
> > > > 'tools/apidoc/target/xmldoc/html' directory also does not have the
> API
> > > > calls that were added in 4.3.
> > > >
> > > > I am stumped as to how this is happening. It is almost like the
> > > > 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how
> > that
> > > > would be possible.
> > > >
> > > > If someone who understands this piece of the software can have a look
> > and
> > > > verify what we are seeing, we would appreciate the insight...
> > > >
> > > > Thanks,
> > > >
> > > > Will
> > > >
> > >
> >
>
Re: [DOCS] Problem generating 4.4 API docs
Posted by Rohit Yadav <bh...@apache.org>.
Check if git branch or source code in question is correct, git clean -df
(note: this will remove files not tracked by git), check that
the commands.properties has the API you were searching in apidocs, remove
~/.m2 and do mvn clean install, if that does not work check if the
API/plugin in question implements a valid CloudStack API (annotations
etc.). Lastly, if nothing worked probably apidocs needs fixing.
Regards.
On Sat, May 31, 2014 at 1:55 AM, Will Stevens <ws...@cloudops.com> wrote:
> By clean install you mean starting from scratch, not 'mvn clean install'
> right? I have been doing mvn clean installs...
>
> Will
>
>
> On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org> wrote:
>
> > Hi Will,
> >
> > Based on my last memories of the apidocs tool and maven poms, I think it
> > used to scan built jar artifacts and reference them against something
> like
> > a properties file (commands.properties?) and internally scans bunch of
> > annotations in available class files to find apis and create apidocs. The
> > ApiDiscovery plugin uses the same approach to discover available apis but
> > during load time instead of build time.
> >
> > I would also recommend a clean install in case there are any caching
> > issues. See if this helps.
> >
> > Regards.
> >
> >
> > On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com>
> > wrote:
> >
> > > Hey All,
> > > Paul Angus and I have both tested this and this is what we are seeing.
> > >
> > > When we compile the the 'master' branch, the docs in
> > > 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong
> docs.
> > > Yes, we know that the versions that appear in the output is hardcoded
> in
> > > the XSL files, but that is not what we are using as our reference.
> > >
> > > So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I
> > have
> > > also confirmed that when a build is done, the
> > 'tools/apidoc/log/@AGENTLOG@
> > > '
> > > shows that the
> > 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
> > > directory is being referenced.
> > >
> > > However, when I check the 'tools/apidoc/target/commands.xml', it does
> not
> > > include API calls which were added in 4.3 (I can verify with the
> > published
> > > 4.3 API docs). Also, the docs that are generated in the
> > > 'tools/apidoc/target/xmldoc/html' directory also does not have the API
> > > calls that were added in 4.3.
> > >
> > > I am stumped as to how this is happening. It is almost like the
> > > 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how
> that
> > > would be possible.
> > >
> > > If someone who understands this piece of the software can have a look
> and
> > > verify what we are seeing, we would appreciate the insight...
> > >
> > > Thanks,
> > >
> > > Will
> > >
> >
>
Re: [DOCS] Problem generating 4.4 API docs
Posted by Will Stevens <ws...@cloudops.com>.
By clean install you mean starting from scratch, not 'mvn clean install'
right? I have been doing mvn clean installs...
Will
On Fri, May 30, 2014 at 4:22 PM, Rohit Yadav <bh...@apache.org> wrote:
> Hi Will,
>
> Based on my last memories of the apidocs tool and maven poms, I think it
> used to scan built jar artifacts and reference them against something like
> a properties file (commands.properties?) and internally scans bunch of
> annotations in available class files to find apis and create apidocs. The
> ApiDiscovery plugin uses the same approach to discover available apis but
> during load time instead of build time.
>
> I would also recommend a clean install in case there are any caching
> issues. See if this helps.
>
> Regards.
>
>
> On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com>
> wrote:
>
> > Hey All,
> > Paul Angus and I have both tested this and this is what we are seeing.
> >
> > When we compile the the 'master' branch, the docs in
> > 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong docs.
> > Yes, we know that the versions that appear in the output is hardcoded in
> > the XSL files, but that is not what we are using as our reference.
> >
> > So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I
> have
> > also confirmed that when a build is done, the
> 'tools/apidoc/log/@AGENTLOG@
> > '
> > shows that the
> 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
> > directory is being referenced.
> >
> > However, when I check the 'tools/apidoc/target/commands.xml', it does not
> > include API calls which were added in 4.3 (I can verify with the
> published
> > 4.3 API docs). Also, the docs that are generated in the
> > 'tools/apidoc/target/xmldoc/html' directory also does not have the API
> > calls that were added in 4.3.
> >
> > I am stumped as to how this is happening. It is almost like the
> > 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how that
> > would be possible.
> >
> > If someone who understands this piece of the software can have a look and
> > verify what we are seeing, we would appreciate the insight...
> >
> > Thanks,
> >
> > Will
> >
>
Re: [DOCS] Problem generating 4.4 API docs
Posted by Rohit Yadav <bh...@apache.org>.
Hi Will,
Based on my last memories of the apidocs tool and maven poms, I think it
used to scan built jar artifacts and reference them against something like
a properties file (commands.properties?) and internally scans bunch of
annotations in available class files to find apis and create apidocs. The
ApiDiscovery plugin uses the same approach to discover available apis but
during load time instead of build time.
I would also recommend a clean install in case there are any caching
issues. See if this helps.
Regards.
On Sat, May 31, 2014 at 1:14 AM, Will Stevens <ws...@cloudops.com> wrote:
> Hey All,
> Paul Angus and I have both tested this and this is what we are seeing.
>
> When we compile the the 'master' branch, the docs in
> 'tools/apidoc/target/xmldoc/html', but they appear to be the wrong docs.
> Yes, we know that the versions that appear in the output is hardcoded in
> the XSL files, but that is not what we are using as our reference.
>
> So in the 'tools/apidoc/pom.xml' the 4.4.0-SNAPSHOT is referenced. I have
> also confirmed that when a build is done, the 'tools/apidoc/log/@AGENTLOG@
> '
> shows that the 'client/target/cloud-client-ui-4.4.0-SNAPSHOT/WEB-INF/lib/'
> directory is being referenced.
>
> However, when I check the 'tools/apidoc/target/commands.xml', it does not
> include API calls which were added in 4.3 (I can verify with the published
> 4.3 API docs). Also, the docs that are generated in the
> 'tools/apidoc/target/xmldoc/html' directory also does not have the API
> calls that were added in 4.3.
>
> I am stumped as to how this is happening. It is almost like the
> 4.4.0-SNAPSHOT is actually the 4.2.0-SNAPSHOT, but I am not sure how that
> would be possible.
>
> If someone who understands this piece of the software can have a look and
> verify what we are seeing, we would appreciate the insight...
>
> Thanks,
>
> Will
>