You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by Ananth Gundabattula <ag...@gmail.com> on 2016/04/28 23:53:36 UTC
Best practises to specify the Application
Hello All,
I am having the following issue with my application. I have a maven project
that has a class called XYZ that extends StreamingApplication.
My maven packaging is ensuring that the final "apa" extension package has
the right folder structure if exploded on a disk.
However I am not able to locate the Application name and the "launch"
button next to it after uploading the "apa" package via the browser.
I tried looking at the docs to see how to specify a class as an App that
can be launched from a packaged application but could not find much. Could
anyone of you please point me to the right way of associating a class as an
application that will get enabled with a launch button in the web ui ? The
class XYZ has the annotation called @ApplicationAnnotation on top of it.
Regards,
Ananth
Re: Best practises to specify the Application
Posted by David Yan <da...@datatorrent.com>.
Ah, never mind. -target 1.7 is already there.
https://github.com/apache/incubator-apex-core/blob/master/apex-app-archetype/src/main/resources/archetype-resources/pom.xml
It boils down to the user not using the archetype.
David
On Fri, Apr 29, 2016 at 10:47 AM, Munagala Ramanath <ra...@datatorrent.com>
wrote:
> A bunch of us are using JDK 1.8 to compile apps and deploying to our
> cluster and/or sandbox and not
> seeing this issue; how come ?
>
> Ram
>
> On Fri, Apr 29, 2016 at 10:37 AM, David Yan <da...@datatorrent.com> wrote:
>
> > [switching to dev list]
> >
> > It appears Ananth is using JDK 1.8 to compile the apa and our distributed
> > binary, compiled with 1.7 by us, cannot read class files compiled with
> 1.8.
> >
> > I think we may need to add target=1.7 in the compiler plugin config in
> the
> > pom.xml file of apex-app-archetype. Thoughts?
> >
> > David
> >
> > ---------- Forwarded message ----------
> > From: Ananth Gundabattula <ag...@gmail.com>
> > Date: Fri, Apr 29, 2016 at 1:20 AM
> > Subject: Re: Best practises to specify the Application
> > To: users@apex.incubator.apache.org
> >
> >
> > I had a look at the dtgateway.log and observed the following stacktrace
> : I
> > guess the root cause might be Apex running on a lower version of java and
> > the application code compiled with a higher version. I shall update
> > everyone after the upgrade to see if it fixes the issue.
> >
> >
> > Fatal error encountered
> > >
> > > 2016-04-29 08:04:45,181 ERROR
> > > com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> > > com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> > > com/tx/y/z/Operator : Unsupported major.minor version 52.0
> > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> > > at
> > > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> > > at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > at
> > >
> >
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> > > at
> > >
> >
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> > > at
> > >
> >
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> > > at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
> > >
> >
> >
> > On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
> > agundabattula@gmail.com> wrote:
> >
> > > Thanks Shubham.
> > >
> > > Those properties were being set by the parent pom which are inherited
> by
> > > the current pom.xml ( Kind of wanted all apex apps to have same version
> > > structure etc) and hence added that property as part of parent pom.
> > >
> > > I am still stumped as to what is causing this.
> > >
> > > Regards,
> > > Ananth
> > >
> > >
> > >
> > > On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <
> shubham@datatorrent.com
> > >
> > > wrote:
> > >
> > >> Hi Ananth,
> > >>
> > >> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> > >> -DarchetypeArtifactId=apex-app-archetype
> > -DarchetypeVersion=3.3.0-incubating
> > >> -DgroupId=com.example -Dpackage=com.example.mydtapp
> > -DartifactId=mydtapp
> > >> -Dversion=1.0-SNAPSHOT
> > >> and it works for me as well.
> > >>
> > >> I compared the pom.xml file and noticed values for following
> properties
> > >> were missing.
> > >> <manifestEntries>
> > >>
> > >>
> > <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
> > >>
> > <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
> > >>
> > >> </manifestEntries>
> > >>
> > >> Could you add the following to pom.xml and try again.
> > >>
> > >> <properties>
> > >> <!-- change this if you desire to use a different version of Apex
> > Core -->
> > >>
> >
> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
> > >>
> >
> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
> > >> </properties>
> > >>
> > >>
> > >> Thanks,
> > >> Shubham
> > >>
> > >> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
> > >> agundabattula@gmail.com> wrote:
> > >>
> > >>> Hello David,
> > >>>
> > >>> I reattempted the packaging with the archetype based approach. I
> guess
> > >>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks
> > for
> > >>> pointing it out.
> > >>>
> > >>> The default package generated by the archetype command seems to be
> > >>> working fine when uploading it but the moment I add in my code and
> > >>> dependencies it goes back the issue I am seeing :
> > >>>
> > >>>
> > >>> My app package looks like app.zip ( I removed all of the
> > >>> application.class for brevity sake ) . Please note that I deleted all
> > the
> > >>> jars under /lib folder and the contents of the /app folder in the apa
> > >>> pacakge.
> > >>>
> > >>> I have also attached a copy of my pom.xml file.
> > >>>
> > >>> Could you please advise if you something amiss right away in the
> > >>> structure below ? It may be noted that I retained the classes of the
> > >>> original simple project generated by the archetype and even they
> > stopped
> > >>> showing with the launch button the moment I added in all of the
> > application
> > >>> dependencies in maven.
> > >>>
> > >>> Regards,
> > >>> Ananth
> > >>>
> > >>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
> > >>> wrote:
> > >>>
> > >>>> Hi Ananth,
> > >>>>
> > >>>> I just tried with 3.3.0-incubating with the following and it's
> > working:
> > >>>>
> > >>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> > >>>> -DarchetypeArtifactId=apex-app-archetype
> > >>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
> > >>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp
> > -Dversion=1.0-SNAPSHOT
> > >>>>
> > >>>> Let me know whether this works for you or not.
> > >>>>
> > >>>> The system uses the MANIFEST.MF file to get the basic meta
> information
> > >>>> on the app package and it looks at all the jars in the app directory
> > and
> > >>>> searches for classes that subclass
> > com.datatorrent.api.StreamingApplication.
> > >>>>
> > >>>> David
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
> > >>>> agundabattula@gmail.com> wrote:
> > >>>>
> > >>>>> Hello David,
> > >>>>>
> > >>>>> Regarding as to why I did not use the archetype, I was building the
> > >>>>> app for 3.3.0 version and I was not able to specify an archetype
> > with the
> > >>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype
> for
> > >>>>> version 3.3.0 ? I was not sure if the archetype version should
> match
> > the
> > >>>>> exact apex engine version . My presumption was archetype does not
> > matter if
> > >>>>> the packaging is done right. So i went ahead and created a simple
> > maven
> > >>>>> project.
> > >>>>>
> > >>>>> The maven assembly plugin used the appPackage.xml and the related
> > >>>>> constructs to build the exact file layout of the app as mentioned
> in
> > the
> > >>>>> output above.
> > >>>>>
> > >>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
> > >>>>>
> > >>>>> This brings me to a question. How does the engine know which is the
> > >>>>> main class that is to be used for launching the Streaming App. I do
> > have a
> > >>>>> class that extends the StreaminApplication that is annotated with
> > >>>>> ApplicationAnnotation but does not seem to be picking up. By this I
> > mean, I
> > >>>>> do not see a "launch" icon next to it. All I see is the top level
> > packaged
> > >>>>> name in the web console but nothing beneath it for me to trigger a
> > launch.
> > >>>>>
> > >>>>> Regards,
> > >>>>> Ananth
> > >>>>>
> > >>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi Ananth,
> > >>>>>>
> > >>>>>> The MANIFEST.MF file is important, and it's generated
> automatically
> > >>>>>> if the archetype is used. For example, the MANFEST.MF file from
> the
> > Pi demo
> > >>>>>> in Malhar has this content:
> > >>>>>>
> > >>>>>> Manifest-Version: 1.0
> > >>>>>> Archiver-Version: Plexus Archiver
> > >>>>>> Built-By: david
> > >>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi
> Demo
> > >>>>>> Class-Path: lib/*.jar
> > >>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
> > >>>>>> Created-By: Apache Maven 3.3.9
> > >>>>>> Build-Jdk: 1.8.0_92
> > >>>>>> DT-App-Package-Description: Apex demo applications that calculate
> > the
> > >>>>>> value of Pi. This is a starting point to understand how Apex
> works.
> > >>>>>> DT-App-Package-Group-Id: org.apache.apex
> > >>>>>> DT-App-Package-Name: pi-demo
> > >>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
> > >>>>>>
> > >>>>>> Is there a reason why the archetype is not used in your project?
> > >>>>>>
> > >>>>>> David
> > >>>>>>
> > >>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
> > >>>>>> agundabattula@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> Hello David,
> > >>>>>>>
> > >>>>>>> Please find the unzip result at the end of this email. The jar is
> > in
> > >>>>>>> the app directory inside the apa.
> > >>>>>>> As to why I did not use the archetype, I was building the app for
> > >>>>>>> 3.3.0 version and I was not able to specify an archetype with the
> > version
> > >>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not
> > matter if
> > >>>>>>> the packaging is done right. So i went ahead and created a simple
> > maven
> > >>>>>>> project.
> > >>>>>>>
> > >>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
> > >>>>>>> listing below ) . I also pasted the contents of the
> properties.xml
> > file at
> > >>>>>>> the end of this email for your reference.
> > >>>>>>>
> > >>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
> > >>>>>>> testing: META-INF/ OK
> > >>>>>>> testing: META-INF/MANIFEST.MF OK
> > >>>>>>> testing: app/ OK
> > >>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/ OK
> > >>>>>>> testing: lib/activation-1.1.jar OK
> > >>>>>>> testing: lib/activemq-client-5.8.0.jar OK
> > >>>>>>> testing: lib/antlr-2.7.7.jar OK
> > >>>>>>> testing: lib/aopalliance-1.0.jar OK
> > >>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
> > >>>>>>> testing: lib/btf-1.2.jar OK
> > >>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
> > >>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
> > >>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
> > >>>>>>> testing: lib/commons-lang3-3.1.jar OK
> > >>>>>>> testing: lib/commons-logging-1.1.3.jar OK
> > >>>>>>> testing: lib/curator-client-2.7.0.jar OK
> > >>>>>>> testing: lib/curator-framework-2.7.0.jar OK
> > >>>>>>> testing: lib/dom4j-1.6.1.jar OK
> > >>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
> > >>>>>>> testing: lib/fastutil-6.6.4.jar OK
> > >>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
> > >>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
> > >>>>>>> testing: lib/guava-14.0.1.jar OK
> > >>>>>>> testing: lib/hawtbuf-1.9.jar OK
> > >>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar
> OK
> > >>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
> > >>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
> > >>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
> > >>>>>>> testing: lib/HikariCP-2.4.3.jar OK
> > >>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
> > >>>>>>> testing: lib/jackson-core-2.6.4.jar OK
> > >>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
> > >>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
> > >>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
> > >>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
> > >>>>>>> testing: lib/javax.mail-1.5.0.jar OK
> > >>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
> > >>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
> > >>>>>>> testing: lib/jersey-client-1.9.jar OK
> > >>>>>>> testing: lib/jersey-core-1.9.jar OK
> > >>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jline-2.11.jar OK
> > >>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
> > >>>>>>> testing: lib/joda-time-2.7.jar OK
> > >>>>>>> testing: lib/json-patch-1.7.jar OK
> > >>>>>>> testing: lib/jsr305-2.0.1.jar OK
> > >>>>>>> testing: lib/jta-1.1.jar OK
> > >>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
> > >>>>>>> testing: lib/log4j-1.2.17.jar OK
> > >>>>>>> testing: lib/logback-classic-1.1.5.jar OK
> > >>>>>>> testing: lib/logback-core-1.1.5.jar OK
> > >>>>>>> testing: lib/lz4-1.2.0.jar OK
> > >>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
> > >>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
> > >>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
> > >>>>>>> testing: lib/metrics-core-3.0.2.jar OK
> > >>>>>>> testing: lib/msg-simple-1.1.jar OK
> > >>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
> > >>>>>>> testing: lib/named-regexp-0.2.3.jar OK
> > >>>>>>> testing: lib/netty-3.6.2.Final.jar OK
> > >>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/opencsv-2.3.jar OK
> > >>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
> > >>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
> > >>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/super-csv-2.3.1.jar OK
> > >>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
> > >>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
> > >>>>>>> testing: lib/zookeeper-3.4.6.jar OK
> > >>>>>>> testing: conf/ OK
> > >>>>>>> testing: conf/properties.xml OK
> > >>>>>>> No errors detected in compressed data of
> > >>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
> > >>>>>>>
> > >>>>>>> Contents of the properties.xml file under the conf folder is
> here:
> > >>>>>>>
> > >>>>>>> <?xml version="1.0"?>
> > >>>>>>> <configuration>
> > >>>>>>> <property>
> > >>>>>>> <name>dt.application.CED.class</name>
> > >>>>>>>
> >
> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
> > >>>>>>> <description>An alias for the application</description>
> > >>>>>>> </property>
> > >>>>>>> <property>
> > >>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
> > >>>>>>> <value>512</value>
> > >>>>>>> </property>
> > >>>>>>>
> > >>>>>>> <property>
> > >>>>>>>
> >
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
> > >>>>>>>
> >
> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
> > >>>>>>> </property>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> <property>
> > >>>>>>>
> >
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
> > >>>>>>> <value>20</value>
> > >>>>>>> </property>
> > >>>>>>>
> > >>>>>>> </configuration>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Regards,
> > >>>>>>> Ananth
> > >>>>>>>
> > >>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <
> david@datatorrent.com>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Hi Ananth,
> > >>>>>>>>
> > >>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
> > here?
> > >>>>>>>> Make sure the jar containing the application classes should be
> in
> > >>>>>>>> the app directory inside the apa.
> > >>>>>>>>
> > >>>>>>>> Can I know why the apex-app-archetype is not used in your apa
> > >>>>>>>> project?
> > >>>>>>>>
> > >>>>>>>> David
> > >>>>>>>>
> > >>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
> > >>>>>>>> agundabattula@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hello All,
> > >>>>>>>>>
> > >>>>>>>>> I am having the following issue with my application. I have a
> > >>>>>>>>> maven project that has a class called XYZ that extends
> > >>>>>>>>> StreamingApplication.
> > >>>>>>>>>
> > >>>>>>>>> My maven packaging is ensuring that the final "apa" extension
> > >>>>>>>>> package has the right folder structure if exploded on a disk.
> > >>>>>>>>>
> > >>>>>>>>> However I am not able to locate the Application name and the
> > >>>>>>>>> "launch" button next to it after uploading the "apa" package
> via
> > the
> > >>>>>>>>> browser.
> > >>>>>>>>>
> > >>>>>>>>> I tried looking at the docs to see how to specify a class as an
> > >>>>>>>>> App that can be launched from a packaged application but could
> > not find
> > >>>>>>>>> much. Could anyone of you please point me to the right way of
> > associating a
> > >>>>>>>>> class as an application that will get enabled with a launch
> > button in the
> > >>>>>>>>> web ui ? The class XYZ has the annotation called
> > @ApplicationAnnotation on
> > >>>>>>>>> top of it.
> > >>>>>>>>>
> > >>>>>>>>> Regards,
> > >>>>>>>>> Ananth
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >
> >
>
Re: Best practises to specify the Application
Posted by Munagala Ramanath <ra...@datatorrent.com>.
A bunch of us are using JDK 1.8 to compile apps and deploying to our
cluster and/or sandbox and not
seeing this issue; how come ?
Ram
On Fri, Apr 29, 2016 at 10:37 AM, David Yan <da...@datatorrent.com> wrote:
> [switching to dev list]
>
> It appears Ananth is using JDK 1.8 to compile the apa and our distributed
> binary, compiled with 1.7 by us, cannot read class files compiled with 1.8.
>
> I think we may need to add target=1.7 in the compiler plugin config in the
> pom.xml file of apex-app-archetype. Thoughts?
>
> David
>
> ---------- Forwarded message ----------
> From: Ananth Gundabattula <ag...@gmail.com>
> Date: Fri, Apr 29, 2016 at 1:20 AM
> Subject: Re: Best practises to specify the Application
> To: users@apex.incubator.apache.org
>
>
> I had a look at the dtgateway.log and observed the following stacktrace : I
> guess the root cause might be Apex running on a lower version of java and
> the application code compiled with a higher version. I shall update
> everyone after the upgrade to see if it fixes the issue.
>
>
> Fatal error encountered
> >
> > 2016-04-29 08:04:45,181 ERROR
> > com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> > com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> > com/tx/y/z/Operator : Unsupported major.minor version 52.0
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> > at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> > at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > at
> >
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> > at
> >
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> > at
> >
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> > at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
> >
>
>
> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
> > Thanks Shubham.
> >
> > Those properties were being set by the parent pom which are inherited by
> > the current pom.xml ( Kind of wanted all apex apps to have same version
> > structure etc) and hence added that property as part of parent pom.
> >
> > I am still stumped as to what is causing this.
> >
> > Regards,
> > Ananth
> >
> >
> >
> > On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <shubham@datatorrent.com
> >
> > wrote:
> >
> >> Hi Ananth,
> >>
> >> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> >> -DarchetypeArtifactId=apex-app-archetype
> -DarchetypeVersion=3.3.0-incubating
> >> -DgroupId=com.example -Dpackage=com.example.mydtapp
> -DartifactId=mydtapp
> >> -Dversion=1.0-SNAPSHOT
> >> and it works for me as well.
> >>
> >> I compared the pom.xml file and noticed values for following properties
> >> were missing.
> >> <manifestEntries>
> >>
> >>
> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
> >>
> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
> >>
> >> </manifestEntries>
> >>
> >> Could you add the following to pom.xml and try again.
> >>
> >> <properties>
> >> <!-- change this if you desire to use a different version of Apex
> Core -->
> >>
> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
> >>
> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
> >> </properties>
> >>
> >>
> >> Thanks,
> >> Shubham
> >>
> >> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
> >> agundabattula@gmail.com> wrote:
> >>
> >>> Hello David,
> >>>
> >>> I reattempted the packaging with the archetype based approach. I guess
> >>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks
> for
> >>> pointing it out.
> >>>
> >>> The default package generated by the archetype command seems to be
> >>> working fine when uploading it but the moment I add in my code and
> >>> dependencies it goes back the issue I am seeing :
> >>>
> >>>
> >>> My app package looks like app.zip ( I removed all of the
> >>> application.class for brevity sake ) . Please note that I deleted all
> the
> >>> jars under /lib folder and the contents of the /app folder in the apa
> >>> pacakge.
> >>>
> >>> I have also attached a copy of my pom.xml file.
> >>>
> >>> Could you please advise if you something amiss right away in the
> >>> structure below ? It may be noted that I retained the classes of the
> >>> original simple project generated by the archetype and even they
> stopped
> >>> showing with the launch button the moment I added in all of the
> application
> >>> dependencies in maven.
> >>>
> >>> Regards,
> >>> Ananth
> >>>
> >>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
> >>> wrote:
> >>>
> >>>> Hi Ananth,
> >>>>
> >>>> I just tried with 3.3.0-incubating with the following and it's
> working:
> >>>>
> >>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> >>>> -DarchetypeArtifactId=apex-app-archetype
> >>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
> >>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp
> -Dversion=1.0-SNAPSHOT
> >>>>
> >>>> Let me know whether this works for you or not.
> >>>>
> >>>> The system uses the MANIFEST.MF file to get the basic meta information
> >>>> on the app package and it looks at all the jars in the app directory
> and
> >>>> searches for classes that subclass
> com.datatorrent.api.StreamingApplication.
> >>>>
> >>>> David
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
> >>>> agundabattula@gmail.com> wrote:
> >>>>
> >>>>> Hello David,
> >>>>>
> >>>>> Regarding as to why I did not use the archetype, I was building the
> >>>>> app for 3.3.0 version and I was not able to specify an archetype
> with the
> >>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
> >>>>> version 3.3.0 ? I was not sure if the archetype version should match
> the
> >>>>> exact apex engine version . My presumption was archetype does not
> matter if
> >>>>> the packaging is done right. So i went ahead and created a simple
> maven
> >>>>> project.
> >>>>>
> >>>>> The maven assembly plugin used the appPackage.xml and the related
> >>>>> constructs to build the exact file layout of the app as mentioned in
> the
> >>>>> output above.
> >>>>>
> >>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
> >>>>>
> >>>>> This brings me to a question. How does the engine know which is the
> >>>>> main class that is to be used for launching the Streaming App. I do
> have a
> >>>>> class that extends the StreaminApplication that is annotated with
> >>>>> ApplicationAnnotation but does not seem to be picking up. By this I
> mean, I
> >>>>> do not see a "launch" icon next to it. All I see is the top level
> packaged
> >>>>> name in the web console but nothing beneath it for me to trigger a
> launch.
> >>>>>
> >>>>> Regards,
> >>>>> Ananth
> >>>>>
> >>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Ananth,
> >>>>>>
> >>>>>> The MANIFEST.MF file is important, and it's generated automatically
> >>>>>> if the archetype is used. For example, the MANFEST.MF file from the
> Pi demo
> >>>>>> in Malhar has this content:
> >>>>>>
> >>>>>> Manifest-Version: 1.0
> >>>>>> Archiver-Version: Plexus Archiver
> >>>>>> Built-By: david
> >>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
> >>>>>> Class-Path: lib/*.jar
> >>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
> >>>>>> Created-By: Apache Maven 3.3.9
> >>>>>> Build-Jdk: 1.8.0_92
> >>>>>> DT-App-Package-Description: Apex demo applications that calculate
> the
> >>>>>> value of Pi. This is a starting point to understand how Apex works.
> >>>>>> DT-App-Package-Group-Id: org.apache.apex
> >>>>>> DT-App-Package-Name: pi-demo
> >>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
> >>>>>>
> >>>>>> Is there a reason why the archetype is not used in your project?
> >>>>>>
> >>>>>> David
> >>>>>>
> >>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
> >>>>>> agundabattula@gmail.com> wrote:
> >>>>>>
> >>>>>>> Hello David,
> >>>>>>>
> >>>>>>> Please find the unzip result at the end of this email. The jar is
> in
> >>>>>>> the app directory inside the apa.
> >>>>>>> As to why I did not use the archetype, I was building the app for
> >>>>>>> 3.3.0 version and I was not able to specify an archetype with the
> version
> >>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not
> matter if
> >>>>>>> the packaging is done right. So i went ahead and created a simple
> maven
> >>>>>>> project.
> >>>>>>>
> >>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
> >>>>>>> listing below ) . I also pasted the contents of the properties.xml
> file at
> >>>>>>> the end of this email for your reference.
> >>>>>>>
> >>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
> >>>>>>> testing: META-INF/ OK
> >>>>>>> testing: META-INF/MANIFEST.MF OK
> >>>>>>> testing: app/ OK
> >>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/ OK
> >>>>>>> testing: lib/activation-1.1.jar OK
> >>>>>>> testing: lib/activemq-client-5.8.0.jar OK
> >>>>>>> testing: lib/antlr-2.7.7.jar OK
> >>>>>>> testing: lib/aopalliance-1.0.jar OK
> >>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
> >>>>>>> testing: lib/btf-1.2.jar OK
> >>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
> >>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
> >>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
> >>>>>>> testing: lib/commons-lang3-3.1.jar OK
> >>>>>>> testing: lib/commons-logging-1.1.3.jar OK
> >>>>>>> testing: lib/curator-client-2.7.0.jar OK
> >>>>>>> testing: lib/curator-framework-2.7.0.jar OK
> >>>>>>> testing: lib/dom4j-1.6.1.jar OK
> >>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
> >>>>>>> testing: lib/fastutil-6.6.4.jar OK
> >>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
> >>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
> >>>>>>> testing: lib/guava-14.0.1.jar OK
> >>>>>>> testing: lib/hawtbuf-1.9.jar OK
> >>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
> >>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
> >>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
> >>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
> >>>>>>> testing: lib/HikariCP-2.4.3.jar OK
> >>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
> >>>>>>> testing: lib/jackson-core-2.6.4.jar OK
> >>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
> >>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
> >>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
> >>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
> >>>>>>> testing: lib/javax.mail-1.5.0.jar OK
> >>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
> >>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
> >>>>>>> testing: lib/jersey-client-1.9.jar OK
> >>>>>>> testing: lib/jersey-core-1.9.jar OK
> >>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jline-2.11.jar OK
> >>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
> >>>>>>> testing: lib/joda-time-2.7.jar OK
> >>>>>>> testing: lib/json-patch-1.7.jar OK
> >>>>>>> testing: lib/jsr305-2.0.1.jar OK
> >>>>>>> testing: lib/jta-1.1.jar OK
> >>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
> >>>>>>> testing: lib/log4j-1.2.17.jar OK
> >>>>>>> testing: lib/logback-classic-1.1.5.jar OK
> >>>>>>> testing: lib/logback-core-1.1.5.jar OK
> >>>>>>> testing: lib/lz4-1.2.0.jar OK
> >>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/metrics-core-3.0.2.jar OK
> >>>>>>> testing: lib/msg-simple-1.1.jar OK
> >>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
> >>>>>>> testing: lib/named-regexp-0.2.3.jar OK
> >>>>>>> testing: lib/netty-3.6.2.Final.jar OK
> >>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
> >>>>>>> testing: lib/opencsv-2.3.jar OK
> >>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
> >>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
> >>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
> >>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
> >>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
> >>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/super-csv-2.3.1.jar OK
> >>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
> >>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
> >>>>>>> testing: lib/zookeeper-3.4.6.jar OK
> >>>>>>> testing: conf/ OK
> >>>>>>> testing: conf/properties.xml OK
> >>>>>>> No errors detected in compressed data of
> >>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
> >>>>>>>
> >>>>>>> Contents of the properties.xml file under the conf folder is here:
> >>>>>>>
> >>>>>>> <?xml version="1.0"?>
> >>>>>>> <configuration>
> >>>>>>> <property>
> >>>>>>> <name>dt.application.CED.class</name>
> >>>>>>>
> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
> >>>>>>> <description>An alias for the application</description>
> >>>>>>> </property>
> >>>>>>> <property>
> >>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
> >>>>>>> <value>512</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
> >>>>>>>
> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
> >>>>>>> <value>20</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> </configuration>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Ananth
> >>>>>>>
> >>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Ananth,
> >>>>>>>>
> >>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
> here?
> >>>>>>>> Make sure the jar containing the application classes should be in
> >>>>>>>> the app directory inside the apa.
> >>>>>>>>
> >>>>>>>> Can I know why the apex-app-archetype is not used in your apa
> >>>>>>>> project?
> >>>>>>>>
> >>>>>>>> David
> >>>>>>>>
> >>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
> >>>>>>>> agundabattula@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hello All,
> >>>>>>>>>
> >>>>>>>>> I am having the following issue with my application. I have a
> >>>>>>>>> maven project that has a class called XYZ that extends
> >>>>>>>>> StreamingApplication.
> >>>>>>>>>
> >>>>>>>>> My maven packaging is ensuring that the final "apa" extension
> >>>>>>>>> package has the right folder structure if exploded on a disk.
> >>>>>>>>>
> >>>>>>>>> However I am not able to locate the Application name and the
> >>>>>>>>> "launch" button next to it after uploading the "apa" package via
> the
> >>>>>>>>> browser.
> >>>>>>>>>
> >>>>>>>>> I tried looking at the docs to see how to specify a class as an
> >>>>>>>>> App that can be launched from a packaged application but could
> not find
> >>>>>>>>> much. Could anyone of you please point me to the right way of
> associating a
> >>>>>>>>> class as an application that will get enabled with a launch
> button in the
> >>>>>>>>> web ui ? The class XYZ has the annotation called
> @ApplicationAnnotation on
> >>>>>>>>> top of it.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Ananth
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
Re: Best practises to specify the Application
Posted by Munagala Ramanath <ra...@datatorrent.com>.
In the pom.xml file of the 3.3 archetype generated project I'm already
seeing:
<source>1.7</source>
<target>1.7</target>
for the maven-compiler-plugin configuration
Ram
On Fri, Apr 29, 2016 at 10:37 AM, David Yan <da...@datatorrent.com> wrote:
> [switching to dev list]
>
> It appears Ananth is using JDK 1.8 to compile the apa and our distributed
> binary, compiled with 1.7 by us, cannot read class files compiled with 1.8.
>
> I think we may need to add target=1.7 in the compiler plugin config in the
> pom.xml file of apex-app-archetype. Thoughts?
>
> David
>
> ---------- Forwarded message ----------
> From: Ananth Gundabattula <ag...@gmail.com>
> Date: Fri, Apr 29, 2016 at 1:20 AM
> Subject: Re: Best practises to specify the Application
> To: users@apex.incubator.apache.org
>
>
> I had a look at the dtgateway.log and observed the following stacktrace : I
> guess the root cause might be Apex running on a lower version of java and
> the application code compiled with a higher version. I shall update
> everyone after the upgrade to see if it fixes the issue.
>
>
> Fatal error encountered
> >
> > 2016-04-29 08:04:45,181 ERROR
> > com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> > com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> > com/tx/y/z/Operator : Unsupported major.minor version 52.0
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> > at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> > at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > at
> >
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> > at
> >
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> > at
> >
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> > at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
> >
>
>
> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
> > Thanks Shubham.
> >
> > Those properties were being set by the parent pom which are inherited by
> > the current pom.xml ( Kind of wanted all apex apps to have same version
> > structure etc) and hence added that property as part of parent pom.
> >
> > I am still stumped as to what is causing this.
> >
> > Regards,
> > Ananth
> >
> >
> >
> > On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <shubham@datatorrent.com
> >
> > wrote:
> >
> >> Hi Ananth,
> >>
> >> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> >> -DarchetypeArtifactId=apex-app-archetype
> -DarchetypeVersion=3.3.0-incubating
> >> -DgroupId=com.example -Dpackage=com.example.mydtapp
> -DartifactId=mydtapp
> >> -Dversion=1.0-SNAPSHOT
> >> and it works for me as well.
> >>
> >> I compared the pom.xml file and noticed values for following properties
> >> were missing.
> >> <manifestEntries>
> >>
> >>
> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
> >>
> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
> >>
> >> </manifestEntries>
> >>
> >> Could you add the following to pom.xml and try again.
> >>
> >> <properties>
> >> <!-- change this if you desire to use a different version of Apex
> Core -->
> >>
> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
> >>
> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
> >> </properties>
> >>
> >>
> >> Thanks,
> >> Shubham
> >>
> >> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
> >> agundabattula@gmail.com> wrote:
> >>
> >>> Hello David,
> >>>
> >>> I reattempted the packaging with the archetype based approach. I guess
> >>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks
> for
> >>> pointing it out.
> >>>
> >>> The default package generated by the archetype command seems to be
> >>> working fine when uploading it but the moment I add in my code and
> >>> dependencies it goes back the issue I am seeing :
> >>>
> >>>
> >>> My app package looks like app.zip ( I removed all of the
> >>> application.class for brevity sake ) . Please note that I deleted all
> the
> >>> jars under /lib folder and the contents of the /app folder in the apa
> >>> pacakge.
> >>>
> >>> I have also attached a copy of my pom.xml file.
> >>>
> >>> Could you please advise if you something amiss right away in the
> >>> structure below ? It may be noted that I retained the classes of the
> >>> original simple project generated by the archetype and even they
> stopped
> >>> showing with the launch button the moment I added in all of the
> application
> >>> dependencies in maven.
> >>>
> >>> Regards,
> >>> Ananth
> >>>
> >>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
> >>> wrote:
> >>>
> >>>> Hi Ananth,
> >>>>
> >>>> I just tried with 3.3.0-incubating with the following and it's
> working:
> >>>>
> >>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> >>>> -DarchetypeArtifactId=apex-app-archetype
> >>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
> >>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp
> -Dversion=1.0-SNAPSHOT
> >>>>
> >>>> Let me know whether this works for you or not.
> >>>>
> >>>> The system uses the MANIFEST.MF file to get the basic meta information
> >>>> on the app package and it looks at all the jars in the app directory
> and
> >>>> searches for classes that subclass
> com.datatorrent.api.StreamingApplication.
> >>>>
> >>>> David
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
> >>>> agundabattula@gmail.com> wrote:
> >>>>
> >>>>> Hello David,
> >>>>>
> >>>>> Regarding as to why I did not use the archetype, I was building the
> >>>>> app for 3.3.0 version and I was not able to specify an archetype
> with the
> >>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
> >>>>> version 3.3.0 ? I was not sure if the archetype version should match
> the
> >>>>> exact apex engine version . My presumption was archetype does not
> matter if
> >>>>> the packaging is done right. So i went ahead and created a simple
> maven
> >>>>> project.
> >>>>>
> >>>>> The maven assembly plugin used the appPackage.xml and the related
> >>>>> constructs to build the exact file layout of the app as mentioned in
> the
> >>>>> output above.
> >>>>>
> >>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
> >>>>>
> >>>>> This brings me to a question. How does the engine know which is the
> >>>>> main class that is to be used for launching the Streaming App. I do
> have a
> >>>>> class that extends the StreaminApplication that is annotated with
> >>>>> ApplicationAnnotation but does not seem to be picking up. By this I
> mean, I
> >>>>> do not see a "launch" icon next to it. All I see is the top level
> packaged
> >>>>> name in the web console but nothing beneath it for me to trigger a
> launch.
> >>>>>
> >>>>> Regards,
> >>>>> Ananth
> >>>>>
> >>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Ananth,
> >>>>>>
> >>>>>> The MANIFEST.MF file is important, and it's generated automatically
> >>>>>> if the archetype is used. For example, the MANFEST.MF file from the
> Pi demo
> >>>>>> in Malhar has this content:
> >>>>>>
> >>>>>> Manifest-Version: 1.0
> >>>>>> Archiver-Version: Plexus Archiver
> >>>>>> Built-By: david
> >>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
> >>>>>> Class-Path: lib/*.jar
> >>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
> >>>>>> Created-By: Apache Maven 3.3.9
> >>>>>> Build-Jdk: 1.8.0_92
> >>>>>> DT-App-Package-Description: Apex demo applications that calculate
> the
> >>>>>> value of Pi. This is a starting point to understand how Apex works.
> >>>>>> DT-App-Package-Group-Id: org.apache.apex
> >>>>>> DT-App-Package-Name: pi-demo
> >>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
> >>>>>>
> >>>>>> Is there a reason why the archetype is not used in your project?
> >>>>>>
> >>>>>> David
> >>>>>>
> >>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
> >>>>>> agundabattula@gmail.com> wrote:
> >>>>>>
> >>>>>>> Hello David,
> >>>>>>>
> >>>>>>> Please find the unzip result at the end of this email. The jar is
> in
> >>>>>>> the app directory inside the apa.
> >>>>>>> As to why I did not use the archetype, I was building the app for
> >>>>>>> 3.3.0 version and I was not able to specify an archetype with the
> version
> >>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not
> matter if
> >>>>>>> the packaging is done right. So i went ahead and created a simple
> maven
> >>>>>>> project.
> >>>>>>>
> >>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
> >>>>>>> listing below ) . I also pasted the contents of the properties.xml
> file at
> >>>>>>> the end of this email for your reference.
> >>>>>>>
> >>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
> >>>>>>> testing: META-INF/ OK
> >>>>>>> testing: META-INF/MANIFEST.MF OK
> >>>>>>> testing: app/ OK
> >>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/ OK
> >>>>>>> testing: lib/activation-1.1.jar OK
> >>>>>>> testing: lib/activemq-client-5.8.0.jar OK
> >>>>>>> testing: lib/antlr-2.7.7.jar OK
> >>>>>>> testing: lib/aopalliance-1.0.jar OK
> >>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
> >>>>>>> testing: lib/btf-1.2.jar OK
> >>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
> >>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
> >>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
> >>>>>>> testing: lib/commons-lang3-3.1.jar OK
> >>>>>>> testing: lib/commons-logging-1.1.3.jar OK
> >>>>>>> testing: lib/curator-client-2.7.0.jar OK
> >>>>>>> testing: lib/curator-framework-2.7.0.jar OK
> >>>>>>> testing: lib/dom4j-1.6.1.jar OK
> >>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
> >>>>>>> testing: lib/fastutil-6.6.4.jar OK
> >>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
> >>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
> >>>>>>> testing: lib/guava-14.0.1.jar OK
> >>>>>>> testing: lib/hawtbuf-1.9.jar OK
> >>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
> >>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
> >>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
> >>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
> >>>>>>> testing: lib/HikariCP-2.4.3.jar OK
> >>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
> >>>>>>> testing: lib/jackson-core-2.6.4.jar OK
> >>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
> >>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
> >>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
> >>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
> >>>>>>> testing: lib/javax.mail-1.5.0.jar OK
> >>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
> >>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
> >>>>>>> testing: lib/jersey-client-1.9.jar OK
> >>>>>>> testing: lib/jersey-core-1.9.jar OK
> >>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jline-2.11.jar OK
> >>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
> >>>>>>> testing: lib/joda-time-2.7.jar OK
> >>>>>>> testing: lib/json-patch-1.7.jar OK
> >>>>>>> testing: lib/jsr305-2.0.1.jar OK
> >>>>>>> testing: lib/jta-1.1.jar OK
> >>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
> >>>>>>> testing: lib/log4j-1.2.17.jar OK
> >>>>>>> testing: lib/logback-classic-1.1.5.jar OK
> >>>>>>> testing: lib/logback-core-1.1.5.jar OK
> >>>>>>> testing: lib/lz4-1.2.0.jar OK
> >>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/metrics-core-3.0.2.jar OK
> >>>>>>> testing: lib/msg-simple-1.1.jar OK
> >>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
> >>>>>>> testing: lib/named-regexp-0.2.3.jar OK
> >>>>>>> testing: lib/netty-3.6.2.Final.jar OK
> >>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
> >>>>>>> testing: lib/opencsv-2.3.jar OK
> >>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
> >>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
> >>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
> >>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
> >>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
> >>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/super-csv-2.3.1.jar OK
> >>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
> >>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
> >>>>>>> testing: lib/zookeeper-3.4.6.jar OK
> >>>>>>> testing: conf/ OK
> >>>>>>> testing: conf/properties.xml OK
> >>>>>>> No errors detected in compressed data of
> >>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
> >>>>>>>
> >>>>>>> Contents of the properties.xml file under the conf folder is here:
> >>>>>>>
> >>>>>>> <?xml version="1.0"?>
> >>>>>>> <configuration>
> >>>>>>> <property>
> >>>>>>> <name>dt.application.CED.class</name>
> >>>>>>>
> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
> >>>>>>> <description>An alias for the application</description>
> >>>>>>> </property>
> >>>>>>> <property>
> >>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
> >>>>>>> <value>512</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
> >>>>>>>
> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
> >>>>>>> <value>20</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> </configuration>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Ananth
> >>>>>>>
> >>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Ananth,
> >>>>>>>>
> >>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
> here?
> >>>>>>>> Make sure the jar containing the application classes should be in
> >>>>>>>> the app directory inside the apa.
> >>>>>>>>
> >>>>>>>> Can I know why the apex-app-archetype is not used in your apa
> >>>>>>>> project?
> >>>>>>>>
> >>>>>>>> David
> >>>>>>>>
> >>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
> >>>>>>>> agundabattula@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hello All,
> >>>>>>>>>
> >>>>>>>>> I am having the following issue with my application. I have a
> >>>>>>>>> maven project that has a class called XYZ that extends
> >>>>>>>>> StreamingApplication.
> >>>>>>>>>
> >>>>>>>>> My maven packaging is ensuring that the final "apa" extension
> >>>>>>>>> package has the right folder structure if exploded on a disk.
> >>>>>>>>>
> >>>>>>>>> However I am not able to locate the Application name and the
> >>>>>>>>> "launch" button next to it after uploading the "apa" package via
> the
> >>>>>>>>> browser.
> >>>>>>>>>
> >>>>>>>>> I tried looking at the docs to see how to specify a class as an
> >>>>>>>>> App that can be launched from a packaged application but could
> not find
> >>>>>>>>> much. Could anyone of you please point me to the right way of
> associating a
> >>>>>>>>> class as an application that will get enabled with a launch
> button in the
> >>>>>>>>> web ui ? The class XYZ has the annotation called
> @ApplicationAnnotation on
> >>>>>>>>> top of it.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Ananth
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
Fwd: Best practises to specify the Application
Posted by David Yan <da...@datatorrent.com>.
[switching to dev list]
It appears Ananth is using JDK 1.8 to compile the apa and our distributed
binary, compiled with 1.7 by us, cannot read class files compiled with 1.8.
I think we may need to add target=1.7 in the compiler plugin config in the
pom.xml file of apex-app-archetype. Thoughts?
David
---------- Forwarded message ----------
From: Ananth Gundabattula <ag...@gmail.com>
Date: Fri, Apr 29, 2016 at 1:20 AM
Subject: Re: Best practises to specify the Application
To: users@apex.incubator.apache.org
I had a look at the dtgateway.log and observed the following stacktrace : I
guess the root cause might be Apex running on a lower version of java and
the application code compiled with a higher version. I shall update
everyone after the upgrade to see if it fixes the issue.
Fatal error encountered
>
> 2016-04-29 08:04:45,181 ERROR
> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> com/tx/y/z/Operator : Unsupported major.minor version 52.0
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> at
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> at
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>
On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> Thanks Shubham.
>
> Those properties were being set by the parent pom which are inherited by
> the current pom.xml ( Kind of wanted all apex apps to have same version
> structure etc) and hence added that property as part of parent pom.
>
> I am still stumped as to what is causing this.
>
> Regards,
> Ananth
>
>
>
> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <sh...@datatorrent.com>
> wrote:
>
>> Hi Ananth,
>>
>> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>> -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
>> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
>> -Dversion=1.0-SNAPSHOT
>> and it works for me as well.
>>
>> I compared the pom.xml file and noticed values for following properties
>> were missing.
>> <manifestEntries>
>>
>> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
>> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
>>
>> </manifestEntries>
>>
>> Could you add the following to pom.xml and try again.
>>
>> <properties>
>> <!-- change this if you desire to use a different version of Apex Core -->
>> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
>> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
>> </properties>
>>
>>
>> Thanks,
>> Shubham
>>
>> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Hello David,
>>>
>>> I reattempted the packaging with the archetype based approach. I guess
>>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
>>> pointing it out.
>>>
>>> The default package generated by the archetype command seems to be
>>> working fine when uploading it but the moment I add in my code and
>>> dependencies it goes back the issue I am seeing :
>>>
>>>
>>> My app package looks like app.zip ( I removed all of the
>>> application.class for brevity sake ) . Please note that I deleted all the
>>> jars under /lib folder and the contents of the /app folder in the apa
>>> pacakge.
>>>
>>> I have also attached a copy of my pom.xml file.
>>>
>>> Could you please advise if you something amiss right away in the
>>> structure below ? It may be noted that I retained the classes of the
>>> original simple project generated by the archetype and even they stopped
>>> showing with the launch button the moment I added in all of the application
>>> dependencies in maven.
>>>
>>> Regards,
>>> Ananth
>>>
>>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
>>> wrote:
>>>
>>>> Hi Ananth,
>>>>
>>>> I just tried with 3.3.0-incubating with the following and it's working:
>>>>
>>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>>> -DarchetypeArtifactId=apex-app-archetype
>>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
>>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>>>>
>>>> Let me know whether this works for you or not.
>>>>
>>>> The system uses the MANIFEST.MF file to get the basic meta information
>>>> on the app package and it looks at all the jars in the app directory and
>>>> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>>>>
>>>> David
>>>>
>>>>
>>>>
>>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>>>> agundabattula@gmail.com> wrote:
>>>>
>>>>> Hello David,
>>>>>
>>>>> Regarding as to why I did not use the archetype, I was building the
>>>>> app for 3.3.0 version and I was not able to specify an archetype with the
>>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>>>>> version 3.3.0 ? I was not sure if the archetype version should match the
>>>>> exact apex engine version . My presumption was archetype does not matter if
>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>> project.
>>>>>
>>>>> The maven assembly plugin used the appPackage.xml and the related
>>>>> constructs to build the exact file layout of the app as mentioned in the
>>>>> output above.
>>>>>
>>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>>>
>>>>> This brings me to a question. How does the engine know which is the
>>>>> main class that is to be used for launching the Streaming App. I do have a
>>>>> class that extends the StreaminApplication that is annotated with
>>>>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>>>>> do not see a "launch" icon next to it. All I see is the top level packaged
>>>>> name in the web console but nothing beneath it for me to trigger a launch.
>>>>>
>>>>> Regards,
>>>>> Ananth
>>>>>
>>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ananth,
>>>>>>
>>>>>> The MANIFEST.MF file is important, and it's generated automatically
>>>>>> if the archetype is used. For example, the MANFEST.MF file from the Pi demo
>>>>>> in Malhar has this content:
>>>>>>
>>>>>> Manifest-Version: 1.0
>>>>>> Archiver-Version: Plexus Archiver
>>>>>> Built-By: david
>>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>>>>> Class-Path: lib/*.jar
>>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>>>> Created-By: Apache Maven 3.3.9
>>>>>> Build-Jdk: 1.8.0_92
>>>>>> DT-App-Package-Description: Apex demo applications that calculate the
>>>>>> value of Pi. This is a starting point to understand how Apex works.
>>>>>> DT-App-Package-Group-Id: org.apache.apex
>>>>>> DT-App-Package-Name: pi-demo
>>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>
>>>>>> Is there a reason why the archetype is not used in your project?
>>>>>>
>>>>>> David
>>>>>>
>>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>>>> agundabattula@gmail.com> wrote:
>>>>>>
>>>>>>> Hello David,
>>>>>>>
>>>>>>> Please find the unzip result at the end of this email. The jar is in
>>>>>>> the app directory inside the apa.
>>>>>>> As to why I did not use the archetype, I was building the app for
>>>>>>> 3.3.0 version and I was not able to specify an archetype with the version
>>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not matter if
>>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>>> project.
>>>>>>>
>>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
>>>>>>> listing below ) . I also pasted the contents of the properties.xml file at
>>>>>>> the end of this email for your reference.
>>>>>>>
>>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>>>>> testing: META-INF/ OK
>>>>>>> testing: META-INF/MANIFEST.MF OK
>>>>>>> testing: app/ OK
>>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/ OK
>>>>>>> testing: lib/activation-1.1.jar OK
>>>>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>>>>> testing: lib/antlr-2.7.7.jar OK
>>>>>>> testing: lib/aopalliance-1.0.jar OK
>>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>>>>> testing: lib/btf-1.2.jar OK
>>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>>>>> testing: lib/commons-lang3-3.1.jar OK
>>>>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>>>>> testing: lib/curator-client-2.7.0.jar OK
>>>>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>>>>> testing: lib/dom4j-1.6.1.jar OK
>>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>>>>> testing: lib/fastutil-6.6.4.jar OK
>>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>>>>> testing: lib/guava-14.0.1.jar OK
>>>>>>> testing: lib/hawtbuf-1.9.jar OK
>>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>>>>> testing: lib/jersey-client-1.9.jar OK
>>>>>>> testing: lib/jersey-core-1.9.jar OK
>>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jline-2.11.jar OK
>>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>>>>> testing: lib/joda-time-2.7.jar OK
>>>>>>> testing: lib/json-patch-1.7.jar OK
>>>>>>> testing: lib/jsr305-2.0.1.jar OK
>>>>>>> testing: lib/jta-1.1.jar OK
>>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>>>>> testing: lib/log4j-1.2.17.jar OK
>>>>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>>>>> testing: lib/logback-core-1.1.5.jar OK
>>>>>>> testing: lib/lz4-1.2.0.jar OK
>>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>>>>> testing: lib/msg-simple-1.1.jar OK
>>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>>>>> testing: lib/opencsv-2.3.jar OK
>>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/super-csv-2.3.1.jar OK
>>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>>>>> testing: conf/ OK
>>>>>>> testing: conf/properties.xml OK
>>>>>>> No errors detected in compressed data of
>>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>>>
>>>>>>> Contents of the properties.xml file under the conf folder is here:
>>>>>>>
>>>>>>> <?xml version="1.0"?>
>>>>>>> <configuration>
>>>>>>> <property>
>>>>>>> <name>dt.application.CED.class</name>
>>>>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>>>> <description>An alias for the application</description>
>>>>>>> </property>
>>>>>>> <property>
>>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>>>> <value>512</value>
>>>>>>> </property>
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>>>> </property>
>>>>>>>
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>>>> <value>20</value>
>>>>>>> </property>
>>>>>>>
>>>>>>> </configuration>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ananth
>>>>>>>
>>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Ananth,
>>>>>>>>
>>>>>>>> Can you do an "unzip -t" on your apa file and paste the output here?
>>>>>>>> Make sure the jar containing the application classes should be in
>>>>>>>> the app directory inside the apa.
>>>>>>>>
>>>>>>>> Can I know why the apex-app-archetype is not used in your apa
>>>>>>>> project?
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello All,
>>>>>>>>>
>>>>>>>>> I am having the following issue with my application. I have a
>>>>>>>>> maven project that has a class called XYZ that extends
>>>>>>>>> StreamingApplication.
>>>>>>>>>
>>>>>>>>> My maven packaging is ensuring that the final "apa" extension
>>>>>>>>> package has the right folder structure if exploded on a disk.
>>>>>>>>>
>>>>>>>>> However I am not able to locate the Application name and the
>>>>>>>>> "launch" button next to it after uploading the "apa" package via the
>>>>>>>>> browser.
>>>>>>>>>
>>>>>>>>> I tried looking at the docs to see how to specify a class as an
>>>>>>>>> App that can be launched from a packaged application but could not find
>>>>>>>>> much. Could anyone of you please point me to the right way of associating a
>>>>>>>>> class as an application that will get enabled with a launch button in the
>>>>>>>>> web ui ? The class XYZ has the annotation called @ApplicationAnnotation on
>>>>>>>>> top of it.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Ananth
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by Ananth Gundabattula <ag...@gmail.com>.
Just wanted to update you all that upgrading the JDK on Apex server did the
trick.
Thanks a lot for your time David/Shubham/Sandeep/Ram.
On Sat, Apr 30, 2016 at 6:51 AM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
>
> I guess the issue is related to the fact that the runtime jdk which Apex
> server engine is running on a lower version than that of the compiled app
> and not the archetype issue at all. I did attempt with the archetype as
> well but came up with the same effect. The app is compiled using 1.8
> source + target but the Apex server is running on a machine with jdk 1.7
> installed. That was the root cause. By the way, the original pom project
> artifact had the exact project structure as that was generated by the
> archetype. I shall make sure I use the archetype from hence on. The pom
> attached also explictly defines the jdk versions for both source and
> target.
>
> The web UI while installing the apa package failed with a Fatal error and
> on refreshing showed the entry for the apa package but did not show any
> applications inside it. I guess we have a bug in the UI wherein an improper
> installation of an app results in an entry being shown in the UI. I believe
> it should not show the app entry at all if the installation fails.
>
> Regards,
> Ananth
>
> On Sat, Apr 30, 2016 at 3:54 AM, David Yan <da...@datatorrent.com> wrote:
>
>> Hi Ananth,
>>
>> Please always use the archetype to generate your project.
>> In the archetype, among many other things, it also enforces that the
>> target java class files to be compatible with JDK 1.7.
>> And it seems that your project is missing that as well and that's why you
>> see the UnsupportedClassVersionError error.
>>
>> David
>>
>> On Fri, Apr 29, 2016 at 1:20 AM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> I had a look at the dtgateway.log and observed the following stacktrace
>>> : I guess the root cause might be Apex running on a lower version of java
>>> and the application code compiled with a higher version. I shall update
>>> everyone after the upgrade to see if it fixes the issue.
>>>
>>>
>>> Fatal error encountered
>>>>
>>>> 2016-04-29 08:04:45,181 ERROR
>>>> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
>>>> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
>>>> com/tx/y/z/Operator : Unsupported major.minor version 52.0
>>>> at java.lang.ClassLoader.defineClass1(Native Method)
>>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>>>> at
>>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>>> at
>>>> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
>>>> at
>>>> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
>>>> at
>>>> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
>>>> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>>>>
>>>
>>>
>>> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Thanks Shubham.
>>>>
>>>> Those properties were being set by the parent pom which are inherited
>>>> by the current pom.xml ( Kind of wanted all apex apps to have same version
>>>> structure etc) and hence added that property as part of parent pom.
>>>>
>>>> I am still stumped as to what is causing this.
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>>
>>>>
>>>> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <
>>>> shubham@datatorrent.com> wrote:
>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>>>> -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
>>>>> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
>>>>> -Dversion=1.0-SNAPSHOT
>>>>> and it works for me as well.
>>>>>
>>>>> I compared the pom.xml file and noticed values for following
>>>>> properties were missing.
>>>>> <manifestEntries>
>>>>>
>>>>> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
>>>>> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
>>>>>
>>>>> </manifestEntries>
>>>>>
>>>>> Could you add the following to pom.xml and try again.
>>>>>
>>>>> <properties>
>>>>> <!-- change this if you desire to use a different version of Apex Core -->
>>>>> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
>>>>> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
>>>>> </properties>
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Shubham
>>>>>
>>>>> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
>>>>> agundabattula@gmail.com> wrote:
>>>>>
>>>>>> Hello David,
>>>>>>
>>>>>> I reattempted the packaging with the archetype based approach. I
>>>>>> guess the 3.3.0 archetype did not exist when I attempted it earlier. Thanks
>>>>>> for pointing it out.
>>>>>>
>>>>>> The default package generated by the archetype command seems to be
>>>>>> working fine when uploading it but the moment I add in my code and
>>>>>> dependencies it goes back the issue I am seeing :
>>>>>>
>>>>>>
>>>>>> My app package looks like app.zip ( I removed all of the
>>>>>> application.class for brevity sake ) . Please note that I deleted all the
>>>>>> jars under /lib folder and the contents of the /app folder in the apa
>>>>>> pacakge.
>>>>>>
>>>>>> I have also attached a copy of my pom.xml file.
>>>>>>
>>>>>> Could you please advise if you something amiss right away in the
>>>>>> structure below ? It may be noted that I retained the classes of the
>>>>>> original simple project generated by the archetype and even they stopped
>>>>>> showing with the launch button the moment I added in all of the application
>>>>>> dependencies in maven.
>>>>>>
>>>>>> Regards,
>>>>>> Ananth
>>>>>>
>>>>>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ananth,
>>>>>>>
>>>>>>> I just tried with 3.3.0-incubating with the following and it's
>>>>>>> working:
>>>>>>>
>>>>>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>>>>>> -DarchetypeArtifactId=apex-app-archetype
>>>>>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
>>>>>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>>>>>>>
>>>>>>> Let me know whether this works for you or not.
>>>>>>>
>>>>>>> The system uses the MANIFEST.MF file to get the basic meta
>>>>>>> information on the app package and it looks at all the jars in the app
>>>>>>> directory and searches for classes that subclass
>>>>>>> com.datatorrent.api.StreamingApplication.
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello David,
>>>>>>>>
>>>>>>>> Regarding as to why I did not use the archetype, I was building the
>>>>>>>> app for 3.3.0 version and I was not able to specify an archetype with the
>>>>>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>>>>>>>> version 3.3.0 ? I was not sure if the archetype version should match the
>>>>>>>> exact apex engine version . My presumption was archetype does not matter if
>>>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>>>> project.
>>>>>>>>
>>>>>>>> The maven assembly plugin used the appPackage.xml and the related
>>>>>>>> constructs to build the exact file layout of the app as mentioned in the
>>>>>>>> output above.
>>>>>>>>
>>>>>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>>>>>>
>>>>>>>> This brings me to a question. How does the engine know which is the
>>>>>>>> main class that is to be used for launching the Streaming App. I do have a
>>>>>>>> class that extends the StreaminApplication that is annotated with
>>>>>>>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>>>>>>>> do not see a "launch" icon next to it. All I see is the top level packaged
>>>>>>>> name in the web console but nothing beneath it for me to trigger a launch.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Ananth
>>>>>>>>
>>>>>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Ananth,
>>>>>>>>>
>>>>>>>>> The MANIFEST.MF file is important, and it's generated
>>>>>>>>> automatically if the archetype is used. For example, the MANFEST.MF file
>>>>>>>>> from the Pi demo in Malhar has this content:
>>>>>>>>>
>>>>>>>>> Manifest-Version: 1.0
>>>>>>>>> Archiver-Version: Plexus Archiver
>>>>>>>>> Built-By: david
>>>>>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi
>>>>>>>>> Demo
>>>>>>>>> Class-Path: lib/*.jar
>>>>>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>>>> Created-By: Apache Maven 3.3.9
>>>>>>>>> Build-Jdk: 1.8.0_92
>>>>>>>>> DT-App-Package-Description: Apex demo applications that calculate
>>>>>>>>> the
>>>>>>>>> value of Pi. This is a starting point to understand how Apex
>>>>>>>>> works.
>>>>>>>>> DT-App-Package-Group-Id: org.apache.apex
>>>>>>>>> DT-App-Package-Name: pi-demo
>>>>>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>>>>
>>>>>>>>> Is there a reason why the archetype is not used in your project?
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello David,
>>>>>>>>>>
>>>>>>>>>> Please find the unzip result at the end of this email. The jar is
>>>>>>>>>> in the app directory inside the apa.
>>>>>>>>>> As to why I did not use the archetype, I was building the app for
>>>>>>>>>> 3.3.0 version and I was not able to specify an archetype with the version
>>>>>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not matter if
>>>>>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>>>>>> project.
>>>>>>>>>>
>>>>>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
>>>>>>>>>> listing below ) . I also pasted the contents of the properties.xml file at
>>>>>>>>>> the end of this email for your reference.
>>>>>>>>>>
>>>>>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>>>>>>>> testing: META-INF/ OK
>>>>>>>>>> testing: META-INF/MANIFEST.MF OK
>>>>>>>>>> testing: app/ OK
>>>>>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>>>>>>>> testing: lib/ OK
>>>>>>>>>> testing: lib/activation-1.1.jar OK
>>>>>>>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>>>>>>>> testing: lib/antlr-2.7.7.jar OK
>>>>>>>>>> testing: lib/aopalliance-1.0.jar OK
>>>>>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>>>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>>>>>>>> testing: lib/btf-1.2.jar OK
>>>>>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>>>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>>>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>>>>>>>> testing: lib/commons-lang3-3.1.jar OK
>>>>>>>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>>>>>>>> testing: lib/curator-client-2.7.0.jar OK
>>>>>>>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>>>>>>>> testing: lib/dom4j-1.6.1.jar OK
>>>>>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>>>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>>>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>>>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>>>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>>>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>>>>>>>> testing: lib/fastutil-6.6.4.jar OK
>>>>>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>>>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>>>>>>>> testing: lib/guava-14.0.1.jar OK
>>>>>>>>>> testing: lib/hawtbuf-1.9.jar OK
>>>>>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar
>>>>>>>>>> OK
>>>>>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>>>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>>>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>>>>>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>>>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>>>>>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>>>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>>>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>>>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>>>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>>>>>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>>>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>>>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>>>>>>>> testing: lib/jersey-client-1.9.jar OK
>>>>>>>>>> testing: lib/jersey-core-1.9.jar OK
>>>>>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>>>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>>>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>>>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>>>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>>>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>>>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>>>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>>>>>>>> testing: lib/jline-2.11.jar OK
>>>>>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>>>>>>>> testing: lib/joda-time-2.7.jar OK
>>>>>>>>>> testing: lib/json-patch-1.7.jar OK
>>>>>>>>>> testing: lib/jsr305-2.0.1.jar OK
>>>>>>>>>> testing: lib/jta-1.1.jar OK
>>>>>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>>>>>>>> testing: lib/log4j-1.2.17.jar OK
>>>>>>>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>>>>>>>> testing: lib/logback-core-1.1.5.jar OK
>>>>>>>>>> testing: lib/lz4-1.2.0.jar OK
>>>>>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>>>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>>>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>>>>>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>>>>>>>> testing: lib/msg-simple-1.1.jar OK
>>>>>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>>>>>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>>>>>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>>>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>>>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>>>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>>>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>>>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>>>>>>>> testing: lib/opencsv-2.3.jar OK
>>>>>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>>>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>>>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>>>>>>>> testing: lib/super-csv-2.3.1.jar OK
>>>>>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>>>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>>>>>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>>>>>>>> testing: conf/ OK
>>>>>>>>>> testing: conf/properties.xml OK
>>>>>>>>>> No errors detected in compressed data of
>>>>>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>>>>>>
>>>>>>>>>> Contents of the properties.xml file under the conf folder is
>>>>>>>>>> here:
>>>>>>>>>>
>>>>>>>>>> <?xml version="1.0"?>
>>>>>>>>>> <configuration>
>>>>>>>>>> <property>
>>>>>>>>>> <name>dt.application.CED.class</name>
>>>>>>>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>>>>>>> <description>An alias for the application</description>
>>>>>>>>>> </property>
>>>>>>>>>> <property>
>>>>>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>>>>>>> <value>512</value>
>>>>>>>>>> </property>
>>>>>>>>>>
>>>>>>>>>> <property>
>>>>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>>>>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>>>>>>> </property>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> <property>
>>>>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>>>>>>> <value>20</value>
>>>>>>>>>> </property>
>>>>>>>>>>
>>>>>>>>>> </configuration>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Ananth
>>>>>>>>>>
>>>>>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <david@datatorrent.com
>>>>>>>>>> > wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Ananth,
>>>>>>>>>>>
>>>>>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
>>>>>>>>>>> here?
>>>>>>>>>>> Make sure the jar containing the application classes should be
>>>>>>>>>>> in the app directory inside the apa.
>>>>>>>>>>>
>>>>>>>>>>> Can I know why the apex-app-archetype is not used in your apa
>>>>>>>>>>> project?
>>>>>>>>>>>
>>>>>>>>>>> David
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello All,
>>>>>>>>>>>>
>>>>>>>>>>>> I am having the following issue with my application. I have a
>>>>>>>>>>>> maven project that has a class called XYZ that extends
>>>>>>>>>>>> StreamingApplication.
>>>>>>>>>>>>
>>>>>>>>>>>> My maven packaging is ensuring that the final "apa" extension
>>>>>>>>>>>> package has the right folder structure if exploded on a disk.
>>>>>>>>>>>>
>>>>>>>>>>>> However I am not able to locate the Application name and the
>>>>>>>>>>>> "launch" button next to it after uploading the "apa" package via the
>>>>>>>>>>>> browser.
>>>>>>>>>>>>
>>>>>>>>>>>> I tried looking at the docs to see how to specify a class as an
>>>>>>>>>>>> App that can be launched from a packaged application but could not find
>>>>>>>>>>>> much. Could anyone of you please point me to the right way of associating a
>>>>>>>>>>>> class as an application that will get enabled with a launch button in the
>>>>>>>>>>>> web ui ? The class XYZ has the annotation called @ApplicationAnnotation on
>>>>>>>>>>>> top of it.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Ananth
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by Ananth Gundabattula <ag...@gmail.com>.
I guess the issue is related to the fact that the runtime jdk which Apex
server engine is running on a lower version than that of the compiled app
and not the archetype issue at all. I did attempt with the archetype as
well but came up with the same effect. The app is compiled using 1.8
source + target but the Apex server is running on a machine with jdk 1.7
installed. That was the root cause. By the way, the original pom project
artifact had the exact project structure as that was generated by the
archetype. I shall make sure I use the archetype from hence on. The pom
attached also explictly defines the jdk versions for both source and
target.
The web UI while installing the apa package failed with a Fatal error and
on refreshing showed the entry for the apa package but did not show any
applications inside it. I guess we have a bug in the UI wherein an improper
installation of an app results in an entry being shown in the UI. I believe
it should not show the app entry at all if the installation fails.
Regards,
Ananth
On Sat, Apr 30, 2016 at 3:54 AM, David Yan <da...@datatorrent.com> wrote:
> Hi Ananth,
>
> Please always use the archetype to generate your project.
> In the archetype, among many other things, it also enforces that the
> target java class files to be compatible with JDK 1.7.
> And it seems that your project is missing that as well and that's why you
> see the UnsupportedClassVersionError error.
>
> David
>
> On Fri, Apr 29, 2016 at 1:20 AM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> I had a look at the dtgateway.log and observed the following stacktrace :
>> I guess the root cause might be Apex running on a lower version of java and
>> the application code compiled with a higher version. I shall update
>> everyone after the upgrade to see if it fixes the issue.
>>
>>
>> Fatal error encountered
>>>
>>> 2016-04-29 08:04:45,181 ERROR
>>> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
>>> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
>>> com/tx/y/z/Operator : Unsupported major.minor version 52.0
>>> at java.lang.ClassLoader.defineClass1(Native Method)
>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>>> at
>>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>>> at
>>> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
>>> at
>>> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
>>> at
>>> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
>>> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>>>
>>
>>
>> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Thanks Shubham.
>>>
>>> Those properties were being set by the parent pom which are inherited by
>>> the current pom.xml ( Kind of wanted all apex apps to have same version
>>> structure etc) and hence added that property as part of parent pom.
>>>
>>> I am still stumped as to what is causing this.
>>>
>>> Regards,
>>> Ananth
>>>
>>>
>>>
>>> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <shubham@datatorrent.com
>>> > wrote:
>>>
>>>> Hi Ananth,
>>>>
>>>> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>>> -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
>>>> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
>>>> -Dversion=1.0-SNAPSHOT
>>>> and it works for me as well.
>>>>
>>>> I compared the pom.xml file and noticed values for following properties
>>>> were missing.
>>>> <manifestEntries>
>>>>
>>>> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
>>>> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
>>>>
>>>> </manifestEntries>
>>>>
>>>> Could you add the following to pom.xml and try again.
>>>>
>>>> <properties>
>>>> <!-- change this if you desire to use a different version of Apex Core -->
>>>> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
>>>> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
>>>> </properties>
>>>>
>>>>
>>>> Thanks,
>>>> Shubham
>>>>
>>>> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
>>>> agundabattula@gmail.com> wrote:
>>>>
>>>>> Hello David,
>>>>>
>>>>> I reattempted the packaging with the archetype based approach. I guess
>>>>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
>>>>> pointing it out.
>>>>>
>>>>> The default package generated by the archetype command seems to be
>>>>> working fine when uploading it but the moment I add in my code and
>>>>> dependencies it goes back the issue I am seeing :
>>>>>
>>>>>
>>>>> My app package looks like app.zip ( I removed all of the
>>>>> application.class for brevity sake ) . Please note that I deleted all the
>>>>> jars under /lib folder and the contents of the /app folder in the apa
>>>>> pacakge.
>>>>>
>>>>> I have also attached a copy of my pom.xml file.
>>>>>
>>>>> Could you please advise if you something amiss right away in the
>>>>> structure below ? It may be noted that I retained the classes of the
>>>>> original simple project generated by the archetype and even they stopped
>>>>> showing with the launch button the moment I added in all of the application
>>>>> dependencies in maven.
>>>>>
>>>>> Regards,
>>>>> Ananth
>>>>>
>>>>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ananth,
>>>>>>
>>>>>> I just tried with 3.3.0-incubating with the following and it's
>>>>>> working:
>>>>>>
>>>>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>>>>> -DarchetypeArtifactId=apex-app-archetype
>>>>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
>>>>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>>>>>>
>>>>>> Let me know whether this works for you or not.
>>>>>>
>>>>>> The system uses the MANIFEST.MF file to get the basic meta
>>>>>> information on the app package and it looks at all the jars in the app
>>>>>> directory and searches for classes that subclass
>>>>>> com.datatorrent.api.StreamingApplication.
>>>>>>
>>>>>> David
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>>>>>> agundabattula@gmail.com> wrote:
>>>>>>
>>>>>>> Hello David,
>>>>>>>
>>>>>>> Regarding as to why I did not use the archetype, I was building the
>>>>>>> app for 3.3.0 version and I was not able to specify an archetype with the
>>>>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>>>>>>> version 3.3.0 ? I was not sure if the archetype version should match the
>>>>>>> exact apex engine version . My presumption was archetype does not matter if
>>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>>> project.
>>>>>>>
>>>>>>> The maven assembly plugin used the appPackage.xml and the related
>>>>>>> constructs to build the exact file layout of the app as mentioned in the
>>>>>>> output above.
>>>>>>>
>>>>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>>>>>
>>>>>>> This brings me to a question. How does the engine know which is the
>>>>>>> main class that is to be used for launching the Streaming App. I do have a
>>>>>>> class that extends the StreaminApplication that is annotated with
>>>>>>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>>>>>>> do not see a "launch" icon next to it. All I see is the top level packaged
>>>>>>> name in the web console but nothing beneath it for me to trigger a launch.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ananth
>>>>>>>
>>>>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Ananth,
>>>>>>>>
>>>>>>>> The MANIFEST.MF file is important, and it's generated automatically
>>>>>>>> if the archetype is used. For example, the MANFEST.MF file from the Pi demo
>>>>>>>> in Malhar has this content:
>>>>>>>>
>>>>>>>> Manifest-Version: 1.0
>>>>>>>> Archiver-Version: Plexus Archiver
>>>>>>>> Built-By: david
>>>>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>>>>>>> Class-Path: lib/*.jar
>>>>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>>> Created-By: Apache Maven 3.3.9
>>>>>>>> Build-Jdk: 1.8.0_92
>>>>>>>> DT-App-Package-Description: Apex demo applications that calculate
>>>>>>>> the
>>>>>>>> value of Pi. This is a starting point to understand how Apex works.
>>>>>>>> DT-App-Package-Group-Id: org.apache.apex
>>>>>>>> DT-App-Package-Name: pi-demo
>>>>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>>>
>>>>>>>> Is there a reason why the archetype is not used in your project?
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello David,
>>>>>>>>>
>>>>>>>>> Please find the unzip result at the end of this email. The jar is
>>>>>>>>> in the app directory inside the apa.
>>>>>>>>> As to why I did not use the archetype, I was building the app for
>>>>>>>>> 3.3.0 version and I was not able to specify an archetype with the version
>>>>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not matter if
>>>>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>>>>> project.
>>>>>>>>>
>>>>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
>>>>>>>>> listing below ) . I also pasted the contents of the properties.xml file at
>>>>>>>>> the end of this email for your reference.
>>>>>>>>>
>>>>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>>>>>>> testing: META-INF/ OK
>>>>>>>>> testing: META-INF/MANIFEST.MF OK
>>>>>>>>> testing: app/ OK
>>>>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>>>>>>> testing: lib/ OK
>>>>>>>>> testing: lib/activation-1.1.jar OK
>>>>>>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>>>>>>> testing: lib/antlr-2.7.7.jar OK
>>>>>>>>> testing: lib/aopalliance-1.0.jar OK
>>>>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>>>>>>> testing: lib/btf-1.2.jar OK
>>>>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>>>>>>> testing: lib/commons-lang3-3.1.jar OK
>>>>>>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>>>>>>> testing: lib/curator-client-2.7.0.jar OK
>>>>>>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>>>>>>> testing: lib/dom4j-1.6.1.jar OK
>>>>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>>>>>>> testing: lib/fastutil-6.6.4.jar OK
>>>>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>>>>>>> testing: lib/guava-14.0.1.jar OK
>>>>>>>>> testing: lib/hawtbuf-1.9.jar OK
>>>>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>>>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>>>>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>>>>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>>>>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>>>>>>> testing: lib/jersey-client-1.9.jar OK
>>>>>>>>> testing: lib/jersey-core-1.9.jar OK
>>>>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>>>>>>> testing: lib/jline-2.11.jar OK
>>>>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>>>>>>> testing: lib/joda-time-2.7.jar OK
>>>>>>>>> testing: lib/json-patch-1.7.jar OK
>>>>>>>>> testing: lib/jsr305-2.0.1.jar OK
>>>>>>>>> testing: lib/jta-1.1.jar OK
>>>>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>>>>>>> testing: lib/log4j-1.2.17.jar OK
>>>>>>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>>>>>>> testing: lib/logback-core-1.1.5.jar OK
>>>>>>>>> testing: lib/lz4-1.2.0.jar OK
>>>>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>>>>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>>>>>>> testing: lib/msg-simple-1.1.jar OK
>>>>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>>>>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>>>>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>>>>>>> testing: lib/opencsv-2.3.jar OK
>>>>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>>>>>>> testing: lib/super-csv-2.3.1.jar OK
>>>>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>>>>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>>>>>>> testing: conf/ OK
>>>>>>>>> testing: conf/properties.xml OK
>>>>>>>>> No errors detected in compressed data of
>>>>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>>>>>
>>>>>>>>> Contents of the properties.xml file under the conf folder is here:
>>>>>>>>>
>>>>>>>>> <?xml version="1.0"?>
>>>>>>>>> <configuration>
>>>>>>>>> <property>
>>>>>>>>> <name>dt.application.CED.class</name>
>>>>>>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>>>>>> <description>An alias for the application</description>
>>>>>>>>> </property>
>>>>>>>>> <property>
>>>>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>>>>>> <value>512</value>
>>>>>>>>> </property>
>>>>>>>>>
>>>>>>>>> <property>
>>>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>>>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>>>>>> </property>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> <property>
>>>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>>>>>> <value>20</value>
>>>>>>>>> </property>
>>>>>>>>>
>>>>>>>>> </configuration>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Ananth
>>>>>>>>>
>>>>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Ananth,
>>>>>>>>>>
>>>>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
>>>>>>>>>> here?
>>>>>>>>>> Make sure the jar containing the application classes should be in
>>>>>>>>>> the app directory inside the apa.
>>>>>>>>>>
>>>>>>>>>> Can I know why the apex-app-archetype is not used in your apa
>>>>>>>>>> project?
>>>>>>>>>>
>>>>>>>>>> David
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello All,
>>>>>>>>>>>
>>>>>>>>>>> I am having the following issue with my application. I have a
>>>>>>>>>>> maven project that has a class called XYZ that extends
>>>>>>>>>>> StreamingApplication.
>>>>>>>>>>>
>>>>>>>>>>> My maven packaging is ensuring that the final "apa" extension
>>>>>>>>>>> package has the right folder structure if exploded on a disk.
>>>>>>>>>>>
>>>>>>>>>>> However I am not able to locate the Application name and the
>>>>>>>>>>> "launch" button next to it after uploading the "apa" package via the
>>>>>>>>>>> browser.
>>>>>>>>>>>
>>>>>>>>>>> I tried looking at the docs to see how to specify a class as an
>>>>>>>>>>> App that can be launched from a packaged application but could not find
>>>>>>>>>>> much. Could anyone of you please point me to the right way of associating a
>>>>>>>>>>> class as an application that will get enabled with a launch button in the
>>>>>>>>>>> web ui ? The class XYZ has the annotation called @ApplicationAnnotation on
>>>>>>>>>>> top of it.
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Ananth
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by David Yan <da...@datatorrent.com>.
Hi Ananth,
Please always use the archetype to generate your project.
In the archetype, among many other things, it also enforces that the target
java class files to be compatible with JDK 1.7.
And it seems that your project is missing that as well and that's why you
see the UnsupportedClassVersionError error.
David
On Fri, Apr 29, 2016 at 1:20 AM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> I had a look at the dtgateway.log and observed the following stacktrace :
> I guess the root cause might be Apex running on a lower version of java and
> the application code compiled with a higher version. I shall update
> everyone after the upgrade to see if it fixes the issue.
>
>
> Fatal error encountered
>>
>> 2016-04-29 08:04:45,181 ERROR
>> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
>> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
>> com/tx/y/z/Operator : Unsupported major.minor version 52.0
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>> at
>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> at
>> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
>> at
>> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
>> at
>> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
>> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>>
>
>
> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Thanks Shubham.
>>
>> Those properties were being set by the parent pom which are inherited by
>> the current pom.xml ( Kind of wanted all apex apps to have same version
>> structure etc) and hence added that property as part of parent pom.
>>
>> I am still stumped as to what is causing this.
>>
>> Regards,
>> Ananth
>>
>>
>>
>> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <sh...@datatorrent.com>
>> wrote:
>>
>>> Hi Ananth,
>>>
>>> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>> -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
>>> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
>>> -Dversion=1.0-SNAPSHOT
>>> and it works for me as well.
>>>
>>> I compared the pom.xml file and noticed values for following properties
>>> were missing.
>>> <manifestEntries>
>>>
>>> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
>>> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
>>>
>>> </manifestEntries>
>>>
>>> Could you add the following to pom.xml and try again.
>>>
>>> <properties>
>>> <!-- change this if you desire to use a different version of Apex Core -->
>>> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
>>> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
>>> </properties>
>>>
>>>
>>> Thanks,
>>> Shubham
>>>
>>> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello David,
>>>>
>>>> I reattempted the packaging with the archetype based approach. I guess
>>>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
>>>> pointing it out.
>>>>
>>>> The default package generated by the archetype command seems to be
>>>> working fine when uploading it but the moment I add in my code and
>>>> dependencies it goes back the issue I am seeing :
>>>>
>>>>
>>>> My app package looks like app.zip ( I removed all of the
>>>> application.class for brevity sake ) . Please note that I deleted all the
>>>> jars under /lib folder and the contents of the /app folder in the apa
>>>> pacakge.
>>>>
>>>> I have also attached a copy of my pom.xml file.
>>>>
>>>> Could you please advise if you something amiss right away in the
>>>> structure below ? It may be noted that I retained the classes of the
>>>> original simple project generated by the archetype and even they stopped
>>>> showing with the launch button the moment I added in all of the application
>>>> dependencies in maven.
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> I just tried with 3.3.0-incubating with the following and it's
>>>>> working:
>>>>>
>>>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>>>> -DarchetypeArtifactId=apex-app-archetype
>>>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
>>>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>>>>>
>>>>> Let me know whether this works for you or not.
>>>>>
>>>>> The system uses the MANIFEST.MF file to get the basic meta information
>>>>> on the app package and it looks at all the jars in the app directory and
>>>>> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>>>>> agundabattula@gmail.com> wrote:
>>>>>
>>>>>> Hello David,
>>>>>>
>>>>>> Regarding as to why I did not use the archetype, I was building the
>>>>>> app for 3.3.0 version and I was not able to specify an archetype with the
>>>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>>>>>> version 3.3.0 ? I was not sure if the archetype version should match the
>>>>>> exact apex engine version . My presumption was archetype does not matter if
>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>> project.
>>>>>>
>>>>>> The maven assembly plugin used the appPackage.xml and the related
>>>>>> constructs to build the exact file layout of the app as mentioned in the
>>>>>> output above.
>>>>>>
>>>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>>>>
>>>>>> This brings me to a question. How does the engine know which is the
>>>>>> main class that is to be used for launching the Streaming App. I do have a
>>>>>> class that extends the StreaminApplication that is annotated with
>>>>>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>>>>>> do not see a "launch" icon next to it. All I see is the top level packaged
>>>>>> name in the web console but nothing beneath it for me to trigger a launch.
>>>>>>
>>>>>> Regards,
>>>>>> Ananth
>>>>>>
>>>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ananth,
>>>>>>>
>>>>>>> The MANIFEST.MF file is important, and it's generated automatically
>>>>>>> if the archetype is used. For example, the MANFEST.MF file from the Pi demo
>>>>>>> in Malhar has this content:
>>>>>>>
>>>>>>> Manifest-Version: 1.0
>>>>>>> Archiver-Version: Plexus Archiver
>>>>>>> Built-By: david
>>>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>>>>>> Class-Path: lib/*.jar
>>>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>> Created-By: Apache Maven 3.3.9
>>>>>>> Build-Jdk: 1.8.0_92
>>>>>>> DT-App-Package-Description: Apex demo applications that calculate
>>>>>>> the
>>>>>>> value of Pi. This is a starting point to understand how Apex works.
>>>>>>> DT-App-Package-Group-Id: org.apache.apex
>>>>>>> DT-App-Package-Name: pi-demo
>>>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>>
>>>>>>> Is there a reason why the archetype is not used in your project?
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello David,
>>>>>>>>
>>>>>>>> Please find the unzip result at the end of this email. The jar is
>>>>>>>> in the app directory inside the apa.
>>>>>>>> As to why I did not use the archetype, I was building the app for
>>>>>>>> 3.3.0 version and I was not able to specify an archetype with the version
>>>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not matter if
>>>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>>>> project.
>>>>>>>>
>>>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
>>>>>>>> listing below ) . I also pasted the contents of the properties.xml file at
>>>>>>>> the end of this email for your reference.
>>>>>>>>
>>>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>>>>>> testing: META-INF/ OK
>>>>>>>> testing: META-INF/MANIFEST.MF OK
>>>>>>>> testing: app/ OK
>>>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/ OK
>>>>>>>> testing: lib/activation-1.1.jar OK
>>>>>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>>>>>> testing: lib/antlr-2.7.7.jar OK
>>>>>>>> testing: lib/aopalliance-1.0.jar OK
>>>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>>>>>> testing: lib/btf-1.2.jar OK
>>>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>>>>>> testing: lib/commons-lang3-3.1.jar OK
>>>>>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>>>>>> testing: lib/curator-client-2.7.0.jar OK
>>>>>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>>>>>> testing: lib/dom4j-1.6.1.jar OK
>>>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>>>>>> testing: lib/fastutil-6.6.4.jar OK
>>>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>>>>>> testing: lib/guava-14.0.1.jar OK
>>>>>>>> testing: lib/hawtbuf-1.9.jar OK
>>>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>>>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>>>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>>>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>>>>>> testing: lib/jersey-client-1.9.jar OK
>>>>>>>> testing: lib/jersey-core-1.9.jar OK
>>>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jline-2.11.jar OK
>>>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>>>>>> testing: lib/joda-time-2.7.jar OK
>>>>>>>> testing: lib/json-patch-1.7.jar OK
>>>>>>>> testing: lib/jsr305-2.0.1.jar OK
>>>>>>>> testing: lib/jta-1.1.jar OK
>>>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>>>>>> testing: lib/log4j-1.2.17.jar OK
>>>>>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>>>>>> testing: lib/logback-core-1.1.5.jar OK
>>>>>>>> testing: lib/lz4-1.2.0.jar OK
>>>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>>>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>>>>>> testing: lib/msg-simple-1.1.jar OK
>>>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>>>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>>>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>>>>>> testing: lib/opencsv-2.3.jar OK
>>>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/super-csv-2.3.1.jar OK
>>>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>>>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>>>>>> testing: conf/ OK
>>>>>>>> testing: conf/properties.xml OK
>>>>>>>> No errors detected in compressed data of
>>>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>>>>
>>>>>>>> Contents of the properties.xml file under the conf folder is here:
>>>>>>>>
>>>>>>>> <?xml version="1.0"?>
>>>>>>>> <configuration>
>>>>>>>> <property>
>>>>>>>> <name>dt.application.CED.class</name>
>>>>>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>>>>> <description>An alias for the application</description>
>>>>>>>> </property>
>>>>>>>> <property>
>>>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>>>>> <value>512</value>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>> <property>
>>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>>
>>>>>>>> <property>
>>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>>>>> <value>20</value>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>> </configuration>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Ananth
>>>>>>>>
>>>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Ananth,
>>>>>>>>>
>>>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
>>>>>>>>> here?
>>>>>>>>> Make sure the jar containing the application classes should be in
>>>>>>>>> the app directory inside the apa.
>>>>>>>>>
>>>>>>>>> Can I know why the apex-app-archetype is not used in your apa
>>>>>>>>> project?
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello All,
>>>>>>>>>>
>>>>>>>>>> I am having the following issue with my application. I have a
>>>>>>>>>> maven project that has a class called XYZ that extends
>>>>>>>>>> StreamingApplication.
>>>>>>>>>>
>>>>>>>>>> My maven packaging is ensuring that the final "apa" extension
>>>>>>>>>> package has the right folder structure if exploded on a disk.
>>>>>>>>>>
>>>>>>>>>> However I am not able to locate the Application name and the
>>>>>>>>>> "launch" button next to it after uploading the "apa" package via the
>>>>>>>>>> browser.
>>>>>>>>>>
>>>>>>>>>> I tried looking at the docs to see how to specify a class as an
>>>>>>>>>> App that can be launched from a packaged application but could not find
>>>>>>>>>> much. Could anyone of you please point me to the right way of associating a
>>>>>>>>>> class as an application that will get enabled with a launch button in the
>>>>>>>>>> web ui ? The class XYZ has the annotation called @ApplicationAnnotation on
>>>>>>>>>> top of it.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Ananth
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by Sandeep Deshmukh <sa...@datatorrent.com>.
Apex needs Java 1.7 min.
What is the Java version you are using?
Regards,
Sandeep
On Fri, Apr 29, 2016 at 1:50 PM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> I had a look at the dtgateway.log and observed the following stacktrace :
> I guess the root cause might be Apex running on a lower version of java and
> the application code compiled with a higher version. I shall update
> everyone after the upgrade to see if it fixes the issue.
>
>
> Fatal error encountered
>>
>> 2016-04-29 08:04:45,181 ERROR
>> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
>> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
>> com/tx/y/z/Operator : Unsupported major.minor version 52.0
>> at java.lang.ClassLoader.defineClass1(Native Method)
>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
>> at
>> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
>> at
>> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
>> at
>> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
>> at
>> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
>> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>>
>
>
> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Thanks Shubham.
>>
>> Those properties were being set by the parent pom which are inherited by
>> the current pom.xml ( Kind of wanted all apex apps to have same version
>> structure etc) and hence added that property as part of parent pom.
>>
>> I am still stumped as to what is causing this.
>>
>> Regards,
>> Ananth
>>
>>
>>
>> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <sh...@datatorrent.com>
>> wrote:
>>
>>> Hi Ananth,
>>>
>>> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>> -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
>>> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
>>> -Dversion=1.0-SNAPSHOT
>>> and it works for me as well.
>>>
>>> I compared the pom.xml file and noticed values for following properties
>>> were missing.
>>> <manifestEntries>
>>>
>>> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
>>> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
>>>
>>> </manifestEntries>
>>>
>>> Could you add the following to pom.xml and try again.
>>>
>>> <properties>
>>> <!-- change this if you desire to use a different version of Apex Core -->
>>> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
>>> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
>>> </properties>
>>>
>>>
>>> Thanks,
>>> Shubham
>>>
>>> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello David,
>>>>
>>>> I reattempted the packaging with the archetype based approach. I guess
>>>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
>>>> pointing it out.
>>>>
>>>> The default package generated by the archetype command seems to be
>>>> working fine when uploading it but the moment I add in my code and
>>>> dependencies it goes back the issue I am seeing :
>>>>
>>>>
>>>> My app package looks like app.zip ( I removed all of the
>>>> application.class for brevity sake ) . Please note that I deleted all the
>>>> jars under /lib folder and the contents of the /app folder in the apa
>>>> pacakge.
>>>>
>>>> I have also attached a copy of my pom.xml file.
>>>>
>>>> Could you please advise if you something amiss right away in the
>>>> structure below ? It may be noted that I retained the classes of the
>>>> original simple project generated by the archetype and even they stopped
>>>> showing with the launch button the moment I added in all of the application
>>>> dependencies in maven.
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> I just tried with 3.3.0-incubating with the following and it's
>>>>> working:
>>>>>
>>>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>>>> -DarchetypeArtifactId=apex-app-archetype
>>>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
>>>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>>>>>
>>>>> Let me know whether this works for you or not.
>>>>>
>>>>> The system uses the MANIFEST.MF file to get the basic meta information
>>>>> on the app package and it looks at all the jars in the app directory and
>>>>> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>>>>> agundabattula@gmail.com> wrote:
>>>>>
>>>>>> Hello David,
>>>>>>
>>>>>> Regarding as to why I did not use the archetype, I was building the
>>>>>> app for 3.3.0 version and I was not able to specify an archetype with the
>>>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>>>>>> version 3.3.0 ? I was not sure if the archetype version should match the
>>>>>> exact apex engine version . My presumption was archetype does not matter if
>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>> project.
>>>>>>
>>>>>> The maven assembly plugin used the appPackage.xml and the related
>>>>>> constructs to build the exact file layout of the app as mentioned in the
>>>>>> output above.
>>>>>>
>>>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>>>>
>>>>>> This brings me to a question. How does the engine know which is the
>>>>>> main class that is to be used for launching the Streaming App. I do have a
>>>>>> class that extends the StreaminApplication that is annotated with
>>>>>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>>>>>> do not see a "launch" icon next to it. All I see is the top level packaged
>>>>>> name in the web console but nothing beneath it for me to trigger a launch.
>>>>>>
>>>>>> Regards,
>>>>>> Ananth
>>>>>>
>>>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ananth,
>>>>>>>
>>>>>>> The MANIFEST.MF file is important, and it's generated automatically
>>>>>>> if the archetype is used. For example, the MANFEST.MF file from the Pi demo
>>>>>>> in Malhar has this content:
>>>>>>>
>>>>>>> Manifest-Version: 1.0
>>>>>>> Archiver-Version: Plexus Archiver
>>>>>>> Built-By: david
>>>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>>>>>> Class-Path: lib/*.jar
>>>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>> Created-By: Apache Maven 3.3.9
>>>>>>> Build-Jdk: 1.8.0_92
>>>>>>> DT-App-Package-Description: Apex demo applications that calculate
>>>>>>> the
>>>>>>> value of Pi. This is a starting point to understand how Apex works.
>>>>>>> DT-App-Package-Group-Id: org.apache.apex
>>>>>>> DT-App-Package-Name: pi-demo
>>>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>>
>>>>>>> Is there a reason why the archetype is not used in your project?
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello David,
>>>>>>>>
>>>>>>>> Please find the unzip result at the end of this email. The jar is
>>>>>>>> in the app directory inside the apa.
>>>>>>>> As to why I did not use the archetype, I was building the app for
>>>>>>>> 3.3.0 version and I was not able to specify an archetype with the version
>>>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not matter if
>>>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>>>> project.
>>>>>>>>
>>>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
>>>>>>>> listing below ) . I also pasted the contents of the properties.xml file at
>>>>>>>> the end of this email for your reference.
>>>>>>>>
>>>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>>>>>> testing: META-INF/ OK
>>>>>>>> testing: META-INF/MANIFEST.MF OK
>>>>>>>> testing: app/ OK
>>>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/ OK
>>>>>>>> testing: lib/activation-1.1.jar OK
>>>>>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>>>>>> testing: lib/antlr-2.7.7.jar OK
>>>>>>>> testing: lib/aopalliance-1.0.jar OK
>>>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>>>>>> testing: lib/btf-1.2.jar OK
>>>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>>>>>> testing: lib/commons-lang3-3.1.jar OK
>>>>>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>>>>>> testing: lib/curator-client-2.7.0.jar OK
>>>>>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>>>>>> testing: lib/dom4j-1.6.1.jar OK
>>>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>>>>>> testing: lib/fastutil-6.6.4.jar OK
>>>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>>>>>> testing: lib/guava-14.0.1.jar OK
>>>>>>>> testing: lib/hawtbuf-1.9.jar OK
>>>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>>>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>>>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>>>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>>>>>> testing: lib/jersey-client-1.9.jar OK
>>>>>>>> testing: lib/jersey-core-1.9.jar OK
>>>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>>>>>> testing: lib/jline-2.11.jar OK
>>>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>>>>>> testing: lib/joda-time-2.7.jar OK
>>>>>>>> testing: lib/json-patch-1.7.jar OK
>>>>>>>> testing: lib/jsr305-2.0.1.jar OK
>>>>>>>> testing: lib/jta-1.1.jar OK
>>>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>>>>>> testing: lib/log4j-1.2.17.jar OK
>>>>>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>>>>>> testing: lib/logback-core-1.1.5.jar OK
>>>>>>>> testing: lib/lz4-1.2.0.jar OK
>>>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>>>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>>>>>> testing: lib/msg-simple-1.1.jar OK
>>>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>>>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>>>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>>>>>> testing: lib/opencsv-2.3.jar OK
>>>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>>>>>> testing: lib/super-csv-2.3.1.jar OK
>>>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>>>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>>>>>> testing: conf/ OK
>>>>>>>> testing: conf/properties.xml OK
>>>>>>>> No errors detected in compressed data of
>>>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>>>>
>>>>>>>> Contents of the properties.xml file under the conf folder is here:
>>>>>>>>
>>>>>>>> <?xml version="1.0"?>
>>>>>>>> <configuration>
>>>>>>>> <property>
>>>>>>>> <name>dt.application.CED.class</name>
>>>>>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>>>>> <description>An alias for the application</description>
>>>>>>>> </property>
>>>>>>>> <property>
>>>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>>>>> <value>512</value>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>> <property>
>>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>>
>>>>>>>> <property>
>>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>>>>> <value>20</value>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>> </configuration>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Ananth
>>>>>>>>
>>>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi Ananth,
>>>>>>>>>
>>>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
>>>>>>>>> here?
>>>>>>>>> Make sure the jar containing the application classes should be in
>>>>>>>>> the app directory inside the apa.
>>>>>>>>>
>>>>>>>>> Can I know why the apex-app-archetype is not used in your apa
>>>>>>>>> project?
>>>>>>>>>
>>>>>>>>> David
>>>>>>>>>
>>>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello All,
>>>>>>>>>>
>>>>>>>>>> I am having the following issue with my application. I have a
>>>>>>>>>> maven project that has a class called XYZ that extends
>>>>>>>>>> StreamingApplication.
>>>>>>>>>>
>>>>>>>>>> My maven packaging is ensuring that the final "apa" extension
>>>>>>>>>> package has the right folder structure if exploded on a disk.
>>>>>>>>>>
>>>>>>>>>> However I am not able to locate the Application name and the
>>>>>>>>>> "launch" button next to it after uploading the "apa" package via the
>>>>>>>>>> browser.
>>>>>>>>>>
>>>>>>>>>> I tried looking at the docs to see how to specify a class as an
>>>>>>>>>> App that can be launched from a packaged application but could not find
>>>>>>>>>> much. Could anyone of you please point me to the right way of associating a
>>>>>>>>>> class as an application that will get enabled with a launch button in the
>>>>>>>>>> web ui ? The class XYZ has the annotation called @ApplicationAnnotation on
>>>>>>>>>> top of it.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Ananth
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by Ananth Gundabattula <ag...@gmail.com>.
I had a look at the dtgateway.log and observed the following stacktrace : I
guess the root cause might be Apex running on a lower version of java and
the application code compiled with a higher version. I shall update
everyone after the upgrade to see if it fixes the issue.
Fatal error encountered
>
> 2016-04-29 08:04:45,181 ERROR
> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> com/tx/y/z/Operator : Unsupported major.minor version 52.0
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> at
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> at
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>
On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> Thanks Shubham.
>
> Those properties were being set by the parent pom which are inherited by
> the current pom.xml ( Kind of wanted all apex apps to have same version
> structure etc) and hence added that property as part of parent pom.
>
> I am still stumped as to what is causing this.
>
> Regards,
> Ananth
>
>
>
> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <sh...@datatorrent.com>
> wrote:
>
>> Hi Ananth,
>>
>> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>> -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
>> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
>> -Dversion=1.0-SNAPSHOT
>> and it works for me as well.
>>
>> I compared the pom.xml file and noticed values for following properties
>> were missing.
>> <manifestEntries>
>>
>> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
>> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
>>
>> </manifestEntries>
>>
>> Could you add the following to pom.xml and try again.
>>
>> <properties>
>> <!-- change this if you desire to use a different version of Apex Core -->
>> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
>> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
>> </properties>
>>
>>
>> Thanks,
>> Shubham
>>
>> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Hello David,
>>>
>>> I reattempted the packaging with the archetype based approach. I guess
>>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
>>> pointing it out.
>>>
>>> The default package generated by the archetype command seems to be
>>> working fine when uploading it but the moment I add in my code and
>>> dependencies it goes back the issue I am seeing :
>>>
>>>
>>> My app package looks like app.zip ( I removed all of the
>>> application.class for brevity sake ) . Please note that I deleted all the
>>> jars under /lib folder and the contents of the /app folder in the apa
>>> pacakge.
>>>
>>> I have also attached a copy of my pom.xml file.
>>>
>>> Could you please advise if you something amiss right away in the
>>> structure below ? It may be noted that I retained the classes of the
>>> original simple project generated by the archetype and even they stopped
>>> showing with the launch button the moment I added in all of the application
>>> dependencies in maven.
>>>
>>> Regards,
>>> Ananth
>>>
>>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
>>> wrote:
>>>
>>>> Hi Ananth,
>>>>
>>>> I just tried with 3.3.0-incubating with the following and it's working:
>>>>
>>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>>> -DarchetypeArtifactId=apex-app-archetype
>>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
>>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>>>>
>>>> Let me know whether this works for you or not.
>>>>
>>>> The system uses the MANIFEST.MF file to get the basic meta information
>>>> on the app package and it looks at all the jars in the app directory and
>>>> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>>>>
>>>> David
>>>>
>>>>
>>>>
>>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>>>> agundabattula@gmail.com> wrote:
>>>>
>>>>> Hello David,
>>>>>
>>>>> Regarding as to why I did not use the archetype, I was building the
>>>>> app for 3.3.0 version and I was not able to specify an archetype with the
>>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>>>>> version 3.3.0 ? I was not sure if the archetype version should match the
>>>>> exact apex engine version . My presumption was archetype does not matter if
>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>> project.
>>>>>
>>>>> The maven assembly plugin used the appPackage.xml and the related
>>>>> constructs to build the exact file layout of the app as mentioned in the
>>>>> output above.
>>>>>
>>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>>>
>>>>> This brings me to a question. How does the engine know which is the
>>>>> main class that is to be used for launching the Streaming App. I do have a
>>>>> class that extends the StreaminApplication that is annotated with
>>>>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>>>>> do not see a "launch" icon next to it. All I see is the top level packaged
>>>>> name in the web console but nothing beneath it for me to trigger a launch.
>>>>>
>>>>> Regards,
>>>>> Ananth
>>>>>
>>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ananth,
>>>>>>
>>>>>> The MANIFEST.MF file is important, and it's generated automatically
>>>>>> if the archetype is used. For example, the MANFEST.MF file from the Pi demo
>>>>>> in Malhar has this content:
>>>>>>
>>>>>> Manifest-Version: 1.0
>>>>>> Archiver-Version: Plexus Archiver
>>>>>> Built-By: david
>>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>>>>> Class-Path: lib/*.jar
>>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>>>> Created-By: Apache Maven 3.3.9
>>>>>> Build-Jdk: 1.8.0_92
>>>>>> DT-App-Package-Description: Apex demo applications that calculate the
>>>>>> value of Pi. This is a starting point to understand how Apex works.
>>>>>> DT-App-Package-Group-Id: org.apache.apex
>>>>>> DT-App-Package-Name: pi-demo
>>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>
>>>>>> Is there a reason why the archetype is not used in your project?
>>>>>>
>>>>>> David
>>>>>>
>>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>>>> agundabattula@gmail.com> wrote:
>>>>>>
>>>>>>> Hello David,
>>>>>>>
>>>>>>> Please find the unzip result at the end of this email. The jar is in
>>>>>>> the app directory inside the apa.
>>>>>>> As to why I did not use the archetype, I was building the app for
>>>>>>> 3.3.0 version and I was not able to specify an archetype with the version
>>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not matter if
>>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>>> project.
>>>>>>>
>>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
>>>>>>> listing below ) . I also pasted the contents of the properties.xml file at
>>>>>>> the end of this email for your reference.
>>>>>>>
>>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>>>>> testing: META-INF/ OK
>>>>>>> testing: META-INF/MANIFEST.MF OK
>>>>>>> testing: app/ OK
>>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/ OK
>>>>>>> testing: lib/activation-1.1.jar OK
>>>>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>>>>> testing: lib/antlr-2.7.7.jar OK
>>>>>>> testing: lib/aopalliance-1.0.jar OK
>>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>>>>> testing: lib/btf-1.2.jar OK
>>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>>>>> testing: lib/commons-lang3-3.1.jar OK
>>>>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>>>>> testing: lib/curator-client-2.7.0.jar OK
>>>>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>>>>> testing: lib/dom4j-1.6.1.jar OK
>>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>>>>> testing: lib/fastutil-6.6.4.jar OK
>>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>>>>> testing: lib/guava-14.0.1.jar OK
>>>>>>> testing: lib/hawtbuf-1.9.jar OK
>>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>>>>> testing: lib/jersey-client-1.9.jar OK
>>>>>>> testing: lib/jersey-core-1.9.jar OK
>>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jline-2.11.jar OK
>>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>>>>> testing: lib/joda-time-2.7.jar OK
>>>>>>> testing: lib/json-patch-1.7.jar OK
>>>>>>> testing: lib/jsr305-2.0.1.jar OK
>>>>>>> testing: lib/jta-1.1.jar OK
>>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>>>>> testing: lib/log4j-1.2.17.jar OK
>>>>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>>>>> testing: lib/logback-core-1.1.5.jar OK
>>>>>>> testing: lib/lz4-1.2.0.jar OK
>>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>>>>> testing: lib/msg-simple-1.1.jar OK
>>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>>>>> testing: lib/opencsv-2.3.jar OK
>>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/super-csv-2.3.1.jar OK
>>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>>>>> testing: conf/ OK
>>>>>>> testing: conf/properties.xml OK
>>>>>>> No errors detected in compressed data of
>>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>>>
>>>>>>> Contents of the properties.xml file under the conf folder is here:
>>>>>>>
>>>>>>> <?xml version="1.0"?>
>>>>>>> <configuration>
>>>>>>> <property>
>>>>>>> <name>dt.application.CED.class</name>
>>>>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>>>> <description>An alias for the application</description>
>>>>>>> </property>
>>>>>>> <property>
>>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>>>> <value>512</value>
>>>>>>> </property>
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>>>> </property>
>>>>>>>
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>>>> <value>20</value>
>>>>>>> </property>
>>>>>>>
>>>>>>> </configuration>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ananth
>>>>>>>
>>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Ananth,
>>>>>>>>
>>>>>>>> Can you do an "unzip -t" on your apa file and paste the output here?
>>>>>>>> Make sure the jar containing the application classes should be in
>>>>>>>> the app directory inside the apa.
>>>>>>>>
>>>>>>>> Can I know why the apex-app-archetype is not used in your apa
>>>>>>>> project?
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello All,
>>>>>>>>>
>>>>>>>>> I am having the following issue with my application. I have a
>>>>>>>>> maven project that has a class called XYZ that extends
>>>>>>>>> StreamingApplication.
>>>>>>>>>
>>>>>>>>> My maven packaging is ensuring that the final "apa" extension
>>>>>>>>> package has the right folder structure if exploded on a disk.
>>>>>>>>>
>>>>>>>>> However I am not able to locate the Application name and the
>>>>>>>>> "launch" button next to it after uploading the "apa" package via the
>>>>>>>>> browser.
>>>>>>>>>
>>>>>>>>> I tried looking at the docs to see how to specify a class as an
>>>>>>>>> App that can be launched from a packaged application but could not find
>>>>>>>>> much. Could anyone of you please point me to the right way of associating a
>>>>>>>>> class as an application that will get enabled with a launch button in the
>>>>>>>>> web ui ? The class XYZ has the annotation called @ApplicationAnnotation on
>>>>>>>>> top of it.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Ananth
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by Ananth Gundabattula <ag...@gmail.com>.
Thanks Shubham.
Those properties were being set by the parent pom which are inherited by
the current pom.xml ( Kind of wanted all apex apps to have same version
structure etc) and hence added that property as part of parent pom.
I am still stumped as to what is causing this.
Regards,
Ananth
On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <sh...@datatorrent.com>
wrote:
> Hi Ananth,
>
> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
> -Dversion=1.0-SNAPSHOT
> and it works for me as well.
>
> I compared the pom.xml file and noticed values for following properties
> were missing.
> <manifestEntries>
>
> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
>
> </manifestEntries>
>
> Could you add the following to pom.xml and try again.
>
> <properties>
> <!-- change this if you desire to use a different version of Apex Core -->
> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
> </properties>
>
>
> Thanks,
> Shubham
>
> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Hello David,
>>
>> I reattempted the packaging with the archetype based approach. I guess
>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
>> pointing it out.
>>
>> The default package generated by the archetype command seems to be
>> working fine when uploading it but the moment I add in my code and
>> dependencies it goes back the issue I am seeing :
>>
>>
>> My app package looks like app.zip ( I removed all of the
>> application.class for brevity sake ) . Please note that I deleted all the
>> jars under /lib folder and the contents of the /app folder in the apa
>> pacakge.
>>
>> I have also attached a copy of my pom.xml file.
>>
>> Could you please advise if you something amiss right away in the
>> structure below ? It may be noted that I retained the classes of the
>> original simple project generated by the archetype and even they stopped
>> showing with the launch button the moment I added in all of the application
>> dependencies in maven.
>>
>> Regards,
>> Ananth
>>
>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
>> wrote:
>>
>>> Hi Ananth,
>>>
>>> I just tried with 3.3.0-incubating with the following and it's working:
>>>
>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>> -DarchetypeArtifactId=apex-app-archetype
>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>>>
>>> Let me know whether this works for you or not.
>>>
>>> The system uses the MANIFEST.MF file to get the basic meta information
>>> on the app package and it looks at all the jars in the app directory and
>>> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>>>
>>> David
>>>
>>>
>>>
>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello David,
>>>>
>>>> Regarding as to why I did not use the archetype, I was building the app
>>>> for 3.3.0 version and I was not able to specify an archetype with the
>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>>>> version 3.3.0 ? I was not sure if the archetype version should match the
>>>> exact apex engine version . My presumption was archetype does not matter if
>>>> the packaging is done right. So i went ahead and created a simple maven
>>>> project.
>>>>
>>>> The maven assembly plugin used the appPackage.xml and the related
>>>> constructs to build the exact file layout of the app as mentioned in the
>>>> output above.
>>>>
>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>>
>>>> This brings me to a question. How does the engine know which is the
>>>> main class that is to be used for launching the Streaming App. I do have a
>>>> class that extends the StreaminApplication that is annotated with
>>>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>>>> do not see a "launch" icon next to it. All I see is the top level packaged
>>>> name in the web console but nothing beneath it for me to trigger a launch.
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> The MANIFEST.MF file is important, and it's generated automatically if
>>>>> the archetype is used. For example, the MANFEST.MF file from the Pi demo in
>>>>> Malhar has this content:
>>>>>
>>>>> Manifest-Version: 1.0
>>>>> Archiver-Version: Plexus Archiver
>>>>> Built-By: david
>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>>>> Class-Path: lib/*.jar
>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>>> Created-By: Apache Maven 3.3.9
>>>>> Build-Jdk: 1.8.0_92
>>>>> DT-App-Package-Description: Apex demo applications that calculate the
>>>>> value of Pi. This is a starting point to understand how Apex works.
>>>>> DT-App-Package-Group-Id: org.apache.apex
>>>>> DT-App-Package-Name: pi-demo
>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>>
>>>>> Is there a reason why the archetype is not used in your project?
>>>>>
>>>>> David
>>>>>
>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>>> agundabattula@gmail.com> wrote:
>>>>>
>>>>>> Hello David,
>>>>>>
>>>>>> Please find the unzip result at the end of this email. The jar is in
>>>>>> the app directory inside the apa.
>>>>>> As to why I did not use the archetype, I was building the app for
>>>>>> 3.3.0 version and I was not able to specify an archetype with the version
>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not matter if
>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>> project.
>>>>>>
>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the listing
>>>>>> below ) . I also pasted the contents of the properties.xml file at the end
>>>>>> of this email for your reference.
>>>>>>
>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>>>> testing: META-INF/ OK
>>>>>> testing: META-INF/MANIFEST.MF OK
>>>>>> testing: app/ OK
>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>>>> testing: lib/ OK
>>>>>> testing: lib/activation-1.1.jar OK
>>>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>>>> testing: lib/antlr-2.7.7.jar OK
>>>>>> testing: lib/aopalliance-1.0.jar OK
>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>>>> testing: lib/btf-1.2.jar OK
>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>>>> testing: lib/commons-lang3-3.1.jar OK
>>>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>>>> testing: lib/curator-client-2.7.0.jar OK
>>>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>>>> testing: lib/dom4j-1.6.1.jar OK
>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>>>> testing: lib/fastutil-6.6.4.jar OK
>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>>>> testing: lib/guava-14.0.1.jar OK
>>>>>> testing: lib/hawtbuf-1.9.jar OK
>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>>>> testing: lib/jersey-client-1.9.jar OK
>>>>>> testing: lib/jersey-core-1.9.jar OK
>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>>>> testing: lib/jline-2.11.jar OK
>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>>>> testing: lib/joda-time-2.7.jar OK
>>>>>> testing: lib/json-patch-1.7.jar OK
>>>>>> testing: lib/jsr305-2.0.1.jar OK
>>>>>> testing: lib/jta-1.1.jar OK
>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>>>> testing: lib/log4j-1.2.17.jar OK
>>>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>>>> testing: lib/logback-core-1.1.5.jar OK
>>>>>> testing: lib/lz4-1.2.0.jar OK
>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>>>> testing: lib/msg-simple-1.1.jar OK
>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>>>> testing: lib/opencsv-2.3.jar OK
>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>>>> testing: lib/super-csv-2.3.1.jar OK
>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>>>> testing: conf/ OK
>>>>>> testing: conf/properties.xml OK
>>>>>> No errors detected in compressed data of
>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>>
>>>>>> Contents of the properties.xml file under the conf folder is here:
>>>>>>
>>>>>> <?xml version="1.0"?>
>>>>>> <configuration>
>>>>>> <property>
>>>>>> <name>dt.application.CED.class</name>
>>>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>>> <description>An alias for the application</description>
>>>>>> </property>
>>>>>> <property>
>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>>> <value>512</value>
>>>>>> </property>
>>>>>>
>>>>>> <property>
>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>>> </property>
>>>>>>
>>>>>>
>>>>>> <property>
>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>>> <value>20</value>
>>>>>> </property>
>>>>>>
>>>>>> </configuration>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Regards,
>>>>>> Ananth
>>>>>>
>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Ananth,
>>>>>>>
>>>>>>> Can you do an "unzip -t" on your apa file and paste the output here?
>>>>>>> Make sure the jar containing the application classes should be in
>>>>>>> the app directory inside the apa.
>>>>>>>
>>>>>>> Can I know why the apex-app-archetype is not used in your apa
>>>>>>> project?
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hello All,
>>>>>>>>
>>>>>>>> I am having the following issue with my application. I have a maven
>>>>>>>> project that has a class called XYZ that extends StreamingApplication.
>>>>>>>>
>>>>>>>> My maven packaging is ensuring that the final "apa" extension
>>>>>>>> package has the right folder structure if exploded on a disk.
>>>>>>>>
>>>>>>>> However I am not able to locate the Application name and the
>>>>>>>> "launch" button next to it after uploading the "apa" package via the
>>>>>>>> browser.
>>>>>>>>
>>>>>>>> I tried looking at the docs to see how to specify a class as an App
>>>>>>>> that can be launched from a packaged application but could not find much.
>>>>>>>> Could anyone of you please point me to the right way of associating a class
>>>>>>>> as an application that will get enabled with a launch button in the web ui
>>>>>>>> ? The class XYZ has the annotation called @ApplicationAnnotation on top of
>>>>>>>> it.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Ananth
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by Shubham Pathak <sh...@datatorrent.com>.
Hi Ananth,
I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
-DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
-DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
-Dversion=1.0-SNAPSHOT
and it works for me as well.
I compared the pom.xml file and noticed values for following properties
were missing.
<manifestEntries>
<Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
<DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
</manifestEntries>
Could you add the following to pom.xml and try again.
<properties>
<!-- change this if you desire to use a different version of Apex Core -->
<apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
<apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
</properties>
Thanks,
Shubham
On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> Hello David,
>
> I reattempted the packaging with the archetype based approach. I guess the
> 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
> pointing it out.
>
> The default package generated by the archetype command seems to be working
> fine when uploading it but the moment I add in my code and dependencies it
> goes back the issue I am seeing :
>
>
> My app package looks like app.zip ( I removed all of the
> application.class for brevity sake ) . Please note that I deleted all the
> jars under /lib folder and the contents of the /app folder in the apa
> pacakge.
>
> I have also attached a copy of my pom.xml file.
>
> Could you please advise if you something amiss right away in the structure
> below ? It may be noted that I retained the classes of the original simple
> project generated by the archetype and even they stopped showing with the
> launch button the moment I added in all of the application dependencies in
> maven.
>
> Regards,
> Ananth
>
> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com> wrote:
>
>> Hi Ananth,
>>
>> I just tried with 3.3.0-incubating with the following and it's working:
>>
>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>> -DarchetypeArtifactId=apex-app-archetype
>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>>
>> Let me know whether this works for you or not.
>>
>> The system uses the MANIFEST.MF file to get the basic meta information on
>> the app package and it looks at all the jars in the app directory and
>> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>>
>> David
>>
>>
>>
>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Hello David,
>>>
>>> Regarding as to why I did not use the archetype, I was building the app
>>> for 3.3.0 version and I was not able to specify an archetype with the
>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>>> version 3.3.0 ? I was not sure if the archetype version should match the
>>> exact apex engine version . My presumption was archetype does not matter if
>>> the packaging is done right. So i went ahead and created a simple maven
>>> project.
>>>
>>> The maven assembly plugin used the appPackage.xml and the related
>>> constructs to build the exact file layout of the app as mentioned in the
>>> output above.
>>>
>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>
>>> This brings me to a question. How does the engine know which is the main
>>> class that is to be used for launching the Streaming App. I do have a class
>>> that extends the StreaminApplication that is annotated with
>>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>>> do not see a "launch" icon next to it. All I see is the top level packaged
>>> name in the web console but nothing beneath it for me to trigger a launch.
>>>
>>> Regards,
>>> Ananth
>>>
>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
>>> wrote:
>>>
>>>> Hi Ananth,
>>>>
>>>> The MANIFEST.MF file is important, and it's generated automatically if
>>>> the archetype is used. For example, the MANFEST.MF file from the Pi demo in
>>>> Malhar has this content:
>>>>
>>>> Manifest-Version: 1.0
>>>> Archiver-Version: Plexus Archiver
>>>> Built-By: david
>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>>> Class-Path: lib/*.jar
>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>> Created-By: Apache Maven 3.3.9
>>>> Build-Jdk: 1.8.0_92
>>>> DT-App-Package-Description: Apex demo applications that calculate the
>>>> value of Pi. This is a starting point to understand how Apex works.
>>>> DT-App-Package-Group-Id: org.apache.apex
>>>> DT-App-Package-Name: pi-demo
>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>
>>>> Is there a reason why the archetype is not used in your project?
>>>>
>>>> David
>>>>
>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>> agundabattula@gmail.com> wrote:
>>>>
>>>>> Hello David,
>>>>>
>>>>> Please find the unzip result at the end of this email. The jar is in
>>>>> the app directory inside the apa.
>>>>> As to why I did not use the archetype, I was building the app for
>>>>> 3.3.0 version and I was not able to specify an archetype with the version
>>>>> 3.3.0 when I attempted it. My presumption was archetype does not matter if
>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>> project.
>>>>>
>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the listing
>>>>> below ) . I also pasted the contents of the properties.xml file at the end
>>>>> of this email for your reference.
>>>>>
>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>>> testing: META-INF/ OK
>>>>> testing: META-INF/MANIFEST.MF OK
>>>>> testing: app/ OK
>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>>> testing: lib/ OK
>>>>> testing: lib/activation-1.1.jar OK
>>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>>> testing: lib/antlr-2.7.7.jar OK
>>>>> testing: lib/aopalliance-1.0.jar OK
>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>>> testing: lib/btf-1.2.jar OK
>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>>> testing: lib/commons-lang3-3.1.jar OK
>>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>>> testing: lib/curator-client-2.7.0.jar OK
>>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>>> testing: lib/dom4j-1.6.1.jar OK
>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>>> testing: lib/fastutil-6.6.4.jar OK
>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>>> testing: lib/guava-14.0.1.jar OK
>>>>> testing: lib/hawtbuf-1.9.jar OK
>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>>> testing: lib/jersey-client-1.9.jar OK
>>>>> testing: lib/jersey-core-1.9.jar OK
>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>>> testing: lib/jline-2.11.jar OK
>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>>> testing: lib/joda-time-2.7.jar OK
>>>>> testing: lib/json-patch-1.7.jar OK
>>>>> testing: lib/jsr305-2.0.1.jar OK
>>>>> testing: lib/jta-1.1.jar OK
>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>>> testing: lib/log4j-1.2.17.jar OK
>>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>>> testing: lib/logback-core-1.1.5.jar OK
>>>>> testing: lib/lz4-1.2.0.jar OK
>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>>> testing: lib/msg-simple-1.1.jar OK
>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>>> testing: lib/opencsv-2.3.jar OK
>>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>>> testing: lib/super-csv-2.3.1.jar OK
>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>>> testing: conf/ OK
>>>>> testing: conf/properties.xml OK
>>>>> No errors detected in compressed data of
>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>
>>>>> Contents of the properties.xml file under the conf folder is here:
>>>>>
>>>>> <?xml version="1.0"?>
>>>>> <configuration>
>>>>> <property>
>>>>> <name>dt.application.CED.class</name>
>>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>> <description>An alias for the application</description>
>>>>> </property>
>>>>> <property>
>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>> <value>512</value>
>>>>> </property>
>>>>>
>>>>> <property>
>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>> </property>
>>>>>
>>>>>
>>>>> <property>
>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>> <value>20</value>
>>>>> </property>
>>>>>
>>>>> </configuration>
>>>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>> Ananth
>>>>>
>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ananth,
>>>>>>
>>>>>> Can you do an "unzip -t" on your apa file and paste the output here?
>>>>>> Make sure the jar containing the application classes should be in the
>>>>>> app directory inside the apa.
>>>>>>
>>>>>> Can I know why the apex-app-archetype is not used in your apa project?
>>>>>>
>>>>>> David
>>>>>>
>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>>> agundabattula@gmail.com> wrote:
>>>>>>
>>>>>>> Hello All,
>>>>>>>
>>>>>>> I am having the following issue with my application. I have a maven
>>>>>>> project that has a class called XYZ that extends StreamingApplication.
>>>>>>>
>>>>>>> My maven packaging is ensuring that the final "apa" extension
>>>>>>> package has the right folder structure if exploded on a disk.
>>>>>>>
>>>>>>> However I am not able to locate the Application name and the
>>>>>>> "launch" button next to it after uploading the "apa" package via the
>>>>>>> browser.
>>>>>>>
>>>>>>> I tried looking at the docs to see how to specify a class as an App
>>>>>>> that can be launched from a packaged application but could not find much.
>>>>>>> Could anyone of you please point me to the right way of associating a class
>>>>>>> as an application that will get enabled with a launch button in the web ui
>>>>>>> ? The class XYZ has the annotation called @ApplicationAnnotation on top of
>>>>>>> it.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ananth
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by Ananth Gundabattula <ag...@gmail.com>.
Hello David,
I reattempted the packaging with the archetype based approach. I guess the
3.3.0 archetype did not exist when I attempted it earlier. Thanks for
pointing it out.
The default package generated by the archetype command seems to be working
fine when uploading it but the moment I add in my code and dependencies it
goes back the issue I am seeing :
My app package looks like app.zip ( I removed all of the application.class
for brevity sake ) . Please note that I deleted all the jars under /lib
folder and the contents of the /app folder in the apa pacakge.
I have also attached a copy of my pom.xml file.
Could you please advise if you something amiss right away in the structure
below ? It may be noted that I retained the classes of the original simple
project generated by the archetype and even they stopped showing with the
launch button the moment I added in all of the application dependencies in
maven.
Regards,
Ananth
On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com> wrote:
> Hi Ananth,
>
> I just tried with 3.3.0-incubating with the following and it's working:
>
> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> -DarchetypeArtifactId=apex-app-archetype
> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>
> Let me know whether this works for you or not.
>
> The system uses the MANIFEST.MF file to get the basic meta information on
> the app package and it looks at all the jars in the app directory and
> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>
> David
>
>
>
> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Hello David,
>>
>> Regarding as to why I did not use the archetype, I was building the app
>> for 3.3.0 version and I was not able to specify an archetype with the
>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>> version 3.3.0 ? I was not sure if the archetype version should match the
>> exact apex engine version . My presumption was archetype does not matter if
>> the packaging is done right. So i went ahead and created a simple maven
>> project.
>>
>> The maven assembly plugin used the appPackage.xml and the related
>> constructs to build the exact file layout of the app as mentioned in the
>> output above.
>>
>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>
>> This brings me to a question. How does the engine know which is the main
>> class that is to be used for launching the Streaming App. I do have a class
>> that extends the StreaminApplication that is annotated with
>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>> do not see a "launch" icon next to it. All I see is the top level packaged
>> name in the web console but nothing beneath it for me to trigger a launch.
>>
>> Regards,
>> Ananth
>>
>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com> wrote:
>>
>>> Hi Ananth,
>>>
>>> The MANIFEST.MF file is important, and it's generated automatically if
>>> the archetype is used. For example, the MANFEST.MF file from the Pi demo in
>>> Malhar has this content:
>>>
>>> Manifest-Version: 1.0
>>> Archiver-Version: Plexus Archiver
>>> Built-By: david
>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>> Class-Path: lib/*.jar
>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>> Created-By: Apache Maven 3.3.9
>>> Build-Jdk: 1.8.0_92
>>> DT-App-Package-Description: Apex demo applications that calculate the
>>> value of Pi. This is a starting point to understand how Apex works.
>>> DT-App-Package-Group-Id: org.apache.apex
>>> DT-App-Package-Name: pi-demo
>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>
>>> Is there a reason why the archetype is not used in your project?
>>>
>>> David
>>>
>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello David,
>>>>
>>>> Please find the unzip result at the end of this email. The jar is in
>>>> the app directory inside the apa.
>>>> As to why I did not use the archetype, I was building the app for 3.3.0
>>>> version and I was not able to specify an archetype with the version 3.3.0
>>>> when I attempted it. My presumption was archetype does not matter if the
>>>> packaging is done right. So i went ahead and created a simple maven
>>>> project.
>>>>
>>>> Here is the output: ( The MANIFEST.MF is an empty file in the listing
>>>> below ) . I also pasted the contents of the properties.xml file at the end
>>>> of this email for your reference.
>>>>
>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>> testing: META-INF/ OK
>>>> testing: META-INF/MANIFEST.MF OK
>>>> testing: app/ OK
>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>> testing: lib/ OK
>>>> testing: lib/activation-1.1.jar OK
>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>> testing: lib/antlr-2.7.7.jar OK
>>>> testing: lib/aopalliance-1.0.jar OK
>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>> testing: lib/btf-1.2.jar OK
>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>> testing: lib/commons-lang3-3.1.jar OK
>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>> testing: lib/curator-client-2.7.0.jar OK
>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>> testing: lib/dom4j-1.6.1.jar OK
>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>> testing: lib/fastutil-6.6.4.jar OK
>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>> testing: lib/guava-14.0.1.jar OK
>>>> testing: lib/hawtbuf-1.9.jar OK
>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>> testing: lib/jersey-client-1.9.jar OK
>>>> testing: lib/jersey-core-1.9.jar OK
>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>> testing: lib/jline-2.11.jar OK
>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>> testing: lib/joda-time-2.7.jar OK
>>>> testing: lib/json-patch-1.7.jar OK
>>>> testing: lib/jsr305-2.0.1.jar OK
>>>> testing: lib/jta-1.1.jar OK
>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>> testing: lib/log4j-1.2.17.jar OK
>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>> testing: lib/logback-core-1.1.5.jar OK
>>>> testing: lib/lz4-1.2.0.jar OK
>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>> testing: lib/msg-simple-1.1.jar OK
>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>> testing: lib/opencsv-2.3.jar OK
>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>> testing: lib/super-csv-2.3.1.jar OK
>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>> testing: conf/ OK
>>>> testing: conf/properties.xml OK
>>>> No errors detected in compressed data of
>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>
>>>> Contents of the properties.xml file under the conf folder is here:
>>>>
>>>> <?xml version="1.0"?>
>>>> <configuration>
>>>> <property>
>>>> <name>dt.application.CED.class</name>
>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>> <description>An alias for the application</description>
>>>> </property>
>>>> <property>
>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>> <value>512</value>
>>>> </property>
>>>>
>>>> <property>
>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>> </property>
>>>>
>>>>
>>>> <property>
>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>> <value>20</value>
>>>> </property>
>>>>
>>>> </configuration>
>>>>
>>>>
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Hi Ananth,
>>>>>
>>>>> Can you do an "unzip -t" on your apa file and paste the output here?
>>>>> Make sure the jar containing the application classes should be in the
>>>>> app directory inside the apa.
>>>>>
>>>>> Can I know why the apex-app-archetype is not used in your apa project?
>>>>>
>>>>> David
>>>>>
>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>> agundabattula@gmail.com> wrote:
>>>>>
>>>>>> Hello All,
>>>>>>
>>>>>> I am having the following issue with my application. I have a maven
>>>>>> project that has a class called XYZ that extends StreamingApplication.
>>>>>>
>>>>>> My maven packaging is ensuring that the final "apa" extension package
>>>>>> has the right folder structure if exploded on a disk.
>>>>>>
>>>>>> However I am not able to locate the Application name and the "launch"
>>>>>> button next to it after uploading the "apa" package via the browser.
>>>>>>
>>>>>> I tried looking at the docs to see how to specify a class as an App
>>>>>> that can be launched from a packaged application but could not find much.
>>>>>> Could anyone of you please point me to the right way of associating a class
>>>>>> as an application that will get enabled with a launch button in the web ui
>>>>>> ? The class XYZ has the annotation called @ApplicationAnnotation on top of
>>>>>> it.
>>>>>>
>>>>>> Regards,
>>>>>> Ananth
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by David Yan <da...@datatorrent.com>.
Hi Ananth,
I just tried with 3.3.0-incubating with the following and it's working:
$ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
-DarchetypeArtifactId=apex-app-archetype
-DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
-Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
Let me know whether this works for you or not.
The system uses the MANIFEST.MF file to get the basic meta information on
the app package and it looks at all the jars in the app directory and
searches for classes that subclass com.datatorrent.api.StreamingApplication.
David
On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> Hello David,
>
> Regarding as to why I did not use the archetype, I was building the app
> for 3.3.0 version and I was not able to specify an archetype with the
> version 3.3.0 when I attempted it. Perhaps there is no archetype for
> version 3.3.0 ? I was not sure if the archetype version should match the
> exact apex engine version . My presumption was archetype does not matter if
> the packaging is done right. So i went ahead and created a simple maven
> project.
>
> The maven assembly plugin used the appPackage.xml and the related
> constructs to build the exact file layout of the app as mentioned in the
> output above.
>
> I even attempted this time with MANIFEST.MF entry but to no avail.
>
> This brings me to a question. How does the engine know which is the main
> class that is to be used for launching the Streaming App. I do have a class
> that extends the StreaminApplication that is annotated with
> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
> do not see a "launch" icon next to it. All I see is the top level packaged
> name in the web console but nothing beneath it for me to trigger a launch.
>
> Regards,
> Ananth
>
> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com> wrote:
>
>> Hi Ananth,
>>
>> The MANIFEST.MF file is important, and it's generated automatically if
>> the archetype is used. For example, the MANFEST.MF file from the Pi demo in
>> Malhar has this content:
>>
>> Manifest-Version: 1.0
>> Archiver-Version: Plexus Archiver
>> Built-By: david
>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>> Class-Path: lib/*.jar
>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>> Created-By: Apache Maven 3.3.9
>> Build-Jdk: 1.8.0_92
>> DT-App-Package-Description: Apex demo applications that calculate the
>> value of Pi. This is a starting point to understand how Apex works.
>> DT-App-Package-Group-Id: org.apache.apex
>> DT-App-Package-Name: pi-demo
>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>
>> Is there a reason why the archetype is not used in your project?
>>
>> David
>>
>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Hello David,
>>>
>>> Please find the unzip result at the end of this email. The jar is in the
>>> app directory inside the apa.
>>> As to why I did not use the archetype, I was building the app for 3.3.0
>>> version and I was not able to specify an archetype with the version 3.3.0
>>> when I attempted it. My presumption was archetype does not matter if the
>>> packaging is done right. So i went ahead and created a simple maven
>>> project.
>>>
>>> Here is the output: ( The MANIFEST.MF is an empty file in the listing
>>> below ) . I also pasted the contents of the properties.xml file at the end
>>> of this email for your reference.
>>>
>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>> testing: META-INF/ OK
>>> testing: META-INF/MANIFEST.MF OK
>>> testing: app/ OK
>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>> testing: lib/ OK
>>> testing: lib/activation-1.1.jar OK
>>> testing: lib/activemq-client-5.8.0.jar OK
>>> testing: lib/antlr-2.7.7.jar OK
>>> testing: lib/aopalliance-1.0.jar OK
>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>> testing: lib/aspectjrt-1.8.7.jar OK
>>> testing: lib/btf-1.2.jar OK
>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>> testing: lib/commons-compiler-2.7.8.jar OK
>>> testing: lib/commons-lang3-3.1.jar OK
>>> testing: lib/commons-logging-1.1.3.jar OK
>>> testing: lib/curator-client-2.7.0.jar OK
>>> testing: lib/curator-framework-2.7.0.jar OK
>>> testing: lib/dom4j-1.6.1.jar OK
>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>> testing: lib/evo-inflector-1.2.1.jar OK
>>> testing: lib/fastutil-6.6.4.jar OK
>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>> testing: lib/guava-14.0.1.jar OK
>>> testing: lib/hawtbuf-1.9.jar OK
>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>> testing: lib/HikariCP-2.4.3.jar OK
>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>> testing: lib/jackson-core-2.6.4.jar OK
>>> testing: lib/jackson-coreutils-1.6.jar OK
>>> testing: lib/jackson-databind-2.6.4.jar OK
>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>> testing: lib/javassist-3.18.1-GA.jar OK
>>> testing: lib/javax.mail-1.5.0.jar OK
>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>> testing: lib/jersey-client-1.9.jar OK
>>> testing: lib/jersey-core-1.9.jar OK
>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>> testing: lib/jline-2.11.jar OK
>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>> testing: lib/joda-time-2.7.jar OK
>>> testing: lib/json-patch-1.7.jar OK
>>> testing: lib/jsr305-2.0.1.jar OK
>>> testing: lib/jta-1.1.jar OK
>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>> testing: lib/log4j-1.2.17.jar OK
>>> testing: lib/logback-classic-1.1.5.jar OK
>>> testing: lib/logback-core-1.1.5.jar OK
>>> testing: lib/lz4-1.2.0.jar OK
>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>> testing: lib/metrics-core-3.0.2.jar OK
>>> testing: lib/msg-simple-1.1.jar OK
>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>> testing: lib/named-regexp-0.2.3.jar OK
>>> testing: lib/netty-3.6.2.Final.jar OK
>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>> testing: lib/opencsv-2.3.jar OK
>>> testing: lib/slf4j-api-1.7.10.jar OK
>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>> testing: lib/super-csv-2.3.1.jar OK
>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>> testing: lib/xml-apis-1.0.b2.jar OK
>>> testing: lib/zookeeper-3.4.6.jar OK
>>> testing: conf/ OK
>>> testing: conf/properties.xml OK
>>> No errors detected in compressed data of
>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>
>>> Contents of the properties.xml file under the conf folder is here:
>>>
>>> <?xml version="1.0"?>
>>> <configuration>
>>> <property>
>>> <name>dt.application.CED.class</name>
>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>> <description>An alias for the application</description>
>>> </property>
>>> <property>
>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>> <value>512</value>
>>> </property>
>>>
>>> <property>
>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>> </property>
>>>
>>>
>>> <property>
>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>> <value>20</value>
>>> </property>
>>>
>>> </configuration>
>>>
>>>
>>>
>>> Regards,
>>> Ananth
>>>
>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>> wrote:
>>>
>>>> Hi Ananth,
>>>>
>>>> Can you do an "unzip -t" on your apa file and paste the output here?
>>>> Make sure the jar containing the application classes should be in the
>>>> app directory inside the apa.
>>>>
>>>> Can I know why the apex-app-archetype is not used in your apa project?
>>>>
>>>> David
>>>>
>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>> agundabattula@gmail.com> wrote:
>>>>
>>>>> Hello All,
>>>>>
>>>>> I am having the following issue with my application. I have a maven
>>>>> project that has a class called XYZ that extends StreamingApplication.
>>>>>
>>>>> My maven packaging is ensuring that the final "apa" extension package
>>>>> has the right folder structure if exploded on a disk.
>>>>>
>>>>> However I am not able to locate the Application name and the "launch"
>>>>> button next to it after uploading the "apa" package via the browser.
>>>>>
>>>>> I tried looking at the docs to see how to specify a class as an App
>>>>> that can be launched from a packaged application but could not find much.
>>>>> Could anyone of you please point me to the right way of associating a class
>>>>> as an application that will get enabled with a launch button in the web ui
>>>>> ? The class XYZ has the annotation called @ApplicationAnnotation on top of
>>>>> it.
>>>>>
>>>>> Regards,
>>>>> Ananth
>>>>>
>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by Ananth Gundabattula <ag...@gmail.com>.
Hello David,
Regarding as to why I did not use the archetype, I was building the app for
3.3.0 version and I was not able to specify an archetype with the version
3.3.0 when I attempted it. Perhaps there is no archetype for version 3.3.0
? I was not sure if the archetype version should match the exact apex
engine version . My presumption was archetype does not matter if the
packaging is done right. So i went ahead and created a simple maven
project.
The maven assembly plugin used the appPackage.xml and the related
constructs to build the exact file layout of the app as mentioned in the
output above.
I even attempted this time with MANIFEST.MF entry but to no avail.
This brings me to a question. How does the engine know which is the main
class that is to be used for launching the Streaming App. I do have a class
that extends the StreaminApplication that is annotated with
ApplicationAnnotation but does not seem to be picking up. By this I mean, I
do not see a "launch" icon next to it. All I see is the top level packaged
name in the web console but nothing beneath it for me to trigger a launch.
Regards,
Ananth
On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com> wrote:
> Hi Ananth,
>
> The MANIFEST.MF file is important, and it's generated automatically if the
> archetype is used. For example, the MANFEST.MF file from the Pi demo in
> Malhar has this content:
>
> Manifest-Version: 1.0
> Archiver-Version: Plexus Archiver
> Built-By: david
> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
> Class-Path: lib/*.jar
> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
> Created-By: Apache Maven 3.3.9
> Build-Jdk: 1.8.0_92
> DT-App-Package-Description: Apex demo applications that calculate the
> value of Pi. This is a starting point to understand how Apex works.
> DT-App-Package-Group-Id: org.apache.apex
> DT-App-Package-Name: pi-demo
> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>
> Is there a reason why the archetype is not used in your project?
>
> David
>
> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Hello David,
>>
>> Please find the unzip result at the end of this email. The jar is in the
>> app directory inside the apa.
>> As to why I did not use the archetype, I was building the app for 3.3.0
>> version and I was not able to specify an archetype with the version 3.3.0
>> when I attempted it. My presumption was archetype does not matter if the
>> packaging is done right. So i went ahead and created a simple maven
>> project.
>>
>> Here is the output: ( The MANIFEST.MF is an empty file in the listing
>> below ) . I also pasted the contents of the properties.xml file at the end
>> of this email for your reference.
>>
>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>> testing: META-INF/ OK
>> testing: META-INF/MANIFEST.MF OK
>> testing: app/ OK
>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>> testing: lib/ OK
>> testing: lib/activation-1.1.jar OK
>> testing: lib/activemq-client-5.8.0.jar OK
>> testing: lib/antlr-2.7.7.jar OK
>> testing: lib/aopalliance-1.0.jar OK
>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>> testing: lib/aspectjrt-1.8.7.jar OK
>> testing: lib/btf-1.2.jar OK
>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>> testing: lib/commons-beanutils-1.8.3.jar OK
>> testing: lib/commons-compiler-2.7.8.jar OK
>> testing: lib/commons-lang3-3.1.jar OK
>> testing: lib/commons-logging-1.1.3.jar OK
>> testing: lib/curator-client-2.7.0.jar OK
>> testing: lib/curator-framework-2.7.0.jar OK
>> testing: lib/dom4j-1.6.1.jar OK
>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>> testing: lib/evo-inflector-1.2.1.jar OK
>> testing: lib/fastutil-6.6.4.jar OK
>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>> testing: lib/guava-14.0.1.jar OK
>> testing: lib/hawtbuf-1.9.jar OK
>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>> testing: lib/HikariCP-2.4.3.jar OK
>> testing: lib/jackson-annotations-2.6.4.jar OK
>> testing: lib/jackson-core-2.6.4.jar OK
>> testing: lib/jackson-coreutils-1.6.jar OK
>> testing: lib/jackson-databind-2.6.4.jar OK
>> testing: lib/jandex-2.0.0.CR1.jar OK
>> testing: lib/javassist-3.18.1-GA.jar OK
>> testing: lib/javax.mail-1.5.0.jar OK
>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>> testing: lib/jersey-client-1.9.jar OK
>> testing: lib/jersey-core-1.9.jar OK
>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>> testing: lib/jline-2.11.jar OK
>> testing: lib/jms-api-1.1-rev-1.jar OK
>> testing: lib/joda-time-2.7.jar OK
>> testing: lib/json-patch-1.7.jar OK
>> testing: lib/jsr305-2.0.1.jar OK
>> testing: lib/jta-1.1.jar OK
>> testing: lib/kafka-clients-0.9.0.0.jar OK
>> testing: lib/log4j-1.2.17.jar OK
>> testing: lib/logback-classic-1.1.5.jar OK
>> testing: lib/logback-core-1.1.5.jar OK
>> testing: lib/lz4-1.2.0.jar OK
>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>> testing: lib/metrics-core-3.0.2.jar OK
>> testing: lib/msg-simple-1.1.jar OK
>> testing: lib/mysql-connector-java-5.1.36.jar OK
>> testing: lib/named-regexp-0.2.3.jar OK
>> testing: lib/netty-3.6.2.Final.jar OK
>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>> testing: lib/netty-codec-4.0.27.Final.jar OK
>> testing: lib/netty-common-4.0.27.Final.jar OK
>> testing: lib/netty-handler-4.0.27.Final.jar OK
>> testing: lib/netty-transport-4.0.27.Final.jar OK
>> testing: lib/opencsv-2.3.jar OK
>> testing: lib/slf4j-api-1.7.10.jar OK
>> testing: lib/snappy-java-1.1.1.7.jar OK
>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>> testing: lib/super-csv-2.3.1.jar OK
>> testing: lib/super-csv-joda-2.3.1.jar OK
>> testing: lib/xml-apis-1.0.b2.jar OK
>> testing: lib/zookeeper-3.4.6.jar OK
>> testing: conf/ OK
>> testing: conf/properties.xml OK
>> No errors detected in compressed data of
>> cassandra-events-1.2-SNAPSHOT.apa.
>>
>> Contents of the properties.xml file under the conf folder is here:
>>
>> <?xml version="1.0"?>
>> <configuration>
>> <property>
>> <name>dt.application.CED.class</name>
>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>> <description>An alias for the application</description>
>> </property>
>> <property>
>> <name>dt.operator.*.attr.MEMORY_MB</name>
>> <value>512</value>
>> </property>
>>
>> <property>
>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>> </property>
>>
>>
>> <property>
>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>> <value>20</value>
>> </property>
>>
>> </configuration>
>>
>>
>>
>> Regards,
>> Ananth
>>
>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com> wrote:
>>
>>> Hi Ananth,
>>>
>>> Can you do an "unzip -t" on your apa file and paste the output here?
>>> Make sure the jar containing the application classes should be in the
>>> app directory inside the apa.
>>>
>>> Can I know why the apex-app-archetype is not used in your apa project?
>>>
>>> David
>>>
>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>> agundabattula@gmail.com> wrote:
>>>
>>>> Hello All,
>>>>
>>>> I am having the following issue with my application. I have a maven
>>>> project that has a class called XYZ that extends StreamingApplication.
>>>>
>>>> My maven packaging is ensuring that the final "apa" extension package
>>>> has the right folder structure if exploded on a disk.
>>>>
>>>> However I am not able to locate the Application name and the "launch"
>>>> button next to it after uploading the "apa" package via the browser.
>>>>
>>>> I tried looking at the docs to see how to specify a class as an App
>>>> that can be launched from a packaged application but could not find much.
>>>> Could anyone of you please point me to the right way of associating a class
>>>> as an application that will get enabled with a launch button in the web ui
>>>> ? The class XYZ has the annotation called @ApplicationAnnotation on top of
>>>> it.
>>>>
>>>> Regards,
>>>> Ananth
>>>>
>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by David Yan <da...@datatorrent.com>.
Hi Ananth,
The MANIFEST.MF file is important, and it's generated automatically if the
archetype is used. For example, the MANFEST.MF file from the Pi demo in
Malhar has this content:
Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Built-By: david
DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
Class-Path: lib/*.jar
DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
Created-By: Apache Maven 3.3.9
Build-Jdk: 1.8.0_92
DT-App-Package-Description: Apex demo applications that calculate the
value of Pi. This is a starting point to understand how Apex works.
DT-App-Package-Group-Id: org.apache.apex
DT-App-Package-Name: pi-demo
DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
Is there a reason why the archetype is not used in your project?
David
On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> Hello David,
>
> Please find the unzip result at the end of this email. The jar is in the
> app directory inside the apa.
> As to why I did not use the archetype, I was building the app for 3.3.0
> version and I was not able to specify an archetype with the version 3.3.0
> when I attempted it. My presumption was archetype does not matter if the
> packaging is done right. So i went ahead and created a simple maven
> project.
>
> Here is the output: ( The MANIFEST.MF is an empty file in the listing
> below ) . I also pasted the contents of the properties.xml file at the end
> of this email for your reference.
>
> Archive: cassandra-events-1.2-SNAPSHOT.apa
> testing: META-INF/ OK
> testing: META-INF/MANIFEST.MF OK
> testing: app/ OK
> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
> testing: lib/ OK
> testing: lib/activation-1.1.jar OK
> testing: lib/activemq-client-5.8.0.jar OK
> testing: lib/antlr-2.7.7.jar OK
> testing: lib/aopalliance-1.0.jar OK
> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
> testing: lib/aspectjrt-1.8.7.jar OK
> testing: lib/btf-1.2.jar OK
> testing: lib/cassandra-driver-core-2.1.8.jar OK
> testing: lib/commons-beanutils-1.8.3.jar OK
> testing: lib/commons-compiler-2.7.8.jar OK
> testing: lib/commons-lang3-3.1.jar OK
> testing: lib/commons-logging-1.1.3.jar OK
> testing: lib/curator-client-2.7.0.jar OK
> testing: lib/curator-framework-2.7.0.jar OK
> testing: lib/dom4j-1.6.1.jar OK
> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
> testing: lib/evo-inflector-1.2.1.jar OK
> testing: lib/fastutil-6.6.4.jar OK
> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
> testing: lib/guava-14.0.1.jar OK
> testing: lib/hawtbuf-1.9.jar OK
> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
> testing: lib/hibernate-core-5.0.4.Final.jar OK
> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
> testing: lib/HikariCP-2.4.3.jar OK
> testing: lib/jackson-annotations-2.6.4.jar OK
> testing: lib/jackson-core-2.6.4.jar OK
> testing: lib/jackson-coreutils-1.6.jar OK
> testing: lib/jackson-databind-2.6.4.jar OK
> testing: lib/jandex-2.0.0.CR1.jar OK
> testing: lib/javassist-3.18.1-GA.jar OK
> testing: lib/javax.mail-1.5.0.jar OK
> testing: lib/jboss-logging-3.3.0.Final.jar OK
> testing: lib/jcl-over-slf4j-1.7.13.jar OK
> testing: lib/jersey-client-1.9.jar OK
> testing: lib/jersey-core-1.9.jar OK
> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
> testing: lib/jetty-http-8.1.10.v20130312.jar OK
> testing: lib/jetty-io-8.1.10.v20130312.jar OK
> testing: lib/jetty-security-8.1.10.v20130312.jar OK
> testing: lib/jetty-server-8.1.10.v20130312.jar OK
> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
> testing: lib/jetty-util-8.1.10.v20130312.jar OK
> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
> testing: lib/jline-2.11.jar OK
> testing: lib/jms-api-1.1-rev-1.jar OK
> testing: lib/joda-time-2.7.jar OK
> testing: lib/json-patch-1.7.jar OK
> testing: lib/jsr305-2.0.1.jar OK
> testing: lib/jta-1.1.jar OK
> testing: lib/kafka-clients-0.9.0.0.jar OK
> testing: lib/log4j-1.2.17.jar OK
> testing: lib/logback-classic-1.1.5.jar OK
> testing: lib/logback-core-1.1.5.jar OK
> testing: lib/lz4-1.2.0.jar OK
> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
> testing: lib/malhar-library-3.3.1-incubating.jar OK
> testing: lib/metrics-core-3.0.2.jar OK
> testing: lib/msg-simple-1.1.jar OK
> testing: lib/mysql-connector-java-5.1.36.jar OK
> testing: lib/named-regexp-0.2.3.jar OK
> testing: lib/netty-3.6.2.Final.jar OK
> testing: lib/netty-buffer-4.0.27.Final.jar OK
> testing: lib/netty-codec-4.0.27.Final.jar OK
> testing: lib/netty-common-4.0.27.Final.jar OK
> testing: lib/netty-handler-4.0.27.Final.jar OK
> testing: lib/netty-transport-4.0.27.Final.jar OK
> testing: lib/opencsv-2.3.jar OK
> testing: lib/slf4j-api-1.7.10.jar OK
> testing: lib/snappy-java-1.1.1.7.jar OK
> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
> testing: lib/spring-context-4.1.9.RELEASE.jar OK
> testing: lib/spring-core-4.1.9.RELEASE.jar OK
> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
> testing: lib/spring-web-4.1.9.RELEASE.jar OK
> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
> testing: lib/super-csv-2.3.1.jar OK
> testing: lib/super-csv-joda-2.3.1.jar OK
> testing: lib/xml-apis-1.0.b2.jar OK
> testing: lib/zookeeper-3.4.6.jar OK
> testing: conf/ OK
> testing: conf/properties.xml OK
> No errors detected in compressed data of cassandra-events-1.2-SNAPSHOT.apa.
>
> Contents of the properties.xml file under the conf folder is here:
>
> <?xml version="1.0"?>
> <configuration>
> <property>
> <name>dt.application.CED.class</name>
> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
> <description>An alias for the application</description>
> </property>
> <property>
> <name>dt.operator.*.attr.MEMORY_MB</name>
> <value>512</value>
> </property>
>
> <property>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
> </property>
>
>
> <property>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
> <value>20</value>
> </property>
>
> </configuration>
>
>
>
> Regards,
> Ananth
>
> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com> wrote:
>
>> Hi Ananth,
>>
>> Can you do an "unzip -t" on your apa file and paste the output here?
>> Make sure the jar containing the application classes should be in the app
>> directory inside the apa.
>>
>> Can I know why the apex-app-archetype is not used in your apa project?
>>
>> David
>>
>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Hello All,
>>>
>>> I am having the following issue with my application. I have a maven
>>> project that has a class called XYZ that extends StreamingApplication.
>>>
>>> My maven packaging is ensuring that the final "apa" extension package
>>> has the right folder structure if exploded on a disk.
>>>
>>> However I am not able to locate the Application name and the "launch"
>>> button next to it after uploading the "apa" package via the browser.
>>>
>>> I tried looking at the docs to see how to specify a class as an App that
>>> can be launched from a packaged application but could not find much. Could
>>> anyone of you please point me to the right way of associating a class as an
>>> application that will get enabled with a launch button in the web ui ? The
>>> class XYZ has the annotation called @ApplicationAnnotation on top of it.
>>>
>>> Regards,
>>> Ananth
>>>
>>
>>
>
Re: Best practises to specify the Application
Posted by Ananth Gundabattula <ag...@gmail.com>.
Hello David,
Please find the unzip result at the end of this email. The jar is in the
app directory inside the apa.
As to why I did not use the archetype, I was building the app for 3.3.0
version and I was not able to specify an archetype with the version 3.3.0
when I attempted it. My presumption was archetype does not matter if the
packaging is done right. So i went ahead and created a simple maven
project.
Here is the output: ( The MANIFEST.MF is an empty file in the listing below
) . I also pasted the contents of the properties.xml file at the end of
this email for your reference.
Archive: cassandra-events-1.2-SNAPSHOT.apa
testing: META-INF/ OK
testing: META-INF/MANIFEST.MF OK
testing: app/ OK
testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
testing: lib/ OK
testing: lib/activation-1.1.jar OK
testing: lib/activemq-client-5.8.0.jar OK
testing: lib/antlr-2.7.7.jar OK
testing: lib/aopalliance-1.0.jar OK
testing: lib/app-commons-1.2-SNAPSHOT.jar OK
testing: lib/aspectjrt-1.8.7.jar OK
testing: lib/btf-1.2.jar OK
testing: lib/cassandra-driver-core-2.1.8.jar OK
testing: lib/commons-beanutils-1.8.3.jar OK
testing: lib/commons-compiler-2.7.8.jar OK
testing: lib/commons-lang3-3.1.jar OK
testing: lib/commons-logging-1.1.3.jar OK
testing: lib/curator-client-2.7.0.jar OK
testing: lib/curator-framework-2.7.0.jar OK
testing: lib/dom4j-1.6.1.jar OK
testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
testing: lib/evo-inflector-1.2.1.jar OK
testing: lib/fastutil-6.6.4.jar OK
testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
testing: lib/guava-14.0.1.jar OK
testing: lib/hawtbuf-1.9.jar OK
testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
testing: lib/hibernate-core-5.0.4.Final.jar OK
testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
testing: lib/HikariCP-2.4.3.jar OK
testing: lib/jackson-annotations-2.6.4.jar OK
testing: lib/jackson-core-2.6.4.jar OK
testing: lib/jackson-coreutils-1.6.jar OK
testing: lib/jackson-databind-2.6.4.jar OK
testing: lib/jandex-2.0.0.CR1.jar OK
testing: lib/javassist-3.18.1-GA.jar OK
testing: lib/javax.mail-1.5.0.jar OK
testing: lib/jboss-logging-3.3.0.Final.jar OK
testing: lib/jcl-over-slf4j-1.7.13.jar OK
testing: lib/jersey-client-1.9.jar OK
testing: lib/jersey-core-1.9.jar OK
testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
testing: lib/jetty-http-8.1.10.v20130312.jar OK
testing: lib/jetty-io-8.1.10.v20130312.jar OK
testing: lib/jetty-security-8.1.10.v20130312.jar OK
testing: lib/jetty-server-8.1.10.v20130312.jar OK
testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
testing: lib/jetty-util-8.1.10.v20130312.jar OK
testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
testing: lib/jline-2.11.jar OK
testing: lib/jms-api-1.1-rev-1.jar OK
testing: lib/joda-time-2.7.jar OK
testing: lib/json-patch-1.7.jar OK
testing: lib/jsr305-2.0.1.jar OK
testing: lib/jta-1.1.jar OK
testing: lib/kafka-clients-0.9.0.0.jar OK
testing: lib/log4j-1.2.17.jar OK
testing: lib/logback-classic-1.1.5.jar OK
testing: lib/logback-core-1.1.5.jar OK
testing: lib/lz4-1.2.0.jar OK
testing: lib/malhar-contrib-3.3.1-incubating.jar OK
testing: lib/malhar-kafka-3.3.1-incubating.jar OK
testing: lib/malhar-library-3.3.1-incubating.jar OK
testing: lib/metrics-core-3.0.2.jar OK
testing: lib/msg-simple-1.1.jar OK
testing: lib/mysql-connector-java-5.1.36.jar OK
testing: lib/named-regexp-0.2.3.jar OK
testing: lib/netty-3.6.2.Final.jar OK
testing: lib/netty-buffer-4.0.27.Final.jar OK
testing: lib/netty-codec-4.0.27.Final.jar OK
testing: lib/netty-common-4.0.27.Final.jar OK
testing: lib/netty-handler-4.0.27.Final.jar OK
testing: lib/netty-transport-4.0.27.Final.jar OK
testing: lib/opencsv-2.3.jar OK
testing: lib/slf4j-api-1.7.10.jar OK
testing: lib/snappy-java-1.1.1.7.jar OK
testing: lib/spring-aop-4.1.9.RELEASE.jar OK
testing: lib/spring-beans-4.1.9.RELEASE.jar OK
testing: lib/spring-context-4.1.9.RELEASE.jar OK
testing: lib/spring-core-4.1.9.RELEASE.jar OK
testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
testing: lib/spring-expression-4.1.9.RELEASE.jar OK
testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
testing: lib/spring-orm-4.1.9.RELEASE.jar OK
testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
testing: lib/spring-tx-4.1.9.RELEASE.jar OK
testing: lib/spring-web-4.1.9.RELEASE.jar OK
testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
testing: lib/super-csv-2.3.1.jar OK
testing: lib/super-csv-joda-2.3.1.jar OK
testing: lib/xml-apis-1.0.b2.jar OK
testing: lib/zookeeper-3.4.6.jar OK
testing: conf/ OK
testing: conf/properties.xml OK
No errors detected in compressed data of cassandra-events-1.2-SNAPSHOT.apa.
Contents of the properties.xml file under the conf folder is here:
<?xml version="1.0"?>
<configuration>
<property>
<name>dt.application.CED.class</name>
<value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
<description>An alias for the application</description>
</property>
<property>
<name>dt.operator.*.attr.MEMORY_MB</name>
<value>512</value>
</property>
<property>
<name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
<value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
</property>
<property>
<name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
<value>20</value>
</property>
</configuration>
Regards,
Ananth
On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com> wrote:
> Hi Ananth,
>
> Can you do an "unzip -t" on your apa file and paste the output here?
> Make sure the jar containing the application classes should be in the app
> directory inside the apa.
>
> Can I know why the apex-app-archetype is not used in your apa project?
>
> David
>
> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
>> Hello All,
>>
>> I am having the following issue with my application. I have a maven
>> project that has a class called XYZ that extends StreamingApplication.
>>
>> My maven packaging is ensuring that the final "apa" extension package has
>> the right folder structure if exploded on a disk.
>>
>> However I am not able to locate the Application name and the "launch"
>> button next to it after uploading the "apa" package via the browser.
>>
>> I tried looking at the docs to see how to specify a class as an App that
>> can be launched from a packaged application but could not find much. Could
>> anyone of you please point me to the right way of associating a class as an
>> application that will get enabled with a launch button in the web ui ? The
>> class XYZ has the annotation called @ApplicationAnnotation on top of it.
>>
>> Regards,
>> Ananth
>>
>
>
Re: Best practises to specify the Application
Posted by David Yan <da...@datatorrent.com>.
Hi Ananth,
Can you do an "unzip -t" on your apa file and paste the output here?
Make sure the jar containing the application classes should be in the app
directory inside the apa.
Can I know why the apex-app-archetype is not used in your apa project?
David
On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> Hello All,
>
> I am having the following issue with my application. I have a maven
> project that has a class called XYZ that extends StreamingApplication.
>
> My maven packaging is ensuring that the final "apa" extension package has
> the right folder structure if exploded on a disk.
>
> However I am not able to locate the Application name and the "launch"
> button next to it after uploading the "apa" package via the browser.
>
> I tried looking at the docs to see how to specify a class as an App that
> can be launched from a packaged application but could not find much. Could
> anyone of you please point me to the right way of associating a class as an
> application that will get enabled with a launch button in the web ui ? The
> class XYZ has the annotation called @ApplicationAnnotation on top of it.
>
> Regards,
> Ananth
>