You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apex.apache.org by David Yan <da...@datatorrent.com> on 2016/04/29 19:37:32 UTC
Fwd: Best practises to specify the Application
[switching to dev list]
It appears Ananth is using JDK 1.8 to compile the apa and our distributed
binary, compiled with 1.7 by us, cannot read class files compiled with 1.8.
I think we may need to add target=1.7 in the compiler plugin config in the
pom.xml file of apex-app-archetype. Thoughts?
David
---------- Forwarded message ----------
From: Ananth Gundabattula <ag...@gmail.com>
Date: Fri, Apr 29, 2016 at 1:20 AM
Subject: Re: Best practises to specify the Application
To: users@apex.incubator.apache.org
I had a look at the dtgateway.log and observed the following stacktrace : I
guess the root cause might be Apex running on a lower version of java and
the application code compiled with a higher version. I shall update
everyone after the upgrade to see if it fixes the issue.
Fatal error encountered
>
> 2016-04-29 08:04:45,181 ERROR
> com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> com/tx/y/z/Operator : Unsupported major.minor version 52.0
> at java.lang.ClassLoader.defineClass1(Native Method)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> at
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> at
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
>
On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
agundabattula@gmail.com> wrote:
> Thanks Shubham.
>
> Those properties were being set by the parent pom which are inherited by
> the current pom.xml ( Kind of wanted all apex apps to have same version
> structure etc) and hence added that property as part of parent pom.
>
> I am still stumped as to what is causing this.
>
> Regards,
> Ananth
>
>
>
> On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <sh...@datatorrent.com>
> wrote:
>
>> Hi Ananth,
>>
>> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>> -DarchetypeArtifactId=apex-app-archetype -DarchetypeVersion=3.3.0-incubating
>> -DgroupId=com.example -Dpackage=com.example.mydtapp -DartifactId=mydtapp
>> -Dversion=1.0-SNAPSHOT
>> and it works for me as well.
>>
>> I compared the pom.xml file and noticed values for following properties
>> were missing.
>> <manifestEntries>
>>
>> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
>> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
>>
>> </manifestEntries>
>>
>> Could you add the following to pom.xml and try again.
>>
>> <properties>
>> <!-- change this if you desire to use a different version of Apex Core -->
>> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
>> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
>> </properties>
>>
>>
>> Thanks,
>> Shubham
>>
>> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
>> agundabattula@gmail.com> wrote:
>>
>>> Hello David,
>>>
>>> I reattempted the packaging with the archetype based approach. I guess
>>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks for
>>> pointing it out.
>>>
>>> The default package generated by the archetype command seems to be
>>> working fine when uploading it but the moment I add in my code and
>>> dependencies it goes back the issue I am seeing :
>>>
>>>
>>> My app package looks like app.zip ( I removed all of the
>>> application.class for brevity sake ) . Please note that I deleted all the
>>> jars under /lib folder and the contents of the /app folder in the apa
>>> pacakge.
>>>
>>> I have also attached a copy of my pom.xml file.
>>>
>>> Could you please advise if you something amiss right away in the
>>> structure below ? It may be noted that I retained the classes of the
>>> original simple project generated by the archetype and even they stopped
>>> showing with the launch button the moment I added in all of the application
>>> dependencies in maven.
>>>
>>> Regards,
>>> Ananth
>>>
>>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
>>> wrote:
>>>
>>>> Hi Ananth,
>>>>
>>>> I just tried with 3.3.0-incubating with the following and it's working:
>>>>
>>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
>>>> -DarchetypeArtifactId=apex-app-archetype
>>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
>>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp -Dversion=1.0-SNAPSHOT
>>>>
>>>> Let me know whether this works for you or not.
>>>>
>>>> The system uses the MANIFEST.MF file to get the basic meta information
>>>> on the app package and it looks at all the jars in the app directory and
>>>> searches for classes that subclass com.datatorrent.api.StreamingApplication.
>>>>
>>>> David
>>>>
>>>>
>>>>
>>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
>>>> agundabattula@gmail.com> wrote:
>>>>
>>>>> Hello David,
>>>>>
>>>>> Regarding as to why I did not use the archetype, I was building the
>>>>> app for 3.3.0 version and I was not able to specify an archetype with the
>>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
>>>>> version 3.3.0 ? I was not sure if the archetype version should match the
>>>>> exact apex engine version . My presumption was archetype does not matter if
>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>> project.
>>>>>
>>>>> The maven assembly plugin used the appPackage.xml and the related
>>>>> constructs to build the exact file layout of the app as mentioned in the
>>>>> output above.
>>>>>
>>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
>>>>>
>>>>> This brings me to a question. How does the engine know which is the
>>>>> main class that is to be used for launching the Streaming App. I do have a
>>>>> class that extends the StreaminApplication that is annotated with
>>>>> ApplicationAnnotation but does not seem to be picking up. By this I mean, I
>>>>> do not see a "launch" icon next to it. All I see is the top level packaged
>>>>> name in the web console but nothing beneath it for me to trigger a launch.
>>>>>
>>>>> Regards,
>>>>> Ananth
>>>>>
>>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Ananth,
>>>>>>
>>>>>> The MANIFEST.MF file is important, and it's generated automatically
>>>>>> if the archetype is used. For example, the MANFEST.MF file from the Pi demo
>>>>>> in Malhar has this content:
>>>>>>
>>>>>> Manifest-Version: 1.0
>>>>>> Archiver-Version: Plexus Archiver
>>>>>> Built-By: david
>>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
>>>>>> Class-Path: lib/*.jar
>>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
>>>>>> Created-By: Apache Maven 3.3.9
>>>>>> Build-Jdk: 1.8.0_92
>>>>>> DT-App-Package-Description: Apex demo applications that calculate the
>>>>>> value of Pi. This is a starting point to understand how Apex works.
>>>>>> DT-App-Package-Group-Id: org.apache.apex
>>>>>> DT-App-Package-Name: pi-demo
>>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
>>>>>>
>>>>>> Is there a reason why the archetype is not used in your project?
>>>>>>
>>>>>> David
>>>>>>
>>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
>>>>>> agundabattula@gmail.com> wrote:
>>>>>>
>>>>>>> Hello David,
>>>>>>>
>>>>>>> Please find the unzip result at the end of this email. The jar is in
>>>>>>> the app directory inside the apa.
>>>>>>> As to why I did not use the archetype, I was building the app for
>>>>>>> 3.3.0 version and I was not able to specify an archetype with the version
>>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not matter if
>>>>>>> the packaging is done right. So i went ahead and created a simple maven
>>>>>>> project.
>>>>>>>
>>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
>>>>>>> listing below ) . I also pasted the contents of the properties.xml file at
>>>>>>> the end of this email for your reference.
>>>>>>>
>>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
>>>>>>> testing: META-INF/ OK
>>>>>>> testing: META-INF/MANIFEST.MF OK
>>>>>>> testing: app/ OK
>>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/ OK
>>>>>>> testing: lib/activation-1.1.jar OK
>>>>>>> testing: lib/activemq-client-5.8.0.jar OK
>>>>>>> testing: lib/antlr-2.7.7.jar OK
>>>>>>> testing: lib/aopalliance-1.0.jar OK
>>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
>>>>>>> testing: lib/btf-1.2.jar OK
>>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
>>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
>>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
>>>>>>> testing: lib/commons-lang3-3.1.jar OK
>>>>>>> testing: lib/commons-logging-1.1.3.jar OK
>>>>>>> testing: lib/curator-client-2.7.0.jar OK
>>>>>>> testing: lib/curator-framework-2.7.0.jar OK
>>>>>>> testing: lib/dom4j-1.6.1.jar OK
>>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
>>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
>>>>>>> testing: lib/fastutil-6.6.4.jar OK
>>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
>>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
>>>>>>> testing: lib/guava-14.0.1.jar OK
>>>>>>> testing: lib/hawtbuf-1.9.jar OK
>>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
>>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
>>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
>>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
>>>>>>> testing: lib/HikariCP-2.4.3.jar OK
>>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
>>>>>>> testing: lib/jackson-core-2.6.4.jar OK
>>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
>>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
>>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
>>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
>>>>>>> testing: lib/javax.mail-1.5.0.jar OK
>>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
>>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
>>>>>>> testing: lib/jersey-client-1.9.jar OK
>>>>>>> testing: lib/jersey-core-1.9.jar OK
>>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
>>>>>>> testing: lib/jline-2.11.jar OK
>>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
>>>>>>> testing: lib/joda-time-2.7.jar OK
>>>>>>> testing: lib/json-patch-1.7.jar OK
>>>>>>> testing: lib/jsr305-2.0.1.jar OK
>>>>>>> testing: lib/jta-1.1.jar OK
>>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
>>>>>>> testing: lib/log4j-1.2.17.jar OK
>>>>>>> testing: lib/logback-classic-1.1.5.jar OK
>>>>>>> testing: lib/logback-core-1.1.5.jar OK
>>>>>>> testing: lib/lz4-1.2.0.jar OK
>>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
>>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
>>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
>>>>>>> testing: lib/metrics-core-3.0.2.jar OK
>>>>>>> testing: lib/msg-simple-1.1.jar OK
>>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
>>>>>>> testing: lib/named-regexp-0.2.3.jar OK
>>>>>>> testing: lib/netty-3.6.2.Final.jar OK
>>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
>>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
>>>>>>> testing: lib/opencsv-2.3.jar OK
>>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
>>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
>>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
>>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
>>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
>>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
>>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
>>>>>>> testing: lib/super-csv-2.3.1.jar OK
>>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
>>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
>>>>>>> testing: lib/zookeeper-3.4.6.jar OK
>>>>>>> testing: conf/ OK
>>>>>>> testing: conf/properties.xml OK
>>>>>>> No errors detected in compressed data of
>>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
>>>>>>>
>>>>>>> Contents of the properties.xml file under the conf folder is here:
>>>>>>>
>>>>>>> <?xml version="1.0"?>
>>>>>>> <configuration>
>>>>>>> <property>
>>>>>>> <name>dt.application.CED.class</name>
>>>>>>> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
>>>>>>> <description>An alias for the application</description>
>>>>>>> </property>
>>>>>>> <property>
>>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
>>>>>>> <value>512</value>
>>>>>>> </property>
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
>>>>>>> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
>>>>>>> </property>
>>>>>>>
>>>>>>>
>>>>>>> <property>
>>>>>>> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
>>>>>>> <value>20</value>
>>>>>>> </property>
>>>>>>>
>>>>>>> </configuration>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Ananth
>>>>>>>
>>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Ananth,
>>>>>>>>
>>>>>>>> Can you do an "unzip -t" on your apa file and paste the output here?
>>>>>>>> Make sure the jar containing the application classes should be in
>>>>>>>> the app directory inside the apa.
>>>>>>>>
>>>>>>>> Can I know why the apex-app-archetype is not used in your apa
>>>>>>>> project?
>>>>>>>>
>>>>>>>> David
>>>>>>>>
>>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
>>>>>>>> agundabattula@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello All,
>>>>>>>>>
>>>>>>>>> I am having the following issue with my application. I have a
>>>>>>>>> maven project that has a class called XYZ that extends
>>>>>>>>> StreamingApplication.
>>>>>>>>>
>>>>>>>>> My maven packaging is ensuring that the final "apa" extension
>>>>>>>>> package has the right folder structure if exploded on a disk.
>>>>>>>>>
>>>>>>>>> However I am not able to locate the Application name and the
>>>>>>>>> "launch" button next to it after uploading the "apa" package via the
>>>>>>>>> browser.
>>>>>>>>>
>>>>>>>>> I tried looking at the docs to see how to specify a class as an
>>>>>>>>> App that can be launched from a packaged application but could not find
>>>>>>>>> much. Could anyone of you please point me to the right way of associating a
>>>>>>>>> class as an application that will get enabled with a launch button in the
>>>>>>>>> web ui ? The class XYZ has the annotation called @ApplicationAnnotation on
>>>>>>>>> top of it.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Ananth
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
Re: Best practises to specify the Application
Posted by David Yan <da...@datatorrent.com>.
Ah, never mind. -target 1.7 is already there.
https://github.com/apache/incubator-apex-core/blob/master/apex-app-archetype/src/main/resources/archetype-resources/pom.xml
It boils down to the user not using the archetype.
David
On Fri, Apr 29, 2016 at 10:47 AM, Munagala Ramanath <ra...@datatorrent.com>
wrote:
> A bunch of us are using JDK 1.8 to compile apps and deploying to our
> cluster and/or sandbox and not
> seeing this issue; how come ?
>
> Ram
>
> On Fri, Apr 29, 2016 at 10:37 AM, David Yan <da...@datatorrent.com> wrote:
>
> > [switching to dev list]
> >
> > It appears Ananth is using JDK 1.8 to compile the apa and our distributed
> > binary, compiled with 1.7 by us, cannot read class files compiled with
> 1.8.
> >
> > I think we may need to add target=1.7 in the compiler plugin config in
> the
> > pom.xml file of apex-app-archetype. Thoughts?
> >
> > David
> >
> > ---------- Forwarded message ----------
> > From: Ananth Gundabattula <ag...@gmail.com>
> > Date: Fri, Apr 29, 2016 at 1:20 AM
> > Subject: Re: Best practises to specify the Application
> > To: users@apex.incubator.apache.org
> >
> >
> > I had a look at the dtgateway.log and observed the following stacktrace
> : I
> > guess the root cause might be Apex running on a lower version of java and
> > the application code compiled with a higher version. I shall update
> > everyone after the upgrade to see if it fixes the issue.
> >
> >
> > Fatal error encountered
> > >
> > > 2016-04-29 08:04:45,181 ERROR
> > > com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> > > com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> > > com/tx/y/z/Operator : Unsupported major.minor version 52.0
> > > at java.lang.ClassLoader.defineClass1(Native Method)
> > > at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> > > at
> > > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> > > at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > > at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > > at
> > >
> >
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> > > at
> > >
> >
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> > > at
> > >
> >
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> > > at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
> > >
> >
> >
> > On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
> > agundabattula@gmail.com> wrote:
> >
> > > Thanks Shubham.
> > >
> > > Those properties were being set by the parent pom which are inherited
> by
> > > the current pom.xml ( Kind of wanted all apex apps to have same version
> > > structure etc) and hence added that property as part of parent pom.
> > >
> > > I am still stumped as to what is causing this.
> > >
> > > Regards,
> > > Ananth
> > >
> > >
> > >
> > > On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <
> shubham@datatorrent.com
> > >
> > > wrote:
> > >
> > >> Hi Ananth,
> > >>
> > >> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> > >> -DarchetypeArtifactId=apex-app-archetype
> > -DarchetypeVersion=3.3.0-incubating
> > >> -DgroupId=com.example -Dpackage=com.example.mydtapp
> > -DartifactId=mydtapp
> > >> -Dversion=1.0-SNAPSHOT
> > >> and it works for me as well.
> > >>
> > >> I compared the pom.xml file and noticed values for following
> properties
> > >> were missing.
> > >> <manifestEntries>
> > >>
> > >>
> > <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
> > >>
> > <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
> > >>
> > >> </manifestEntries>
> > >>
> > >> Could you add the following to pom.xml and try again.
> > >>
> > >> <properties>
> > >> <!-- change this if you desire to use a different version of Apex
> > Core -->
> > >>
> >
> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
> > >>
> >
> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
> > >> </properties>
> > >>
> > >>
> > >> Thanks,
> > >> Shubham
> > >>
> > >> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
> > >> agundabattula@gmail.com> wrote:
> > >>
> > >>> Hello David,
> > >>>
> > >>> I reattempted the packaging with the archetype based approach. I
> guess
> > >>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks
> > for
> > >>> pointing it out.
> > >>>
> > >>> The default package generated by the archetype command seems to be
> > >>> working fine when uploading it but the moment I add in my code and
> > >>> dependencies it goes back the issue I am seeing :
> > >>>
> > >>>
> > >>> My app package looks like app.zip ( I removed all of the
> > >>> application.class for brevity sake ) . Please note that I deleted all
> > the
> > >>> jars under /lib folder and the contents of the /app folder in the apa
> > >>> pacakge.
> > >>>
> > >>> I have also attached a copy of my pom.xml file.
> > >>>
> > >>> Could you please advise if you something amiss right away in the
> > >>> structure below ? It may be noted that I retained the classes of the
> > >>> original simple project generated by the archetype and even they
> > stopped
> > >>> showing with the launch button the moment I added in all of the
> > application
> > >>> dependencies in maven.
> > >>>
> > >>> Regards,
> > >>> Ananth
> > >>>
> > >>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
> > >>> wrote:
> > >>>
> > >>>> Hi Ananth,
> > >>>>
> > >>>> I just tried with 3.3.0-incubating with the following and it's
> > working:
> > >>>>
> > >>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> > >>>> -DarchetypeArtifactId=apex-app-archetype
> > >>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
> > >>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp
> > -Dversion=1.0-SNAPSHOT
> > >>>>
> > >>>> Let me know whether this works for you or not.
> > >>>>
> > >>>> The system uses the MANIFEST.MF file to get the basic meta
> information
> > >>>> on the app package and it looks at all the jars in the app directory
> > and
> > >>>> searches for classes that subclass
> > com.datatorrent.api.StreamingApplication.
> > >>>>
> > >>>> David
> > >>>>
> > >>>>
> > >>>>
> > >>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
> > >>>> agundabattula@gmail.com> wrote:
> > >>>>
> > >>>>> Hello David,
> > >>>>>
> > >>>>> Regarding as to why I did not use the archetype, I was building the
> > >>>>> app for 3.3.0 version and I was not able to specify an archetype
> > with the
> > >>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype
> for
> > >>>>> version 3.3.0 ? I was not sure if the archetype version should
> match
> > the
> > >>>>> exact apex engine version . My presumption was archetype does not
> > matter if
> > >>>>> the packaging is done right. So i went ahead and created a simple
> > maven
> > >>>>> project.
> > >>>>>
> > >>>>> The maven assembly plugin used the appPackage.xml and the related
> > >>>>> constructs to build the exact file layout of the app as mentioned
> in
> > the
> > >>>>> output above.
> > >>>>>
> > >>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
> > >>>>>
> > >>>>> This brings me to a question. How does the engine know which is the
> > >>>>> main class that is to be used for launching the Streaming App. I do
> > have a
> > >>>>> class that extends the StreaminApplication that is annotated with
> > >>>>> ApplicationAnnotation but does not seem to be picking up. By this I
> > mean, I
> > >>>>> do not see a "launch" icon next to it. All I see is the top level
> > packaged
> > >>>>> name in the web console but nothing beneath it for me to trigger a
> > launch.
> > >>>>>
> > >>>>> Regards,
> > >>>>> Ananth
> > >>>>>
> > >>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi Ananth,
> > >>>>>>
> > >>>>>> The MANIFEST.MF file is important, and it's generated
> automatically
> > >>>>>> if the archetype is used. For example, the MANFEST.MF file from
> the
> > Pi demo
> > >>>>>> in Malhar has this content:
> > >>>>>>
> > >>>>>> Manifest-Version: 1.0
> > >>>>>> Archiver-Version: Plexus Archiver
> > >>>>>> Built-By: david
> > >>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi
> Demo
> > >>>>>> Class-Path: lib/*.jar
> > >>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
> > >>>>>> Created-By: Apache Maven 3.3.9
> > >>>>>> Build-Jdk: 1.8.0_92
> > >>>>>> DT-App-Package-Description: Apex demo applications that calculate
> > the
> > >>>>>> value of Pi. This is a starting point to understand how Apex
> works.
> > >>>>>> DT-App-Package-Group-Id: org.apache.apex
> > >>>>>> DT-App-Package-Name: pi-demo
> > >>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
> > >>>>>>
> > >>>>>> Is there a reason why the archetype is not used in your project?
> > >>>>>>
> > >>>>>> David
> > >>>>>>
> > >>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
> > >>>>>> agundabattula@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> Hello David,
> > >>>>>>>
> > >>>>>>> Please find the unzip result at the end of this email. The jar is
> > in
> > >>>>>>> the app directory inside the apa.
> > >>>>>>> As to why I did not use the archetype, I was building the app for
> > >>>>>>> 3.3.0 version and I was not able to specify an archetype with the
> > version
> > >>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not
> > matter if
> > >>>>>>> the packaging is done right. So i went ahead and created a simple
> > maven
> > >>>>>>> project.
> > >>>>>>>
> > >>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
> > >>>>>>> listing below ) . I also pasted the contents of the
> properties.xml
> > file at
> > >>>>>>> the end of this email for your reference.
> > >>>>>>>
> > >>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
> > >>>>>>> testing: META-INF/ OK
> > >>>>>>> testing: META-INF/MANIFEST.MF OK
> > >>>>>>> testing: app/ OK
> > >>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/ OK
> > >>>>>>> testing: lib/activation-1.1.jar OK
> > >>>>>>> testing: lib/activemq-client-5.8.0.jar OK
> > >>>>>>> testing: lib/antlr-2.7.7.jar OK
> > >>>>>>> testing: lib/aopalliance-1.0.jar OK
> > >>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
> > >>>>>>> testing: lib/btf-1.2.jar OK
> > >>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
> > >>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
> > >>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
> > >>>>>>> testing: lib/commons-lang3-3.1.jar OK
> > >>>>>>> testing: lib/commons-logging-1.1.3.jar OK
> > >>>>>>> testing: lib/curator-client-2.7.0.jar OK
> > >>>>>>> testing: lib/curator-framework-2.7.0.jar OK
> > >>>>>>> testing: lib/dom4j-1.6.1.jar OK
> > >>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
> > >>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
> > >>>>>>> testing: lib/fastutil-6.6.4.jar OK
> > >>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
> > >>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
> > >>>>>>> testing: lib/guava-14.0.1.jar OK
> > >>>>>>> testing: lib/hawtbuf-1.9.jar OK
> > >>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar
> OK
> > >>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
> > >>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
> > >>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
> > >>>>>>> testing: lib/HikariCP-2.4.3.jar OK
> > >>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
> > >>>>>>> testing: lib/jackson-core-2.6.4.jar OK
> > >>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
> > >>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
> > >>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
> > >>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
> > >>>>>>> testing: lib/javax.mail-1.5.0.jar OK
> > >>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
> > >>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
> > >>>>>>> testing: lib/jersey-client-1.9.jar OK
> > >>>>>>> testing: lib/jersey-core-1.9.jar OK
> > >>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
> > >>>>>>> testing: lib/jline-2.11.jar OK
> > >>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
> > >>>>>>> testing: lib/joda-time-2.7.jar OK
> > >>>>>>> testing: lib/json-patch-1.7.jar OK
> > >>>>>>> testing: lib/jsr305-2.0.1.jar OK
> > >>>>>>> testing: lib/jta-1.1.jar OK
> > >>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
> > >>>>>>> testing: lib/log4j-1.2.17.jar OK
> > >>>>>>> testing: lib/logback-classic-1.1.5.jar OK
> > >>>>>>> testing: lib/logback-core-1.1.5.jar OK
> > >>>>>>> testing: lib/lz4-1.2.0.jar OK
> > >>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
> > >>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
> > >>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
> > >>>>>>> testing: lib/metrics-core-3.0.2.jar OK
> > >>>>>>> testing: lib/msg-simple-1.1.jar OK
> > >>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
> > >>>>>>> testing: lib/named-regexp-0.2.3.jar OK
> > >>>>>>> testing: lib/netty-3.6.2.Final.jar OK
> > >>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
> > >>>>>>> testing: lib/opencsv-2.3.jar OK
> > >>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
> > >>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
> > >>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
> > >>>>>>> testing: lib/super-csv-2.3.1.jar OK
> > >>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
> > >>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
> > >>>>>>> testing: lib/zookeeper-3.4.6.jar OK
> > >>>>>>> testing: conf/ OK
> > >>>>>>> testing: conf/properties.xml OK
> > >>>>>>> No errors detected in compressed data of
> > >>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
> > >>>>>>>
> > >>>>>>> Contents of the properties.xml file under the conf folder is
> here:
> > >>>>>>>
> > >>>>>>> <?xml version="1.0"?>
> > >>>>>>> <configuration>
> > >>>>>>> <property>
> > >>>>>>> <name>dt.application.CED.class</name>
> > >>>>>>>
> >
> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
> > >>>>>>> <description>An alias for the application</description>
> > >>>>>>> </property>
> > >>>>>>> <property>
> > >>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
> > >>>>>>> <value>512</value>
> > >>>>>>> </property>
> > >>>>>>>
> > >>>>>>> <property>
> > >>>>>>>
> >
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
> > >>>>>>>
> >
> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
> > >>>>>>> </property>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> <property>
> > >>>>>>>
> >
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
> > >>>>>>> <value>20</value>
> > >>>>>>> </property>
> > >>>>>>>
> > >>>>>>> </configuration>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Regards,
> > >>>>>>> Ananth
> > >>>>>>>
> > >>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <
> david@datatorrent.com>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Hi Ananth,
> > >>>>>>>>
> > >>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
> > here?
> > >>>>>>>> Make sure the jar containing the application classes should be
> in
> > >>>>>>>> the app directory inside the apa.
> > >>>>>>>>
> > >>>>>>>> Can I know why the apex-app-archetype is not used in your apa
> > >>>>>>>> project?
> > >>>>>>>>
> > >>>>>>>> David
> > >>>>>>>>
> > >>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
> > >>>>>>>> agundabattula@gmail.com> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hello All,
> > >>>>>>>>>
> > >>>>>>>>> I am having the following issue with my application. I have a
> > >>>>>>>>> maven project that has a class called XYZ that extends
> > >>>>>>>>> StreamingApplication.
> > >>>>>>>>>
> > >>>>>>>>> My maven packaging is ensuring that the final "apa" extension
> > >>>>>>>>> package has the right folder structure if exploded on a disk.
> > >>>>>>>>>
> > >>>>>>>>> However I am not able to locate the Application name and the
> > >>>>>>>>> "launch" button next to it after uploading the "apa" package
> via
> > the
> > >>>>>>>>> browser.
> > >>>>>>>>>
> > >>>>>>>>> I tried looking at the docs to see how to specify a class as an
> > >>>>>>>>> App that can be launched from a packaged application but could
> > not find
> > >>>>>>>>> much. Could anyone of you please point me to the right way of
> > associating a
> > >>>>>>>>> class as an application that will get enabled with a launch
> > button in the
> > >>>>>>>>> web ui ? The class XYZ has the annotation called
> > @ApplicationAnnotation on
> > >>>>>>>>> top of it.
> > >>>>>>>>>
> > >>>>>>>>> Regards,
> > >>>>>>>>> Ananth
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> > >
> >
>
Re: Best practises to specify the Application
Posted by Munagala Ramanath <ra...@datatorrent.com>.
A bunch of us are using JDK 1.8 to compile apps and deploying to our
cluster and/or sandbox and not
seeing this issue; how come ?
Ram
On Fri, Apr 29, 2016 at 10:37 AM, David Yan <da...@datatorrent.com> wrote:
> [switching to dev list]
>
> It appears Ananth is using JDK 1.8 to compile the apa and our distributed
> binary, compiled with 1.7 by us, cannot read class files compiled with 1.8.
>
> I think we may need to add target=1.7 in the compiler plugin config in the
> pom.xml file of apex-app-archetype. Thoughts?
>
> David
>
> ---------- Forwarded message ----------
> From: Ananth Gundabattula <ag...@gmail.com>
> Date: Fri, Apr 29, 2016 at 1:20 AM
> Subject: Re: Best practises to specify the Application
> To: users@apex.incubator.apache.org
>
>
> I had a look at the dtgateway.log and observed the following stacktrace : I
> guess the root cause might be Apex running on a lower version of java and
> the application code compiled with a higher version. I shall update
> everyone after the upgrade to see if it fixes the issue.
>
>
> Fatal error encountered
> >
> > 2016-04-29 08:04:45,181 ERROR
> > com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> > com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> > com/tx/y/z/Operator : Unsupported major.minor version 52.0
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> > at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> > at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > at
> >
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> > at
> >
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> > at
> >
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> > at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
> >
>
>
> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
> > Thanks Shubham.
> >
> > Those properties were being set by the parent pom which are inherited by
> > the current pom.xml ( Kind of wanted all apex apps to have same version
> > structure etc) and hence added that property as part of parent pom.
> >
> > I am still stumped as to what is causing this.
> >
> > Regards,
> > Ananth
> >
> >
> >
> > On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <shubham@datatorrent.com
> >
> > wrote:
> >
> >> Hi Ananth,
> >>
> >> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> >> -DarchetypeArtifactId=apex-app-archetype
> -DarchetypeVersion=3.3.0-incubating
> >> -DgroupId=com.example -Dpackage=com.example.mydtapp
> -DartifactId=mydtapp
> >> -Dversion=1.0-SNAPSHOT
> >> and it works for me as well.
> >>
> >> I compared the pom.xml file and noticed values for following properties
> >> were missing.
> >> <manifestEntries>
> >>
> >>
> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
> >>
> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
> >>
> >> </manifestEntries>
> >>
> >> Could you add the following to pom.xml and try again.
> >>
> >> <properties>
> >> <!-- change this if you desire to use a different version of Apex
> Core -->
> >>
> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
> >>
> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
> >> </properties>
> >>
> >>
> >> Thanks,
> >> Shubham
> >>
> >> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
> >> agundabattula@gmail.com> wrote:
> >>
> >>> Hello David,
> >>>
> >>> I reattempted the packaging with the archetype based approach. I guess
> >>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks
> for
> >>> pointing it out.
> >>>
> >>> The default package generated by the archetype command seems to be
> >>> working fine when uploading it but the moment I add in my code and
> >>> dependencies it goes back the issue I am seeing :
> >>>
> >>>
> >>> My app package looks like app.zip ( I removed all of the
> >>> application.class for brevity sake ) . Please note that I deleted all
> the
> >>> jars under /lib folder and the contents of the /app folder in the apa
> >>> pacakge.
> >>>
> >>> I have also attached a copy of my pom.xml file.
> >>>
> >>> Could you please advise if you something amiss right away in the
> >>> structure below ? It may be noted that I retained the classes of the
> >>> original simple project generated by the archetype and even they
> stopped
> >>> showing with the launch button the moment I added in all of the
> application
> >>> dependencies in maven.
> >>>
> >>> Regards,
> >>> Ananth
> >>>
> >>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
> >>> wrote:
> >>>
> >>>> Hi Ananth,
> >>>>
> >>>> I just tried with 3.3.0-incubating with the following and it's
> working:
> >>>>
> >>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> >>>> -DarchetypeArtifactId=apex-app-archetype
> >>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
> >>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp
> -Dversion=1.0-SNAPSHOT
> >>>>
> >>>> Let me know whether this works for you or not.
> >>>>
> >>>> The system uses the MANIFEST.MF file to get the basic meta information
> >>>> on the app package and it looks at all the jars in the app directory
> and
> >>>> searches for classes that subclass
> com.datatorrent.api.StreamingApplication.
> >>>>
> >>>> David
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
> >>>> agundabattula@gmail.com> wrote:
> >>>>
> >>>>> Hello David,
> >>>>>
> >>>>> Regarding as to why I did not use the archetype, I was building the
> >>>>> app for 3.3.0 version and I was not able to specify an archetype
> with the
> >>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
> >>>>> version 3.3.0 ? I was not sure if the archetype version should match
> the
> >>>>> exact apex engine version . My presumption was archetype does not
> matter if
> >>>>> the packaging is done right. So i went ahead and created a simple
> maven
> >>>>> project.
> >>>>>
> >>>>> The maven assembly plugin used the appPackage.xml and the related
> >>>>> constructs to build the exact file layout of the app as mentioned in
> the
> >>>>> output above.
> >>>>>
> >>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
> >>>>>
> >>>>> This brings me to a question. How does the engine know which is the
> >>>>> main class that is to be used for launching the Streaming App. I do
> have a
> >>>>> class that extends the StreaminApplication that is annotated with
> >>>>> ApplicationAnnotation but does not seem to be picking up. By this I
> mean, I
> >>>>> do not see a "launch" icon next to it. All I see is the top level
> packaged
> >>>>> name in the web console but nothing beneath it for me to trigger a
> launch.
> >>>>>
> >>>>> Regards,
> >>>>> Ananth
> >>>>>
> >>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Ananth,
> >>>>>>
> >>>>>> The MANIFEST.MF file is important, and it's generated automatically
> >>>>>> if the archetype is used. For example, the MANFEST.MF file from the
> Pi demo
> >>>>>> in Malhar has this content:
> >>>>>>
> >>>>>> Manifest-Version: 1.0
> >>>>>> Archiver-Version: Plexus Archiver
> >>>>>> Built-By: david
> >>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
> >>>>>> Class-Path: lib/*.jar
> >>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
> >>>>>> Created-By: Apache Maven 3.3.9
> >>>>>> Build-Jdk: 1.8.0_92
> >>>>>> DT-App-Package-Description: Apex demo applications that calculate
> the
> >>>>>> value of Pi. This is a starting point to understand how Apex works.
> >>>>>> DT-App-Package-Group-Id: org.apache.apex
> >>>>>> DT-App-Package-Name: pi-demo
> >>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
> >>>>>>
> >>>>>> Is there a reason why the archetype is not used in your project?
> >>>>>>
> >>>>>> David
> >>>>>>
> >>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
> >>>>>> agundabattula@gmail.com> wrote:
> >>>>>>
> >>>>>>> Hello David,
> >>>>>>>
> >>>>>>> Please find the unzip result at the end of this email. The jar is
> in
> >>>>>>> the app directory inside the apa.
> >>>>>>> As to why I did not use the archetype, I was building the app for
> >>>>>>> 3.3.0 version and I was not able to specify an archetype with the
> version
> >>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not
> matter if
> >>>>>>> the packaging is done right. So i went ahead and created a simple
> maven
> >>>>>>> project.
> >>>>>>>
> >>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
> >>>>>>> listing below ) . I also pasted the contents of the properties.xml
> file at
> >>>>>>> the end of this email for your reference.
> >>>>>>>
> >>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
> >>>>>>> testing: META-INF/ OK
> >>>>>>> testing: META-INF/MANIFEST.MF OK
> >>>>>>> testing: app/ OK
> >>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/ OK
> >>>>>>> testing: lib/activation-1.1.jar OK
> >>>>>>> testing: lib/activemq-client-5.8.0.jar OK
> >>>>>>> testing: lib/antlr-2.7.7.jar OK
> >>>>>>> testing: lib/aopalliance-1.0.jar OK
> >>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
> >>>>>>> testing: lib/btf-1.2.jar OK
> >>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
> >>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
> >>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
> >>>>>>> testing: lib/commons-lang3-3.1.jar OK
> >>>>>>> testing: lib/commons-logging-1.1.3.jar OK
> >>>>>>> testing: lib/curator-client-2.7.0.jar OK
> >>>>>>> testing: lib/curator-framework-2.7.0.jar OK
> >>>>>>> testing: lib/dom4j-1.6.1.jar OK
> >>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
> >>>>>>> testing: lib/fastutil-6.6.4.jar OK
> >>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
> >>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
> >>>>>>> testing: lib/guava-14.0.1.jar OK
> >>>>>>> testing: lib/hawtbuf-1.9.jar OK
> >>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
> >>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
> >>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
> >>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
> >>>>>>> testing: lib/HikariCP-2.4.3.jar OK
> >>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
> >>>>>>> testing: lib/jackson-core-2.6.4.jar OK
> >>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
> >>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
> >>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
> >>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
> >>>>>>> testing: lib/javax.mail-1.5.0.jar OK
> >>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
> >>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
> >>>>>>> testing: lib/jersey-client-1.9.jar OK
> >>>>>>> testing: lib/jersey-core-1.9.jar OK
> >>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jline-2.11.jar OK
> >>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
> >>>>>>> testing: lib/joda-time-2.7.jar OK
> >>>>>>> testing: lib/json-patch-1.7.jar OK
> >>>>>>> testing: lib/jsr305-2.0.1.jar OK
> >>>>>>> testing: lib/jta-1.1.jar OK
> >>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
> >>>>>>> testing: lib/log4j-1.2.17.jar OK
> >>>>>>> testing: lib/logback-classic-1.1.5.jar OK
> >>>>>>> testing: lib/logback-core-1.1.5.jar OK
> >>>>>>> testing: lib/lz4-1.2.0.jar OK
> >>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/metrics-core-3.0.2.jar OK
> >>>>>>> testing: lib/msg-simple-1.1.jar OK
> >>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
> >>>>>>> testing: lib/named-regexp-0.2.3.jar OK
> >>>>>>> testing: lib/netty-3.6.2.Final.jar OK
> >>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
> >>>>>>> testing: lib/opencsv-2.3.jar OK
> >>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
> >>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
> >>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
> >>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
> >>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
> >>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/super-csv-2.3.1.jar OK
> >>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
> >>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
> >>>>>>> testing: lib/zookeeper-3.4.6.jar OK
> >>>>>>> testing: conf/ OK
> >>>>>>> testing: conf/properties.xml OK
> >>>>>>> No errors detected in compressed data of
> >>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
> >>>>>>>
> >>>>>>> Contents of the properties.xml file under the conf folder is here:
> >>>>>>>
> >>>>>>> <?xml version="1.0"?>
> >>>>>>> <configuration>
> >>>>>>> <property>
> >>>>>>> <name>dt.application.CED.class</name>
> >>>>>>>
> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
> >>>>>>> <description>An alias for the application</description>
> >>>>>>> </property>
> >>>>>>> <property>
> >>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
> >>>>>>> <value>512</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
> >>>>>>>
> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
> >>>>>>> <value>20</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> </configuration>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Ananth
> >>>>>>>
> >>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Ananth,
> >>>>>>>>
> >>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
> here?
> >>>>>>>> Make sure the jar containing the application classes should be in
> >>>>>>>> the app directory inside the apa.
> >>>>>>>>
> >>>>>>>> Can I know why the apex-app-archetype is not used in your apa
> >>>>>>>> project?
> >>>>>>>>
> >>>>>>>> David
> >>>>>>>>
> >>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
> >>>>>>>> agundabattula@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hello All,
> >>>>>>>>>
> >>>>>>>>> I am having the following issue with my application. I have a
> >>>>>>>>> maven project that has a class called XYZ that extends
> >>>>>>>>> StreamingApplication.
> >>>>>>>>>
> >>>>>>>>> My maven packaging is ensuring that the final "apa" extension
> >>>>>>>>> package has the right folder structure if exploded on a disk.
> >>>>>>>>>
> >>>>>>>>> However I am not able to locate the Application name and the
> >>>>>>>>> "launch" button next to it after uploading the "apa" package via
> the
> >>>>>>>>> browser.
> >>>>>>>>>
> >>>>>>>>> I tried looking at the docs to see how to specify a class as an
> >>>>>>>>> App that can be launched from a packaged application but could
> not find
> >>>>>>>>> much. Could anyone of you please point me to the right way of
> associating a
> >>>>>>>>> class as an application that will get enabled with a launch
> button in the
> >>>>>>>>> web ui ? The class XYZ has the annotation called
> @ApplicationAnnotation on
> >>>>>>>>> top of it.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Ananth
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>
Re: Best practises to specify the Application
Posted by Munagala Ramanath <ra...@datatorrent.com>.
In the pom.xml file of the 3.3 archetype generated project I'm already
seeing:
<source>1.7</source>
<target>1.7</target>
for the maven-compiler-plugin configuration
Ram
On Fri, Apr 29, 2016 at 10:37 AM, David Yan <da...@datatorrent.com> wrote:
> [switching to dev list]
>
> It appears Ananth is using JDK 1.8 to compile the apa and our distributed
> binary, compiled with 1.7 by us, cannot read class files compiled with 1.8.
>
> I think we may need to add target=1.7 in the compiler plugin config in the
> pom.xml file of apex-app-archetype. Thoughts?
>
> David
>
> ---------- Forwarded message ----------
> From: Ananth Gundabattula <ag...@gmail.com>
> Date: Fri, Apr 29, 2016 at 1:20 AM
> Subject: Re: Best practises to specify the Application
> To: users@apex.incubator.apache.org
>
>
> I had a look at the dtgateway.log and observed the following stacktrace : I
> guess the root cause might be Apex running on a lower version of java and
> the application code compiled with a higher version. I shall update
> everyone after the upgrade to see if it fixes the issue.
>
>
> Fatal error encountered
> >
> > 2016-04-29 08:04:45,181 ERROR
> > com.datatorrent.gateway.resources.ws.v2.WSResource: Caught exception
> > com.datatorrent.gateway.y: java.lang.UnsupportedClassVersionError:
> > com/tx/y/z/Operator : Unsupported major.minor version 52.0
> > at java.lang.ClassLoader.defineClass1(Native Method)
> > at java.lang.ClassLoader.defineClass(ClassLoader.java:800)
> > at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
> > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
> > at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> > at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> > at
> >
> com.datatorrent.stram.webapp.OperatorDiscoverer.addDefaultValue(OperatorDiscoverer.java:297)
> > at
> >
> com.datatorrent.stram.cli.DTCli$GetJarOperatorClassesCommand.execute(DTCli.java:3010)
> > at
> >
> com.datatorrent.stram.cli.DTCli$GetAppPackageOperatorsCommand.execute(DTCli.java:3755)
> > at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1449)
> >
>
>
> On Fri, Apr 29, 2016 at 5:49 PM, Ananth Gundabattula <
> agundabattula@gmail.com> wrote:
>
> > Thanks Shubham.
> >
> > Those properties were being set by the parent pom which are inherited by
> > the current pom.xml ( Kind of wanted all apex apps to have same version
> > structure etc) and hence added that property as part of parent pom.
> >
> > I am still stumped as to what is causing this.
> >
> > Regards,
> > Ananth
> >
> >
> >
> > On Fri, Apr 29, 2016 at 4:23 PM, Shubham Pathak <shubham@datatorrent.com
> >
> > wrote:
> >
> >> Hi Ananth,
> >>
> >> I tried $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> >> -DarchetypeArtifactId=apex-app-archetype
> -DarchetypeVersion=3.3.0-incubating
> >> -DgroupId=com.example -Dpackage=com.example.mydtapp
> -DartifactId=mydtapp
> >> -Dversion=1.0-SNAPSHOT
> >> and it works for me as well.
> >>
> >> I compared the pom.xml file and noticed values for following properties
> >> were missing.
> >> <manifestEntries>
> >>
> >>
> <Class-Path>${apache.apex.apppackage.classpath}</Class-Path>
> >>
> <DT-Engine-Version>${apache.apex.engine.version}</DT-Engine-Version>
> >>
> >> </manifestEntries>
> >>
> >> Could you add the following to pom.xml and try again.
> >>
> >> <properties>
> >> <!-- change this if you desire to use a different version of Apex
> Core -->
> >>
> <apache.apex.engine.version>3.3.0-incubating</apache.apex.engine.version>
> >>
> <apache.apex.apppackage.classpath>lib/*.jar</apache.apex.apppackage.classpath>
> >> </properties>
> >>
> >>
> >> Thanks,
> >> Shubham
> >>
> >> On Fri, Apr 29, 2016 at 11:12 AM, Ananth Gundabattula <
> >> agundabattula@gmail.com> wrote:
> >>
> >>> Hello David,
> >>>
> >>> I reattempted the packaging with the archetype based approach. I guess
> >>> the 3.3.0 archetype did not exist when I attempted it earlier. Thanks
> for
> >>> pointing it out.
> >>>
> >>> The default package generated by the archetype command seems to be
> >>> working fine when uploading it but the moment I add in my code and
> >>> dependencies it goes back the issue I am seeing :
> >>>
> >>>
> >>> My app package looks like app.zip ( I removed all of the
> >>> application.class for brevity sake ) . Please note that I deleted all
> the
> >>> jars under /lib folder and the contents of the /app folder in the apa
> >>> pacakge.
> >>>
> >>> I have also attached a copy of my pom.xml file.
> >>>
> >>> Could you please advise if you something amiss right away in the
> >>> structure below ? It may be noted that I retained the classes of the
> >>> original simple project generated by the archetype and even they
> stopped
> >>> showing with the launch button the moment I added in all of the
> application
> >>> dependencies in maven.
> >>>
> >>> Regards,
> >>> Ananth
> >>>
> >>> On Fri, Apr 29, 2016 at 12:01 PM, David Yan <da...@datatorrent.com>
> >>> wrote:
> >>>
> >>>> Hi Ananth,
> >>>>
> >>>> I just tried with 3.3.0-incubating with the following and it's
> working:
> >>>>
> >>>> $ mvn archetype:generate -DarchetypeGroupId=org.apache.apex
> >>>> -DarchetypeArtifactId=apex-app-archetype
> >>>> -DarchetypeVersion=3.3.0-incubating -DgroupId=com.example
> >>>> -Dpackage=com.example.mydtapp -DartifactId=mydtapp
> -Dversion=1.0-SNAPSHOT
> >>>>
> >>>> Let me know whether this works for you or not.
> >>>>
> >>>> The system uses the MANIFEST.MF file to get the basic meta information
> >>>> on the app package and it looks at all the jars in the app directory
> and
> >>>> searches for classes that subclass
> com.datatorrent.api.StreamingApplication.
> >>>>
> >>>> David
> >>>>
> >>>>
> >>>>
> >>>> On Thu, Apr 28, 2016 at 6:32 PM, Ananth Gundabattula <
> >>>> agundabattula@gmail.com> wrote:
> >>>>
> >>>>> Hello David,
> >>>>>
> >>>>> Regarding as to why I did not use the archetype, I was building the
> >>>>> app for 3.3.0 version and I was not able to specify an archetype
> with the
> >>>>> version 3.3.0 when I attempted it. Perhaps there is no archetype for
> >>>>> version 3.3.0 ? I was not sure if the archetype version should match
> the
> >>>>> exact apex engine version . My presumption was archetype does not
> matter if
> >>>>> the packaging is done right. So i went ahead and created a simple
> maven
> >>>>> project.
> >>>>>
> >>>>> The maven assembly plugin used the appPackage.xml and the related
> >>>>> constructs to build the exact file layout of the app as mentioned in
> the
> >>>>> output above.
> >>>>>
> >>>>> I even attempted this time with MANIFEST.MF entry but to no avail.
> >>>>>
> >>>>> This brings me to a question. How does the engine know which is the
> >>>>> main class that is to be used for launching the Streaming App. I do
> have a
> >>>>> class that extends the StreaminApplication that is annotated with
> >>>>> ApplicationAnnotation but does not seem to be picking up. By this I
> mean, I
> >>>>> do not see a "launch" icon next to it. All I see is the top level
> packaged
> >>>>> name in the web console but nothing beneath it for me to trigger a
> launch.
> >>>>>
> >>>>> Regards,
> >>>>> Ananth
> >>>>>
> >>>>> On Fri, Apr 29, 2016 at 8:27 AM, David Yan <da...@datatorrent.com>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Ananth,
> >>>>>>
> >>>>>> The MANIFEST.MF file is important, and it's generated automatically
> >>>>>> if the archetype is used. For example, the MANFEST.MF file from the
> Pi demo
> >>>>>> in Malhar has this content:
> >>>>>>
> >>>>>> Manifest-Version: 1.0
> >>>>>> Archiver-Version: Plexus Archiver
> >>>>>> Built-By: david
> >>>>>> DT-App-Package-Display-Name: Apache Apex Malhar (incubating) Pi Demo
> >>>>>> Class-Path: lib/*.jar
> >>>>>> DT-Engine-Version: 3.4.0-incubating-SNAPSHOT
> >>>>>> Created-By: Apache Maven 3.3.9
> >>>>>> Build-Jdk: 1.8.0_92
> >>>>>> DT-App-Package-Description: Apex demo applications that calculate
> the
> >>>>>> value of Pi. This is a starting point to understand how Apex works.
> >>>>>> DT-App-Package-Group-Id: org.apache.apex
> >>>>>> DT-App-Package-Name: pi-demo
> >>>>>> DT-App-Package-Version: 3.4.0-incubating-SNAPSHOT
> >>>>>>
> >>>>>> Is there a reason why the archetype is not used in your project?
> >>>>>>
> >>>>>> David
> >>>>>>
> >>>>>> On Thu, Apr 28, 2016 at 3:10 PM, Ananth Gundabattula <
> >>>>>> agundabattula@gmail.com> wrote:
> >>>>>>
> >>>>>>> Hello David,
> >>>>>>>
> >>>>>>> Please find the unzip result at the end of this email. The jar is
> in
> >>>>>>> the app directory inside the apa.
> >>>>>>> As to why I did not use the archetype, I was building the app for
> >>>>>>> 3.3.0 version and I was not able to specify an archetype with the
> version
> >>>>>>> 3.3.0 when I attempted it. My presumption was archetype does not
> matter if
> >>>>>>> the packaging is done right. So i went ahead and created a simple
> maven
> >>>>>>> project.
> >>>>>>>
> >>>>>>> Here is the output: ( The MANIFEST.MF is an empty file in the
> >>>>>>> listing below ) . I also pasted the contents of the properties.xml
> file at
> >>>>>>> the end of this email for your reference.
> >>>>>>>
> >>>>>>> Archive: cassandra-events-1.2-SNAPSHOT.apa
> >>>>>>> testing: META-INF/ OK
> >>>>>>> testing: META-INF/MANIFEST.MF OK
> >>>>>>> testing: app/ OK
> >>>>>>> testing: app/cassandra-events-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/ OK
> >>>>>>> testing: lib/activation-1.1.jar OK
> >>>>>>> testing: lib/activemq-client-5.8.0.jar OK
> >>>>>>> testing: lib/antlr-2.7.7.jar OK
> >>>>>>> testing: lib/aopalliance-1.0.jar OK
> >>>>>>> testing: lib/app-commons-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/aspectjrt-1.8.7.jar OK
> >>>>>>> testing: lib/btf-1.2.jar OK
> >>>>>>> testing: lib/cassandra-driver-core-2.1.8.jar OK
> >>>>>>> testing: lib/commons-beanutils-1.8.3.jar OK
> >>>>>>> testing: lib/commons-compiler-2.7.8.jar OK
> >>>>>>> testing: lib/commons-lang3-3.1.jar OK
> >>>>>>> testing: lib/commons-logging-1.1.3.jar OK
> >>>>>>> testing: lib/curator-client-2.7.0.jar OK
> >>>>>>> testing: lib/curator-framework-2.7.0.jar OK
> >>>>>>> testing: lib/dom4j-1.6.1.jar OK
> >>>>>>> testing: lib/dwh-cassandra-store-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-dd-db-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-kafka-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-metastoredb-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/dwh-schemas-dd-1.2-SNAPSHOT.jar OK
> >>>>>>> testing: lib/evo-inflector-1.2.1.jar OK
> >>>>>>> testing: lib/fastutil-6.6.4.jar OK
> >>>>>>> testing: lib/geronimo-j2ee-management_1.1_spec-1.0.1.jar OK
> >>>>>>> testing: lib/geronimo-jms_1.1_spec-1.1.1.jar OK
> >>>>>>> testing: lib/guava-14.0.1.jar OK
> >>>>>>> testing: lib/hawtbuf-1.9.jar OK
> >>>>>>> testing: lib/hibernate-commons-annotations-5.0.0.Final.jar OK
> >>>>>>> testing: lib/hibernate-core-5.0.4.Final.jar OK
> >>>>>>> testing: lib/hibernate-entitymanager-5.0.4.Final.jar OK
> >>>>>>> testing: lib/hibernate-jpa-2.1-api-1.0.0.Final.jar OK
> >>>>>>> testing: lib/HikariCP-2.4.3.jar OK
> >>>>>>> testing: lib/jackson-annotations-2.6.4.jar OK
> >>>>>>> testing: lib/jackson-core-2.6.4.jar OK
> >>>>>>> testing: lib/jackson-coreutils-1.6.jar OK
> >>>>>>> testing: lib/jackson-databind-2.6.4.jar OK
> >>>>>>> testing: lib/jandex-2.0.0.CR1.jar OK
> >>>>>>> testing: lib/javassist-3.18.1-GA.jar OK
> >>>>>>> testing: lib/javax.mail-1.5.0.jar OK
> >>>>>>> testing: lib/jboss-logging-3.3.0.Final.jar OK
> >>>>>>> testing: lib/jcl-over-slf4j-1.7.13.jar OK
> >>>>>>> testing: lib/jersey-client-1.9.jar OK
> >>>>>>> testing: lib/jersey-core-1.9.jar OK
> >>>>>>> testing: lib/jetty-continuation-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-http-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-io-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-security-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-server-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-servlet-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-util-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jetty-websocket-8.1.10.v20130312.jar OK
> >>>>>>> testing: lib/jline-2.11.jar OK
> >>>>>>> testing: lib/jms-api-1.1-rev-1.jar OK
> >>>>>>> testing: lib/joda-time-2.7.jar OK
> >>>>>>> testing: lib/json-patch-1.7.jar OK
> >>>>>>> testing: lib/jsr305-2.0.1.jar OK
> >>>>>>> testing: lib/jta-1.1.jar OK
> >>>>>>> testing: lib/kafka-clients-0.9.0.0.jar OK
> >>>>>>> testing: lib/log4j-1.2.17.jar OK
> >>>>>>> testing: lib/logback-classic-1.1.5.jar OK
> >>>>>>> testing: lib/logback-core-1.1.5.jar OK
> >>>>>>> testing: lib/lz4-1.2.0.jar OK
> >>>>>>> testing: lib/malhar-contrib-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/malhar-kafka-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/malhar-library-3.3.1-incubating.jar OK
> >>>>>>> testing: lib/metrics-core-3.0.2.jar OK
> >>>>>>> testing: lib/msg-simple-1.1.jar OK
> >>>>>>> testing: lib/mysql-connector-java-5.1.36.jar OK
> >>>>>>> testing: lib/named-regexp-0.2.3.jar OK
> >>>>>>> testing: lib/netty-3.6.2.Final.jar OK
> >>>>>>> testing: lib/netty-buffer-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-codec-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-common-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-handler-4.0.27.Final.jar OK
> >>>>>>> testing: lib/netty-transport-4.0.27.Final.jar OK
> >>>>>>> testing: lib/opencsv-2.3.jar OK
> >>>>>>> testing: lib/slf4j-api-1.7.10.jar OK
> >>>>>>> testing: lib/snappy-java-1.1.1.7.jar OK
> >>>>>>> testing: lib/spring-aop-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-beans-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-context-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-core-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-commons-1.11.4.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-jpa-1.9.4.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-rest-core-2.4.2.RELEASE.jar OK
> >>>>>>> testing: lib/spring-data-rest-webmvc-2.4.2.RELEASE.jar OK
> >>>>>>> testing: lib/spring-expression-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-hateoas-0.19.0.RELEASE.jar OK
> >>>>>>> testing: lib/spring-jdbc-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-orm-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-plugin-core-1.2.0.RELEASE.jar OK
> >>>>>>> testing: lib/spring-tx-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-web-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/spring-webmvc-4.1.9.RELEASE.jar OK
> >>>>>>> testing: lib/super-csv-2.3.1.jar OK
> >>>>>>> testing: lib/super-csv-joda-2.3.1.jar OK
> >>>>>>> testing: lib/xml-apis-1.0.b2.jar OK
> >>>>>>> testing: lib/zookeeper-3.4.6.jar OK
> >>>>>>> testing: conf/ OK
> >>>>>>> testing: conf/properties.xml OK
> >>>>>>> No errors detected in compressed data of
> >>>>>>> cassandra-events-1.2-SNAPSHOT.apa.
> >>>>>>>
> >>>>>>> Contents of the properties.xml file under the conf folder is here:
> >>>>>>>
> >>>>>>> <?xml version="1.0"?>
> >>>>>>> <configuration>
> >>>>>>> <property>
> >>>>>>> <name>dt.application.CED.class</name>
> >>>>>>>
> <value>com.threatmetrix.dataplatform.storage.apps.cassandra.eventdetails.CassandraEventDetailsStreamingApp</value>
> >>>>>>> <description>An alias for the application</description>
> >>>>>>> </property>
> >>>>>>> <property>
> >>>>>>> <name>dt.operator.*.attr.MEMORY_MB</name>
> >>>>>>> <value>512</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.PARTITIONER</name>
> >>>>>>>
> <value>com.datatorrent.common.partitioner.StatelessPartitioner:10</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>>
> >>>>>>> <property>
> >>>>>>>
> <name>dt.application.CED.operator.eventDetailsFormatter.attr.APPLICATION_WINDOW_COUNT</name>
> >>>>>>> <value>20</value>
> >>>>>>> </property>
> >>>>>>>
> >>>>>>> </configuration>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> Regards,
> >>>>>>> Ananth
> >>>>>>>
> >>>>>>> On Fri, Apr 29, 2016 at 8:00 AM, David Yan <da...@datatorrent.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi Ananth,
> >>>>>>>>
> >>>>>>>> Can you do an "unzip -t" on your apa file and paste the output
> here?
> >>>>>>>> Make sure the jar containing the application classes should be in
> >>>>>>>> the app directory inside the apa.
> >>>>>>>>
> >>>>>>>> Can I know why the apex-app-archetype is not used in your apa
> >>>>>>>> project?
> >>>>>>>>
> >>>>>>>> David
> >>>>>>>>
> >>>>>>>> On Thu, Apr 28, 2016 at 2:53 PM, Ananth Gundabattula <
> >>>>>>>> agundabattula@gmail.com> wrote:
> >>>>>>>>
> >>>>>>>>> Hello All,
> >>>>>>>>>
> >>>>>>>>> I am having the following issue with my application. I have a
> >>>>>>>>> maven project that has a class called XYZ that extends
> >>>>>>>>> StreamingApplication.
> >>>>>>>>>
> >>>>>>>>> My maven packaging is ensuring that the final "apa" extension
> >>>>>>>>> package has the right folder structure if exploded on a disk.
> >>>>>>>>>
> >>>>>>>>> However I am not able to locate the Application name and the
> >>>>>>>>> "launch" button next to it after uploading the "apa" package via
> the
> >>>>>>>>> browser.
> >>>>>>>>>
> >>>>>>>>> I tried looking at the docs to see how to specify a class as an
> >>>>>>>>> App that can be launched from a packaged application but could
> not find
> >>>>>>>>> much. Could anyone of you please point me to the right way of
> associating a
> >>>>>>>>> class as an application that will get enabled with a launch
> button in the
> >>>>>>>>> web ui ? The class XYZ has the annotation called
> @ApplicationAnnotation on
> >>>>>>>>> top of it.
> >>>>>>>>>
> >>>>>>>>> Regards,
> >>>>>>>>> Ananth
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> >
>