You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@asterixdb.apache.org by Yingyi Bu <bu...@gmail.com> on 2016/10/14 21:22:04 UTC
Re: [jira] [Commented] (ASTERIXDB-1694) Fail running Tweet Feed on
Cluster of 16 nodes (while succeed on 4 nodes)
>> Log4J in Twitter4j doesn't initialize properly.
Can you explain that a little bit more?
What exceptions do you hit, "class not found", or sth. else?
Thx!
On Fri, Oct 14, 2016 at 2:20 PM, Wail Alkowaileet (JIRA) <ji...@apache.org>
wrote:
>
> [ https://issues.apache.org/jira/browse/ASTERIXDB-1694?
> page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&
> focusedCommentId=15576516#comment-15576516 ]
>
> Wail Alkowaileet commented on ASTERIXDB-1694:
> ---------------------------------------------
>
> After a long meeting, Mingda and I located the problem. Log4J in Twitter4j
> doesn't initialize properly.
> I don't know why that's the case. I suspect that there's something wrong
> in the deployment of AsterixDB.
>
> > Fail running Tweet Feed on Cluster of 16 nodes (while succeed on 4 nodes)
> > ------------------------------------------------------------
> -------------
> >
> > Key: ASTERIXDB-1694
> > URL: https://issues.apache.org/
> jira/browse/ASTERIXDB-1694
> > Project: Apache AsterixDB
> > Issue Type: Bug
> > Components: Feeds
> > Environment: asterix-server-0.8.9-SNAPSHOT-binary-assembly run
> on cluster
> > Reporter: Mingda Li
> > Assignee: Xikui Wang
> > Original Estimate: 24h
> > Remaining Estimate: 24h
> >
> > Recently, I am trying to run the data feed query on a cluster of 16
> nodes. All the query run well without error. But each time after I
> disconnect the feed, there is none of tweet data stored in dataverse.
> However, when I try to run other AQL queries, the cluster can work well.
> And I have ever used 4 nodes cluster to load Tweet data successfully. I
> also checked the log file and find no error there. This is wired. Does
> anyone know why? Has anyone ever used the data feed function on a cluster
> of 16 nodes or more?
> > I am using a asterix-server-0.8.9-SNAPSHOT-binary-assembly to configure
> cluster compiled by myself.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>
Re: [jira] [Commented] (ASTERIXDB-1694) Fail running Tweet Feed on
Cluster of 16 nodes (while succeed on 4 nodes)
Posted by Xikui Wang <xi...@uci.edu>.
Hi Devs,
I'd like to put note on this problem, in case anyone hit this problem again
or have an insight of what's causing the problem.
Basically, this problem is caused by the initialization fail of Log4J in
Twitter4j(so not our problem :D). The solution is enclosed in Mingda's last
reply.
However, the cause for this problem is still unclear to us, especially the
case that it works on 4 nodes but not on 16 nodes.
Code snippet [1] shows how Twitter4J creates logger. It scans all possible
libraries and found the one that is available first. The feed adaptor
in AsterixDB will run on only one of nodes in cluster. According to Mingda,
if we shutdown the node with log problem, the new node that
adaptor running on will have the same problem.
As for a permanent solution, probably I can turn off the Twitter4j logger
in code, or hardcode the configuration to avoid this problem in the future.
If anyone have better idea, please let me know. Thanks! :)
[1]
https://github.com/yusuke/twitter4j/blob/4ebca9da71b271775624b11b5197af99a57bf175/twitter4j-core/src/internal-logging/java/twitter4j/Logger.java
On Wed, Oct 19, 2016 at 8:17 PM, mingda li <li...@gmail.com> wrote:
> Dear all,
> Good news!
> For the official version of AsterixDB, the datafeed problem for Twitter can
> be solved for 16 nodes by adding a log4j.properties
> to asterix-server-0.8.9-SNAPSHOT-binary-assembly/etc and /repo. I will try
> Wail's version. And see why this can work for 4 nodes without adding the
> log4j.properties file.
>
> BTW, the log4j.properties file is as following, if you may need someday:
> # Set root logger level to DEBUG and its only appender to A1.
> log4j.rootLogger=DEBUG, A1
>
> # A1 is set to be a ConsoleAppender.
> log4j.appender.A1=org.apache.log4j.ConsoleAppender
>
> # A1 uses PatternLayout.
> log4j.appender.A1.layout=org.apache.log4j.PatternLayout
> log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
>
>
>
> On Wed, Oct 19, 2016 at 8:07 PM, mingda li <li...@gmail.com> wrote:
>
> > BTW, I tried to run AsterixDB's official version of Tweet feed and also
> > meet similar problem in the Node 14.
> > I follow Xikui's suggestion to add a log4j.properties
> > in asterix-server-0.8.9-SNAPSHOT-binary-assembly/etc and /repo. I
> checked
> > the log file and find it is changed to the attachment. It seems begin
> > catch tweet but failed.
> >
> > nc-red15.log
> > <https://drive.google.com/file/d/0B-3JraLWXVVGVVJ0T0VHaVpGYzA/
> view?usp=drive_web>
> >
> >
> >
> > On Wed, Oct 19, 2016 at 12:51 PM, mingda li <li...@gmail.com>
> > wrote:
> >
> >> En, that is good suggestion.
> >> Since this is not my version of AsterixDB, we should ask Wail if he have
> >> ever set something related to twitter4j's log.
> >>
> >> @Wail, have you ever set such thing?
> >>
> >> On Tue, Oct 18, 2016 at 6:19 PM, Xikui Wang <xi...@uci.edu> wrote:
> >>
> >>> It looks like the log4j in Twitter4J is not correctly initialized[1].
> Did
> >>> you customize the log4j in Twitter4J configuration in your system like
> >>> this[2]? By default, it's printed to standard output.
> >>>
> >>>
> >>> [1]
> >>> http://activemq.apache.org/log4j-warn-no-appenders-could-be-
> >>> found-for-logger.html
> >>> [2] http://twitter4j.org/en/configuration.html#logger
> >>>
> >>> On Tue, Oct 18, 2016 at 5:42 PM, mingda li <li...@gmail.com>
> >>> wrote:
> >>>
> >>> > Hi,
> >>> >
> >>> > When I start 16 nodes, I found the 15th node has log file different
> >>> from
> >>> > others as following.
> >>> >
> >>> > Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.NCDriver main
> >>> > SEVERE: Setting uncaught exception handler
> >>> org.apache.hyracks.api.lifecyc
> >>> > le.LifeCycleComponentManager@73f792cf
> >>> > Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.
> >>> > NodeControllerService
> >>> > start
> >>> > INFO: Starting NodeControllerService
> >>> > Oct 18, 2016 5:23:10 PM
> >>> > org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint
> >>> > start
> >>> > INFO: Starting Asterix node controller: red15
> >>> > log4j:WARN No appenders could be found for logger
> >>> > (twitter4j.TwitterStreamImpl).
> >>> > log4j:WARN Please initialize the log4j system properly.
> >>> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
> >>> for
> >>> > more info.
> >>> > Oct 18, 2016 5:25:20 PM org.apache.hyracks.control.com
> >>> mon.work.WorkQueue$
> >>> > WorkerThread auditWaitsAndBlocks
> >>> > WARNING: Work StartTasks waited 0 times (~0ms), blocked 1 times
> (~0ms)
> >>> > Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 229376 bytes
> >>> >
> >>> > The other nodes' log files are without the problem: ( I have tried to
> >>> give
> >>> > up the node, but there will be a new node with the same problem)
> >>> > Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.NCDriver main
> >>> > SEVERE: Setting uncaught exception handler
> >>> > org.apache.hyracks.api.lifecycle.LifeCycleComponentManager@73f792cf
> >>> > Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.
> >>> > NodeControllerService
> >>> > start
> >>> > INFO: Starting NodeControllerService
> >>> > Oct 18, 2016 5:23:09 PM
> >>> > org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint start
> >>> > INFO: Starting Asterix node controller: red6
> >>> > Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 229376 bytes
> >>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> > Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
> >>> > WARNING: Freeing leaked 131072 bytes
> >>> >
> >>> > The cluster can work well except the data feeding function on Twitter
> >>> (load
> >>> > for a while but no items loaded from Twitter). The data feeding
> >>> function of
> >>> > RSS can also work.
> >>> > The problem is wired. Does anyone have idea?
> >>> >
> >>> > Bests,
> >>> > Mingda
> >>> >
> >>> >
> >>> > On Tue, Oct 18, 2016 at 10:17 AM, mingda li <li...@gmail.com>
> >>> > wrote:
> >>> >
> >>> > > Oh, sure. I just came back to school and could repeat the
> experiment
> >>> on
> >>> > > our cluster today. I will try.
> >>> > >
> >>> > > Bests,
> >>> > > Mingda
> >>> > >
> >>> > > On Tue, Oct 18, 2016 at 12:25 AM, Wail Alkowaileet <
> >>> wael.y.k@gmail.com>
> >>> > > wrote:
> >>> > >
> >>> > >> Unfortunately I couldn't reproduce the issue in our tiny cluster.
> >>> > >>
> >>> > >> @Mingda if you can post the logs it might be helpful.
> >>> > >>
> >>> > >> On Sat, Oct 15, 2016 at 12:22 AM, Yingyi Bu <bu...@gmail.com>
> >>> wrote:
> >>> > >>
> >>> > >> > >> Log4J in Twitter4j doesn't initialize properly.
> >>> > >> >
> >>> > >> > Can you explain that a little bit more?
> >>> > >> > What exceptions do you hit, "class not found", or sth. else?
> >>> > >> > Thx!
> >>> > >> >
> >>> > >> > On Fri, Oct 14, 2016 at 2:20 PM, Wail Alkowaileet (JIRA) <
> >>> > >> jira@apache.org>
> >>> > >> > wrote:
> >>> > >> >
> >>> > >> > >
> >>> > >> > > [ https://issues.apache.org/jira/browse/ASTERIXDB-1694?
> >>> > >> > > page=com.atlassian.jira.plugin.system.issuetabpanels:comment
> >>> > >> -tabpanel&
> >>> > >> > > focusedCommentId=15576516#comment-15576516 ]
> >>> > >> > >
> >>> > >> > > Wail Alkowaileet commented on ASTERIXDB-1694:
> >>> > >> > > ---------------------------------------------
> >>> > >> > >
> >>> > >> > > After a long meeting, Mingda and I located the problem. Log4J
> in
> >>> > >> > Twitter4j
> >>> > >> > > doesn't initialize properly.
> >>> > >> > > I don't know why that's the case. I suspect that there's
> >>> something
> >>> > >> wrong
> >>> > >> > > in the deployment of AsterixDB.
> >>> > >> > >
> >>> > >> > > > Fail running Tweet Feed on Cluster of 16 nodes (while
> succeed
> >>> on 4
> >>> > >> > nodes)
> >>> > >> > > > ------------------------------
> ------------------------------
> >>> > >> > > -------------
> >>> > >> > > >
> >>> > >> > > > Key: ASTERIXDB-1694
> >>> > >> > > > URL: https://issues.apache.org/
> >>> > >> > > jira/browse/ASTERIXDB-1694
> >>> > >> > > > Project: Apache AsterixDB
> >>> > >> > > > Issue Type: Bug
> >>> > >> > > > Components: Feeds
> >>> > >> > > > Environment: asterix-server-0.8.9-SNAPSHOT-
> >>> > binary-assembly
> >>> > >> run
> >>> > >> > > on cluster
> >>> > >> > > > Reporter: Mingda Li
> >>> > >> > > > Assignee: Xikui Wang
> >>> > >> > > > Original Estimate: 24h
> >>> > >> > > > Remaining Estimate: 24h
> >>> > >> > > >
> >>> > >> > > > Recently, I am trying to run the data feed query on a
> cluster
> >>> of
> >>> > 16
> >>> > >> > > nodes. All the query run well without error. But each time
> >>> after I
> >>> > >> > > disconnect the feed, there is none of tweet data stored in
> >>> > dataverse.
> >>> > >> > > However, when I try to run other AQL queries, the cluster can
> >>> work
> >>> > >> well.
> >>> > >> > > And I have ever used 4 nodes cluster to load Tweet data
> >>> > successfully.
> >>> > >> I
> >>> > >> > > also checked the log file and find no error there. This is
> >>> wired.
> >>> > Does
> >>> > >> > > anyone know why? Has anyone ever used the data feed function
> on
> >>> a
> >>> > >> cluster
> >>> > >> > > of 16 nodes or more?
> >>> > >> > > > I am using a asterix-server-0.8.9-SNAPSHOT-binary-assembly
> to
> >>> > >> > configure
> >>> > >> > > cluster compiled by myself.
> >>> > >> > >
> >>> > >> > >
> >>> > >> > >
> >>> > >> > > --
> >>> > >> > > This message was sent by Atlassian JIRA
> >>> > >> > > (v6.3.4#6332)
> >>> > >> > >
> >>> > >> >
> >>> > >>
> >>> > >>
> >>> > >>
> >>> > >> --
> >>> > >>
> >>> > >> *Regards,*
> >>> > >> Wail Alkowaileet
> >>> > >>
> >>> > >
> >>> > >
> >>> >
> >>>
> >>
> >>
> >
>
Re: [jira] [Commented] (ASTERIXDB-1694) Fail running Tweet Feed on
Cluster of 16 nodes (while succeed on 4 nodes)
Posted by mingda li <li...@gmail.com>.
Dear all,
Good news!
For the official version of AsterixDB, the datafeed problem for Twitter can
be solved for 16 nodes by adding a log4j.properties
to asterix-server-0.8.9-SNAPSHOT-binary-assembly/etc and /repo. I will try
Wail's version. And see why this can work for 4 nodes without adding the
log4j.properties file.
BTW, the log4j.properties file is as following, if you may need someday:
# Set root logger level to DEBUG and its only appender to A1.
log4j.rootLogger=DEBUG, A1
# A1 is set to be a ConsoleAppender.
log4j.appender.A1=org.apache.log4j.ConsoleAppender
# A1 uses PatternLayout.
log4j.appender.A1.layout=org.apache.log4j.PatternLayout
log4j.appender.A1.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
On Wed, Oct 19, 2016 at 8:07 PM, mingda li <li...@gmail.com> wrote:
> BTW, I tried to run AsterixDB's official version of Tweet feed and also
> meet similar problem in the Node 14.
> I follow Xikui's suggestion to add a log4j.properties
> in asterix-server-0.8.9-SNAPSHOT-binary-assembly/etc and /repo. I checked
> the log file and find it is changed to the attachment. It seems begin
> catch tweet but failed.
>
> nc-red15.log
> <https://drive.google.com/file/d/0B-3JraLWXVVGVVJ0T0VHaVpGYzA/view?usp=drive_web>
>
>
>
> On Wed, Oct 19, 2016 at 12:51 PM, mingda li <li...@gmail.com>
> wrote:
>
>> En, that is good suggestion.
>> Since this is not my version of AsterixDB, we should ask Wail if he have
>> ever set something related to twitter4j's log.
>>
>> @Wail, have you ever set such thing?
>>
>> On Tue, Oct 18, 2016 at 6:19 PM, Xikui Wang <xi...@uci.edu> wrote:
>>
>>> It looks like the log4j in Twitter4J is not correctly initialized[1]. Did
>>> you customize the log4j in Twitter4J configuration in your system like
>>> this[2]? By default, it's printed to standard output.
>>>
>>>
>>> [1]
>>> http://activemq.apache.org/log4j-warn-no-appenders-could-be-
>>> found-for-logger.html
>>> [2] http://twitter4j.org/en/configuration.html#logger
>>>
>>> On Tue, Oct 18, 2016 at 5:42 PM, mingda li <li...@gmail.com>
>>> wrote:
>>>
>>> > Hi,
>>> >
>>> > When I start 16 nodes, I found the 15th node has log file different
>>> from
>>> > others as following.
>>> >
>>> > Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.NCDriver main
>>> > SEVERE: Setting uncaught exception handler
>>> org.apache.hyracks.api.lifecyc
>>> > le.LifeCycleComponentManager@73f792cf
>>> > Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.
>>> > NodeControllerService
>>> > start
>>> > INFO: Starting NodeControllerService
>>> > Oct 18, 2016 5:23:10 PM
>>> > org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint
>>> > start
>>> > INFO: Starting Asterix node controller: red15
>>> > log4j:WARN No appenders could be found for logger
>>> > (twitter4j.TwitterStreamImpl).
>>> > log4j:WARN Please initialize the log4j system properly.
>>> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
>>> for
>>> > more info.
>>> > Oct 18, 2016 5:25:20 PM org.apache.hyracks.control.com
>>> mon.work.WorkQueue$
>>> > WorkerThread auditWaitsAndBlocks
>>> > WARNING: Work StartTasks waited 0 times (~0ms), blocked 1 times (~0ms)
>>> > Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 229376 bytes
>>> >
>>> > The other nodes' log files are without the problem: ( I have tried to
>>> give
>>> > up the node, but there will be a new node with the same problem)
>>> > Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.NCDriver main
>>> > SEVERE: Setting uncaught exception handler
>>> > org.apache.hyracks.api.lifecycle.LifeCycleComponentManager@73f792cf
>>> > Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.
>>> > NodeControllerService
>>> > start
>>> > INFO: Starting NodeControllerService
>>> > Oct 18, 2016 5:23:09 PM
>>> > org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint start
>>> > INFO: Starting Asterix node controller: red6
>>> > Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 229376 bytes
>>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> > Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
>>> > WARNING: Freeing leaked 131072 bytes
>>> >
>>> > The cluster can work well except the data feeding function on Twitter
>>> (load
>>> > for a while but no items loaded from Twitter). The data feeding
>>> function of
>>> > RSS can also work.
>>> > The problem is wired. Does anyone have idea?
>>> >
>>> > Bests,
>>> > Mingda
>>> >
>>> >
>>> > On Tue, Oct 18, 2016 at 10:17 AM, mingda li <li...@gmail.com>
>>> > wrote:
>>> >
>>> > > Oh, sure. I just came back to school and could repeat the experiment
>>> on
>>> > > our cluster today. I will try.
>>> > >
>>> > > Bests,
>>> > > Mingda
>>> > >
>>> > > On Tue, Oct 18, 2016 at 12:25 AM, Wail Alkowaileet <
>>> wael.y.k@gmail.com>
>>> > > wrote:
>>> > >
>>> > >> Unfortunately I couldn't reproduce the issue in our tiny cluster.
>>> > >>
>>> > >> @Mingda if you can post the logs it might be helpful.
>>> > >>
>>> > >> On Sat, Oct 15, 2016 at 12:22 AM, Yingyi Bu <bu...@gmail.com>
>>> wrote:
>>> > >>
>>> > >> > >> Log4J in Twitter4j doesn't initialize properly.
>>> > >> >
>>> > >> > Can you explain that a little bit more?
>>> > >> > What exceptions do you hit, "class not found", or sth. else?
>>> > >> > Thx!
>>> > >> >
>>> > >> > On Fri, Oct 14, 2016 at 2:20 PM, Wail Alkowaileet (JIRA) <
>>> > >> jira@apache.org>
>>> > >> > wrote:
>>> > >> >
>>> > >> > >
>>> > >> > > [ https://issues.apache.org/jira/browse/ASTERIXDB-1694?
>>> > >> > > page=com.atlassian.jira.plugin.system.issuetabpanels:comment
>>> > >> -tabpanel&
>>> > >> > > focusedCommentId=15576516#comment-15576516 ]
>>> > >> > >
>>> > >> > > Wail Alkowaileet commented on ASTERIXDB-1694:
>>> > >> > > ---------------------------------------------
>>> > >> > >
>>> > >> > > After a long meeting, Mingda and I located the problem. Log4J in
>>> > >> > Twitter4j
>>> > >> > > doesn't initialize properly.
>>> > >> > > I don't know why that's the case. I suspect that there's
>>> something
>>> > >> wrong
>>> > >> > > in the deployment of AsterixDB.
>>> > >> > >
>>> > >> > > > Fail running Tweet Feed on Cluster of 16 nodes (while succeed
>>> on 4
>>> > >> > nodes)
>>> > >> > > > ------------------------------------------------------------
>>> > >> > > -------------
>>> > >> > > >
>>> > >> > > > Key: ASTERIXDB-1694
>>> > >> > > > URL: https://issues.apache.org/
>>> > >> > > jira/browse/ASTERIXDB-1694
>>> > >> > > > Project: Apache AsterixDB
>>> > >> > > > Issue Type: Bug
>>> > >> > > > Components: Feeds
>>> > >> > > > Environment: asterix-server-0.8.9-SNAPSHOT-
>>> > binary-assembly
>>> > >> run
>>> > >> > > on cluster
>>> > >> > > > Reporter: Mingda Li
>>> > >> > > > Assignee: Xikui Wang
>>> > >> > > > Original Estimate: 24h
>>> > >> > > > Remaining Estimate: 24h
>>> > >> > > >
>>> > >> > > > Recently, I am trying to run the data feed query on a cluster
>>> of
>>> > 16
>>> > >> > > nodes. All the query run well without error. But each time
>>> after I
>>> > >> > > disconnect the feed, there is none of tweet data stored in
>>> > dataverse.
>>> > >> > > However, when I try to run other AQL queries, the cluster can
>>> work
>>> > >> well.
>>> > >> > > And I have ever used 4 nodes cluster to load Tweet data
>>> > successfully.
>>> > >> I
>>> > >> > > also checked the log file and find no error there. This is
>>> wired.
>>> > Does
>>> > >> > > anyone know why? Has anyone ever used the data feed function on
>>> a
>>> > >> cluster
>>> > >> > > of 16 nodes or more?
>>> > >> > > > I am using a asterix-server-0.8.9-SNAPSHOT-binary-assembly to
>>> > >> > configure
>>> > >> > > cluster compiled by myself.
>>> > >> > >
>>> > >> > >
>>> > >> > >
>>> > >> > > --
>>> > >> > > This message was sent by Atlassian JIRA
>>> > >> > > (v6.3.4#6332)
>>> > >> > >
>>> > >> >
>>> > >>
>>> > >>
>>> > >>
>>> > >> --
>>> > >>
>>> > >> *Regards,*
>>> > >> Wail Alkowaileet
>>> > >>
>>> > >
>>> > >
>>> >
>>>
>>
>>
>
Re: [jira] [Commented] (ASTERIXDB-1694) Fail running Tweet Feed on
Cluster of 16 nodes (while succeed on 4 nodes)
Posted by mingda li <li...@gmail.com>.
BTW, I tried to run AsterixDB's official version of Tweet feed and also
meet similar problem in the Node 14.
I follow Xikui's suggestion to add a log4j.properties
in asterix-server-0.8.9-SNAPSHOT-binary-assembly/etc and /repo. I checked
the log file and find it is changed to the attachment. It seems begin
catch tweet but failed.
nc-red15.log
<https://drive.google.com/file/d/0B-3JraLWXVVGVVJ0T0VHaVpGYzA/view?usp=drive_web>
On Wed, Oct 19, 2016 at 12:51 PM, mingda li <li...@gmail.com> wrote:
> En, that is good suggestion.
> Since this is not my version of AsterixDB, we should ask Wail if he have
> ever set something related to twitter4j's log.
>
> @Wail, have you ever set such thing?
>
> On Tue, Oct 18, 2016 at 6:19 PM, Xikui Wang <xi...@uci.edu> wrote:
>
>> It looks like the log4j in Twitter4J is not correctly initialized[1]. Did
>> you customize the log4j in Twitter4J configuration in your system like
>> this[2]? By default, it's printed to standard output.
>>
>>
>> [1]
>> http://activemq.apache.org/log4j-warn-no-appenders-could-be-
>> found-for-logger.html
>> [2] http://twitter4j.org/en/configuration.html#logger
>>
>> On Tue, Oct 18, 2016 at 5:42 PM, mingda li <li...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > When I start 16 nodes, I found the 15th node has log file different from
>> > others as following.
>> >
>> > Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.NCDriver main
>> > SEVERE: Setting uncaught exception handler
>> org.apache.hyracks.api.lifecyc
>> > le.LifeCycleComponentManager@73f792cf
>> > Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.
>> > NodeControllerService
>> > start
>> > INFO: Starting NodeControllerService
>> > Oct 18, 2016 5:23:10 PM
>> > org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint
>> > start
>> > INFO: Starting Asterix node controller: red15
>> > log4j:WARN No appenders could be found for logger
>> > (twitter4j.TwitterStreamImpl).
>> > log4j:WARN Please initialize the log4j system properly.
>> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig
>> for
>> > more info.
>> > Oct 18, 2016 5:25:20 PM org.apache.hyracks.control.com
>> mon.work.WorkQueue$
>> > WorkerThread auditWaitsAndBlocks
>> > WARNING: Work StartTasks waited 0 times (~0ms), blocked 1 times (~0ms)
>> > Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 229376 bytes
>> >
>> > The other nodes' log files are without the problem: ( I have tried to
>> give
>> > up the node, but there will be a new node with the same problem)
>> > Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.NCDriver main
>> > SEVERE: Setting uncaught exception handler
>> > org.apache.hyracks.api.lifecycle.LifeCycleComponentManager@73f792cf
>> > Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.
>> > NodeControllerService
>> > start
>> > INFO: Starting NodeControllerService
>> > Oct 18, 2016 5:23:09 PM
>> > org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint start
>> > INFO: Starting Asterix node controller: red6
>> > Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 229376 bytes
>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> > Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
>> > WARNING: Freeing leaked 131072 bytes
>> >
>> > The cluster can work well except the data feeding function on Twitter
>> (load
>> > for a while but no items loaded from Twitter). The data feeding
>> function of
>> > RSS can also work.
>> > The problem is wired. Does anyone have idea?
>> >
>> > Bests,
>> > Mingda
>> >
>> >
>> > On Tue, Oct 18, 2016 at 10:17 AM, mingda li <li...@gmail.com>
>> > wrote:
>> >
>> > > Oh, sure. I just came back to school and could repeat the experiment
>> on
>> > > our cluster today. I will try.
>> > >
>> > > Bests,
>> > > Mingda
>> > >
>> > > On Tue, Oct 18, 2016 at 12:25 AM, Wail Alkowaileet <
>> wael.y.k@gmail.com>
>> > > wrote:
>> > >
>> > >> Unfortunately I couldn't reproduce the issue in our tiny cluster.
>> > >>
>> > >> @Mingda if you can post the logs it might be helpful.
>> > >>
>> > >> On Sat, Oct 15, 2016 at 12:22 AM, Yingyi Bu <bu...@gmail.com>
>> wrote:
>> > >>
>> > >> > >> Log4J in Twitter4j doesn't initialize properly.
>> > >> >
>> > >> > Can you explain that a little bit more?
>> > >> > What exceptions do you hit, "class not found", or sth. else?
>> > >> > Thx!
>> > >> >
>> > >> > On Fri, Oct 14, 2016 at 2:20 PM, Wail Alkowaileet (JIRA) <
>> > >> jira@apache.org>
>> > >> > wrote:
>> > >> >
>> > >> > >
>> > >> > > [ https://issues.apache.org/jira/browse/ASTERIXDB-1694?
>> > >> > > page=com.atlassian.jira.plugin.system.issuetabpanels:comment
>> > >> -tabpanel&
>> > >> > > focusedCommentId=15576516#comment-15576516 ]
>> > >> > >
>> > >> > > Wail Alkowaileet commented on ASTERIXDB-1694:
>> > >> > > ---------------------------------------------
>> > >> > >
>> > >> > > After a long meeting, Mingda and I located the problem. Log4J in
>> > >> > Twitter4j
>> > >> > > doesn't initialize properly.
>> > >> > > I don't know why that's the case. I suspect that there's
>> something
>> > >> wrong
>> > >> > > in the deployment of AsterixDB.
>> > >> > >
>> > >> > > > Fail running Tweet Feed on Cluster of 16 nodes (while succeed
>> on 4
>> > >> > nodes)
>> > >> > > > ------------------------------------------------------------
>> > >> > > -------------
>> > >> > > >
>> > >> > > > Key: ASTERIXDB-1694
>> > >> > > > URL: https://issues.apache.org/
>> > >> > > jira/browse/ASTERIXDB-1694
>> > >> > > > Project: Apache AsterixDB
>> > >> > > > Issue Type: Bug
>> > >> > > > Components: Feeds
>> > >> > > > Environment: asterix-server-0.8.9-SNAPSHOT-
>> > binary-assembly
>> > >> run
>> > >> > > on cluster
>> > >> > > > Reporter: Mingda Li
>> > >> > > > Assignee: Xikui Wang
>> > >> > > > Original Estimate: 24h
>> > >> > > > Remaining Estimate: 24h
>> > >> > > >
>> > >> > > > Recently, I am trying to run the data feed query on a cluster
>> of
>> > 16
>> > >> > > nodes. All the query run well without error. But each time after
>> I
>> > >> > > disconnect the feed, there is none of tweet data stored in
>> > dataverse.
>> > >> > > However, when I try to run other AQL queries, the cluster can
>> work
>> > >> well.
>> > >> > > And I have ever used 4 nodes cluster to load Tweet data
>> > successfully.
>> > >> I
>> > >> > > also checked the log file and find no error there. This is wired.
>> > Does
>> > >> > > anyone know why? Has anyone ever used the data feed function on a
>> > >> cluster
>> > >> > > of 16 nodes or more?
>> > >> > > > I am using a asterix-server-0.8.9-SNAPSHOT-binary-assembly to
>> > >> > configure
>> > >> > > cluster compiled by myself.
>> > >> > >
>> > >> > >
>> > >> > >
>> > >> > > --
>> > >> > > This message was sent by Atlassian JIRA
>> > >> > > (v6.3.4#6332)
>> > >> > >
>> > >> >
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >>
>> > >> *Regards,*
>> > >> Wail Alkowaileet
>> > >>
>> > >
>> > >
>> >
>>
>
>
Re: [jira] [Commented] (ASTERIXDB-1694) Fail running Tweet Feed on
Cluster of 16 nodes (while succeed on 4 nodes)
Posted by mingda li <li...@gmail.com>.
En, that is good suggestion.
Since this is not my version of AsterixDB, we should ask Wail if he have
ever set something related to twitter4j's log.
@Wail, have you ever set such thing?
On Tue, Oct 18, 2016 at 6:19 PM, Xikui Wang <xi...@uci.edu> wrote:
> It looks like the log4j in Twitter4J is not correctly initialized[1]. Did
> you customize the log4j in Twitter4J configuration in your system like
> this[2]? By default, it's printed to standard output.
>
>
> [1]
> http://activemq.apache.org/log4j-warn-no-appenders-could-
> be-found-for-logger.html
> [2] http://twitter4j.org/en/configuration.html#logger
>
> On Tue, Oct 18, 2016 at 5:42 PM, mingda li <li...@gmail.com> wrote:
>
> > Hi,
> >
> > When I start 16 nodes, I found the 15th node has log file different from
> > others as following.
> >
> > Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.NCDriver main
> > SEVERE: Setting uncaught exception handler org.apache.hyracks.api.lifecyc
> > le.LifeCycleComponentManager@73f792cf
> > Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.
> > NodeControllerService
> > start
> > INFO: Starting NodeControllerService
> > Oct 18, 2016 5:23:10 PM
> > org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint
> > start
> > INFO: Starting Asterix node controller: red15
> > log4j:WARN No appenders could be found for logger
> > (twitter4j.TwitterStreamImpl).
> > log4j:WARN Please initialize the log4j system properly.
> > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> > more info.
> > Oct 18, 2016 5:25:20 PM org.apache.hyracks.control.
> common.work.WorkQueue$
> > WorkerThread auditWaitsAndBlocks
> > WARNING: Work StartTasks waited 0 times (~0ms), blocked 1 times (~0ms)
> > Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 229376 bytes
> >
> > The other nodes' log files are without the problem: ( I have tried to
> give
> > up the node, but there will be a new node with the same problem)
> > Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.NCDriver main
> > SEVERE: Setting uncaught exception handler
> > org.apache.hyracks.api.lifecycle.LifeCycleComponentManager@73f792cf
> > Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.
> > NodeControllerService
> > start
> > INFO: Starting NodeControllerService
> > Oct 18, 2016 5:23:09 PM
> > org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint start
> > INFO: Starting Asterix node controller: red6
> > Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 229376 bytes
> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> > Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
> > WARNING: Freeing leaked 131072 bytes
> >
> > The cluster can work well except the data feeding function on Twitter
> (load
> > for a while but no items loaded from Twitter). The data feeding function
> of
> > RSS can also work.
> > The problem is wired. Does anyone have idea?
> >
> > Bests,
> > Mingda
> >
> >
> > On Tue, Oct 18, 2016 at 10:17 AM, mingda li <li...@gmail.com>
> > wrote:
> >
> > > Oh, sure. I just came back to school and could repeat the experiment on
> > > our cluster today. I will try.
> > >
> > > Bests,
> > > Mingda
> > >
> > > On Tue, Oct 18, 2016 at 12:25 AM, Wail Alkowaileet <wael.y.k@gmail.com
> >
> > > wrote:
> > >
> > >> Unfortunately I couldn't reproduce the issue in our tiny cluster.
> > >>
> > >> @Mingda if you can post the logs it might be helpful.
> > >>
> > >> On Sat, Oct 15, 2016 at 12:22 AM, Yingyi Bu <bu...@gmail.com>
> wrote:
> > >>
> > >> > >> Log4J in Twitter4j doesn't initialize properly.
> > >> >
> > >> > Can you explain that a little bit more?
> > >> > What exceptions do you hit, "class not found", or sth. else?
> > >> > Thx!
> > >> >
> > >> > On Fri, Oct 14, 2016 at 2:20 PM, Wail Alkowaileet (JIRA) <
> > >> jira@apache.org>
> > >> > wrote:
> > >> >
> > >> > >
> > >> > > [ https://issues.apache.org/jira/browse/ASTERIXDB-1694?
> > >> > > page=com.atlassian.jira.plugin.system.issuetabpanels:comment
> > >> -tabpanel&
> > >> > > focusedCommentId=15576516#comment-15576516 ]
> > >> > >
> > >> > > Wail Alkowaileet commented on ASTERIXDB-1694:
> > >> > > ---------------------------------------------
> > >> > >
> > >> > > After a long meeting, Mingda and I located the problem. Log4J in
> > >> > Twitter4j
> > >> > > doesn't initialize properly.
> > >> > > I don't know why that's the case. I suspect that there's something
> > >> wrong
> > >> > > in the deployment of AsterixDB.
> > >> > >
> > >> > > > Fail running Tweet Feed on Cluster of 16 nodes (while succeed
> on 4
> > >> > nodes)
> > >> > > > ------------------------------------------------------------
> > >> > > -------------
> > >> > > >
> > >> > > > Key: ASTERIXDB-1694
> > >> > > > URL: https://issues.apache.org/
> > >> > > jira/browse/ASTERIXDB-1694
> > >> > > > Project: Apache AsterixDB
> > >> > > > Issue Type: Bug
> > >> > > > Components: Feeds
> > >> > > > Environment: asterix-server-0.8.9-SNAPSHOT-
> > binary-assembly
> > >> run
> > >> > > on cluster
> > >> > > > Reporter: Mingda Li
> > >> > > > Assignee: Xikui Wang
> > >> > > > Original Estimate: 24h
> > >> > > > Remaining Estimate: 24h
> > >> > > >
> > >> > > > Recently, I am trying to run the data feed query on a cluster of
> > 16
> > >> > > nodes. All the query run well without error. But each time after I
> > >> > > disconnect the feed, there is none of tweet data stored in
> > dataverse.
> > >> > > However, when I try to run other AQL queries, the cluster can work
> > >> well.
> > >> > > And I have ever used 4 nodes cluster to load Tweet data
> > successfully.
> > >> I
> > >> > > also checked the log file and find no error there. This is wired.
> > Does
> > >> > > anyone know why? Has anyone ever used the data feed function on a
> > >> cluster
> > >> > > of 16 nodes or more?
> > >> > > > I am using a asterix-server-0.8.9-SNAPSHOT-binary-assembly to
> > >> > configure
> > >> > > cluster compiled by myself.
> > >> > >
> > >> > >
> > >> > >
> > >> > > --
> > >> > > This message was sent by Atlassian JIRA
> > >> > > (v6.3.4#6332)
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >>
> > >> *Regards,*
> > >> Wail Alkowaileet
> > >>
> > >
> > >
> >
>
Re: [jira] [Commented] (ASTERIXDB-1694) Fail running Tweet Feed on
Cluster of 16 nodes (while succeed on 4 nodes)
Posted by Xikui Wang <xi...@uci.edu>.
It looks like the log4j in Twitter4J is not correctly initialized[1]. Did
you customize the log4j in Twitter4J configuration in your system like
this[2]? By default, it's printed to standard output.
[1]
http://activemq.apache.org/log4j-warn-no-appenders-could-be-found-for-logger.html
[2] http://twitter4j.org/en/configuration.html#logger
On Tue, Oct 18, 2016 at 5:42 PM, mingda li <li...@gmail.com> wrote:
> Hi,
>
> When I start 16 nodes, I found the 15th node has log file different from
> others as following.
>
> Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.NCDriver main
> SEVERE: Setting uncaught exception handler org.apache.hyracks.api.lifecyc
> le.LifeCycleComponentManager@73f792cf
> Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.
> NodeControllerService
> start
> INFO: Starting NodeControllerService
> Oct 18, 2016 5:23:10 PM
> org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint
> start
> INFO: Starting Asterix node controller: red15
> log4j:WARN No appenders could be found for logger
> (twitter4j.TwitterStreamImpl).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
> Oct 18, 2016 5:25:20 PM org.apache.hyracks.control.common.work.WorkQueue$
> WorkerThread auditWaitsAndBlocks
> WARNING: Work StartTasks waited 0 times (~0ms), blocked 1 times (~0ms)
> Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 229376 bytes
>
> The other nodes' log files are without the problem: ( I have tried to give
> up the node, but there will be a new node with the same problem)
> Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.NCDriver main
> SEVERE: Setting uncaught exception handler
> org.apache.hyracks.api.lifecycle.LifeCycleComponentManager@73f792cf
> Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.
> NodeControllerService
> start
> INFO: Starting NodeControllerService
> Oct 18, 2016 5:23:09 PM
> org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint start
> INFO: Starting Asterix node controller: red6
> Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 229376 bytes
> Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
> Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
> WARNING: Freeing leaked 131072 bytes
>
> The cluster can work well except the data feeding function on Twitter (load
> for a while but no items loaded from Twitter). The data feeding function of
> RSS can also work.
> The problem is wired. Does anyone have idea?
>
> Bests,
> Mingda
>
>
> On Tue, Oct 18, 2016 at 10:17 AM, mingda li <li...@gmail.com>
> wrote:
>
> > Oh, sure. I just came back to school and could repeat the experiment on
> > our cluster today. I will try.
> >
> > Bests,
> > Mingda
> >
> > On Tue, Oct 18, 2016 at 12:25 AM, Wail Alkowaileet <wa...@gmail.com>
> > wrote:
> >
> >> Unfortunately I couldn't reproduce the issue in our tiny cluster.
> >>
> >> @Mingda if you can post the logs it might be helpful.
> >>
> >> On Sat, Oct 15, 2016 at 12:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
> >>
> >> > >> Log4J in Twitter4j doesn't initialize properly.
> >> >
> >> > Can you explain that a little bit more?
> >> > What exceptions do you hit, "class not found", or sth. else?
> >> > Thx!
> >> >
> >> > On Fri, Oct 14, 2016 at 2:20 PM, Wail Alkowaileet (JIRA) <
> >> jira@apache.org>
> >> > wrote:
> >> >
> >> > >
> >> > > [ https://issues.apache.org/jira/browse/ASTERIXDB-1694?
> >> > > page=com.atlassian.jira.plugin.system.issuetabpanels:comment
> >> -tabpanel&
> >> > > focusedCommentId=15576516#comment-15576516 ]
> >> > >
> >> > > Wail Alkowaileet commented on ASTERIXDB-1694:
> >> > > ---------------------------------------------
> >> > >
> >> > > After a long meeting, Mingda and I located the problem. Log4J in
> >> > Twitter4j
> >> > > doesn't initialize properly.
> >> > > I don't know why that's the case. I suspect that there's something
> >> wrong
> >> > > in the deployment of AsterixDB.
> >> > >
> >> > > > Fail running Tweet Feed on Cluster of 16 nodes (while succeed on 4
> >> > nodes)
> >> > > > ------------------------------------------------------------
> >> > > -------------
> >> > > >
> >> > > > Key: ASTERIXDB-1694
> >> > > > URL: https://issues.apache.org/
> >> > > jira/browse/ASTERIXDB-1694
> >> > > > Project: Apache AsterixDB
> >> > > > Issue Type: Bug
> >> > > > Components: Feeds
> >> > > > Environment: asterix-server-0.8.9-SNAPSHOT-
> binary-assembly
> >> run
> >> > > on cluster
> >> > > > Reporter: Mingda Li
> >> > > > Assignee: Xikui Wang
> >> > > > Original Estimate: 24h
> >> > > > Remaining Estimate: 24h
> >> > > >
> >> > > > Recently, I am trying to run the data feed query on a cluster of
> 16
> >> > > nodes. All the query run well without error. But each time after I
> >> > > disconnect the feed, there is none of tweet data stored in
> dataverse.
> >> > > However, when I try to run other AQL queries, the cluster can work
> >> well.
> >> > > And I have ever used 4 nodes cluster to load Tweet data
> successfully.
> >> I
> >> > > also checked the log file and find no error there. This is wired.
> Does
> >> > > anyone know why? Has anyone ever used the data feed function on a
> >> cluster
> >> > > of 16 nodes or more?
> >> > > > I am using a asterix-server-0.8.9-SNAPSHOT-binary-assembly to
> >> > configure
> >> > > cluster compiled by myself.
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > This message was sent by Atlassian JIRA
> >> > > (v6.3.4#6332)
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >>
> >> *Regards,*
> >> Wail Alkowaileet
> >>
> >
> >
>
Re: [jira] [Commented] (ASTERIXDB-1694) Fail running Tweet Feed on
Cluster of 16 nodes (while succeed on 4 nodes)
Posted by mingda li <li...@gmail.com>.
Hi,
When I start 16 nodes, I found the 15th node has log file different from
others as following.
Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.NCDriver main
SEVERE: Setting uncaught exception handler org.apache.hyracks.api.lifecyc
le.LifeCycleComponentManager@73f792cf
Oct 18, 2016 5:23:10 PM org.apache.hyracks.control.nc.NodeControllerService
start
INFO: Starting NodeControllerService
Oct 18, 2016 5:23:10 PM
org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint
start
INFO: Starting Asterix node controller: red15
log4j:WARN No appenders could be found for logger
(twitter4j.TwitterStreamImpl).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
more info.
Oct 18, 2016 5:25:20 PM org.apache.hyracks.control.common.work.WorkQueue$
WorkerThread auditWaitsAndBlocks
WARNING: Work StartTasks waited 0 times (~0ms), blocked 1 times (~0ms)
Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 229376 bytes
The other nodes' log files are without the problem: ( I have tried to give
up the node, but there will be a new node with the same problem)
Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.NCDriver main
SEVERE: Setting uncaught exception handler
org.apache.hyracks.api.lifecycle.LifeCycleComponentManager@73f792cf
Oct 18, 2016 5:23:09 PM org.apache.hyracks.control.nc.NodeControllerService
start
INFO: Starting NodeControllerService
Oct 18, 2016 5:23:09 PM
org.apache.asterix.hyracks.bootstrap.NCApplicationEntryPoint start
INFO: Starting Asterix node controller: red6
Oct 18, 2016 5:25:31 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:32 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 229376 bytes
Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:33 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
Oct 18, 2016 5:25:47 PM org.apache.hyracks.control.nc.Joblet close
WARNING: Freeing leaked 131072 bytes
The cluster can work well except the data feeding function on Twitter (load
for a while but no items loaded from Twitter). The data feeding function of
RSS can also work.
The problem is wired. Does anyone have idea?
Bests,
Mingda
On Tue, Oct 18, 2016 at 10:17 AM, mingda li <li...@gmail.com> wrote:
> Oh, sure. I just came back to school and could repeat the experiment on
> our cluster today. I will try.
>
> Bests,
> Mingda
>
> On Tue, Oct 18, 2016 at 12:25 AM, Wail Alkowaileet <wa...@gmail.com>
> wrote:
>
>> Unfortunately I couldn't reproduce the issue in our tiny cluster.
>>
>> @Mingda if you can post the logs it might be helpful.
>>
>> On Sat, Oct 15, 2016 at 12:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
>>
>> > >> Log4J in Twitter4j doesn't initialize properly.
>> >
>> > Can you explain that a little bit more?
>> > What exceptions do you hit, "class not found", or sth. else?
>> > Thx!
>> >
>> > On Fri, Oct 14, 2016 at 2:20 PM, Wail Alkowaileet (JIRA) <
>> jira@apache.org>
>> > wrote:
>> >
>> > >
>> > > [ https://issues.apache.org/jira/browse/ASTERIXDB-1694?
>> > > page=com.atlassian.jira.plugin.system.issuetabpanels:comment
>> -tabpanel&
>> > > focusedCommentId=15576516#comment-15576516 ]
>> > >
>> > > Wail Alkowaileet commented on ASTERIXDB-1694:
>> > > ---------------------------------------------
>> > >
>> > > After a long meeting, Mingda and I located the problem. Log4J in
>> > Twitter4j
>> > > doesn't initialize properly.
>> > > I don't know why that's the case. I suspect that there's something
>> wrong
>> > > in the deployment of AsterixDB.
>> > >
>> > > > Fail running Tweet Feed on Cluster of 16 nodes (while succeed on 4
>> > nodes)
>> > > > ------------------------------------------------------------
>> > > -------------
>> > > >
>> > > > Key: ASTERIXDB-1694
>> > > > URL: https://issues.apache.org/
>> > > jira/browse/ASTERIXDB-1694
>> > > > Project: Apache AsterixDB
>> > > > Issue Type: Bug
>> > > > Components: Feeds
>> > > > Environment: asterix-server-0.8.9-SNAPSHOT-binary-assembly
>> run
>> > > on cluster
>> > > > Reporter: Mingda Li
>> > > > Assignee: Xikui Wang
>> > > > Original Estimate: 24h
>> > > > Remaining Estimate: 24h
>> > > >
>> > > > Recently, I am trying to run the data feed query on a cluster of 16
>> > > nodes. All the query run well without error. But each time after I
>> > > disconnect the feed, there is none of tweet data stored in dataverse.
>> > > However, when I try to run other AQL queries, the cluster can work
>> well.
>> > > And I have ever used 4 nodes cluster to load Tweet data successfully.
>> I
>> > > also checked the log file and find no error there. This is wired. Does
>> > > anyone know why? Has anyone ever used the data feed function on a
>> cluster
>> > > of 16 nodes or more?
>> > > > I am using a asterix-server-0.8.9-SNAPSHOT-binary-assembly to
>> > configure
>> > > cluster compiled by myself.
>> > >
>> > >
>> > >
>> > > --
>> > > This message was sent by Atlassian JIRA
>> > > (v6.3.4#6332)
>> > >
>> >
>>
>>
>>
>> --
>>
>> *Regards,*
>> Wail Alkowaileet
>>
>
>
Re: [jira] [Commented] (ASTERIXDB-1694) Fail running Tweet Feed on
Cluster of 16 nodes (while succeed on 4 nodes)
Posted by mingda li <li...@gmail.com>.
Oh, sure. I just came back to school and could repeat the experiment on our
cluster today. I will try.
Bests,
Mingda
On Tue, Oct 18, 2016 at 12:25 AM, Wail Alkowaileet <wa...@gmail.com>
wrote:
> Unfortunately I couldn't reproduce the issue in our tiny cluster.
>
> @Mingda if you can post the logs it might be helpful.
>
> On Sat, Oct 15, 2016 at 12:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
>
> > >> Log4J in Twitter4j doesn't initialize properly.
> >
> > Can you explain that a little bit more?
> > What exceptions do you hit, "class not found", or sth. else?
> > Thx!
> >
> > On Fri, Oct 14, 2016 at 2:20 PM, Wail Alkowaileet (JIRA) <
> jira@apache.org>
> > wrote:
> >
> > >
> > > [ https://issues.apache.org/jira/browse/ASTERIXDB-1694?
> > > page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&
> > > focusedCommentId=15576516#comment-15576516 ]
> > >
> > > Wail Alkowaileet commented on ASTERIXDB-1694:
> > > ---------------------------------------------
> > >
> > > After a long meeting, Mingda and I located the problem. Log4J in
> > Twitter4j
> > > doesn't initialize properly.
> > > I don't know why that's the case. I suspect that there's something
> wrong
> > > in the deployment of AsterixDB.
> > >
> > > > Fail running Tweet Feed on Cluster of 16 nodes (while succeed on 4
> > nodes)
> > > > ------------------------------------------------------------
> > > -------------
> > > >
> > > > Key: ASTERIXDB-1694
> > > > URL: https://issues.apache.org/
> > > jira/browse/ASTERIXDB-1694
> > > > Project: Apache AsterixDB
> > > > Issue Type: Bug
> > > > Components: Feeds
> > > > Environment: asterix-server-0.8.9-SNAPSHOT-binary-assembly
> run
> > > on cluster
> > > > Reporter: Mingda Li
> > > > Assignee: Xikui Wang
> > > > Original Estimate: 24h
> > > > Remaining Estimate: 24h
> > > >
> > > > Recently, I am trying to run the data feed query on a cluster of 16
> > > nodes. All the query run well without error. But each time after I
> > > disconnect the feed, there is none of tweet data stored in dataverse.
> > > However, when I try to run other AQL queries, the cluster can work
> well.
> > > And I have ever used 4 nodes cluster to load Tweet data successfully. I
> > > also checked the log file and find no error there. This is wired. Does
> > > anyone know why? Has anyone ever used the data feed function on a
> cluster
> > > of 16 nodes or more?
> > > > I am using a asterix-server-0.8.9-SNAPSHOT-binary-assembly to
> > configure
> > > cluster compiled by myself.
> > >
> > >
> > >
> > > --
> > > This message was sent by Atlassian JIRA
> > > (v6.3.4#6332)
> > >
> >
>
>
>
> --
>
> *Regards,*
> Wail Alkowaileet
>
Re: [jira] [Commented] (ASTERIXDB-1694) Fail running Tweet Feed on
Cluster of 16 nodes (while succeed on 4 nodes)
Posted by Wail Alkowaileet <wa...@gmail.com>.
Unfortunately I couldn't reproduce the issue in our tiny cluster.
@Mingda if you can post the logs it might be helpful.
On Sat, Oct 15, 2016 at 12:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
> >> Log4J in Twitter4j doesn't initialize properly.
>
> Can you explain that a little bit more?
> What exceptions do you hit, "class not found", or sth. else?
> Thx!
>
> On Fri, Oct 14, 2016 at 2:20 PM, Wail Alkowaileet (JIRA) <ji...@apache.org>
> wrote:
>
> >
> > [ https://issues.apache.org/jira/browse/ASTERIXDB-1694?
> > page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&
> > focusedCommentId=15576516#comment-15576516 ]
> >
> > Wail Alkowaileet commented on ASTERIXDB-1694:
> > ---------------------------------------------
> >
> > After a long meeting, Mingda and I located the problem. Log4J in
> Twitter4j
> > doesn't initialize properly.
> > I don't know why that's the case. I suspect that there's something wrong
> > in the deployment of AsterixDB.
> >
> > > Fail running Tweet Feed on Cluster of 16 nodes (while succeed on 4
> nodes)
> > > ------------------------------------------------------------
> > -------------
> > >
> > > Key: ASTERIXDB-1694
> > > URL: https://issues.apache.org/
> > jira/browse/ASTERIXDB-1694
> > > Project: Apache AsterixDB
> > > Issue Type: Bug
> > > Components: Feeds
> > > Environment: asterix-server-0.8.9-SNAPSHOT-binary-assembly run
> > on cluster
> > > Reporter: Mingda Li
> > > Assignee: Xikui Wang
> > > Original Estimate: 24h
> > > Remaining Estimate: 24h
> > >
> > > Recently, I am trying to run the data feed query on a cluster of 16
> > nodes. All the query run well without error. But each time after I
> > disconnect the feed, there is none of tweet data stored in dataverse.
> > However, when I try to run other AQL queries, the cluster can work well.
> > And I have ever used 4 nodes cluster to load Tweet data successfully. I
> > also checked the log file and find no error there. This is wired. Does
> > anyone know why? Has anyone ever used the data feed function on a cluster
> > of 16 nodes or more?
> > > I am using a asterix-server-0.8.9-SNAPSHOT-binary-assembly to
> configure
> > cluster compiled by myself.
> >
> >
> >
> > --
> > This message was sent by Atlassian JIRA
> > (v6.3.4#6332)
> >
>
--
*Regards,*
Wail Alkowaileet