You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@oodt.apache.org by "Mallder, Valerie" <Va...@jhuapl.edu> on 2014/11/19 22:41:50 UTC
logger flushing all the time
Hi All,
Hopefully this is an easy question .... How do I get the logger to stop "flush"ing in the middle of my log messages? It's becoming very difficult to read the information that I'm logging while debugging. Is there a output buffer size that I can increase? If so, where and how to do I increase it? If not, what else can I do to prevent it?
Thanks,
Val
OUTPUT: JEDI-INFO: feiToAr
Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream flush
INFO: chive.pl: executing command: cp 3D-061BE9A3DA-2014-307T15.19.01.out /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/ops/sci/2014/307/3D-061BE9A3DA-2014-307T15.19.01.out
OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp JEDI_2014307000237000_2014307003234999.sfdu /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/ops/eng/2014/307/JEDI_2014307000237000_2014307003234999.sfdu
OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp JEDI_2014307000311000_
Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream flush
INFO: 2014307003308999.ecsv /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/ops/eng/2014/307/JEDI_2014307000311000_2014307003308999.ecsv
OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp JEDI_2014307003235000_2014307010234999.sfdu /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/ops/eng/2014/307/JEDI_2014307003235000_2014307010234999.sfdu
OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp JEDI_2014307003309000_2014307010308999.ecsv /homes/malldva1/
Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream flush
INFO: project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/ops/eng/2014/307/JEDI_2014307003309000_2014307010308999.ecsv
OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp JEDI_2014307010235000_2014307013221999.sfdu /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/ops/eng/2014/307/JEDI_2014307010235000_2014307013221999.sfdu
OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp JEDI_2014307010309000_2014307013254999.ecsv /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-
Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream flush
INFO: deploy/data/archive/ops/eng/2014/307/JEDI_2014307010309000_2014307013254999.ecsv
OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp JEDI_2014307093130000_2014307100214999.sfdu /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/ops/eng/2014/307/JEDI_2014307093130000_2014307100214999.sfdu
OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp JEDI_2014307093158000_2014307100247999.ecsv /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/ops/eng/2014/307/J
Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream flush
INFO: EDI_2014307093158000_2014307100247999.ecsv
Nov 19, 2014 2:25:42 PM org.apache.oodt.cas.pge.PGETaskInstance createProductCrawler
INFO: pgeMetadata.getAllMetadata(PCS_ActionsIds) = [[MoveMetadataFileToBackupDir, MoveDataFileToFailureDir, MoveMetadataFileToFailureDir]]
Nov 19, 2014 2:25:42 PM org.apache.oodt.cas.pge.PGETaskInstance createProductCrawler
INFO: Num in list of actionIds = 3]
Nov 19, 2014 2:25:42 PM org.apache.oodt.cas.pge.PGETaskInstance updateStatus
Valerie A. Mallder
New Horizons Deputy Mission System Engineer
The Johns Hopkins University/Applied Physics Laboratory
11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
240-228-7846 (Office) 410-504-2233 (Blackberry)
Re: logger flushing all the time
Posted by "Mattmann, Chris A (3980)" <ch...@jpl.nasa.gov>.
Hey Val,
This is a tricky one. Because the output stream from jobs isn¹t
deterministic,
aka, we don¹t know when the PGE will generate output, we buffer and stream
the log output. It¹s not something that can be ³turned on² or off right
now,
but we could certainly think about doing that in the future.
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message-----
From: <Mallder>, Valerie <Va...@jhuapl.edu>
Reply-To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Date: Wednesday, November 19, 2014 at 10:41 PM
To: "dev@oodt.apache.org" <de...@oodt.apache.org>
Subject: logger flushing all the time
>Hi All,
>
>Hopefully this is an easy question .... How do I get the logger to stop
>"flush"ing in the middle of my log messages? It's becoming very
>difficult to read the information that I'm logging while debugging. Is
>there a output buffer size that I can increase? If so, where and how to
>do I increase it? If not, what else can I do to prevent it?
>
>Thanks,
>Val
>
>
>OUTPUT: JEDI-INFO: feiToAr
>Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream
>flush
>INFO: chive.pl: executing command: cp 3D-061BE9A3DA-2014-307T15.19.01.out
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/
>ops/sci/2014/307/3D-061BE9A3DA-2014-307T15.19.01.out
>OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp
>JEDI_2014307000237000_2014307003234999.sfdu
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/
>ops/eng/2014/307/JEDI_2014307000237000_2014307003234999.sfdu
>OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp
>JEDI_2014307000311000_
>Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream
>flush
>INFO: 2014307003308999.ecsv
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/
>ops/eng/2014/307/JEDI_2014307000311000_2014307003308999.ecsv
>OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp
>JEDI_2014307003235000_2014307010234999.sfdu
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/
>ops/eng/2014/307/JEDI_2014307003235000_2014307010234999.sfdu
>OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp
>JEDI_2014307003309000_2014307010308999.ecsv /homes/malldva1/
>Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream
>flush
>INFO:
>project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/ops/eng/2014/307
>/JEDI_2014307003309000_2014307010308999.ecsv
>OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp
>JEDI_2014307010235000_2014307013221999.sfdu
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/
>ops/eng/2014/307/JEDI_2014307010235000_2014307013221999.sfdu
>OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp
>JEDI_2014307010309000_2014307013254999.ecsv
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-
>Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream
>flush
>INFO:
>deploy/data/archive/ops/eng/2014/307/JEDI_2014307010309000_201430701325499
>9.ecsv
>OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp
>JEDI_2014307093130000_2014307100214999.sfdu
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/
>ops/eng/2014/307/JEDI_2014307093130000_2014307100214999.sfdu
>OUTPUT: JEDI-INFO: feiToArchive.pl: executing command: cp
>JEDI_2014307093158000_2014307100247999.ecsv
>/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive/
>ops/eng/2014/307/J
>Nov 19, 2014 2:25:40 PM org.apache.oodt.commons.io.LoggerOutputStream
>flush
>INFO: EDI_2014307093158000_2014307100247999.ecsv
>
>Nov 19, 2014 2:25:42 PM org.apache.oodt.cas.pge.PGETaskInstance
>createProductCrawler
>INFO: pgeMetadata.getAllMetadata(PCS_ActionsIds) =
>[[MoveMetadataFileToBackupDir, MoveDataFileToFailureDir,
>MoveMetadataFileToFailureDir]]
>Nov 19, 2014 2:25:42 PM org.apache.oodt.cas.pge.PGETaskInstance
>createProductCrawler
>INFO: Num in list of actionIds = 3]
>Nov 19, 2014 2:25:42 PM org.apache.oodt.cas.pge.PGETaskInstance
>updateStatus
>
>Valerie A. Mallder
>
>New Horizons Deputy Mission System Engineer
>The Johns Hopkins University/Applied Physics Laboratory
>11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
>240-228-7846 (Office) 410-504-2233 (Blackberry)
>