[jira] [Commented] (JCR-3568) Property.getBinary().getStream() files in tempDir not removed by InputStream.close() nor by Binary.dispose()

["Kevin Koslowski (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/JCR-3568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16012193#comment-16012193 ] 

Kevin Koslowski commented on JCR-3568:
--------------------------------------

I also stumbled upon this sneaky little -bastard- bug (jackrabbit 2.12.1). Job has written up to 9 GB of temp space until it crashed due to hard drive limit.

I've written a reflection-based "hack" that clears all of the tracked transient files, that I now call once in a while.

{code}
private int purgeTransientFiles()
  {
    try
    {
      TransientFileFactory factory = TransientFileFactory.getInstance();

      Field field = factory.getClass().getDeclaredField("trackedRefs");
      field.setAccessible(true);

      @SuppressWarnings("unchecked")
      Collection<PhantomReference<File>> fileRefs =
          (Collection<PhantomReference<File>>) field.get(factory);

      int count = 0;
      for (PhantomReference<?> ref : fileRefs.toArray(new PhantomReference[0]))
      {
        Method method = ref.getClass().getDeclaredMethod("delete");
        method.setAccessible(true);

        boolean trashed = (Boolean) method.invoke(ref);

        if (trashed)
        {
          count++;

          ref.clear();

          fileRefs.remove(ref);
        }
      }
      return count;
    }
    catch (ReflectiveOperationException e)
    {
      throw new RuntimeException(e);
    }
  }
{code}

This *cannot* used with concurrent access though, because from this point of view it is indistinguishable which files are still in use and should only be called on a clean session.

I would appreciate when this unnecessary behavior can be resolved any time soon.

> Property.getBinary().getStream() files in tempDir not removed by InputStream.close() nor by Binary.dispose() 
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: JCR-3568
>                 URL: https://issues.apache.org/jira/browse/JCR-3568
>             Project: Jackrabbit Content Repository
>          Issue Type: Bug
>          Components: jackrabbit-spi-commons, jackrabbit-webdav
>    Affects Versions: 2.4, 2.6, 2.7
>         Environment: Windows 7 Pro, Java 6.0.39, WebDAV, JCR 2.0
>            Reporter: Ulrich Schmidt
>         Attachments: jcr3568-r1482639.patch
>
>
> I need to inspect the the files stored to the jcr:data-Property in Node jcr:content which is a subnode of a nt:file-Node. Access mode is WebDAV using JCR 2.0-API.
> Jackrabbit does not drop the temp files created by the command Property.getBinary().getStream() by the closing instructions InputStream.close() nor Binary.dispose(). I get a RepositoryException "No space left on device" when the  tempspace becomes full.
> The executed code is;
> {code}
> public class DownloadLoopMain {
> 	private final static Logger LOGGER = LoggerFactory.getLogger("Test.DownloadLoopMain");
> 	String repository = "http://localhost:8080/server";
> 	String user="admin";
> 	String password="admin";
> 	Session session;
> 	File temp = new File(System.getProperty("java.io.tmpdir"));
> 	List<String> nodeList = new ArrayList<String>();
> 	public DownloadLoopMain() throws Exception {
> 		LOGGER.info("TempDir=" + temp.getPath());
> 		long totalsize=0;
> 		
> 		connectRepository();
> 		buildNodeList();
> 		List<String[]> tempfiles = getTempFiles(temp.listFiles());
> 		LOGGER.info("Start with number of files in Tempdir:" + tempfiles.size());
> 		for (String node : nodeList) {			
> 			LOGGER.info("Retrieve node " + node);
> 			Node currentNode=session.getNode(node);
> 			Node fileNode = currentNode.getNode("jcr:content");
> 			Property jcrdata = fileNode.getProperty("jcr:data");
> 			Binary fileBin=jcrdata.getBinary();
> 			long filesize=fileBin.getSize();
> 			totalsize+=filesize;
> 			InputStream file = fileBin.getStream();
> 			
> 			LOGGER.info("Now we have number of files in Tempdir:" + tempfiles.size());		
> 			
> 			List<String[]> newTempfiles = getTempFiles(temp.listFiles());
> 			// Display new files in temp-directory
> 			compareTempfiles("new", newTempfiles, tempfiles);
> 			
> 			// Display files gone from temp-directory
> 			compareTempfiles("gone", tempfiles, newTempfiles);
> 			
> 			tempfiles=newTempfiles;
> 			
> 			file.close();
> 			fileBin.dispose();
> 		}
> 	}
> 	
> 	
> 	/**
> 	 * Compare List of tempfiles.
> 	 * @param intend
> 	 * @param list1
> 	 * @param list2
> 	 */
> 	public void compareTempfiles(String intend, List<String[]> list1, List<String[]> list2 ) {
> 		for (String[] list1file : list1) {
> 			boolean known=false;
> 			for (int i=0; i< list2.size(); i++) {
> 				String[] list2file=list2.get(i);
> 				if (list1file[0].equals(list2file[0])) {
> 					known=true;
> 					break;
> 				}
> 			}
> 			if (!known) {
> 				LOGGER.info(intend + " tempfile=" + list1file[0]+ " " + list1file[1]);
> 			}
> 		}
> 	}
> 	public List<String[]> getTempFiles(File[] files) {
> 		List<String[]> filesList = new ArrayList<String[]>();
> 		for (File file : files) {
> 			String[] filedesc = new String[2];
> 			filedesc[0]=file.getName();
> 			filedesc[1]=file.length()+"";
> 			filesList.add(filedesc);
> 		}
> 		return filesList;
> 	}
> 	
> 	public void buildNodeList() throws IOException {
> 		String path ="E:/Jackrabbit/logs/Populate-Files.log";
> 		File file = new File(path);
> 		BufferedReader br = new BufferedReader(new FileReader(file));
> 		String line;
> 		while ((line=br.readLine())!=null) {
> 			nodeList.add(line);
> 		}
> 		br.close();
> 		String knownPath="/content/dam/ebase/privatebase/en/media/franz/2012/11_november/1211_VV-Interview_en_640x320_VP8.webm/jcr:content/renditions/original";		
> 	}
> 	private void connectRepository() throws Exception {
> 		LOGGER.info("About ot connect to " + this.repository);
> 		Repository repository = org.apache.jackrabbit.commons.JcrUtils.getRepository(this.repository);
> 		char[] password=this.password.toCharArray();
> 		Credentials cred= new SimpleCredentials(this.user, password);
> 		session = repository.login(cred);
> 		LOGGER.info("connected");
> 		LOGGER.info("Workspace: " + 
> 				session.getWorkspace().getName());
> 		boolean writeable = repository.getDescriptorValue(Repository.WRITE_SUPPORTED).getBoolean();
> 		LOGGER.info("Write supported=" + writeable);
> 	}
> 	public static void main(String[] args) throws Exception {
> 		LOGGER.info("Program-Start"); 
> 		DownloadLoopMain dlm = new  DownloadLoopMain();
> 	}
> }
> {code}
> Based on a list of repository nodes it retrieves node by node from the repository an inspects the temp-directory. After every step it displays the number of files in the tempdir, the name and size of the files which are new and those which are gone since last inspection.
> This proofs, that for each Property.getBinary().getStream() jackrabbit creates six identical files from the jcr:data-property in the tempdir and it proofs that these files are not deleted by the instructions:
> InputStream.close();
> Binary.dispose();
> I tested this by installing the latest jackrabbit-jar "org.apache.sling.launchpad-6-standalone.jar" from the Sling-Webpage and populating the repository with several files.
> Please provide a patch to make sure deletion of the temporary files is done in time when issuing the close and/or dispose instruction.
> Please reduce the number of identical temporary files created by the  Property.getBinary().getStream() instruction. In the test, each of the files was 90MB in size. For each of the instructions I need 550MB storage in the tempdir.
> Thanks in advance,
> Ulrich



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)