You are viewing a plain text version of this content. The canonical link for it is here.
Posted to infrastructure-issues@apache.org by "Tony Stevenson (JIRA)" <ji...@apache.org> on 2013/02/05 08:28:15 UTC
[jira] [Comment Edited] (INFRA-4380) Import Flex JIRA
[ https://issues.apache.org/jira/browse/INFRA-4380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13571113#comment-13571113 ]
Tony Stevenson edited comment on INFRA-4380 at 2/5/13 7:26 AM:
---------------------------------------------------------------
{quote}
Alex,
So I took your zip file onto a mac exploded it using the UI, and using unzip on the terminal.
They produced exactly the same output.
I then did a clean install of JIRA 5.0.4, and started the project import. See the attached catalina log file. This is from a clean start, and attempted import.
root@arcas [/x1/flex-jira-dance/jira/logs ] cat catalina.out| grep 'FLEX/FLEX-' | wc -l
28267
That shows we have 28267 missing files. Something isn't right here.
root@arcas [/x1/flex-jira-dance/jira/logs ] cat catalina.out| grep 'FLEX/FLEX-' | head -n3
2013-02-04 10:32:23,283 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143 /secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler] The attachment 'rotateBug.zip' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-75/10000'. It will not be imported.
2013-02-04 10:32:23,284 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143 /secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler] The attachment 'workaround.mxml' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-305/10001'. It will not be imported.
2013-02-04 10:32:23,284 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143 /secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler] The attachment 'BusyCursorTest.zip' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-313/10002'. It will not be imported.
That is 3 missing files at the start of the import.
root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-75/
total 628K
drwxr-xr-x 2 root root 4.0K Feb 4 09:52 .
drwxr-xr-x 12686 root root 356K Feb 4 09:56 ..
-rwxr-xr-x 1 root root 261K Feb 4 09:52 10186
root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-305
total 368K
drwxr-xr-x 2 root root 4.0K Feb 4 09:51 .
drwxr-xr-x 12686 root root 356K Feb 4 09:56 ..
-rwxr-xr-x 1 root root 1.8K Feb 4 09:51 10187
root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-313
total 380K
drwxr-xr-x 2 root root 4.0K Feb 4 09:53 .
drwxr-xr-x 12686 root root 356K Feb 4 09:56 ..
-rwxr-xr-x 1 root root 14K Feb 4 09:53 10188
As you can see none of the blob objects in the zip file match. None. Did you do some kind of transforms on this data?
The file that the import is looking for is taken from the xml import file (in the zip). Did you import the data into another version of jira to export only the FLEX project, i.e. filter out others. If so you have to import the attachments into that instance as they will get renumbered starting at 10000 - if not the attachments we have are from the original and mismatch your exported zip/xml file.
As it stands I cannot use the attachments.
The attachments on disk do not use files names as per the source file. They are stored as blobs on disk starting at 10000, and the filename is stored in SQL.
Alex, as far as I can tell. You did the export from Adobe, into a staging instance and then exported again after some filtering I suspect. That will only work if you import the the files as part of your import/export process.
So far I have spent over 75 hours on this since the start of the year. Is there anyway we can start the process again? You get a clean dump and attachments? If so please can we get it as a tar.gz ?
I hope this all makes sense. If not just shout.
{quote}
was (Author: pctony):
Alex,
So I took your zip file onto a mac exploded it using the UI, and using unzip on the terminal.
They produced exactly the same output.
I then did a clean install of JIRA 5.0.4, and started the project import. See the attached catalina log file. This is from a clean start, and attempted import.
root@arcas [/x1/flex-jira-dance/jira/logs ] cat catalina.out| grep 'FLEX/FLEX-' | wc -l
28267
That shows we have 28267 missing files. Something isn't right here.
root@arcas [/x1/flex-jira-dance/jira/logs ] cat catalina.out| grep 'FLEX/FLEX-' | head -n3
2013-02-04 10:32:23,283 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143 /secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler] The attachment 'rotateBug.zip' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-75/10000'. It will not be imported.
2013-02-04 10:32:23,284 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143 /secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler] The attachment 'workaround.mxml' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-305/10001'. It will not be imported.
2013-02-04 10:32:23,284 JiraTaskExectionThread-2 WARN pctony 632x30x1 pd7roa 128.232.134.143 /secure/admin/ProjectImportSelectProject.jspa [imports.project.handler.AttachmentFileValidatorHandler] The attachment 'BusyCursorTest.zip' does not exist at '/x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-313/10002'. It will not be imported.
That is 3 missing files at the start of the import.
root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-75/
total 628K
drwxr-xr-x 2 root root 4.0K Feb 4 09:52 .
drwxr-xr-x 12686 root root 356K Feb 4 09:56 ..
-rwxr-xr-x 1 root root 261K Feb 4 09:52 10186
root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-305
total 368K
drwxr-xr-x 2 root root 4.0K Feb 4 09:51 .
drwxr-xr-x 12686 root root 356K Feb 4 09:56 ..
-rwxr-xr-x 1 root root 1.8K Feb 4 09:51 10187
root@arcas [/x1/flex-jira-dance/jira/logs ] ls -lah /x1/flex-jira-dance/data/import/attachments/FLEX/FLEX-313
total 380K
drwxr-xr-x 2 root root 4.0K Feb 4 09:53 .
drwxr-xr-x 12686 root root 356K Feb 4 09:56 ..
-rwxr-xr-x 1 root root 14K Feb 4 09:53 10188
As you can see none of the blob objects in the zip file match. None. Did you do some kind of transforms on this data?
The file that the import is looking for is taken from the xml import file (in the zip). Did you import the data into another version of jira to export only the FLEX project, i.e. filter out others. If so you have to import the attachments into that instance as they will get renumbered starting at 10000 - if not the attachments we have are from the original and mismatch your exported zip/xml file.
As it stands I cannot use the attachments.
The attachments on disk do not use files names as per the source file. They are stored as blobs on disk starting at 10000, and the filename is stored in SQL.
Alex, as far as I can tell. You did the export from Adobe, into a staging instance and then exported again after some filtering I suspect. That will only work if you import the the files as part of your import/export process.
So far I have spent over 75 hours on this since the start of the year. Is there anyway we can start the process again? You get a clean dump and attachments? If so please can we get it as a tar.gz ?
I hope this all makes sense. If not just shout.
> Import Flex JIRA
> ----------------
>
> Key: INFRA-4380
> URL: https://issues.apache.org/jira/browse/INFRA-4380
> Project: Infrastructure
> Issue Type: Task
> Security Level: public(Regular issues)
> Components: JIRA
> Reporter: Alex Harui
> Assignee: Tony Stevenson
> Attachments: catalina.out, flex-20120201.xml
>
>
> Files representing the Adobe Flex JIRA database has been made available to Tony Stevenson and infrastructure@a.o via DropBox.
> The FLEX.zip is the 6GB zip of the attachments. According to an MD5 app I got off the internet, the checksum is: 12c4b40f0ada7aa7145e52d338399441
> The FlexSDKENTDMV.zip is the import file. It's checksum is: a8118ac7e2feb452eb6281091ec09582
> Please perform a Project Import of these files into the FLEX project at issues.a.o.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira