You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "C S (JIRA)" <ji...@apache.org> on 2011/05/27 15:35:47 UTC
[jira] [Created] (SOLR-2551) Checking dataimport.properties for
write access
Checking dataimport.properties for write access
-----------------------------------------------
Key: SOLR-2551
URL: https://issues.apache.org/jira/browse/SOLR-2551
Project: Solr
Issue Type: Improvement
Components: contrib - DataImportHandler
Affects Versions: 3.1, 1.4.1
Reporter: C S
Priority: Minor
A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2551) Checking dataimport.properties for
write access during startup
Posted by "C S (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
C S updated SOLR-2551:
----------------------
Summary: Checking dataimport.properties for write access during startup (was: Checking dataimport.properties for write access)
> Checking dataimport.properties for write access during startup
> --------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Priority: Minor
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Checking dataimport.properties for
write access during startup
Posted by "C S (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049000#comment-13049000 ]
C S commented on SOLR-2551:
---------------------------
I might be wrong but isn't the dataimport.properties written even if no delta-query is configured? So even if you'd just be interested in full imports, you'll run into an exception at the end of your full import when solr attempts to write the timestamp into dataimport.properties.
However, if that's not the case (i.e. a non-writable dataimport.properties does not break a full import), i'd suggest that the check if dataimport.properties is writable should only be done if a delta-query is defined, and in this case it should refuse to start the import.
> Checking dataimport.properties for write access during startup
> --------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063346#comment-13063346 ]
Shalin Shekhar Mangar commented on SOLR-2551:
---------------------------------------------
bq. Doesn't every test of delta functionality write to the dataimport.properties file?
Yes, it does but I don't think any of our tests rely on the contents of the properties file.
Ironically, the fact that the tests failed is proof that this feature works :)
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063511#comment-13063511 ]
Steven Rowe commented on SOLR-2551:
-----------------------------------
The [Lucene-Solr-tests-only-trunk Jenkins job|https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/] has run only once since the DIH tests were made to run sequentially (https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/9500/), so I'll delay closing this issue until it's successfully run 15 or 20 more times, which should take less than one day.
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2551) Check dataimport.properties for write
access before starting import
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-2551:
----------------------------------------
Fix Version/s: 4.0
3.3
Summary: Check dataimport.properties for write access before starting import (was: Checking dataimport.properties for write access during startup)
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063335#comment-13063335 ]
Shalin Shekhar Mangar commented on SOLR-2551:
---------------------------------------------
The tests run in parallel so for the brief time that the dataimport.properties is set to read-only, other tests can fail. There is no way to change the location of the properties file so I don't see a way to fix this. Should we ignore this particular test?
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063352#comment-13063352 ]
Chris Male commented on SOLR-2551:
----------------------------------
Okay, so, given that basically every test writes to this file, what are our options?
To me it seems since the file is getting written too (whether we rely on the contents or not), this could get in the way of another test. So perhaps we need to pull the checkWritablePersistFile method out for awhile and re-assess how to achieve the same functionality in a way the tests can handle?
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063366#comment-13063366 ]
Steven Rowe commented on SOLR-2551:
-----------------------------------
I'll switch the DIH tests to run sequentially. The benchmark module does this by setting the {{tests.threadspercpu}} property to zero.
Here's the patch:
{{
Index: solr/contrib/dataimporthandler/build.xml
===================================================================
--- solr/contrib/dataimporthandler/build.xml (revision 1145189)
+++ solr/contrib/dataimporthandler/build.xml (working copy)
@@ -23,6 +23,9 @@
Data Import Handler
</description>
+ <!-- the tests have some parallel problems: writability to single copy of dataimport.properties -->
+ <property name="tests.threadspercpu" value="0"/>
+
<import file="../contrib-build.xml"/>
</project>
}}
Committing shortly.
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Reopened] (SOLR-2551) Check dataimport.properties for write
access before starting import
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Male reopened SOLR-2551:
------------------------------
We're seeing some [test failures|https://builds.apache.org/job/Lucene-Solr-tests-only-trunk/9469/testReport/junit/org.apache.solr.handler.dataimport/TestSqlEntityProcessorDelta2/testCompositePk_DeltaImport_replace_delete/] on Jenkins due to exceptions being thrown by DataImporter.checkWritablePersistFile.
Any chance you can shed some light on the issue Shalin?
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (SOLR-2551) Check dataimport.properties for write
access before starting import
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar resolved SOLR-2551.
-----------------------------------------
Resolution: Fixed
Committed revision 1135954 on trunk and 1135956 on branch_3x.
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063355#comment-13063355 ]
Shalin Shekhar Mangar commented on SOLR-2551:
---------------------------------------------
Yes, lets disable this test for now. I don't think it is even worth testing. I guess I just had too much time that day :)
Another option could be to run the DIH tests sequentially.
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Updated] (SOLR-2551) Checking dataimport.properties for
write access during startup
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar updated SOLR-2551:
----------------------------------------
Attachment: SOLR-2551.patch
Patch to fail imports if the data config supports delta import but the dataimport.properties file is not writable.
Added a test to verify.
> Checking dataimport.properties for write access during startup
> --------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063424#comment-13063424 ]
Shalin Shekhar Mangar commented on SOLR-2551:
---------------------------------------------
Thanks Steven!
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Resolved] (SOLR-2551) Check dataimport.properties for write
access before starting import
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Male resolved SOLR-2551.
------------------------------
Resolution: Fixed
Seems running the tests sequentially has solved the error.
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 4.0, 3.3
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Issue Comment Edited] (SOLR-2551) Check
dataimport.properties for write access before starting import
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063366#comment-13063366 ]
Steven Rowe edited comment on SOLR-2551 at 7/11/11 2:39 PM:
------------------------------------------------------------
I'll switch the DIH tests to run sequentially. The benchmark module does this by setting the {{tests.threadspercpu}} property to zero.
Here's the patch:
{noformat}
Index: solr/contrib/dataimporthandler/build.xml
===================================================================
--- solr/contrib/dataimporthandler/build.xml (revision 1145189)
+++ solr/contrib/dataimporthandler/build.xml (working copy)
@@ -23,6 +23,9 @@
Data Import Handler
</description>
+ <!-- the tests have some parallel problems: writability to single copy of dataimport.properties -->
+ <property name="tests.threadspercpu" value="0"/>
+
<import file="../contrib-build.xml"/>
</project>
{noformat}
Committing shortly.
was (Author: steve_rowe):
I'll switch the DIH tests to run sequentially. The benchmark module does this by setting the {{tests.threadspercpu}} property to zero.
Here's the patch:
{{
Index: solr/contrib/dataimporthandler/build.xml
===================================================================
--- solr/contrib/dataimporthandler/build.xml (revision 1145189)
+++ solr/contrib/dataimporthandler/build.xml (working copy)
@@ -23,6 +23,9 @@
Data Import Handler
</description>
+ <!-- the tests have some parallel problems: writability to single copy of dataimport.properties -->
+ <property name="tests.threadspercpu" value="0"/>
+
<import file="../contrib-build.xml"/>
</project>
}}
Committing shortly.
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13062866#comment-13062866 ]
Shalin Shekhar Mangar commented on SOLR-2551:
---------------------------------------------
I'm taking a look. Thanks Chris.
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063339#comment-13063339 ]
Chris Male commented on SOLR-2551:
----------------------------------
Amazingly, I came to the exact same conclusion at the same time. Doesn't every test of delta functionality write to the dataimport.properties file?
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Chris Male (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063354#comment-13063354 ]
Chris Male commented on SOLR-2551:
----------------------------------
or alternatively, we could make the DIH tests run sequentially, so we don't hit this problem.
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Check dataimport.properties for
write access before starting import
Posted by "Steven Rowe (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13063369#comment-13063369 ]
Steven Rowe commented on SOLR-2551:
-----------------------------------
Committed the patch to run DIH tests sequentially:
- r1145194: trunk
- r1145196: branch_3x
> Check dataimport.properties for write access before starting import
> -------------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
> Fix For: 3.3, 4.0
>
> Attachments: SOLR-2551.patch
>
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Assigned] (SOLR-2551) Checking dataimport.properties for
write access during startup
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shalin Shekhar Mangar reassigned SOLR-2551:
-------------------------------------------
Assignee: Shalin Shekhar Mangar
> Checking dataimport.properties for write access during startup
> --------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Assignee: Shalin Shekhar Mangar
> Priority: Minor
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
[jira] [Commented] (SOLR-2551) Checking dataimport.properties for
write access during startup
Posted by "Shalin Shekhar Mangar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/SOLR-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048662#comment-13048662 ]
Shalin Shekhar Mangar commented on SOLR-2551:
---------------------------------------------
If DIH is unable to write dataimport.properties, it logs a message saying so. We don't want the import to fail in this case because a lot of people use only full-imports which does not need the dataimport.properties at all. What do you suggest?
> Checking dataimport.properties for write access during startup
> --------------------------------------------------------------
>
> Key: SOLR-2551
> URL: https://issues.apache.org/jira/browse/SOLR-2551
> Project: Solr
> Issue Type: Improvement
> Components: contrib - DataImportHandler
> Affects Versions: 1.4.1, 3.1
> Reporter: C S
> Priority: Minor
>
> A common mistake is that the /conf (respectively the dataimport.properties) file is not writable for solr. It would be great if that were detected on starting a dataimport job.
> Currently and import might grind away for days and fail if it can't write its timestamp to the dataimport.properties file.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org