You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Gianmarco De Francisci Morales (JIRA)" <ji...@apache.org> on 2012/10/24 18:14:12 UTC
[jira] [Created] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Gianmarco De Francisci Morales created PIG-3001:
---------------------------------------------------
Summary: TestExecutableManager.testAddJobConfToEnv fails randomly
Key: PIG-3001
URL: https://issues.apache.org/jira/browse/PIG-3001
Project: Pig
Issue Type: Bug
Reporter: Gianmarco De Francisci Morales
Priority: Minor
The test in the Summary fails intermittently.
This is due to using a random number generator without seeding it.
We should avoid stochastic tests.
Furthermore, the test itself is ill conceived.
Here the failure summary:
{code}
12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
{code}
{code}
Error Message:
There should be no remaining pairs in the included map
Stacktrace:
junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Gianmarco De Francisci Morales (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485710#comment-13485710 ]
Gianmarco De Francisci Morales commented on PIG-3001:
-----------------------------------------------------
Thanks Rohini, I had missed this bit.
Then I also vote for removing the conversion.
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483856#comment-13483856 ]
Jonathan Coveney commented on PIG-3001:
---------------------------------------
No worries on the whitespace. Like I said: I wish it weren't so, but the worry is breaking old or in-transit patches.
IMHO, I would prefer for it to simply fail out, since this on the front end. That said, I don't think that is currently an option since it would break a lot of scripts. I propose the following:
- In the case where there is a conversion, we should throw a warning
- There can be a flag which sets whether or not we should instead error out. It'll default turned to no, but then people can set it in their confs.
I do not like things that automagically do things, though in some cases it could be inavoidable
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3001:
-------------------------------
Attachment: PIG-3001-3.patch
Updating a new patch with no whitespace changes.
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485706#comment-13485706 ]
Rohini Palaniswamy commented on PIG-3001:
-----------------------------------------
Gianmarco,
I am all for backward compatibility. But providing env variables to streaming is a new feature in 0.11 [PIG-2900 and PIG-2971] and the convert setting is being introduced with this patch and was not there before. I could not think of a need or a use case where we need to convert the environment setting provided by the user which is going to be consumed by his script. If we have a use case scenario for it, then I have no objections to add the setting.
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3001:
-------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks Jon for reviewing it. Committed to trunk/0.11.
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001-5.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3001:
-------------------------------
Attachment: PIG-3001-4.patch
Updated the patch as follows:
- Added a flag to toggle on/off auto-conversion. By default, auto-conversion is on.
- Added a warning when converting non-alphanumeric chars to underscores.
- Added two unit test cases: 1) auto-conversion, and 2) no auto-conversion.
Thanks!
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3001:
-------------------------------
Fix Version/s: 0.11
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Fix For: 0.11
>
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001-5.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Rohini Palaniswamy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485252#comment-13485252 ]
Rohini Palaniswamy commented on PIG-3001:
-----------------------------------------
My vote is for removing the conversion altogether. Since it is a user setting which will be consumed by the user in his script, it is best left to the user to set it correctly. Converting it might make it unusable.
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3001:
-------------------------------
Issue Type: Sub-task (was: Bug)
Parent: PIG-2972
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Priority: Minor
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483851#comment-13483851 ]
Cheolsoo Park commented on PIG-3001:
------------------------------------
I am sorry. I will upload a new patch without whitespace changes and update my other patches that include whitespace changes.
Btw, I was wondering whether we should keep safeEnvVarName() or not. I can argue both ways:
+ It will be safer since un-proper chars are auto-converted to '_', so we're less likely to see errors like this.
\- The user has to remember non-alphanumeric chars are auto-converted to '_' when passed to the streaming process.
Thoughts?
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3001:
-------------------------------
Attachment: PIG-3001.patch
Uploading a patch that makes the test more stable. I made two changes.
- Looking at the commit history, PIG-2971 made ExecutableManager#safeEnvName() obsolete. Now the user specifies which properties to be added as env. vars via a Pig property, so safeEnvName() is no longer used. I removed it.
- Replaced RandomStringUtils.random() with RandomStringUtils.randomAlphanumeric(). RandomStringUtils.random() generates a string from the set of all characters, and sometimes the returned string is not proper to be used as an env. var. name as shown in this jira.
Testing done:
ant clean test -Dtestcase=TestExecutableManager
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Priority: Minor
> Attachments: PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13487236#comment-13487236 ]
Jonathan Coveney commented on PIG-3001:
---------------------------------------
+1 looks good to me. I agree with Rohini's opinion completely.
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001-5.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3001:
-------------------------------
Attachment: PIG-3001-2.patch
On a second thought, I decided to keep safeEnvVarName(). It looks like it was dropped by PIG-2971 accidentally.
Updating my patch.
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Gianmarco De Francisci Morales (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485357#comment-13485357 ]
Gianmarco De Francisci Morales commented on PIG-3001:
-----------------------------------------------------
I would also remove it, however backwards compatibility is an issue.
I am fine with deprecating it and show a warning to the user.
We can remove it altogether in next release.
(if we choose this way, let's open a Jira to keep track of it).
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13485274#comment-13485274 ]
Cheolsoo Park commented on PIG-3001:
------------------------------------
Thanks for your comment Rohini.
That's definitely reasonable too. I will wait few more days just to see if anyone has a different opinion.
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3001:
-------------------------------
Assignee: Cheolsoo Park
Status: Patch Available (was: Open)
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Cheolsoo Park (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Cheolsoo Park updated PIG-3001:
-------------------------------
Attachment: PIG-3001-5.patch
Then, I will go back to my 1st patch that removes safeEnvVarName().
Updating the patch.
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001-3.patch, PIG-3001-4.patch, PIG-3001-5.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (PIG-3001)
TestExecutableManager.testAddJobConfToEnv fails randomly
Posted by "Jonathan Coveney (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/PIG-3001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13483848#comment-13483848 ]
Jonathan Coveney commented on PIG-3001:
---------------------------------------
Cheolsoo,
I am not a fan of this policy (though I understand why we have it), but we generally make patches without whitespace changes. Can you upload a new version without whitespace changes?
Thanks for the contribution. It looks good to me.
Jon
> TestExecutableManager.testAddJobConfToEnv fails randomly
> --------------------------------------------------------
>
> Key: PIG-3001
> URL: https://issues.apache.org/jira/browse/PIG-3001
> Project: Pig
> Issue Type: Sub-task
> Reporter: Gianmarco De Francisci Morales
> Assignee: Cheolsoo Park
> Priority: Minor
> Attachments: PIG-3001-2.patch, PIG-3001.patch
>
>
> The test in the Summary fails intermittently.
> This is due to using a random number generator without seeding it.
> We should avoid stochastic tests.
> Furthermore, the test itself is ill conceived.
> Here the failure summary:
> {code}
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: ⻨ꢏ切歯
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 狓偝
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 墣챟㌌̀썬鼹騷
> 12/10/23 11:02:48 WARN streaming.ExecutableManager: Property set in pig.streaming.environment not found in Configuration: 훎滼
> {code}
> {code}
> Error Message:
> There should be no remaining pairs in the included map
> Stacktrace:
> junit.framework.AssertionFailedError: There should be no remaining pairs in the included map
> at org.apache.pig.impl.streaming.TestExecutableManager.testAddJobConfToEnv(TestExecutableManager.java:84)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira