You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ambari.apache.org by Tom Beerbower <tb...@hortonworks.com> on 2015/06/16 20:31:24 UTC
Review Request 35528: Views : DataStore persistence error
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/
-----------------------------------------------------------
Review request for Ambari, Jonathan Hurley and Nate Cole.
Bugs: AMBARI-11957
https://issues.apache.org/jira/browse/AMBARI-11957
Repository: ambari
Description
-------
In one case we are seeing this exception while trying to store data from a view ...
java.lang.IllegalStateException:
Exception Description: No transaction is currently active
at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
at
The underlying exception ...
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
Error Code: 0
Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
bind => [6 parameters bound]
This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
Diffs
-----
ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
Diff: https://reviews.apache.org/r/35528/diff/
Testing
-------
Manual tested.
New unit tests added.
mvn clean test
all pass
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 46:25 min
[INFO] Finished at: 2015-06-16T13:42:50-04:00
[INFO] Final Memory: 54M/1453M
[INFO] ------------------------------------------------------------------------
Thanks,
Tom Beerbower
Re: Review Request 35528: Views : DataStore persistence error
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88118
-----------------------------------------------------------
Ship it!
ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java (line 307)
<https://reviews.apache.org/r/35528/#comment140517>
In general, I think this is safer if this is truly a dynamic field. This might map to things like LARGETEXT which is a slight performance hit on some DBs like MySQL, but if this isn't a high traffic field, then it's not a problem.
I'm also not sure of CLOBs are natively searchable on some DBs, like Oracle.
But again, I don't think you're doing 10,000 tx's / minute or using native search queries, so you're fine.
- Jonathan Hurley
On June 16, 2015, 4:21 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 4:21 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88120
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On June 16, 2015, 4:21 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 4:21 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Tom Beerbower <tb...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/
-----------------------------------------------------------
(Updated June 16, 2015, 8:21 p.m.)
Review request for Ambari, Jonathan Hurley and Nate Cole.
Changes
-------
Update diff to map String fields to CLOB type.
Bugs: AMBARI-11957
https://issues.apache.org/jira/browse/AMBARI-11957
Repository: ambari
Description
-------
In one case we are seeing this exception while trying to store data from a view ...
java.lang.IllegalStateException:
Exception Description: No transaction is currently active
at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
at
The underlying exception ...
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
Error Code: 0
Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
bind => [6 parameters bound]
This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
Diffs (updated)
-----
ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
Diff: https://reviews.apache.org/r/35528/diff/
Testing
-------
Manual tested.
New unit tests added.
mvn clean test
all pass
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 46:25 min
[INFO] Finished at: 2015-06-16T13:42:50-04:00
[INFO] Final Memory: 54M/1453M
[INFO] ------------------------------------------------------------------------
Thanks,
Tom Beerbower
Re: Review Request 35528: Views : DataStore persistence error
Posted by Tom Beerbower <tb...@hortonworks.com>.
> On June 16, 2015, 6:46 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java, line 618
> > <https://reviews.apache.org/r/35528/diff/1/?file=985857#file985857line618>
> >
> > It seems like this would truncate the value, thus loosing data. Is this OK? If so, we can close this issue, but I just wanted to verify that this behavior was expected.
Thanks for reviewing!
The DataStore was meant to be a light weight persistence mechanism for views so I think that having reasonable restrictions is okay. But you are correct in that the data would be trunctated and the user might never know, which could be bad. I guess the options are 1) truncate the String and log it. 2) Fail fast if the String is too long. 3) Map the String fields to a CLOB type, if possible. What do you think?
- Tom
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88107
-----------------------------------------------------------
On June 16, 2015, 6:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 6:31 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Jonathan Hurley <jh...@hortonworks.com>.
> On June 16, 2015, 2:46 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java, line 618
> > <https://reviews.apache.org/r/35528/diff/1/?file=985857#file985857line618>
> >
> > It seems like this would truncate the value, thus loosing data. Is this OK? If so, we can close this issue, but I just wanted to verify that this behavior was expected.
>
> Tom Beerbower wrote:
> Thanks for reviewing!
>
> The DataStore was meant to be a light weight persistence mechanism for views so I think that having reasonable restrictions is okay. But you are correct in that the data would be trunctated and the user might never know, which could be bad. I guess the options are 1) truncate the String and log it. 2) Fail fast if the String is too long. 3) Map the String fields to a CLOB type, if possible. What do you think?
Yes, I'd say use a CLOB if this could go beyond 4000. How does this work for different DBs? Do we just rely on the mappings? If we used a CLOB, the persisence layer will map this to things like LARGETEXT, TEXT, etc, which is good.
Unless you really don't think you'd need more than 4000. In which case I'm fine with the truncation.
- Jonathan
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88107
-----------------------------------------------------------
On June 16, 2015, 4:21 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 4:21 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Tom Beerbower <tb...@hortonworks.com>.
> On June 16, 2015, 6:46 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java, line 618
> > <https://reviews.apache.org/r/35528/diff/1/?file=985857#file985857line618>
> >
> > It seems like this would truncate the value, thus loosing data. Is this OK? If so, we can close this issue, but I just wanted to verify that this behavior was expected.
>
> Tom Beerbower wrote:
> Thanks for reviewing!
>
> The DataStore was meant to be a light weight persistence mechanism for views so I think that having reasonable restrictions is okay. But you are correct in that the data would be trunctated and the user might never know, which could be bad. I guess the options are 1) truncate the String and log it. 2) Fail fast if the String is too long. 3) Map the String fields to a CLOB type, if possible. What do you think?
>
> Jonathan Hurley wrote:
> Yes, I'd say use a CLOB if this could go beyond 4000. How does this work for different DBs? Do we just rely on the mappings? If we used a CLOB, the persisence layer will map this to things like LARGETEXT, TEXT, etc, which is good.
>
> Unless you really don't think you'd need more than 4000. In which case I'm fine with the truncation.
We don't really know because the user view could stick any length String into the entity field. I've update the diff to use CLOB. Yeah, the persistence layer should handle the correct mapping. I just tested on Postgres and it mapped to a TEXT type.
Thanks guys!
- Tom
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88107
-----------------------------------------------------------
On June 16, 2015, 8:21 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 8:21 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Jonathan Hurley <jh...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88107
-----------------------------------------------------------
ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java (lines 611 - 612)
<https://reviews.apache.org/r/35528/#comment140510>
Use {} parameters instead of +'s ?
ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java (line 614)
<https://reviews.apache.org/r/35528/#comment140511>
It seems like this would truncate the value, thus loosing data. Is this OK? If so, we can close this issue, but I just wanted to verify that this behavior was expected.
- Jonathan Hurley
On June 16, 2015, 2:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 2:31 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88112
-----------------------------------------------------------
Ship it!
Ship It!
- Nate Cole
On June 16, 2015, 2:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 2:31 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Nate Cole <nc...@hortonworks.com>.
> On June 16, 2015, 3:26 p.m., Nate Cole wrote:
> > The code looks like it caps the length of a value (ok), but where is the definition of that storage - in a db script somewhere? No sql changes to support the bigger field? I thought this might be "viewinstancedata" but I'm not sure.
>
> Nate Cole wrote:
> Oh, I see you said it's a dynamic entity - I thought there was a definition somewhere. You can ignore :) Just Jonathan's suggestions and I'm good.
>
> Tom Beerbower wrote:
> Thanks for the review!
>
> Yeah, the view DataStore uses dynamic entites. By default String types map to VARCHAR(255), which lead to this issue as soon as someone tried to persist an entity with a long String. So, instead of the way I handled it in this patch, I could (I think) map the Strings to a CLOB. Would that be a better solution?
If you can't specify the max length of a string column, then I would say you should probably go CLOB.
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88110
-----------------------------------------------------------
On June 16, 2015, 2:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 2:31 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Nate Cole <nc...@hortonworks.com>.
> On June 16, 2015, 3:26 p.m., Nate Cole wrote:
> > The code looks like it caps the length of a value (ok), but where is the definition of that storage - in a db script somewhere? No sql changes to support the bigger field? I thought this might be "viewinstancedata" but I'm not sure.
Oh, I see you said it's a dynamic entity - I thought there was a definition somewhere. You can ignore :) Just Jonathan's suggestions and I'm good.
- Nate
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88110
-----------------------------------------------------------
On June 16, 2015, 2:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 2:31 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Tom Beerbower <tb...@hortonworks.com>.
> On June 16, 2015, 7:26 p.m., Nate Cole wrote:
> > The code looks like it caps the length of a value (ok), but where is the definition of that storage - in a db script somewhere? No sql changes to support the bigger field? I thought this might be "viewinstancedata" but I'm not sure.
>
> Nate Cole wrote:
> Oh, I see you said it's a dynamic entity - I thought there was a definition somewhere. You can ignore :) Just Jonathan's suggestions and I'm good.
Thanks for the review!
Yeah, the view DataStore uses dynamic entites. By default String types map to VARCHAR(255), which lead to this issue as soon as someone tried to persist an entity with a long String. So, instead of the way I handled it in this patch, I could (I think) map the Strings to a CLOB. Would that be a better solution?
- Tom
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88110
-----------------------------------------------------------
On June 16, 2015, 6:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 6:31 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>
Re: Review Request 35528: Views : DataStore persistence error
Posted by Nate Cole <nc...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35528/#review88110
-----------------------------------------------------------
The code looks like it caps the length of a value (ok), but where is the definition of that storage - in a db script somewhere? No sql changes to support the bigger field? I thought this might be "viewinstancedata" but I'm not sure.
- Nate Cole
On June 16, 2015, 2:31 p.m., Tom Beerbower wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/35528/
> -----------------------------------------------------------
>
> (Updated June 16, 2015, 2:31 p.m.)
>
>
> Review request for Ambari, Jonathan Hurley and Nate Cole.
>
>
> Bugs: AMBARI-11957
> https://issues.apache.org/jira/browse/AMBARI-11957
>
>
> Repository: ambari
>
>
> Description
> -------
>
> In one case we are seeing this exception while trying to store data from a view ...
>
> java.lang.IllegalStateException:
> Exception Description: No transaction is currently active
> at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.rollback(EntityTransactionImpl.java:176)
> at org.apache.ambari.server.view.persistence.DataStoreImpl.store(DataStoreImpl.java:135)
> at org.apache.ambari.view.hive.persistence.DataStoreStorage.store(DataStoreStorage.java:76)
> at org.apache.ambari.view.hive.resources.CRUDResourceManager.save(CRUDResourceManager.java:117)
> at
>
> The underlying exception ...
>
> Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
> Internal Exception: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(255)
> Error Code: 0
> Call: UPDATE DS_JOBIMPL_4 SET DS_applicationId = ?, DS_duration = ?, DS_sqlState = ?, DS_status = ?, DS_statusMessage = ? WHERE (DS_id = ?)
> bind => [6 parameters bound]
>
>
> This occurs because the view tries to save an entity with a String type attribute which is mapped to a VARCHAR(255). The dynamic entity code maps the String to a VARCHAR using the database default length.
>
> Also, once the exception is thrown, the DataStore catches it and tries to rollback the transaction. It looks like the tx is already inactive at this point so a different exception is thrown out. The original exception should at least be logged by the DataStoreImpl.
>
>
> Diffs
> -----
>
> ambari-server/src/main/java/org/apache/ambari/server/view/persistence/DataStoreImpl.java 8353d63
> ambari-server/src/test/java/org/apache/ambari/server/view/persistence/DataStoreImplTest.java 2b19b0a
>
> Diff: https://reviews.apache.org/r/35528/diff/
>
>
> Testing
> -------
>
> Manual tested.
>
> New unit tests added.
>
> mvn clean test
>
> all pass
>
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 46:25 min
> [INFO] Finished at: 2015-06-16T13:42:50-04:00
> [INFO] Final Memory: 54M/1453M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Tom Beerbower
>
>