You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by GitBox <gi...@apache.org> on 2020/04/27 02:21:30 UTC
[GitHub] [carbondata] niuge01 opened a new pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
niuge01 opened a new pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727
### Why is this PR needed?
Change the way of mv module get database location to recommend way.
Currently , the recommend way is using CarbonEvn.getDatabaseLocation，not using carbon.storelocation variable.
### What changes were proposed in this PR?
Change all code which use carbon.storelocation variable to use CarbonEvn.getDatabaseLocation.
### Does this PR introduce any user interface change?
- No
### Is any new testcase added?
- No
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] niuge01 commented on a change in pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
niuge01 commented on a change in pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#discussion_r417016171
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -228,8 +200,8 @@ private static ICarbonLock getStatusLock(String databaseName) {
* @param schemaList schemas of which are need to be updated in mv status
* @param status status to be updated for the mv schemas
*/
- public void updateStatus(List<MVSchema> schemaList,
- MVStatus status) throws IOException {
+ public void updateStatus(MVManager viewManager, List<MVSchema> schemaList, MVStatus status)
Review comment:
OK
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +100,63 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ public MVSchema getSchema(MVManager viewManager, String databaseName, String viewName)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return null;
}
return schemaProvider.retrieveSchema(viewManager, viewName);
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName, CarbonTable carbonTable) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName, CarbonTable carbonTable)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveSchemas(viewManager, carbonTable);
}
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName) throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveAllSchemas(viewManager);
}
}
- void saveSchema(MVManager viewManager, String databaseName,
- MVSchema viewSchema) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ void saveSchema(MVManager viewManager, String databaseName, MVSchema viewSchema)
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-619691357
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2840/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#discussion_r415538262
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +109,69 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ public MVSchema getSchema(MVManager viewManager, String databaseName, String viewName)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return null;
}
return schemaProvider.retrieveSchema(viewManager, viewName);
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName, CarbonTable carbonTable) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName, CarbonTable carbonTable)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveSchemas(viewManager, carbonTable);
}
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName) throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveAllSchemas(viewManager);
}
}
- void saveSchema(MVManager viewManager, String databaseName,
- MVSchema viewSchema) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ void saveSchema(MVManager viewManager, String databaseName, MVSchema viewSchema)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
throw new IOException("Database [" + databaseName + "] is not found.");
}
schemaProvider.saveSchema(viewManager, viewSchema);
}
- public void dropSchema(String databaseName, String viewName)
+ public void dropSchema(MVManager viewManager, String databaseName, String viewName)
throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
throw new IOException("Materialized view with name " + databaseName + "." + viewName +
" does not exists in storage");
}
schemaProvider.dropSchema(viewName);
}
- private String getStatusFileName(String databaseName) {
+ private String getStatusFileName(MVManager viewManager, String databaseName) {
if (databaseName.equalsIgnoreCase("default")) {
- return this.storeLocation +
+ return viewManager.getDatabaseLocation(databaseName) +
Review comment:
Move viewManager.getDatabaseLocation(databaseName) before ifelse and reuse
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] niuge01 commented on a change in pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
niuge01 commented on a change in pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#discussion_r417016239
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +100,63 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ public MVSchema getSchema(MVManager viewManager, String databaseName, String viewName)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return null;
}
return schemaProvider.retrieveSchema(viewManager, viewName);
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName, CarbonTable carbonTable) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName, CarbonTable carbonTable)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveSchemas(viewManager, carbonTable);
}
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName) throws IOException {
Review comment:
OK
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +100,63 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ public MVSchema getSchema(MVManager viewManager, String databaseName, String viewName)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return null;
}
return schemaProvider.retrieveSchema(viewManager, viewName);
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName, CarbonTable carbonTable) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName, CarbonTable carbonTable)
Review comment:
OK
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +100,63 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
Review comment:
OK
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -74,42 +74,21 @@
private static final String STATUS_FILE_NAME = "mv_status";
- private final String storeLocation;
-
private final Map<String, SchemaProvider> schemaProviders = new ConcurrentHashMap<>();
- private MVProvider(String storeLocation) {
- this.storeLocation = storeLocation;
- }
-
- public static MVProvider get() {
- String storeLocation =
- CarbonProperties.getInstance().getProperty(CarbonCommonConstants.STORE_LOCATION);
- if (storeLocation == null) {
- throw new RuntimeException(
- "Property [" + CarbonCommonConstants.STORE_LOCATION + "] is not set.");
- }
- return new MVProvider(storeLocation);
- }
-
private static String getSchemaPath(String schemaRoot, String viewName) {
return schemaRoot + CarbonCommonConstants.FILE_SEPARATOR + "mv_schema." + viewName;
}
- private SchemaProvider getSchemaProvider(String databaseName) {
+ private SchemaProvider getSchemaProvider(MVManager viewManager, String databaseName) {
String databaseNameUpper = databaseName.toUpperCase();
SchemaProvider schemaProvider = this.schemaProviders.get(databaseNameUpper);
if (schemaProvider == null) {
synchronized (this.schemaProviders) {
schemaProvider = this.schemaProviders.get(databaseNameUpper);
if (schemaProvider == null) {
- String databaseLocation;
- if (databaseNameUpper.equalsIgnoreCase(CarbonCommonConstants.DATABASE_DEFAULT_NAME)) {
- databaseLocation = CarbonUtil.checkAndAppendHDFSUrl(this.storeLocation);
- } else {
- databaseLocation = CarbonUtil.checkAndAppendHDFSUrl(this.storeLocation +
- CarbonCommonConstants.FILE_SEPARATOR + databaseName + ".db");
- }
+ final String databaseLocation = FileFactory.getCarbonFile(
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-619820913
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1133/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-620979501
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2882/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-619906041
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1137/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-619685113
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1128/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-620511566
Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1155/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-620389364
@niuge01 please fix the CI
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-619858653
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2849/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-620939863
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2879/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-619833863
Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2845/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on a change in pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on a change in pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#discussion_r415536756
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -105,10 +89,14 @@ private SchemaProvider getSchemaProvider(String databaseName) {
if (schemaProvider == null) {
String databaseLocation;
if (databaseNameUpper.equalsIgnoreCase(CarbonCommonConstants.DATABASE_DEFAULT_NAME)) {
- databaseLocation = CarbonUtil.checkAndAppendHDFSUrl(this.storeLocation);
+ databaseLocation = CarbonUtil.checkAndAppendHDFSUrl(
+ viewManager.getDatabaseLocation(databaseName)
Review comment:
Please move getting database location before ifElse statement and reuse it in if and else block
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] niuge01 commented on a change in pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
niuge01 commented on a change in pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#discussion_r415571503
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -105,10 +89,14 @@ private SchemaProvider getSchemaProvider(String databaseName) {
if (schemaProvider == null) {
String databaseLocation;
if (databaseNameUpper.equalsIgnoreCase(CarbonCommonConstants.DATABASE_DEFAULT_NAME)) {
- databaseLocation = CarbonUtil.checkAndAppendHDFSUrl(this.storeLocation);
+ databaseLocation = CarbonUtil.checkAndAppendHDFSUrl(
+ viewManager.getDatabaseLocation(databaseName)
Review comment:
OK
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +109,69 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ public MVSchema getSchema(MVManager viewManager, String databaseName, String viewName)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return null;
}
return schemaProvider.retrieveSchema(viewManager, viewName);
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName, CarbonTable carbonTable) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName, CarbonTable carbonTable)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveSchemas(viewManager, carbonTable);
}
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName) throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveAllSchemas(viewManager);
}
}
- void saveSchema(MVManager viewManager, String databaseName,
- MVSchema viewSchema) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ void saveSchema(MVManager viewManager, String databaseName, MVSchema viewSchema)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
throw new IOException("Database [" + databaseName + "] is not found.");
}
schemaProvider.saveSchema(viewManager, viewSchema);
}
- public void dropSchema(String databaseName, String viewName)
+ public void dropSchema(MVManager viewManager, String databaseName, String viewName)
throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
throw new IOException("Materialized view with name " + databaseName + "." + viewName +
" does not exists in storage");
}
schemaProvider.dropSchema(viewName);
}
- private String getStatusFileName(String databaseName) {
+ private String getStatusFileName(MVManager viewManager, String databaseName) {
if (databaseName.equalsIgnoreCase("default")) {
- return this.storeLocation +
+ return viewManager.getDatabaseLocation(databaseName) +
Review comment:
OK
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-620971625
Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/1164/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] niuge01 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
niuge01 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-620978136
retest this please
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on a change in pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on a change in pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#discussion_r416616355
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -228,8 +200,8 @@ private static ICarbonLock getStatusLock(String databaseName) {
* @param schemaList schemas of which are need to be updated in mv status
* @param status status to be updated for the mv schemas
*/
- public void updateStatus(List<MVSchema> schemaList,
- MVStatus status) throws IOException {
+ public void updateStatus(MVManager viewManager, List<MVSchema> schemaList, MVStatus status)
Review comment:
correct the style like before
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +100,63 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ public MVSchema getSchema(MVManager viewManager, String databaseName, String viewName)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return null;
}
return schemaProvider.retrieveSchema(viewManager, viewName);
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName, CarbonTable carbonTable) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName, CarbonTable carbonTable)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveSchemas(viewManager, carbonTable);
}
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName) throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveAllSchemas(viewManager);
}
}
- void saveSchema(MVManager viewManager, String databaseName,
- MVSchema viewSchema) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ void saveSchema(MVManager viewManager, String databaseName, MVSchema viewSchema)
Review comment:
please keep the style as before, revert the method signature changes, just change `SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);`
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +100,63 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ public MVSchema getSchema(MVManager viewManager, String databaseName, String viewName)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return null;
}
return schemaProvider.retrieveSchema(viewManager, viewName);
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName, CarbonTable carbonTable) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName, CarbonTable carbonTable)
Review comment:
same as below comment
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +100,63 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
Review comment:
same as below comment
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -121,70 +100,63 @@ private SchemaProvider getSchemaProvider(String databaseName) {
return schemaProvider;
}
- public MVSchema getSchema(MVManager viewManager,
- String databaseName, String viewName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ public MVSchema getSchema(MVManager viewManager, String databaseName, String viewName)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return null;
}
return schemaProvider.retrieveSchema(viewManager, viewName);
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName, CarbonTable carbonTable) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName, CarbonTable carbonTable)
+ throws IOException {
+ SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);
if (schemaProvider == null) {
return Collections.emptyList();
} else {
return schemaProvider.retrieveSchemas(viewManager, carbonTable);
}
}
- List<MVSchema> getSchemas(MVManager viewManager,
- String databaseName) throws IOException {
- SchemaProvider schemaProvider = this.getSchemaProvider(databaseName);
+ List<MVSchema> getSchemas(MVManager viewManager, String databaseName) throws IOException {
Review comment:
please keep the style as before, revert the method signature changes, just change `SchemaProvider schemaProvider = this.getSchemaProvider(viewManager, databaseName);`
##########
File path: core/src/main/java/org/apache/carbondata/core/view/MVProvider.java
##########
@@ -74,42 +74,21 @@
private static final String STATUS_FILE_NAME = "mv_status";
- private final String storeLocation;
-
private final Map<String, SchemaProvider> schemaProviders = new ConcurrentHashMap<>();
- private MVProvider(String storeLocation) {
- this.storeLocation = storeLocation;
- }
-
- public static MVProvider get() {
- String storeLocation =
- CarbonProperties.getInstance().getProperty(CarbonCommonConstants.STORE_LOCATION);
- if (storeLocation == null) {
- throw new RuntimeException(
- "Property [" + CarbonCommonConstants.STORE_LOCATION + "] is not set.");
- }
- return new MVProvider(storeLocation);
- }
-
private static String getSchemaPath(String schemaRoot, String viewName) {
return schemaRoot + CarbonCommonConstants.FILE_SEPARATOR + "mv_schema." + viewName;
}
- private SchemaProvider getSchemaProvider(String databaseName) {
+ private SchemaProvider getSchemaProvider(MVManager viewManager, String databaseName) {
String databaseNameUpper = databaseName.toUpperCase();
SchemaProvider schemaProvider = this.schemaProviders.get(databaseNameUpper);
if (schemaProvider == null) {
synchronized (this.schemaProviders) {
schemaProvider = this.schemaProviders.get(databaseNameUpper);
if (schemaProvider == null) {
- String databaseLocation;
- if (databaseNameUpper.equalsIgnoreCase(CarbonCommonConstants.DATABASE_DEFAULT_NAME)) {
- databaseLocation = CarbonUtil.checkAndAppendHDFSUrl(this.storeLocation);
- } else {
- databaseLocation = CarbonUtil.checkAndAppendHDFSUrl(this.storeLocation +
- CarbonCommonConstants.FILE_SEPARATOR + databaseName + ".db");
- }
+ final String databaseLocation = FileFactory.getCarbonFile(
Review comment:
1. assign `viewManager.getDatabaseLocation(databaseName)` to a variable as databaseLocation.
2.assign ` FileFactory.getCarbonFile(databaseLocation)` to a variable and reuse same in if check condition at line number 92
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] akashrn5 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
akashrn5 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-621138371
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] CarbonDataQA1 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
CarbonDataQA1 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-620511702
Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2870/
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [carbondata] Indhumathi27 commented on pull request #3727: [CARBONDATA-3778] Change conf.getProperty("carbon.storelocation") to CarbonEvn.getDatabaseLocation in mv module
Posted by GitBox <gi...@apache.org>.
Indhumathi27 commented on pull request #3727:
URL: https://github.com/apache/carbondata/pull/3727#issuecomment-620564373
LGTM
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org