You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by GitBox <gi...@apache.org> on 2020/01/08 08:34:24 UTC
[GitHub] [incubator-shardingsphere] avalon566 opened a new pull request
#3899: Sharding scaling
avalon566 opened a new pull request #3899: Sharding scaling
URL: https://github.com/apache/incubator-shardingsphere/pull/3899
For #3699.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] KomachiSion commented on issue #3899:
Sharding scaling
Posted by GitBox <gi...@apache.org>.
KomachiSion commented on issue #3899: Sharding scaling
URL: https://github.com/apache/incubator-shardingsphere/pull/3899#issuecomment-571976539
@avalon566, Please change the title, the title will be set the commit description when squash.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] KomachiSion commented on a change in
pull request #3899: [Sharding Scaling]Refactor to support PostgreSQL
Posted by GitBox <gi...@apache.org>.
KomachiSion commented on a change in pull request #3899: [Sharding Scaling]Refactor to support PostgreSQL
URL: https://github.com/apache/incubator-shardingsphere/pull/3899#discussion_r364519005
##########
File path: sharding-scaling/sharding-scaling-core/src/main/java/org/apache/shardingsphere/shardingscaling/core/execute/executor/writer/SqlBuilder.java
##########
@@ -20,33 +20,28 @@
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
+import org.apache.shardingsphere.shardingscaling.core.metadata.ColumnMetaData;
+import org.apache.shardingsphere.shardingscaling.core.util.DbMetaDataUtil;
import javax.sql.DataSource;
import java.util.List;
import java.util.concurrent.ExecutionException;
-import org.apache.shardingsphere.shardingscaling.core.util.DbMetaDataUtil;
-import org.apache.shardingsphere.shardingscaling.core.metadata.ColumnMetaData;
-
/**
* Sql builder.
*
* @author avalon566
*/
-public final class SqlBuilder {
-
- private static final String LEFT_ESCAPE_QUOTE = "`";
-
- private static final String RIGHT_ESCAPE_QUOTE = "`";
+public abstract class SqlBuilder {
Review comment:
Should we rename the class?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] KomachiSion commented on a change in
pull request #3899: [Sharding Scaling]Refactor to support PostgreSQL
Posted by GitBox <gi...@apache.org>.
KomachiSion commented on a change in pull request #3899: [Sharding Scaling]Refactor to support PostgreSQL
URL: https://github.com/apache/incubator-shardingsphere/pull/3899#discussion_r364518568
##########
File path: sharding-scaling/sharding-scaling-core/src/main/java/org/apache/shardingsphere/shardingscaling/core/execute/executor/reader/AbstractJdbcReader.java
##########
@@ -102,7 +93,19 @@ public final void read(final Channel channel) {
pushRecord(new FinishedRecord(new NopLogPosition()));
}
}
-
+
+ /**
+ * Read value from {@code ResultSet}.
+ *
+ * @param resultSet result set
+ * @param index of read column
+ * @return value
+ * @throws SQLException sql exception
+ */
+ public Object readValue(final ResultSet resultSet, final int index) throws SQLException {
Review comment:
protected may be enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] KomachiSion commented on a change in
pull request #3899: [Sharding Scaling]Refactor to support PostgreSQL
Posted by GitBox <gi...@apache.org>.
KomachiSion commented on a change in pull request #3899: [Sharding Scaling]Refactor to support PostgreSQL
URL: https://github.com/apache/incubator-shardingsphere/pull/3899#discussion_r364520344
##########
File path: sharding-scaling/sharding-scaling-core/src/main/java/org/apache/shardingsphere/shardingscaling/core/execute/executor/writer/SqlBuilder.java
##########
@@ -66,6 +61,20 @@ public String load(final String key) {
}
});
}
+
+ /**
+ * Get left identifier quote string.
+ *
+ * @return string
+ */
+ public abstract String getLeftIdentifierQuoteString();
Review comment:
protected may be enough.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] KomachiSion commented on a change in
pull request #3899: Sharding scaling
Posted by GitBox <gi...@apache.org>.
KomachiSion commented on a change in pull request #3899: Sharding scaling
URL: https://github.com/apache/incubator-shardingsphere/pull/3899#discussion_r364118332
##########
File path: sharding-scaling/sharding-scaling-core/src/main/java/org/apache/shardingsphere/shardingscaling/core/execute/executor/reader/AbstractJdbcReader.java
##########
@@ -48,46 +48,44 @@
*/
@Slf4j
public abstract class AbstractJdbcReader extends AbstractSyncRunner implements JdbcReader {
-
+
@Getter(AccessLevel.PROTECTED)
private final RdbmsConfiguration rdbmsConfiguration;
private final DataSourceFactory dataSourceFactory;
-
+
@Setter
private Channel channel;
-
+
public AbstractJdbcReader(final RdbmsConfiguration rdbmsConfiguration, final DataSourceFactory dataSourceFactory) {
if (!JdbcDataSourceConfiguration.class.equals(rdbmsConfiguration.getDataSourceConfiguration().getClass())) {
throw new UnsupportedOperationException("AbstractJdbcReader only support JdbcDataSourceConfiguration");
}
this.rdbmsConfiguration = rdbmsConfiguration;
this.dataSourceFactory = dataSourceFactory;
}
-
+
@Override
public final void run() {
start();
read(channel);
}
-
+
@Override
public final void read(final Channel channel) {
try (Connection conn = dataSourceFactory.getDataSource(rdbmsConfiguration.getDataSourceConfiguration()).getConnection()) {
String sql = String.format("select * from %s %s", rdbmsConfiguration.getTableName(), rdbmsConfiguration.getWhereCondition());
PreparedStatement ps = conn.prepareStatement(sql, java.sql.ResultSet.TYPE_FORWARD_ONLY, ResultSet.CONCUR_READ_ONLY);
- ps.setFetchSize(100);
- ps.setFetchDirection(ResultSet.FETCH_REVERSE);
+ ps.setFetchSize(Integer.MIN_VALUE);
ResultSet rs = ps.executeQuery();
ResultSetMetaData metaData = rs.getMetaData();
while (isRunning() && rs.next()) {
DataRecord record = new DataRecord(new NopLogPosition(), metaData.getColumnCount());
record.setType("bootstrap-insert");
record.setTableName(rdbmsConfiguration.getTableNameMap().get(rdbmsConfiguration.getTableName()));
for (int i = 1; i <= metaData.getColumnCount(); i++) {
- if (Types.TIME == rs.getMetaData().getColumnType(i)
- || Types.DATE == rs.getMetaData().getColumnType(i)
- || Types.TIMESTAMP == rs.getMetaData().getColumnType(i)) {
+ if ("MySQL".equals(rdbmsConfiguration.getDataSourceConfiguration().getDatabaseType().getName())
+ && isDateTimeValue(rs.getMetaData().getColumnType(i))) {
Review comment:
Abstract class should not perceive the specific type, it can use abstract method replace, and let subClass to implement.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] KomachiSion commented on a change in
pull request #3899: Sharding scaling
Posted by GitBox <gi...@apache.org>.
KomachiSion commented on a change in pull request #3899: Sharding scaling
URL: https://github.com/apache/incubator-shardingsphere/pull/3899#discussion_r364119116
##########
File path: sharding-scaling/sharding-scaling-core/src/main/java/org/apache/shardingsphere/shardingscaling/core/execute/executor/writer/SqlBuilder.java
##########
@@ -66,6 +66,23 @@ public String load(final String key) {
}
});
}
+
+ private void initIdentifierQuoteString(final String databaseType) {
+ switch (databaseType) {
+ case "MySQL":
+ leftIdentifierQuoteString = "`";
+ rightIdentifierQuoteString = "`";
+ break;
+ case "PostgreSQL":
+ leftIdentifierQuoteString = "\"";
+ rightIdentifierQuoteString = "\"";
+ break;
+ default:
+ leftIdentifierQuoteString = "";
+ rightIdentifierQuoteString = "";
+ break;
+ }
+ }
Review comment:
Similar problem as AbstractJdbcReader.
This can abstract a protect method, and let subClass to implement.
Or create a new interface to handle this situation.
each type database has implement, SQLBuilder only call interface.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-shardingsphere] KomachiSion merged pull request #3899:
[Sharding Scaling]Refactor to support PostgreSQL
Posted by GitBox <gi...@apache.org>.
KomachiSion merged pull request #3899: [Sharding Scaling]Refactor to support PostgreSQL
URL: https://github.com/apache/incubator-shardingsphere/pull/3899
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services