You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@shardingsphere.apache.org by "pan1998 (via GitHub)" <gi...@apache.org> on 2026/01/16 09:15:34 UTC
[I] When will it be compatible with Dameng 8 database? [shardingsphere]
pan1998 opened a new issue, #37756:
URL: https://github.com/apache/shardingsphere/issues/37756
I used self-written SPI code to connect to Dameng 8 (in Oracle compatibility mode), but I received the following error: "nested exception is org.apache.shardingsphere.infra.exception.kernel.metadata.TableNotFoundException: **Table or view 'all_sequences' does not exist**."
Development environment: JDK11 + Dameng 8 (Oracle compatibility mode) + hibernate 5.6.15.Final + spring 5.2.20.RELEASE
public final class DmDatabaseType implements DatabaseType {
@Override
public Collection<String> getJdbcUrlPrefixes() {
return Collections.singleton(String.format("jdbc:%s:", getType().toLowerCase()));
}
@Override
public String getType() {
return CommonConstants.DM;
}
}
public class DmParserFacade implements DialectSQLParserFacade {
@Override
public Class<? extends SQLLexer> getLexerClass() {
return OracleLexer.class;
}
@Override
public Class<? extends SQLParser> getParserClass() {
return OracleParser.class;
}
@Override
public String getDatabaseType() {
return CommonConstants.DM;
}
}
public final class DmDatabaseMetaData implements DialectDatabaseMetaData {
private static final Set<String> RESERVED_KEYWORDS = new HashSet<>(Arrays.asList("ACCESS", "ADD", "ALL", "ALTER", "AND", "ANY", "ARRAYLEN", "AS", "ASC", "AUDIT", "BETWEEN", "BY", "CHAR", "CHECK",
"CLUSTER", "COLUMN", "COMMENT", "COMPRESS", "CONNECT", "CREATE", "CURRENT", "DATE", "DECIMAL", "DEFAULT", "DELETE", "DESC", "DISTINCT", "DROP", "ELSE", "EXCLUSIVE", "EXISTS", "FILE",
"FLOAT", "FOR", "FROM", "GRANT", "GROUP", "HAVING", "IDENTIFIED", "IMMEDIATE", "IN", "INCREMENT", "INDEX", "INITIAL", "INSERT", "INTEGER", "INTERSECT", "INTO", "IS", "LEVEL", "LIKE",
"LOCK", "LONG", "MAXEXTENTS", "MINUS", "MODE", "MODIFY", "NOAUDIT", "NOCOMPRESS", "NOT", "NOTFOUND", "NOWAIT", "NULL", "NUMBER", "OF", "OFFLINE", "ON", "ONLINE", "OPTION", "OR", "ORDER",
"PCTFREE", "PRIOR", "PRIVILEGES", "PUBLIC", "RAW", "RENAME", "RESOURCE", "REVOKE", "ROW", "ROWID", "ROWLABEL", "ROWNUM", "ROWS", "START", "SELECT", "SESSION", "SET", "SHARE", "SIZE",
"SMALLINT", "SQLBUF", "SUCCESSFUL", "SYNONYM", "SYSDATE", "TABLE", "THEN", "TO", "TRIGGER", "UID", "UNION", "UNIQUE", "UPDATE", "USER", "VALIDATE", "VALUES", "VARCHAR", "VARCHAR2",
"VIEW", "WHENEVER", "WHERE", "WITH"));
@Override
public QuoteCharacter getQuoteCharacter() {
return QuoteCharacter.QUOTE;
}
@Override
public NullsOrderType getDefaultNullsOrderType() {
return NullsOrderType.HIGH;
}
@Override
public boolean isReservedWord(final String identifier) {
return RESERVED_KEYWORDS.contains(identifier.toUpperCase());
}
@Override
public boolean isSchemaAvailable() {
return true;
}
@Override
public String getSchema(final Connection connection) {
try {
return Optional.ofNullable(connection.getMetaData().getUserName()).map(String::toUpperCase).orElse(null);
} catch (final SQLException ignored) {
return null;
}
}
@Override
public String formatTableNamePattern(final String tableNamePattern) {
return tableNamePattern.toUpperCase();
}
@Override
public boolean isInstanceConnectionAvailable() {
return true;
}
@Override
public String getDatabaseType() {
return CommonConstants.DM;
}
}
public final class DmConnectionPropertiesParser implements ConnectionPropertiesParser {
private static final int DEFAULT_PORT = 5236;
@Override
public ConnectionProperties parse(final String url, final String username, final String catalog) {
JdbcUrl jdbcUrl = new StandardJdbcUrlParser().parse(url);
return new StandardConnectionProperties(jdbcUrl.getHostname(), jdbcUrl.getPort(DEFAULT_PORT),
null == catalog ? jdbcUrl.getDatabase() : catalog, null, jdbcUrl.getQueryProperties(),
new Properties());
}
@Override
public String getDatabaseType() {
return CommonConstants.DM;
}
}
public final class DmMetaDataLoader implements DialectMetaDataLoader {
private static final String TABLE_META_DATA_SQL_NO_ORDER =
"SELECT OWNER AS TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME, NULLABLE, DATA_TYPE, COLUMN_ID, HIDDEN_COLUMN %s FROM ALL_TAB_COLS WHERE OWNER = ?";
private static final String ORDER_BY_COLUMN_ID = " ORDER BY COLUMN_ID";
private static final String TABLE_META_DATA_SQL = TABLE_META_DATA_SQL_NO_ORDER + ORDER_BY_COLUMN_ID;
private static final String TABLE_META_DATA_SQL_IN_TABLES = TABLE_META_DATA_SQL_NO_ORDER + " AND TABLE_NAME IN (%s)" + ORDER_BY_COLUMN_ID;
private static final String INDEX_META_DATA_SQL = "SELECT OWNER AS TABLE_SCHEMA, TABLE_NAME, INDEX_NAME, UNIQUENESS FROM ALL_INDEXES WHERE OWNER = ? AND TABLE_NAME IN (%s)";
private static final String PRIMARY_KEY_META_DATA_SQL = "SELECT A.OWNER AS TABLE_SCHEMA, A.TABLE_NAME AS TABLE_NAME, B.COLUMN_NAME AS COLUMN_NAME FROM ALL_CONSTRAINTS A INNER JOIN"
+ " ALL_CONS_COLUMNS B ON A.CONSTRAINT_NAME = B.CONSTRAINT_NAME WHERE CONSTRAINT_TYPE = 'P' AND A.OWNER = '%s'";
private static final String PRIMARY_KEY_META_DATA_SQL_IN_TABLES = PRIMARY_KEY_META_DATA_SQL + " AND A.TABLE_NAME IN (%s)";
private static final String INDEX_COLUMN_META_DATA_SQL = "SELECT COLUMN_NAME FROM ALL_IND_COLUMNS WHERE INDEX_OWNER = ? AND TABLE_NAME = ? AND INDEX_NAME = ?";
private static final int COLLATION_START_MAJOR_VERSION = 12;
private static final int COLLATION_START_MINOR_VERSION = 2;
private static final int IDENTITY_COLUMN_START_MINOR_VERSION = 1;
private static final int MAX_EXPRESSION_SIZE = 1000;
@Override
public Collection<SchemaMetaData> load(final MetaDataLoaderMaterial material) throws SQLException {
Map<String, Collection<ColumnMetaData>> columnMetaDataMap = new HashMap<>(material.getActualTableNames().size(), 1F);
Map<String, Collection<IndexMetaData>> indexMetaDataMap = new HashMap<>(material.getActualTableNames().size(), 1F);
try (Connection connection = new MetaDataLoaderConnection(TypedSPILoader.getService(DatabaseType.class, CommonConstants.DM), material.getDataSource().getConnection())) {
for (List<String> each : Lists.partition(new ArrayList<>(material.getActualTableNames()), MAX_EXPRESSION_SIZE)) {
columnMetaDataMap.putAll(loadColumnMetaDataMap(connection, each));
indexMetaDataMap.putAll(loadIndexMetaData(connection, each));
}
}
Collection<TableMetaData> tableMetaDataList = new LinkedList<>();
for (Entry<String, Collection<ColumnMetaData>> entry : columnMetaDataMap.entrySet()) {
tableMetaDataList.add(new TableMetaData(entry.getKey(), entry.getValue(), indexMetaDataMap.getOrDefault(entry.getKey(), Collections.emptyList()), Collections.emptyList()));
}
return Collections.singletonList(new SchemaMetaData(material.getDefaultSchemaName(), tableMetaDataList));
}
private Map<String, Collection<ColumnMetaData>> loadColumnMetaDataMap(final Connection connection, final Collection<String> tables) throws SQLException {
Map<String, Collection<ColumnMetaData>> result = new HashMap<>(tables.size(), 1F);
try (PreparedStatement preparedStatement = connection.prepareStatement(getTableMetaDataSQL(tables, connection.getMetaData()))) {
Map<String, Integer> dataTypes = new DataTypeLoader().load(connection.getMetaData(), getType());
Map<String, Collection<String>> tablePrimaryKeys = loadTablePrimaryKeys(connection, tables);
preparedStatement.setString(1, connection.getSchema());
try (ResultSet resultSet = preparedStatement.executeQuery()) {
while (resultSet.next()) {
String tableName = resultSet.getString("TABLE_NAME");
// System.out.println("" + tableName);
//add by pansh
if(tableName.startsWith("SREF_CON_TAB")){
continue;
}
ColumnMetaData columnMetaData = loadColumnMetaData(dataTypes, resultSet, tablePrimaryKeys.getOrDefault(tableName, Collections.emptyList()), connection.getMetaData());
if (!result.containsKey(tableName)) {
result.put(tableName, new LinkedList<>());
}
result.get(tableName).add(columnMetaData);
}
}
}
return result;
}
private ColumnMetaData loadColumnMetaData(final Map<String, Integer> dataTypeMap, final ResultSet resultSet, final Collection<String> primaryKeys,
final DatabaseMetaData databaseMetaData) throws SQLException {
String columnName = resultSet.getString("COLUMN_NAME");
String dataType = getOriginalDataType(resultSet.getString("DATA_TYPE"));
boolean primaryKey = primaryKeys.contains(columnName);
boolean generated = versionContainsIdentityColumn(databaseMetaData) && "YES".equals(resultSet.getString("IDENTITY_COLUMN"));
String collation = versionContainsCollation(databaseMetaData) ? resultSet.getString("COLLATION") : null;
boolean caseSensitive = null != collation && collation.endsWith("_CS");
boolean isVisible = "NO".equals(resultSet.getString("HIDDEN_COLUMN"));
boolean nullable = "Y".equals(resultSet.getString("NULLABLE"));
return new ColumnMetaData(columnName, dataTypeMap.get(dataType), primaryKey, generated, caseSensitive, isVisible, false, nullable);
}
private String getOriginalDataType(final String dataType) {
int index = dataType.indexOf('(');
if (index > 0) {
return dataType.substring(0, index);
}
return dataType;
}
private String getTableMetaDataSQL(final Collection<String> tables, final DatabaseMetaData databaseMetaData) throws SQLException {
StringBuilder stringBuilder = new StringBuilder(28);
if (versionContainsIdentityColumn(databaseMetaData)) {
stringBuilder.append(", IDENTITY_COLUMN");
}
if (versionContainsCollation(databaseMetaData)) {
stringBuilder.append(", COLLATION");
}
String collation = stringBuilder.toString();
return tables.isEmpty() ? String.format(TABLE_META_DATA_SQL, collation)
: String.format(TABLE_META_DATA_SQL_IN_TABLES, collation, tables.stream().map(each -> String.format("'%s'", each)).collect(Collectors.joining(",")));
}
private boolean versionContainsCollation(final DatabaseMetaData databaseMetaData) throws SQLException {
return databaseMetaData.getDatabaseMajorVersion() >= COLLATION_START_MAJOR_VERSION && databaseMetaData.getDatabaseMinorVersion() >= COLLATION_START_MINOR_VERSION;
}
private boolean versionContainsIdentityColumn(final DatabaseMetaData databaseMetaData) throws SQLException {
return databaseMetaData.getDatabaseMajorVersion() >= COLLATION_START_MAJOR_VERSION && databaseMetaData.getDatabaseMinorVersion() >= IDENTITY_COLUMN_START_MINOR_VERSION;
}
private Map<String, Collection<IndexMetaData>> loadIndexMetaData(final Connection connection, final Collection<String> tableNames) throws SQLException {
Map<String, Collection<IndexMetaData>> result = new HashMap<>(tableNames.size(), 1F);
try (PreparedStatement preparedStatement = connection.prepareStatement(getIndexMetaDataSQL(tableNames))) {
preparedStatement.setString(1, connection.getSchema());
try (ResultSet resultSet = preparedStatement.executeQuery()) {
while (resultSet.next()) {
String indexName = resultSet.getString("INDEX_NAME");
String tableName = resultSet.getString("TABLE_NAME");
if(tableName.startsWith("SREF_CON_TAB")){
continue;
}
boolean isUnique = "UNIQUE".equals(resultSet.getString("UNIQUENESS"));
if (!result.containsKey(tableName)) {
result.put(tableName, new LinkedList<>());
}
IndexMetaData indexMetaData = new IndexMetaData(indexName);
indexMetaData.setUnique(isUnique);
indexMetaData.getColumns().addAll(loadIndexColumnNames(connection, tableName, indexName));
result.get(tableName).add(indexMetaData);
}
}
}
return result;
}
private List<String> loadIndexColumnNames(final Connection connection, final String tableName, final String indexName) throws SQLException {
try (PreparedStatement preparedStatement = connection.prepareStatement(INDEX_COLUMN_META_DATA_SQL)) {
preparedStatement.setString(1, connection.getSchema());
preparedStatement.setString(2, tableName);
preparedStatement.setString(3, indexName);
List<String> result = new LinkedList<>();
ResultSet resultSet = preparedStatement.executeQuery();
while (resultSet.next()) {
result.add(resultSet.getString("COLUMN_NAME"));
}
return result;
}
}
private String getIndexMetaDataSQL(final Collection<String> tableNames) {
return String.format(INDEX_META_DATA_SQL, tableNames.stream().map(each -> String.format("'%s'", each)).collect(Collectors.joining(",")));
}
private Map<String, Collection<String>> loadTablePrimaryKeys(final Connection connection, final Collection<String> tableNames) throws SQLException {
Map<String, Collection<String>> result = new HashMap<>();
try (PreparedStatement preparedStatement = connection.prepareStatement(getPrimaryKeyMetaDataSQL(connection.getSchema(), tableNames))) {
try (ResultSet resultSet = preparedStatement.executeQuery()) {
while (resultSet.next()) {
String columnName = resultSet.getString("COLUMN_NAME");
String tableName = resultSet.getString("TABLE_NAME");
result.computeIfAbsent(tableName, k -> new LinkedList<>()).add(columnName);
}
}
}
return result;
}
private String getPrimaryKeyMetaDataSQL(final String schemaName, final Collection<String> tables) {
return tables.isEmpty() ? String.format(PRIMARY_KEY_META_DATA_SQL, schemaName)
: String.format(PRIMARY_KEY_META_DATA_SQL_IN_TABLES, schemaName, tables.stream().map(each -> String.format("'%s'", each)).collect(Collectors.joining(",")));
}
@Override
public String getDatabaseType() {
return CommonConstants.DM;
}
}
public class DmStatementVisitorFacade implements SQLStatementVisitorFacade {
@Override
public Class<? extends DMLStatementVisitor> getDMLVisitorClass() {
return OracleDMLStatementVisitor.class;
}
@Override
public Class<? extends DDLStatementVisitor> getDDLVisitorClass() {
return OracleDDLStatementVisitor.class;
}
@Override
public Class<? extends TCLStatementVisitor> getTCLVisitorClass() {
return OracleTCLStatementVisitor.class;
}
@Override
public Class<? extends DCLStatementVisitor> getDCLVisitorClass() {
return OracleDCLStatementVisitor.class;
}
@Override
public Class<? extends DALStatementVisitor> getDALVisitorClass() {
return OracleDALStatementVisitor.class;
}
@Override
public Class<? extends RLStatementVisitor> getRLVisitorClass() {
return null;
}
@Override
public String getDatabaseType() {
return CommonConstants.DM;
}
}
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] When will it be compatible with Dameng 8 database? [shardingsphere]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #37756:
URL: https://github.com/apache/shardingsphere/issues/37756#issuecomment-3802178872
There hasn't been any activity on this issue recently, and in order to prioritize active issues, it will be marked as stale.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] When will it be compatible with Dameng 8 database? [shardingsphere]
Posted by "terrymanu (via GitHub)" <gi...@apache.org>.
terrymanu commented on issue #37756:
URL: https://github.com/apache/shardingsphere/issues/37756#issuecomment-3769570956
Who knows in open source community :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] When will it be compatible with Dameng 8 database? [shardingsphere]
Posted by "pan1998 (via GitHub)" <gi...@apache.org>.
pan1998 commented on issue #37756:
URL: https://github.com/apache/shardingsphere/issues/37756#issuecomment-3764976876
> ShardingSphere does not officially plan to support Xinchuang (信创) databases. If your enterprise has such requirements, you can contact SphereEx—the company behind ShardingSphere's commercially supported enterprise edition. Their DBPlusEngine product supports various types of non-open-source databases.
>
> Company website: https://www.sphere-ex.com/ For a comparison between the open-source and enterprise editions: https://www.sphere-ex.cn/comparison/
What is the price of SphereEx?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] When will it be compatible with Dameng 8 database? [shardingsphere]
Posted by "terrymanu (via GitHub)" <gi...@apache.org>.
terrymanu commented on issue #37756:
URL: https://github.com/apache/shardingsphere/issues/37756#issuecomment-3762583870
ShardingSphere does not officially plan to support Xinchuang (信创) databases. If your enterprise has such requirements, you can contact SphereEx—the company behind ShardingSphere's commercially supported enterprise edition. Their DBPlusEngine product supports various types of non-open-source databases.
Company website: https://www.sphere-ex.com/
For a comparison between the open-source and enterprise editions: https://www.sphere-ex.cn/comparison/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] When will it be compatible with Dameng 8 database? [shardingsphere]
Posted by "terrymanu (via GitHub)" <gi...@apache.org>.
terrymanu closed issue #37756: When will it be compatible with Dameng 8 database?
URL: https://github.com/apache/shardingsphere/issues/37756
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] When will it be compatible with Dameng 8 database? [shardingsphere]
Posted by "linghengqian (via GitHub)" <gi...@apache.org>.
linghengqian commented on issue #37756:
URL: https://github.com/apache/shardingsphere/issues/37756#issuecomment-3759633867
If dm8 doesn't provide a Docker image on Docker Hub or ghcr, it's impossible to write integration tests for it on the master branch.
Considering that dm8, like Vertica CE, doesn't allow developers to run the database without a license, this also raises legal issues similar to those discussed in https://github.com/testcontainers/testcontainers-java/issues/2496 . If you feel that the legal issues of running dm8 in testcontainers are not a concern, you can certainly consider using the dm8 documentation you have and writing a dm8-compatible database using an AI agent. Then, the master branch can write integration tests against the new database. Considering the current state of Claude Code and OpenCode, this has happened before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@shardingsphere.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org