You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "andrii-kysylevskyi (via GitHub)" <gi...@apache.org> on 2025/11/02 20:46:03 UTC
Re: [I] Add ScyllaDB support documentation and tests [pulsar]
andrii-kysylevskyi commented on issue #24924:
URL: https://github.com/apache/pulsar/issues/24924#issuecomment-3478298746
Hi @lhotari,
Thanks a lot for the instructions. Unfortunately, the provided steps did not help me validate the new tests
The integration tests fail during the sink creation/validation phase, before any actual data processing occurs.
```
/pulsar/bin/pulsar-admin sink create --tenant public --namespace default
--name test-sink-connector-...
--archive /pulsar/connectors/pulsar-io-cassandra-4.2.0-SNAPSHOT.nar
--classname org.apache.pulsar.io.cassandra.CassandraStringSink
--sinkConfig {"keyspace":"...","roots":"scylladb","columnName":"col"}
--inputs test-sink-connector-...-input-topic
--ram 134217728
failed with error code 1
```
It looks like the Pulsar worker/broker container cannot successfully connect to the database container (Cassandra or ScyllaDB) during the validation phase. This seems to be happening both on the new tests as well as existing Cassandra tests.
I've tried creating some standalone tests that wouldn't depend on the existing infra, and they seemed to have passed. Something along the lines of
```
@Test
public void testScyllaDBBasicOperations() {
// 1. Start ScyllaDB container (TestContainers)
scyllaContainer = new GenericContainer<>("scylladb/scylla:2025.1.4")
.withExposedPorts(9042)
.start();
// 2. Connect using DataStax Cassandra driver (same driver Pulsar uses)
cluster = Cluster.builder()
.addContactPoint("localhost")
.withPort(mappedPort)
.build();
session = cluster.connect();
// 3. Execute same CQL commands as Pulsar sink
session.execute("CREATE KEYSPACE test_keyspace WITH replication = {...}");
session.execute("CREATE TABLE test_table (key text PRIMARY KEY, col text)");
// 4. Insert test data (20 key-value pairs)
for (int i = 0; i < 20; i++) {
session.execute("INSERT INTO test_table VALUES (?, ?)", "key-" + i, "value-" + i);
}
// 5. Read and validate data integrity
ResultSet result = session.execute("SELECT * FROM test_table");
assertEquals(result.all().size(), 20); // ✅ All records present and correct
}
```
Basically, testing that commands Pulsar tests are running using the DataStax Cassandra driver that Pulsar uses.
I've tried to make the full testing infra run for a while, but couldn't get it to work. If there is a way to launch CI/CD pipeline for my branch, that would be amazing. Here is the PR for this change in the core project - https://github.com/apache/pulsar/pull/24931
Thanks a lot in advance!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org