You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "David Capwell (Jira)" <ji...@apache.org> on 2021/01/29 20:24:00 UTC
[jira] [Commented] (CASSANDRA-16321) 4.0 Quality: Python Upgrade
dtest Audit
[ https://issues.apache.org/jira/browse/CASSANDRA-16321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275304#comment-17275304 ]
David Capwell commented on CASSANDRA-16321:
-------------------------------------------
Sorry for the wait, here is my audit so far. I did not finish upgrade_tests/cql_tests.py but the rest of the upgrade_tests were completed. I tried to come up with a short hand to show what tests are doing so will be missing some subtle differences, but the main groups of things touched are mostly covered.
TLDR - majority of tests wipe the data after upgrade and restart with a clean state, only a very small amount of tests test against previous data and/or upgraded data. Only a subset of types are used (though not documented), and almost no concurrent operations are performed. There is a decent coverage of COMPACT STORAGE but its a hodgepodge of covered there vs not; majority of tests could be replaced with a simple property test org.apache.cassandra.net.MessageSerializationPropertyTest but with rolling upgrades
Upgrade Test Audit
Scope
* Tests that include 4.0
Not run in 4.0
* upgrade_tests
* bootstrap_upgrade_test.py
* drop_compact_storage_upgrade_test.py
* repair_test.py
upgrade_tests/compatibility_flag_test.py
TestCompatibilityFlag
Action Item: disable for 4.0
* test__compatibility_flag_on_3014 - Relies on -Dcassandra.force_3_0_protocol_version=true which is no longer in the code; grep -r 'force_3_0_protocol_version' src/java/
* test__compatibility_flag_off_3014: write 6 rows then read them...
upgrade_tests/cql_tests.py
TODO - not done
TestCQL (tests=148)
is_40_or_greater is incorrect, so when we have a 5.0 or 4.1 the test will do the wrong thing
All tests have the same pattern: for each node in cluster, upgrade 1 node, truncate data, run test. So the only data that exists pre-upgrade is schema
* test_static_cf - Feature(BATCH), test syntax which pulls keyspace from "USE <ks>"; runs in mixed mode
* test_noncomposite_static_cf - Table(COMPACT STORAGE(CK=0, V=3)), Feature(BATCH); basically the same as test_static_cf, and special cases results for 4.0 when doing DROP COMPACT STORAGE (adds null columns)
* test_dynamic_cf - Table(COMPACT STORAGE(CK=1, V=1)); basically the same as above, but the
* test_dense_cf - Table(COMPACT STORAGE(CK=2, V=1)); more robust version of the previous tests. This also tests COMPACT STORAGE where last clustering column is not defined (and inserts null; non-compact storage tables do not allow this).
* test_sparse_cf - Table(COMPACT STORAGE(CK=2, V=2)); similar test as test_static_cf but without batching
* test_limit_ranges - Table(COMPACT STORAGE(CK=1, V=1)); Clause(LIMIT 1), query by token
* test_limit_multiget - Table(COMPACT STORAGE(CK=1, V=1)); Clause(LIMIT 1) (same as test_limit_ranges but without token)
* test_simple_tuple_query - Feature(ALLOW FILTERING) - 5 column partition key (no value); on partial columns
* test_limit_sparse - test LIMIT 4 returns 4 rows...
* test_counters - Table(COMPACT STORAGE(CK=1, V=1)), Feature(Counter), test counter increment/decrement
* test_indexed_with_eq - Feature(2i). Select partition with column = not present, select partition with column = present
* test_select_key_in - Clause(IN)
* test_exclusive_slice - Table(COMPACT STORAGE(CK=1, V=1)), Clause(Range)
* test_in_clause_wide_rows - Table(COMPACT STORAGE(CK=1, V=1), COMPACT STORAGE(CK=2 V=1)). Clause(IN)
* test_order_by - Table(COMPACT STORAGE(CK=1, V=1), COMPACT STORAGE(CK=2 V=1)), Clause(Order By)
* test_more_order_by - Table(COMPACT STORAGE(CK=1, V=1), COMPACT STORAGE(CK=2 V=1)). Same as test_order_by, but with different data
* test_order_by_validation - Clause(ORDER BY), make sure we block ORDER BY where partition key is missing
* test_order_by_with_in - Clause(Order By, IN)
* test_reversed_comparator - Table(CLUSTERING ORDER BY), Clause(ORDER BY)
* test_null_support - null support for normal columns
* test_nameless_index - Feature(2i). Unnamed index. Comment says to drop, but doesn't drop
* test_deletion - Table(COMPACT STORAGE(CK=1, V=1)) - test delete of a row
* test_count - Aggregate(COUNT(*), COUNT(1)), Clause(IN)
* test_batch - Feature(BATCH) - does not validate correctness; assumes no error == success. This looks to mimic test_static_cf
* test_token_range - Clause(RANGE, TOKEN)
* test_timestamp_and_ttl -
upgrade_tests/paging_test.py
All tests have the same pattern: for each node in cluster, upgrade 1 node, truncate data, run test. So the only data that exists pre-upgrade is schema
* TestPagingSize
* test_with_no_results - Query(fetch_size) - when no data, paging doesn't fail. Also checks paging state (client api)
* test_with_less_results_than_page_size - Query(fetch_size) , checks number of rows match (but doesn't check content)
* test_with_more_results_than_page_size - Query(fetch_size), check paging state (client), validate data
* test_with_equal_results_to_page_size - Query(fetch_size), same as test_with_more_results_than_page_size but with 1 page
* test_undefined_page_size_default - default fetch size
* TestPagingWithModifiers
* test_with_order_by - Table(CLUSTERING ORDER BY), Query(fetch_size). Makes sure we reject paging cross multiple partitions
* test_with_order_by_reversed - Table(CLUSTERING ORDER BY(DESC)), Query(fetch_size), Clause(Order By). Tests pager state, data correctness. Reverse iteration
* test_with_limit - Query(fetch_size), Clause(LIMIT). Tests a combination of cases!
* test_with_allow_filtering - Query(fetch_size), Feature(ALLOW FILTERING)
* TestPagingData
* test_basic_paging - Table(COMPACT STORAGE(CK=1, V=1)), Query(fetch_size). Rerun the test with different fetch sizes; check data equality
* test_basic_compound_paging - Table(COMPACT STORAGE(CK=2, V=1)). Same as test_basic_paging, but with a compact table with composite keys
* test_paging_a_single_wide_row - Query(fetch_size). Tests with 10k rows and large fetch size
* test_paging_across_multi_wide_rows - Query(fetch_size), Clause(IN). 5k rows per partition, IN clause to hit both for the pager
* test_paging_using_secondary_indexes - Feature(2i), Query(fetch_size)
* test_static_columns_paging - Feature(static column), Query(fetch_size), Clause(IN, ORDER BY DESC, >, <, RANGE)
* test_paging_using_secondary_indexes_with_static_cols - Feature(2i, static column), Query(fetch_size)
* TestPagingDatasetChanges
* test_data_change_impacting_earlier_page - Query(fetch_size), Clause(IN). Check what happens after we fetch a page (full partition) and the patcher changes
* test_data_change_impacting_later_page - Query(fetch_size), Clause(IN). Checks that the write that impacts second page is visible once we hit that page (anomaly)
* test_row_TTL_expiry_during_paging - Feature(Row TTL), Query(fetch_size), Clause(IN). TTLs impacting pages not sent yet will be dropped before sending
* test_cell_TTL_expiry_during_paging - Feature(Cell TTL), Query(fetch_size), Clause(IN), Statement(UPDATE). Same as test_row_TTL_expiry_during_paging but at the cell level
* TestPagingQueryIsolation
* test_query_isolation - Query(fetch_size), Clause(IN). Checks that multiple open pages do not pollute each other
* TestPagingWithDeletions
* test_single_partition_deletions - Query(fetch_size), Clause(IN), Statement(DELETE). Delete partitions and make sure the returned data is correct
* test_multiple_partition_deletions - Same as test_single_partition_deletions, but use IN clause while doing the delete
* test_single_row_deletions - same as test_single_partition_deletions but row delete
* test_single_cell_deletions - same as test_single_partition_deletions but cell delete
* test_multiple_cell_deletions - same as test_single_cell_deletions, but delete multiple cells for the row
* test_ttl_deletions - Feature(Row TTL), Query(fetch_size) - after all TTLs complete, query expecting no results
* test_failure_threshold_deletions - Feature(Cell Delete), Query(fetch_size), Config(tombstone_failure_threshold) - makes sure there are enough tombstones to tricker threshold failure, and makes sure paging hits it. This makes sure a single page is large enough to hit the threshold, so does NOT test cross-page tombstones
upgrade_tests/regression_test.py
* TestForRegressions
* test_10822 - Statement(DELETE), Aggregate(COUNT(*)). Write, then delete parts of the data, then verify that COUNT(*) is correct after upgrade
* test13294 - rename sstables (change generation), after upgrading rewrite the first sstable via compaction then verify that before/after compaction has the same number of files.
upgrade_tests/storage_engine_upgrade_test.py
* TestStorageEngineUpgrade
* test_update_and_drop_column - Statement(ALTER TABLE DROP column) - write data, upgrade, drop column, read data
* test_upgrade_with_clustered_CQL_table - data data, upgrade, validate selects return proper data; compact, then repeat
* test_upgrade_with_clustered_compact_table - same as test_upgrade_with_clustered_CQL_table but with compact storage
* test_upgrade_with_unclustered_CQL_table - basically the same as test_upgrade_with_clustered_CQL_table but without a clustering key
* test_upgrade_with_unclustered_compact_table - Table(COMPACT STORAGE(CK=0, V=4)) same as test_upgrade_with_unclustered_CQL_table
* test_upgrade_with_statics - Feature(static column) - write before upgrade, read after upgrade, read after compact
* test_upgrade_with_wide_partition_and_statics - same as test_upgrade_with_statics but with more rows
* test_upgrade_with_wide_partition - Feature(Cell Delete, Row Delete) - write, upgrade read, compact read
* test_upgrade_with_wide_partition_reversed - Table(CLUSTERING ORDER BY(DESC)) same as test_upgrade_with_wide_partition
* test_upgrade_with_index - Feature(2i) - write, upgrade read, compact read
* test_upgrade_with_range_tombstones - Feature(Range Tombstone). Write, upgrade read, compact (no read)
* test_upgrade_with_range_and_collection_tombstones - Feature(Collection(List)) - write, upgrade, compact, read partition key (not collection...)
* test_upgrade_with_range_tombstone_ae - Feature(Range Tombstone). Create table, create range tombstones, upgrade, select (expect no rows)
upgrade_tests/thrift_upgrade_test.py
* TestUpgradeSuperColumnsThrough
* test_dense_supercolumn_3_0_created - Feature(thrift, column_type=SUPER). Write super column, upgrade, read
* test_dense_supercolumn - Feature(thrift, column_type=SUPER(column_metadata=0)). Same as test_dense_supercolumn_3_0_created but start in 2.2, upgrade to 3.0, then upgrade to latest
* test_sparse_supercolumn - Feature(thrift, column_type=SUPER(column_metadata=2)). Start in 2.2, upgrade to 3.0, upgrade to latest; verify reads at each version
* TestUpgradeTo40
* test_keys_index_3_0_created - Feature(thrift). Using CQL, validate read/writes/deletes work for thrift created table, upgrade from 3.0 to latest
* test_keys_index_3_11_created - same as test_keys_index_3_0_created but in 3.11
upgrade_tests/upgrade_compact_storage.py
* TestUpgradeSuperColumnsThrough
* test_mixed_cluster - in mixed mode, create table w/ COMPACT STORAGE fails
* test_upgrade_with_dropped_compact_storage - Table(COMPACT STORAGE(CK=0, V=2)). Create table, drop CS, upgrade, read
* test_upgrade_with_dropped_compact_storage_index - Table(COMPACT STORAGE(CK=0, V=1)), Feature(2i). Insert data w/ index, drop CS, read, upgrade, read
upgrade_tests/upgrade_schema_agreement_test.py
* TestSchemaAgreementUpgrade
* runs several tests but non use the provided version, it relies on hard coded 3.0, 3.11, and a magic snazy/schema-migration-upgrade-bug-3.11 branch...
* TODO disable running the test in 4.0
upgrade_tests/upgrade_supercolumns_test.py
* TestSCUpgrade
* test_upgrade_super_columns_through_all_versions - Table(COMPACT STORAGE(CK=2, V=1)) upgrade from 2.1 -> 2.2 -> 3.0 -> 3.11 -> trunk with pre-created SSTables, calling nodetool upgradesstables -a before querying
* test_upgrade_super_columns_through_limited_versions - same as test_upgrade_super_columns_through_all_versions but only 3.0 -> trunk
upgrade_tests/upgrade_udtfix_test.py
* UpgradeUDTFixTest
* test_udtfix_in_sstable - Feature(frozen UDT) - create table with UDT, add data, upgrade, validate UDT upgrade log is present, read rows
* test_udtfix_in_messaging -same as test_udtfix_in_sstable but with 2 tables
upgrade_tests/upgrade_through_versions_test.py
* TestUpgrade
* test_parallel_upgrade - Feature(2i, counter), Statement(UPDATE, UPDATE counter + 1)
* test_rolling_upgrade - same as test_parallel_upgrade but with a rolling upgrade
* test_parallel_upgrade_with_internode_ssl - same as test_parallel_upgrade but with ssl
* test_rolling_upgrade_with_internode_ssl - same as test_rolling_upgrade but with ssl
* test_bootstrap - same as test_parallel_upgrade but with bootstrapping nodes
* test_bootstrap_multidc - same as test_bootstrap but with multi dc
> 4.0 Quality: Python Upgrade dtest Audit
> ---------------------------------------
>
> Key: CASSANDRA-16321
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16321
> Project: Cassandra
> Issue Type: Task
> Components: CI, Test/dtest/python
> Reporter: David Capwell
> Assignee: David Capwell
> Priority: Normal
> Fix For: 4.0-rc
>
>
> This is a subtask of CASSANDRA-15588 focusing on python dtest upgrade tests.
> This ticket is to document and describe the existing coverage so we have a broad view of what is currently covered. This should be used to come up with new test cases which should be managed in separate tickets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org